MySQL Query 127 seconds of execution - php

I have this SQL Query:
SELECT * FROM magnitudes t1 INNER JOIN value_magnitudes t2 ON t1.id = t2.magnitude_id WHERE t1.floor_id = 42 AND t2.reading_date = (SELECT Max(reading_date) FROM value_magnitudes WHERE t2.magnitude_id = t1.id);
And my DB Schema is:
CREATE TABLE magnitudes (
id INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
name VARCHAR(255) NULL ,
sdi_id VARCHAR(255) NULL ,
sdi_id_floor VARCHAR(255) NULL ,
visible TINYINT(1) UNSIGNED NULL ,
history TINYINT(1) UNSIGNED NULL ,
created_at DATETIME NULL ,
updated_at DATETIME NULL ,
floor_id INT(11) NULL ,
unit VARCHAR(255) NULL ,
kind_id INT(11) NULL ,
sample_time INT(11) NULL ,
history_from DATETIME NULL DEFAULT 2011-02-22 11:18:07 ,
PRIMARY KEY(id));
CREATE TABLE value_magnitudes (
id INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
value FLOAT NULL ,
magnitude_id INT(11) NULL ,
sdi_belongs_id VARCHAR(255) NULL ,
reading_date DATETIME NULL ,
created_at DATETIME NULL ,
updated_at DATETIME NULL ,
PRIMARY KEY(id));
What I actually want is, get the last value_magnitudes depending on the last updated_at by magnitudes.id on values_magnitudes.magnitude_id
Thanks.


SELECT TOP 1 VM.* FROM value_magnitudes VM, magnitudes M WHERE M.id = VM.id ORDER BY updated_at DESC


SELECT VM.* FROM value_magnitudes VM, magnitudes M WHERE M.id = VM.id ORDER BY updated_at DESC LIMIT 0,1;

Related

MySQL Math on "join" function

Not too sure how to explain this, but, basically I have two tables, and I need to do some math on their values. I am having issues with anything other than addition (which it does automatically).
This is the MySQL statement that I am using.
select snippet_id, count(snippet_id) as cnt
from snippets_likes join snippets_engagement on snippet_id = snippets_engagement.snip_id
group by snippet_id
order by cnt desc
This statements pulls the total number of likes + engagements.
However, I want to have it equal likes + ( engagements/1000 ).
Table construction
CREATE TABLE `IOTunes`.`snippets_engagement` (
`ID` int(10) unsigned NOT NULL AUTO_INCREMENT,
`snip_id` int(10) unsigned NOT NULL DEFAULT '0',
`artist_id` int(10) unsigned NOT NULL DEFAULT '0',
`snip_timestamp` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`engagement_type` int(10) unsigned NOT NULL DEFAULT '0',
PRIMARY KEY (`ID`)
) ENGINE=InnoDB AUTO_INCREMENT=6694 DEFAULT CHARSET=latin1;
CREATE TABLE `IOTunes`.`snippets_likes` (
`ID` int(10) unsigned NOT NULL AUTO_INCREMENT,
`snippet_id` int(10) unsigned NOT NULL DEFAULT '0',
`artist_id` int(10) unsigned NOT NULL DEFAULT '0',
`snippet_like` int(10) unsigned NOT NULL DEFAULT '0',
PRIMARY KEY (`ID`)
) ENGINE=InnoDB AUTO_INCREMENT=197 DEFAULT CHARSET=latin1;
I have tried so many different formulas, but, cannot get this to work. Any pointers?

Try this statement:
SELECT snippet_id, SUM(cnt) AS cnt
FROM (
SELECT snippet_id, COUNT(1) AS cnt
FROM snippets_likes
GROUP BY snippet_id
UNION
SELECT snippet_id, COUNT(1)/1000 AS cnt
FROM snippets_engagement
GROUP BY snippet_id
) AS subQ
GROUP BY snippet_id
ORDER BY cnt DESC
or if you insist on a join...
select sl.snippet_id, count(DISTINCT sl.ID) + count(DISTINCT se.ID)/1000 as cnt
from snippets_likes AS sl join snippets_engagement AS se
on sl.snippet_id = se.snip_id
group by snippet_id
order by cnt desc
I wouldn't recommend the join though, you'll only get values for "snippets" that have at least one "like" and one "engagement".

Improve speed of MySQL query [closed]

It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.
Closed 10 years ago.
EDIT:
I reworked the query as follows:
SELECT
a.title, count(*),at.search_code
FROM
`qitz3_attributes_type` at
left join
qitz3_attributes a
on
a.attribute_type_id = at.id
left join
qitz3_attributes_property ap
on
ap.attribute_id = a.id
left join
qitz3_helloworld h
on
h.id = ap.property_id
where
at.id in (1,2,8,9,11)
and
a.search_filter = 1
and
h.area=506
and
h.expiry_date >= '2013-02-20 13:28:04'
group by
a.title
order by search_code
This seems much faster but I am still getting a using temporary and using filesort on the explain...
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE at range PRIMARY PRIMARY 4 NULL 5 Using where; Using temporary; Using filesort
1 SIMPLE a ref PRIMARY,Attribute type ID,Search filter,Attribute ... Attribute type ID 4 password.at.id 6 Using where
1 SIMPLE ap ref Property ID,Attribute ID,Attribute Property Attribute Property 4 password.a.id 142 Using where; Using index
1 SIMPLE h eq_ref PRIMARY,Area indexes,Expiry date PRIMARY 4 password.ap.property_id 1 Using where
ENDEDIT
I am working on a search component for a site I am developing and while it is all working there are a couple of queries that I would like to see running a touch faster.
In order to populate a set of search filters (display the count of the number of properties that have each facility or are of a particular type) I am using the following two queries. The first will get a list of IDs that I then plug into the second query as follows:
Based on the following info can anyone suggest a more efficient way to do this? I would really like to speed the first query up as it seems a little slow compared to the others.
Is 150ms actually that slow? 15ms would be better...
Thanks,
Adam
Query 1 (takes around 150ms):
SELECT h.id, h.parent_id, h.level, h.title as property_title, h.area, h.region, h.department, h.city,
LEFT(h.description, 250) as description, h.thumbnail, h.occupancy, h.swimming, g.path, (single_bedrooms + double_bedrooms + triple_bedrooms + quad_bedrooms + twin_bedrooms) as bedrooms, c.title as location_title, ( select min(tariff)
from qitz3_tariffs
where id = h.id ) as price, e.title as tariff_based_on, f.title as base_currency, a.title as property_type, a2.title as accommodation_type, ( select count(*)
from qitz3_reviews
where property_id = h.id
group by h.id ) as reviews
FROM qitz3_classifications c
LEFT JOIN qitz3_helloworld h
on c.id = h.area
LEFT JOIN qitz3_attributes_property ap
ON ap.property_id = h.id
LEFT JOIN qitz3_attributes_type at
ON at.id = ap.attribute_id
LEFT JOIN qitz3_attributes a
ON a.id = ap.attribute_id
LEFT JOIN qitz3_attributes_property ap2
ON ap2.property_id = h.id
LEFT JOIN qitz3_attributes_type at2
ON at2.id = ap2.attribute_id
LEFT JOIN qitz3_attributes a2
ON a2.id = ap2.attribute_id
LEFT JOIN qitz3_attributes e
ON e.id = h.tariff_based_on
LEFT JOIN qitz3_attributes f
ON f.id = h.base_currency
LEFT JOIN qitz3_classifications g
ON g.id = h.city
WHERE a.attribute_type_id = 1
AND a2.attribute_type_id = 2
AND c.id = 506
AND h.expiry_date >= '2013-02-20 12:05:13'
AND h.id > 1
Explain:
id select_type table type possible_keys key key_len ref rows Extra
1 PRIMARY c const PRIMARY PRIMARY 4 const 1
1 PRIMARY h ref PRIMARY,Area indexes,Expiry date Area indexes 4 const 615 Using where
1 PRIMARY ap ref Property ID,Attribute ID Property ID 4 password.h.id 21 Using where
1 PRIMARY at eq_ref PRIMARY PRIMARY 4 password.ap.attribute_id 1 Using index
1 PRIMARY a eq_ref PRIMARY,Attribute type ID PRIMARY 4 password.ap.attribute_id 1 Using where
1 PRIMARY ap2 ref Property ID,Attribute ID Property ID 4 password.ap.property_id 21 Using where
1 PRIMARY at2 eq_ref PRIMARY PRIMARY 4 password.ap2.attribute_id 1 Using index
1 PRIMARY a2 eq_ref PRIMARY,Attribute type ID PRIMARY 4 password.ap2.attribute_id 1 Using where
1 PRIMARY e eq_ref PRIMARY PRIMARY 4 password.h.tariff_based_on 1
1 PRIMARY f eq_ref PRIMARY PRIMARY 4 password.h.base_currency 1
1 PRIMARY g eq_ref PRIMARY PRIMARY 4 password.h.city 1
3 DEPENDENT SUBQUERY qitz3_reviews ref Property ID Property ID 4 password.h.id 1 Using index; Using temporary; Using filesort
2 DEPENDENT SUBQUERY qitz3_tariffs ref Property ID Property ID 4 password.h.id 2
Query 2 (takes around 30ms):
SELECT a.id,count(attribute_id) as count, a.title AS attribute, a.published, at.title as facility_type, at.search_code
FROM qitz3_attributes AS a
LEFT JOIN qitz3_attributes_type at
on at.id = a.attribute_type_id
LEFT JOIN qitz3_attributes_property ap
on ap.attribute_id = a.id
WHERE search_filter = 1
AND a.published = 1
AND property_id in (152843,103180,152845,4628,5653,3865,107553,155945,106029,107575,149052,837,104264,98635,98636,98637,98638,3667,106838,3672,157278,155743,157791,157792,151153,151155,100725,106109,157569,157576,107145,150666,103310,5780,3480,102041,154016,3490,154018,932,153518,151991,154041,154042,4288,5832,149451,157646,102094,148444,153822,157407,153839,151536,157393,157395,157428,157429,102236,104770,157378,157380,157381,157382,157383,157654,104768,103746,153683,150175,4618,101050,104942,157229,157230,98515,104771,104944,3612,148721,104212,5307,3432,156676,102706,404,5518,156359,5252,102697,5271,98979,101827,149524,102676,107551,5685,101736,156538,152703,4730,151881,95838,3759,149769,5269,98429,153729,5233,703,5579,98943,157433,3157,155661,107347,147545,147547,5216,106345,156533,93833,158091,96403,3491,968,158195,158196,157580,104148,3030,94686,154725,150582,103027,99062,102462,4384,5634,153874,157974,101669,47,105285,102481,102234,5749,156793,153748,96404,151467,154292,147645,97471,100551,102090,4841,3563,155643,4656,98424,157243,150601,157415,4701,102283,100719,100738,5643,98425,98972,103261,531,3105,98108,150592,5719,150616,157532,3974,3212,157581,97469,97470,149183,157638,149730,102114,156395,153621,102560,102913,94684,5609,157578,98423,98971,102151,146734,150585,104287,155296,104956,94592,102433,147575,156325,106344,101766,107058,106560,103026,157848,98973,4303,5620,149767,150563,4407,104268,97876,156784,156786,149922,701,154317,153821,102480,348,102778,102779,102780,102781,102479,352,103025,98677,5254,98697,3995,156322,100305,98532,3833,5374,150172,151435,102368,102380,157228,103171,147740,152870,103579,3870,104037,103016,4995,105104,157605,5811,955,147643,156648,107802,101502,94685,3569,148755,150293,4122,157013,157297,98676,156794,102848,157635,157640,95717,98980,102764,102777,102782,36,101765,154373,149829,154955,107683,158176,102557,157552,103163,5760,104627,153561,266,151335,151176,147620,147379,3085,155760,106339,151424,106759,5145,104990,97877,155495,5241,156407,156625,3236,152782,96066,147617,3860,4614,3497,147883,158207,102985,104622,101816,157275,149037,4792,149226,3496,101825,102538,150571,105015,97874,157391,158192,102562,155032,5383,102558,104194,156740,101446,147615,5815,107081,155992,97473,148817,945,101751,158074,4249,101792,4532,152828,104316,157319,156071,157508,157510,148836,4745,153823,157942,3859,157442,100017,102555,147629,149272,157845,151256,101481,154735,154737,157652,106232,97991,4660,3309,157597,407,157658,154152,157374,153385,148037,158214,100452)
GROUP BY a.id
This query gives a count of the number of properties with each attribute (e.g. Golf, Air conditioning, property type etc). The good thing about this is that only attributes are shown that have properties. So as the user drills down attributes with no properties are not shown. This is basically down to the first query where I get a list of the properties matching a particular set of attributes.
The tables are as follows:
--
-- Table structure for table qitz3_attributes
CREATE TABLE IF NOT EXISTS `qitz3_attributes` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`attribute_type_id` int(11) NOT NULL,
`title` varchar(75) NOT NULL,
`ordering` int(11) NOT NULL,
`state` tinyint(4) NOT NULL DEFAULT '1',
`published` int(11) NOT NULL,
`search_filter` bit(1) NOT NULL,
`language_code` varchar(6) NOT NULL DEFAULT 'en-GB',
PRIMARY KEY (`id`),
KEY `Attribute type ID` (`attribute_type_id`),
KEY `Search filter` (`search_filter`),
KEY `Published` (`published`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=1522 ;
--
-- Table structure for table qitz3_attributes_property
CREATE TABLE IF NOT EXISTS `qitz3_attributes_property` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`property_id` int(11) NOT NULL,
`attribute_id` int(11) NOT NULL,
PRIMARY KEY (`id`),
KEY `Property ID` (`property_id`),
KEY `Attribute ID` (`attribute_id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=66261 ;
--
-- Table structure for table qitz3_attributes_type
CREATE TABLE IF NOT EXISTS `qitz3_attributes_type` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`title` varchar(75) NOT NULL,
`language_code` varchar(6) NOT NULL,
`field_name` varchar(25) NOT NULL,
`search_code` varchar(25) NOT NULL,
`state` int(11) NOT NULL DEFAULT '1',
`published` int(11) NOT NULL DEFAULT '1',
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=34 ;
--
-- Table structure for table qitz3_classifications
CREATE TABLE IF NOT EXISTS `qitz3_classifications` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`parent_id` int(10) unsigned NOT NULL DEFAULT '0',
`lft` int(11) NOT NULL DEFAULT '0',
`rgt` int(11) NOT NULL DEFAULT '0',
`level` int(10) unsigned NOT NULL DEFAULT '0',
`title` varchar(255) NOT NULL,
`description` text NOT NULL,
`path` varchar(255) NOT NULL DEFAULT '',
`alias` varchar(255) NOT NULL,
`access` tinyint(3) unsigned NOT NULL DEFAULT '0',
`published` int(11) NOT NULL,
`longitude` float(10,6) NOT NULL,
`latitude` float(10,6) NOT NULL,
PRIMARY KEY (`id`),
KEY `idx_left_right` (`lft`,`rgt`),
KEY `Alias index` (`alias`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 AUTO_INCREMENT=158052 ;
--
-- Table structure for table qitz3_helloworld (Property table)
CREATE TABLE IF NOT EXISTS `qitz3_helloworld` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`parent_id` int(10) NOT NULL DEFAULT '0',
`lft` int(11) NOT NULL DEFAULT '0',
`rgt` int(11) NOT NULL DEFAULT '0',
`level` int(10) unsigned NOT NULL,
`alias` varchar(250) NOT NULL DEFAULT '',
`access` tinyint(3) unsigned NOT NULL DEFAULT '0',
`path` varchar(255) NOT NULL DEFAULT '',
`title` varchar(120) NOT NULL,
`area` int(11) NOT NULL DEFAULT '0',
`region` int(11) NOT NULL DEFAULT '0',
`department` int(11) NOT NULL DEFAULT '0',
`city` int(11) NOT NULL DEFAULT '0',
`params` text NOT NULL,
`created_by` int(10) NOT NULL DEFAULT '0',
`created_on` datetime NOT NULL,
`modified` datetime DEFAULT NULL,
`expiry_date` date DEFAULT NULL,
`availability_last_updated_on` datetime DEFAULT NULL,
`modified_by` int(11) DEFAULT NULL,
`lang` varchar(5) NOT NULL DEFAULT 'en-GB',
`description` mediumtext NOT NULL COMMENT 'The summary and description for this accommodation',
`internal_facilities_other` varchar(1000) NOT NULL,
`external_facilities_other` varchar(1000) NOT NULL,
`activities_other` varchar(5000) NOT NULL,
`location_details` varchar(5000) NOT NULL,
`getting_there` varchar(5000) NOT NULL,
`thumbnail` varchar(150) NOT NULL,
`occupancy` int(11) DEFAULT NULL,
`single_bedrooms` int(11) NOT NULL,
`double_bedrooms` int(11) NOT NULL,
`triple_bedrooms` int(11) DEFAULT NULL,
`quad_bedrooms` int(11) DEFAULT NULL,
`twin_bedrooms` int(11) DEFAULT NULL,
`childrens_beds` int(11) DEFAULT NULL,
`cots` int(11) DEFAULT NULL,
`extra_beds` int(11) DEFAULT NULL,
`bathrooms` int(11) NOT NULL,
`toilets` int(11) DEFAULT NULL,
`swimming` int(11) NOT NULL,
`latitude` decimal(10,7) DEFAULT NULL,
`longitude` decimal(10,7) DEFAULT NULL,
`nearest_town` varchar(50) DEFAULT NULL,
`distance_to_coast` int(11) DEFAULT NULL,
`additional_price_notes` varchar(3000) DEFAULT NULL,
`base_currency` int(11) DEFAULT NULL,
`tariff_based_on` int(11) DEFAULT NULL,
`linen_costs` varchar(250) DEFAULT NULL,
`changeover_day` int(11) DEFAULT NULL,
`published` tinyint(4) NOT NULL DEFAULT '0',
`video` tinyint(4) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`),
KEY `idx_left_right` (`lft`,`rgt`),
KEY `Area indexes` (`area`,`region`,`department`),
KEY `Expiry date` (`expiry_date`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=158249 ;
--
-- Table structure for table qitz3_reviews
CREATE TABLE IF NOT EXISTS `qitz3_reviews` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`property_id` int(11) NOT NULL,
`title` varchar(150) NOT NULL,
`review_text` varchar(4000) NOT NULL,
`date` date NOT NULL,
`rating` int(11) NOT NULL,
`guest_name` varchar(75) NOT NULL,
`guest_email` varchar(150) NOT NULL,
`state` tinyint(3) NOT NULL DEFAULT '0',
`published` tinyint(1) NOT NULL DEFAULT '0',
`created` datetime NOT NULL,
`created_by` int(11) NOT NULL,
PRIMARY KEY (`id`),
KEY `Property ID` (`property_id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=921 ;
--
-- Table structure for table qitz3_tariffs
CREATE TABLE IF NOT EXISTS `qitz3_tariffs` (
`tariff_id` int(11) NOT NULL AUTO_INCREMENT,
`id` int(11) NOT NULL COMMENT 'Denotes the property listing ID',
`start_date` date NOT NULL,
`end_date` date NOT NULL,
`tariff` int(11) NOT NULL COMMENT 'Price per booking period between the dates specified. dated spec',
PRIMARY KEY (`tariff_id`),
KEY `Property ID` (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=9267 ;

Quick play, but trying to move the subselects out from the field list.
SELECT h.id, h.parent_id, h.level, h.title as property_title, h.area, h.region, h.department, h.city, LEFT(h.description, 250) as description, h.thumbnail, h.occupancy, h.swimming, g.path, (single_bedrooms + double_bedrooms + triple_bedrooms + quad_bedrooms + twin_bedrooms) as bedrooms, c.title as location_title,
Sub1.price,
e.title as tariff_based_on, f.title as base_currency, a.title as property_type, a2.title as accommodation_type,
Sub2.reviews
FROM qitz3_classifications c
LEFT JOIN qitz3_helloworld h on c.id = h.area
LEFT JOIN qitz3_attributes_property ap ON ap.property_id = h.id
LEFT JOIN qitz3_attributes_type at ON at.id = ap.attribute_id
LEFT JOIN qitz3_attributes a ON a.id = ap.attribute_id
LEFT JOIN qitz3_attributes_property ap2 ON ap2.property_id = h.id
LEFT JOIN qitz3_attributes_type at2 ON at2.id = ap2.attribute_id
LEFT JOIN qitz3_attributes a2 ON a2.id = ap2.attribute_id
LEFT JOIN qitz3_attributes e ON e.id = h.tariff_based_on
LEFT JOIN qitz3_attributes f ON f.id = h.base_currency
LEFT JOIN qitz3_classifications g ON g.id = h.city
LEFT JOIN ( SELECT id, MIN(tariff) AS price FROM qitz3_tariffs GROUP BY id) Sub1 ON Sub1.Id = h.id
LEFT JOIN ( SELECT property_id, COUNT(*) AS reviews FROM qitz3_reviews GROUP BY property_id ) as Sub2 ON Sub2.property_id = h.id
WHERE a.attribute_type_id = 1
AND a2.attribute_type_id = 2
AND c.id = 506
AND h.expiry_date >= '2013-02-20 12:05:13'
AND h.id > 1

When you need more speed in searching data you should take a look at Solr or Sphinx. With this index servers you can index your MySQL-Data and query them.
Its much more faster then MySQL.

There's quite a lot wrong in your first query:
The first 7 LEFT JOINs should be INNER JOINs. For qitz3_attributes you should consider normalizing the data less aggressively (use multiple columns instead of multiple rows to describe the data.

This query/database is not working well together

Here's a query:
SELECT
*,
COUNT(*) as `numauth`
FROM `favorites` as `f1`
INNER JOIN `story` as `s1` ON `f1`.`story_id` = `s1`.`story_id`
WHERE `f1`.`story_id` != '".addslashes($_REQUEST['storyid'])."'
AND `f1`.`story_id` != '".addslashes($_REQUEST['storyid2'])."'
AND EXISTS (
SELECT 1 FROM `favorites` as `f2`
WHERE `story_id` = '".addslashes($_REQUEST['storyid'])."'
AND `f2`.`auth_id` = `f1`.`auth_id`)
AND EXISTS (
SELECT 1 FROM `favorites` as `f3`
WHERE `story_id` = '".addslashes($_REQUEST['storyid2'])."'
AND `f3`.`auth_id` = `f1`.`auth_id`)
AND NOT EXISTS (
SELECT 1 FROM `favorites` as `f4`
WHERE `story_id` =
'".addslashes($_REQUEST['exclude'])."'
`f4`.`auth_id` = `f1`.`auth_id`)
GROUP BY `f1`.`story_id`
ORDER BY `numauth` DESC, `story_words` DESC
And here's a description of the tables...
CREATE TABLE IF NOT EXISTS `favorites` (
`fav_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`auth_id` int(10) unsigned NOT NULL,
`story_id` int(10) unsigned NOT NULL,
PRIMARY KEY (`fav_id`),
UNIQUE KEY `auth_id_2` (`auth_id`,`story_id`),
KEY `auth_id` (`auth_id`),
KEY `story_id` (`story_id`),
KEY `fav_id` (`fav_id`,`auth_id`,`story_id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 AUTO_INCREMENT=1577985 ;
CREATE TABLE IF NOT EXISTS `story` (
`story_id` int(10) unsigned NOT NULL,
`story_title` varchar(255) NOT NULL,
`story_desc` text NOT NULL,
`story_authid` int(8) unsigned NOT NULL,
`story_authname` varchar(255) NOT NULL,
`story_fandom` varchar(255) NOT NULL,
`story_genre1` tinyint(2) unsigned NOT NULL,
`story_genre2` tinyint(2) unsigned NOT NULL,
`story_created` int(10) unsigned NOT NULL,
`story_updated` int(10) unsigned NOT NULL,
`story_reviews` smallint(5) unsigned NOT NULL,
`story_chapters` smallint(3) unsigned NOT NULL,
`story_rating` tinyint(2) unsigned NOT NULL,
`story_words` mediumint(7) unsigned NOT NULL,
`story_chars` varchar(255) NOT NULL,
UNIQUE KEY `story_id` (`story_id`),
KEY `story_authid` (`story_authid`),
KEY `story_fandom` (`story_fandom`),
KEY `story_authid_2` (`story_authid`,`story_fandom`),
KEY `story_id_2` (`story_id`,`story_authid`),
KEY `story_id_3` (`story_id`,`story_words`),
KEY `story_id_4` (`story_id`,`story_fandom`,`story_words`),
KEY `story_id_5` (`story_id`,`story_reviews`,`story_words`),
KEY `story_words` (`story_words`),
KEY `story_reviews` (`story_reviews`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
Now I've done a fair bit of optimizing to get the query down to this. I'm running on a dedicated server but the query is still taking 5-7 seconds, which is unacceptable. We're looking at about 800,000 records on favorites and 400,000 records on stories, and I'm lost at this point on where to look to next for improvements.
It seems a bit daunting, so even if someone can point me in the right direction I'll be happy.
EXPLAIN with sample inputs:
id select_type table type possible_keys key key_len ref rows Extra
1 PRIMARY s1 ALL story_id,story_id_2,story_id_3,story_id_4,story_id... NULL NULL NULL 129429 Using where; Using temporary; Using filesort
1 PRIMARY f1 ref story_id story_id 4 fanfic_jordanl_ffrecs.s1.story_id 2 Using where
4 DEPENDENT SUBQUERY f4 eq_ref auth_id_2,auth_id,story_id auth_id_2 8 fanfic_jordanl_ffrecs.f1.auth_id,const 1 Using index
3 DEPENDENT SUBQUERY f3 eq_ref auth_id_2,auth_id,story_id auth_id_2 8 fanfic_jordanl_ffrecs.f1.auth_id,const 1 Using index
2 DEPENDENT SUBQUERY f2 eq_ref auth_id_2,auth_id,story_id auth_id_2 8 fanfic_jordanl_ffrecs.f1.auth_id,const 1 Using index

try this:
SELECT f1.*, s1.*, COUNT(*) as `numauth`
FROM `favorites` as `f1`
INNER JOIN `story` as `s1` ON `f1`.`story_id` = `s1`.`story_id`
INNER JOIN (
SELECT auth_id
FROM favorites
WHERE story_id IN ('".addslashes($_REQUEST['storyid'])."', '".addslashes($_REQUEST['storyid2'])."', '".addslashes($_REQUEST['exclude'])."')
GROUP BY auth_id
HAVING Count(IF(story_id = '".addslashes($_REQUEST['exclude'])."', 1, NULL)) = 0 AND Count(*) = 2
) fv ON f1.auth_id = fv.auth_id
WHERE `f1`.`story_id` != '".addslashes($_REQUEST['storyid'])."'
AND `f1`.`story_id` != '".addslashes($_REQUEST['storyid2'])."'
GROUP BY `f1`.`story_id`
ORDER BY `numauth` DESC, `story_words` DESC
Since you are selecting * but not grouping by auth_id, what exactly are you trying to do?
--- UPDATE
since you do not need all the fav info for the stories, this query should perform better:
SELECT s.*, fv.cnt
FROM story s
JOIN (
SELECT fv.story_id, COUNT(*) cnt
FROM favorites fv
JOIN (
SELECT auth_id
FROM favorites
WHERE story_id IN ('".addslashes($_REQUEST['storyid'])."', '".addslashes($_REQUEST['storyid2'])."', '".addslashes($_REQUEST['exclude'])."')
GROUP BY auth_id
HAVING Count(IF(story_id = '".addslashes($_REQUEST['exclude'])."', 1, NULL)) = 0 AND Count(*) = 2
) ufv ON fv.auth_id = ufv.auth_id
WHERE story_id != '".addslashes($_REQUEST['storyid'])."' AND story_id != '".addslashes($_REQUEST['storyid2'])."'
GROUP BY fv.story_id
ORDER BY COUNT(*) DESC
LIMIT 25
) fv ON s.story_id = fv.story_id
ORDER BY fv.cnt DESC, `story_words` DESC

PHP and SQL Social Operations

I've got a table created using this query.
CREATE TABLE `tbl_friends` (
`friend_id` int(11) NOT NULL auto_increment,
`friend_status` tinyint(4) NOT NULL default '0',
`friend_id1` int(11) NOT NULL default '0',
`friend_id2` int(11) NOT NULL default '0',
`friend_date` datetime NOT NULL default '0000-00-00 00:00:00',
`friend_confirm` datetime NOT NULL default '0000-00-00 00:00:00',
PRIMARY KEY (`friend_id`)
) TYPE=MyISAM AUTO_INCREMENT=10 ;
I want to view the list of my friends, for that I only need to know other user's id - numeric value stored in friend_id1 or friend_id2 rows. My problem is that I don't know if friend_id1 or friend_id2 contains required data.
Please help me create a query to receive the number of another user if my number is $_SESSION['id']

CREATE TABLE `tbl_friends` (
`friend_id` int(11) NOT NULL auto_increment,
`friend_status` tinyint(4) NOT NULL default '0',
`friend_id1` int(11) NOT NULL default '0',
`friend_id2` int(11) NOT NULL default '0',
`friend_date` datetime NOT NULL default '0000-00-00 00:00:00',
`friend_confirm` datetime NOT NULL default '0000-00-00 00:00:00',
PRIMARY KEY (`friend_id`)
) TYPE=MyISAM AUTO_INCREMENT=10 ;
Would be like so:
SELECT * FROM tbl_friends WHERE friend_id2 = %d OR friend_id1 = %d AND friend_status != 0 ORDER BY friend_id
More Visual example:
SELECT
*
FROM
tbl_friends
WHERE
friend_id2 = %$1d
OR
friend_id1 = %$1d
AND
friend_status != 0
ORDER BY
friend_id
DESC
Then just run it threw a sprintf function and your ready.
$sql = sprintf($sql,$_SESSION['id']);

I'm not sure what you exactly mean, but I think your looking for something like:
select distinct * from tbl_friends where friend_id1 = $_SESSION["id"] OR friend_id2 = $_SESSION["id"];
If this is not what you mean, please add some additional information.

Something like:
Select distinct(id) from (
select friend_id1 as id from friends where friend_id2 = :my_id
union
select friend_id2 as id from friends where friend_id1 = :my_id
)
There real problem here is keeping the table from getting weird data. You are better off always putting two records into the table for each reciprocal relationship. Then you only need to say:
select * from friends where friend_id1 = :my_id and status = :whatever
If you do it this way, the control data for when friendship happens might need to move to another table

Getting Popular Topics on a Custom Made Forum

For this website we're working on, we're trying to get the most popular topics (based on how many posts have been made in them within the last 24 hours). We have a medium to large based forum, and the current MySQL query looks like this:
SELECT `forums_topics`.`id`,`forums_topics`.`name`,
(
SELECT COUNT(`id`)
FROM `forums_posts`
WHERE `postdate` > (UNIX_TIMESTAMP()-60*60*24)
AND `topicid`=`forums_topics`.`id`
) AS `trendy_threads`
FROM `forums_topics`
WHERE `deleted`=0
AND `lastpost` > (UNIX_TIMESTAMP()-60*60*24)
ORDER BY `trendy_threads` DESC,`postdate` DESC
LIMIT 3
The SQL is quite sluggish.
How can we get this information as quickly and as efficiently as possible?
forums_topics
Field Type Null Key Default Extra
id int(50) NO PRI NULL auto_increment
uid varchar(255) NO NULL
flag int(1) NO 0
boardid varchar(255) NO NULL
postdate varchar(255) NO NULL
lastpost bigint(255) NO NULL
name varchar(50) NO NULL
description text NO NULL
body text NO NULL
author varchar(25) NO NULL
deleted tinyint(3) unsigned NO 0
deletememberid int(10) unsigned NO 0
pinned tinyint(1) NO 0
flagged text NO NULL
privateaccess text NO NULL
lastposter int(255) NO 1
replycount int(255) NO 0
viewcount int(255) NO 0
movedfrom int(255) NO 0
forums_posts
Field Type Null Key Default Extra
id int(50) NO PRI NULL auto_increment
topicid int(10) unsigned NO 0
author varchar(25) NO NULL
postdate varchar(255) NO NULL
body text NO NULL
lastedit varchar(255) NO NULL
postcount tinyint(1) NO NULL
invincible tinyint(1) NO 0
deleted tinyint(3) unsigned NO 0
deletememberid int(10) unsigned NO 0
thumbsup int(255) NO 0
thumbsdown int(255) NO 0
thumbsupuser text NO NULL
thumbsdownuser text NO NULL

The problem is probably that MySQL evaluates the subquery for every row. You can give MySQL a hint that that it should execute the subquery only once by moving the subquery into a join:
SELECT *
FROM forum_topics ft
JOIN (
SELECT topicid
, COUNT(*) as cnt
FROM forums_posts
WHERE postdate > UNIX_TIMESTAMP()-60*60*24
GROUP BY
topicid
) fpc
ON ft.topicid = fpc.topicid
WHERE ft.deleted = 0
ORDER BY
fpc.cnt DESC
, ft.postdate DESC
LIMIT 3
An index on forum_posts(postdate, topicid) would further improve performance.

I'm going to take a stab in the dark, and I'll edit further if needed. An EXPLAIN query would help.
SELECT `forums_topics`.*
FROM (
SELECT `topicid`, COUNT(*) as num
FROM `forums_posts`
WHERE `postdate` > (UNIX_TIMESTAMP()-60*60*24)
GROUP BY `topicid`
ORDER BY num DESC, `postdate` DESC
LIMIT 3
) `trendy`
LEFT JOIN `forums_topics` ON `id`=`topicid`
WHERE `deleted`=0

Categories