Table "tblvotes"
Field
Type
Null
Key
Default
Extra
id
int(11)
NO
PRI
NULL
auto_increment
candidateid
int(11)
NO
MUL
NULL
districtid
int(11)
NO
NULL
daterecorded
datetime
NO
current_timestamp()
Table "tblcandidate":
Field
Type
Null
Key
Default
Extra
id
int(11)
NO
PRI
NULL
auto_increment
voterid
int(11)
NO
MUL
NULL
partyid
int(11)
NO
MUL
NULL
candidatepositionid
int(11)
NO
MUL
NULL
Table "tbldistricts":
Field
Type
Null
Key
Default
Extra
id
int(11)
NO
PRI
NULL
auto_increment
district_short
varchar(8)
NO
NULL
district_name
varchar(100)
NO
NULL
district_aun
varchar(10)
NO
NULL
district_propVal
tinyint(4)
NO
1
Table "tblvoterlist":
Field
Type
Null
Key
Default
Extra
id
int(11)
NO
PRI
NULL
auto_increment
idno
varchar(15)
YES
NULL
lastname
varchar(30)
NO
NULL
firstname
varchar(30)
NO
NULL
middlename
varchar(30)
NO
NULL
districtid
int(5)
YES
MUL
NULL
image
varchar(30)
NO
NULL
votingcode
varchar(15)
YES
UNI
NULL
votestatus
char(1)
YES
NULL
yearlevelid
int(12)
YES
MUL
NULL
SELECT concat_ws(",", tvl.lastname, tvl.firstname) as candidate, td.district_name as district, count(tv.candidateid)
FROM tblvotes tv
JOIN tblcandidate tc on tv.candidateid = tc.id
JOIN tbldistricts td on tv.districtid = td.id
JOIN tblvoterlist tvl on tc.voterid = tvl.id
Also tried to group by tv.districtid and I am not sure where the COUNT should be.
unfortunately, I really am not understanding how use the keys to get the desired result.
As an example the desired result is:
Candidate1
Canddiate2
Candidate3
Candidate4
District1
5
2
1
0
District2
0
4
2
1
District3
6
2
3
2
I hope this makes sense.
Related
I have below query:
SELECT bank_code, bank_name
FROM system_bank_info
WHERE company_id=1 AND country_id=103 AND status='ACTIVE'
GROUP BY bank_name
ORDER BY bank_name ASC
TABLE SCHEMA:
CREATE TABLE `system_bank_info` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`company_id` int(11) NOT NULL,
`country_id` int(11) NOT NULL,
`bank_code` varchar(50) NOT NULL,
`bank_name` varchar(255) NOT NULL,
`branch_code` varchar(50) NOT NULL,
`branch_name` varchar(255) NOT NULL,
`status` enum('ACTIVE','INACTIVE') NOT NULL DEFAULT 'ACTIVE',
`bank_state` varchar(255) DEFAULT NULL,
`bank_district` varchar(255) DEFAULT NULL,
`bank_city` varchar(255) DEFAULT NULL,
`bank_address` text,
`bank_contact` varchar(255) DEFAULT NULL,
`service_type` enum('INSTA_CREDIT') DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `country_id` (`country_id`),
KEY `bank_code` (`bank_code`),
KEY `branch_code` (`branch_code`),
KEY `company_id_country_id_status` (`company_id`,`country_id`,`status`),
CONSTRAINT `system_bank_info_ibfk_1` FOREIGN KEY (`company_id`) REFERENCES `company` (`id`),
CONSTRAINT `system_bank_info_ibfk_2` FOREIGN KEY (`country_id`) REFERENCES `system_country_list` (`country_id`)
) ENGINE=InnoDB AUTO_INCREMENT=150398 DEFAULT CHARSET=utf8
EXPLAIN:
+----+-------------+------------------+------+-----------------------------------------+------------+---------+-------+-------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+------------------+------+-----------------------------------------+------------+---------+-------+-------+----------------------------------------------+
| 1 | SIMPLE | system_bank_info | ref | country_id,company_id_country_id_status | country_id | 4 | const | 59324 | Using where; Using temporary; Using filesort |
+----+-------------+------------------+------+-----------------------------------------+------------+---------+-------+-------+----------------------------------------------+
As you can see there is an index called company_id_country_id_status. But, query still uses country_id only. How can I make this query more efficient?
I also tried adding indexes bank_code and bank_name. But, same result!
You can give hints to the optimizer so that it uses a specific index. USE INDEX ()
SELECT bank_code, bank_name
FROM system_bank_info USE INDEX (company_id_country_id_status)
WHERE company_id=1 AND country_id=103 AND status='ACTIVE'
I need to update a database table with value that is used in another row.
I have a database with 4000 products, for every product there are 3 rows, 1 for every language (i work with 3 languages).
Now i need to copy the product title from the dutch language into the title field from the English product row and the French product row for the same product id. Some title field in English and Frensh are empty, but not al off them are empty.
So I have this fields
product_id ;
language_id ;
product_title
Anyone can help me with the sql i need to have to do this for all my 4000 products?
Thank you, kind regards
Pete
ps_product_lang is the name of the product table
the database is MySQL
id_product int(10) unsigned NO PRI NULL
id_shop int(11) unsigned NO PRI 1
id_lang int(10) unsigned NO PRI NULL
description text YES NULL
description_short text YES NULL
link_rewrite varchar(128) NO NULL
meta_description varchar(255) YES NULL
meta_keywords varchar(255) YES NULL
meta_title varchar(128) YES NULL
name varchar(128) NO MUL NULL
available_now varchar(255) YES NULL
available_later varchar(255) YES NULL
Something along these lines should work.
UPDATE ps_product_lang p1
SET name =
(SELECT name
FROM (SELECT * FROM ps_product_lang) p2
WHERE p2.`id_product` = p1.`id_product`
AND p2.`id_lang` = 6)
WHERE `id_lang` != 6
You need the extra nested select in there to prevent MySQL from reporting the following error:
Error Code: 1093
You can't specify target table 'p1' for update in FROM clause
I am using the below SQL query but it takes more than 180 sec to execute. Is there a way to speed it up ? This SQL give me the pic_id and of all the females.
SELECT pic_id, small
FROM picture
WHERE hide =0
AND userhide =0
AND `fbid`
IN (
SELECT fbid
FROM user
WHERE gender = "female"
)
ORDER BY `picture`.`pic_id` ASC
LIMIT 1500 , 200
The Explain SQL
id select_type table type possible_keys key key_len ref rows Extra
1 PRIMARY picture index NULL PRIMARY 4 NULL 1700 Using where
2 DEPENDENT SUBQUERY user ALL NULL NULL NULL NULL 7496 Using where
--- Result of explain statement for Tim's sql answer --
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE picture ALL NULL NULL NULL NULL 41443 Using where; Using temporary; Using filesort
1 SIMPLE user ALL NULL NULL NULL NULL 7501 Using where; Using join buffer
-- Structure ---
CREATE TABLE `user` (
`sid` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(255) CHARACTER SET utf8 NOT NULL,
`first_name` varchar(255) CHARACTER SET utf8 NOT NULL,
`username` varchar(255) CHARACTER SET utf8 NOT NULL,
`birthday` date NOT NULL,
`location` varchar(255) CHARACTER SET utf8 NOT NULL,
`gender` varchar(6) CHARACTER SET utf8 NOT NULL,
`created` date NOT NULL,
`fbid` bigint(50) NOT NULL,
`token` varchar(255) CHARACTER SET utf8 NOT NULL,
`relationship_status` varchar(20) CHARACTER SET utf8 NOT NULL,
`smallest` varchar(255) CHARACTER SET utf8 NOT NULL,
`email` varchar(40) CHARACTER SET utf8 NOT NULL,
`ref` varchar(15) NOT NULL,
PRIMARY KEY (`sid`),
KEY `gender` (`gender`),
KEY `fbid` (`fbid`)
) ENGINE=MyISAM AUTO_INCREMENT=7595 DEFAULT CHARSET=latin
---- structure of picture table ---
CREATE TABLE `picture` (
`fbid` bigint(50) NOT NULL,
`pic_id` int(11) NOT NULL AUTO_INCREMENT,
`pic_location` varchar(255) NOT NULL,
`hide` int(1) NOT NULL,
`small` varchar(255) NOT NULL,
`userhide` int(1) NOT NULL,
`likes` int(10) NOT NULL,
`hot` int(1) NOT NULL,
PRIMARY KEY (`pic_id`),
UNIQUE KEY `pic_location` (`pic_location`),
UNIQUE KEY `small` (`small`),
KEY `fbid` (`fbid`),
KEY `hide` (`hide`),
KEY `userhide` (`userhide`)
) ENGINE=MyISAM AUTO_INCREMENT=42749 DEFAULT CHARSET=latin1
try something like this:
SELECT pic_id, small
FROM picture
INNER JOIN user ON ( picture.fbid = user.fbid and user.gender='female' )
WHERE hide =0
AND userhide =0
ORDER BY `picture`.`pic_id` ASC
LIMIT 1500 , 200
I put gender in the join because a query will not return rows that don't have a match on an inner join.
You should also read this stack overflow topic
EDIT:
make sure you have indexed the following fields:
picture.fbid
user.fbid
user.gender
picture.hide
picture.userhide
Try using a join instead:
SELECT p.pic_id, p.small
FROM picture p
INNER JOIN fbid f USING ( fbid )
WHERE p.hide =0
AND p.userhide =0
AND f.gender = 'female'
ORDER BY `picture`.`pic_id` ASC
LIMIT 1500 , 200
I have a query below that joins 4 tables. I have added indexes and the explain output looks good with indexes used (see below).
Can I optimize the query further?
The modelXml is rather big on some records.For a big project where I get back 22 records, each with approximately 2.5 - 3MB of modelXml data, the query is taking long (total of 69MB of data returned).
I suspect this is the issue but not sure how to deal with it.
I was reading on adjusting internal mysql variables e.g key_buffer_size and table_cache. Would any of this help?
key_buffer_size is currently set at 8384512 (~8MB) and table_cache at 64
What should I increase it to?
What other variables should I be looking at to manage to speed up return of such big data?
Any other suggestions are welcome. I am a novice to mysql but really trying to get better.
SELECT `m`.`modelId`, `m`.`modelTypeId`, `m`.`modelXml`, `m`.`xmlSize`, `m`.`createdById`, `m`.`creationDate`, `m`.`modifiedDate`, `u`.`firstName`, `u`.`lastName` FROM `models_1` AS `m`
INNER JOIN `modelFolderAssociations_1` AS `mfa` ON m.modelId = mfa.modelIOId
INNER JOIN `modelFolders_1` AS `mf` ON mfa.folderId = mf.folderId
INNER JOIN `users_1` AS `u` ON m.createdById = u.userId
WHERE (m.projectId = 2) AND (mfa.folderId = 5) AND (mfa.modelIOType = 2) AND (m.modelTypeId = 2)
CREATE TABLE `models` (
`modelId` int(11) NOT NULL auto_increment,
`customerId` int(11) NOT NULL,
`groupId` int(11) NOT NULL,
`projectId` int(11) NOT NULL,
`createdById` int(11) NOT NULL,
`modelTypeId` int(11) NOT NULL,
`modelXml` longtext,
`modelSpecXml` longtext NOT NULL,
`xmlSize` bigint(20) NOT NULL default '0',
`creationDate` datetime NOT NULL,
`modifiedDate` datetime NOT NULL,
PRIMARY KEY (`modelId`,`customerId`)
) ENGINE=InnoDB AUTO_INCREMENT=75 DEFAULT CHARSET=utf8
CREATE TABLE `modelFolders` (
`folderId` int(11) NOT NULL auto_increment,
`customerId` int(11) NOT NULL,
`groupId` int(11) NOT NULL,
`projectId` int(11) NOT NULL,
`parentId` int(11) NOT NULL,
`folderName` varchar(64) NOT NULL,
`folderType` int(11) NOT NULL,
`editable` tinyint(1) NOT NULL default '1',
`nextDefaultNameNumber` int(11) NOT NULL default '1',
`creationDate` datetime NOT NULL,
`modifiedDate` datetime NOT NULL,
PRIMARY KEY (`folderId`,`customerId`),
KEY `parentId` (`parentId`)
) ENGINE=InnoDB AUTO_INCREMENT=16 DEFAULT CHARSET=utf8
CREATE TABLE `modelFolderAssociations` (
`associationId` int(11) NOT NULL auto_increment,
`customerId` int(11) NOT NULL,
`folderId` int(11) NOT NULL,
`projectId` int(11) NOT NULL,
`modelIOId` int(11) NOT NULL,
`modelIOType` tinyint(1) NOT NULL default '1',
`creationDate` datetime NOT NULL,
`modifiedDate` datetime NOT NULL,
PRIMARY KEY (`associationId`,`customerId`),
KEY `folderId` (`folderId`,`modelIOType`)
) ENGINE=InnoDB AUTO_INCREMENT=75 DEFAULT CHARSET=utf8
CREATE TABLE `users` (
`userId` int(11) NOT NULL auto_increment,
`customerId` int(11) NOT NULL,
`userName` varchar(50) NOT NULL,
`password` varchar(256) NOT NULL,
`firstName` varchar(50) default NULL,
`lastName` varchar(50) default NULL,
`creationDate` datetime NOT NULL,
`modifiedDate` datetime NOT NULL,
PRIMARY KEY (`userId`,`customerId`),
UNIQUE KEY `userName` (`userName`)
) ENGINE=InnoDB AUTO_INCREMENT=21 DEFAULT CHARSET=utf8
Explain output
+----+-------------+-------------------------+--------+---------------+----------+---------+---------------------------------------------------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------------------------+--------+---------------+----------+---------+---------------------------------------------------+------+-------------+
| 1 | SIMPLE | modelFolders | const | PRIMARY | PRIMARY | 8 | const,const | 1 | Using index |
| 1 | SIMPLE | modelFolderAssociations | ref | folderId | folderId | 5 | const,const | 22 | Using where |
| 1 | SIMPLE | models | eq_ref | PRIMARY | PRIMARY | 8 | xa_system.modelFolderAssociations.modelIOId,const | 1 | Using where |
| 1 | SIMPLE | users | eq_ref | PRIMARY | PRIMARY | 8 | xa_system.models.createdById,const | 1 | |
+----+-------------+-------------------------+--------+---------------+----------+---------+---------------------------------------------------+------+-------------+
Having large text columns, like those you're using to store XML I presume, can hurt performance regardless of how well your indexes are structured.
It can be better in these cases to move the text columns to a separate table, indexed by string length which you're already storing and CRC32.
CREATE TABLE MODEL_XML (
xmlId INT(11) unsigned NOT NULL auto_increment,
xmlSize BIGINT(20) NOT NULL default '0',
crc32 INT(11) unsigned NOT NULL,
xmlData LONGTEXT,
PRIMARY KEY (xmlId),
UNIQUE KEY (xmlSize, crc32)
)
Then the width of your columns on the table with your important indices becomes constant.
ex.
modelXmlId INT(11) unsigned NOT NULL
specXmlId INT(11) unsigned NOT NULL
It also has the benefit of being more space-efficient for redundant text (empty strings, etc) since they would all share one xmlId and thus one row in the DB.
You should index your foreign keys, this will really help with your joins:
CREATE INDEX IDX_MODELS_CUSTID
on models (customerId)
CREATE INDEX IDX_FLDR_ASSOC_MODELIO
modelFolderAssociations(modelIOId)
CREATE INDEX IDX_FLDR_ASSOC_FLDRID
modelFolderAssociations(folderId)
and so on.
Test your query after taking off the modelXml column from the SELECT clause.
If the speed is then significantly better, then the slowness comes from the volume off data to transfer, not the query itself.
Your indexes don't always match the queries and joins. If all of the columns in your WHERE and JOIN clauses are not in the index, you're forcing mysql to look at the underlying row, which will kill your performance, especially since your model rows are so wide.
For "modelFolderAssociations", you made the correct composite key for the where clause, but you should include modelIOId for the join to model.
For "models", you'd want a composite index on (modelId, projectId, modelTypeId, createdById ) to cover the incoming link from mfa, the two items in the where clause, and the outbound link to users.
For "modelFolders" and "users", you have the incoming join covered in your primary keys.
The engine is only going to use one index, so adding extra index single indexes (as Mike suggested) won't be as good.
For this website we're working on, we're trying to get the most popular topics (based on how many posts have been made in them within the last 24 hours). We have a medium to large based forum, and the current MySQL query looks like this:
SELECT `forums_topics`.`id`,`forums_topics`.`name`,
(
SELECT COUNT(`id`)
FROM `forums_posts`
WHERE `postdate` > (UNIX_TIMESTAMP()-60*60*24)
AND `topicid`=`forums_topics`.`id`
) AS `trendy_threads`
FROM `forums_topics`
WHERE `deleted`=0
AND `lastpost` > (UNIX_TIMESTAMP()-60*60*24)
ORDER BY `trendy_threads` DESC,`postdate` DESC
LIMIT 3
The SQL is quite sluggish.
How can we get this information as quickly and as efficiently as possible?
forums_topics
Field Type Null Key Default Extra
id int(50) NO PRI NULL auto_increment
uid varchar(255) NO NULL
flag int(1) NO 0
boardid varchar(255) NO NULL
postdate varchar(255) NO NULL
lastpost bigint(255) NO NULL
name varchar(50) NO NULL
description text NO NULL
body text NO NULL
author varchar(25) NO NULL
deleted tinyint(3) unsigned NO 0
deletememberid int(10) unsigned NO 0
pinned tinyint(1) NO 0
flagged text NO NULL
privateaccess text NO NULL
lastposter int(255) NO 1
replycount int(255) NO 0
viewcount int(255) NO 0
movedfrom int(255) NO 0
forums_posts
Field Type Null Key Default Extra
id int(50) NO PRI NULL auto_increment
topicid int(10) unsigned NO 0
author varchar(25) NO NULL
postdate varchar(255) NO NULL
body text NO NULL
lastedit varchar(255) NO NULL
postcount tinyint(1) NO NULL
invincible tinyint(1) NO 0
deleted tinyint(3) unsigned NO 0
deletememberid int(10) unsigned NO 0
thumbsup int(255) NO 0
thumbsdown int(255) NO 0
thumbsupuser text NO NULL
thumbsdownuser text NO NULL
The problem is probably that MySQL evaluates the subquery for every row. You can give MySQL a hint that that it should execute the subquery only once by moving the subquery into a join:
SELECT *
FROM forum_topics ft
JOIN (
SELECT topicid
, COUNT(*) as cnt
FROM forums_posts
WHERE postdate > UNIX_TIMESTAMP()-60*60*24
GROUP BY
topicid
) fpc
ON ft.topicid = fpc.topicid
WHERE ft.deleted = 0
ORDER BY
fpc.cnt DESC
, ft.postdate DESC
LIMIT 3
An index on forum_posts(postdate, topicid) would further improve performance.
I'm going to take a stab in the dark, and I'll edit further if needed. An EXPLAIN query would help.
SELECT `forums_topics`.*
FROM (
SELECT `topicid`, COUNT(*) as num
FROM `forums_posts`
WHERE `postdate` > (UNIX_TIMESTAMP()-60*60*24)
GROUP BY `topicid`
ORDER BY num DESC, `postdate` DESC
LIMIT 3
) `trendy`
LEFT JOIN `forums_topics` ON `id`=`topicid`
WHERE `deleted`=0