MySQL Query Optimization (Takes way too long) - php

I use the following prepared statement:
SELECT *
FROM
c_members,c_users,c_positions,c_done_meetings
WHERE
c_positions.POS_ID=c_users.POS_ID
AND c_members.CLUB_ID = ?
AND USER_POINTS >= ?
AND USER_POINTS <= ?
AND c_users.POS_ID LIKE ?
AND MEM_ACADEMY LIKE ?
AND MEM_SEX LIKE ?
AND MEM_GRADELVL LIKE ?
AND MEM_GPA >= ?
AND MEM_GPA <= ?
AND MEM_ARCHIVE = 0
GROUP BY
c_members.MEM_ID, c_members.CLUB_ID
HAVING
SUM(c_done_meetings.MEDONE_ATTEND = 'u') >= 1
ORDER BY
USER_POINTS DESC
However this query takes 21.971405982971 seconds to load 111 records. When I remove "Having SUM(...)" clause, the performance is 100% better. Is there a way I could optimize it better?
Edit: (Table structures)
CREATE TABLE IF NOT EXISTS `c_done_meetings` (
`MEM_ID` int(11) NOT NULL,
`CLUB_ID` int(11) NOT NULL,
`MEETING_ID` int(11) NOT NULL,
`MEDONE_ATTEND` varchar(1) NOT NULL COMMENT 'E=excused, U=unexcused, P=present',
UNIQUE KEY `unique` (`MEM_ID`,`CLUB_ID`,`MEETING_ID`),
KEY `MEETING_ID` (`MEETING_ID`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
CREATE TABLE IF NOT EXISTS `c_members` (
`MEM_ID` int(11) NOT NULL,
`CLUB_ID` int(11) NOT NULL,
`MEM_FIRST` varchar(50) NOT NULL,
`MEM_MIDDLE` varchar(50) DEFAULT NULL,
`MEM_LAST` varchar(50) NOT NULL,
`MEM_SEX` tinyint(1) NOT NULL COMMENT '0-Male 1-Female',
`MEM_EMAIL` varchar(100) DEFAULT NULL,
`MEM_GRADELVL` int(11) NOT NULL,
`MEM_ACADEMY` varchar(50) DEFAULT '',
`MEM_GPA` double DEFAULT '0',
`MEM_ADDRESS` varchar(500) DEFAULT NULL,
`MEM_CITY` varchar(100) DEFAULT NULL,
`MEM_STATE` varchar(100) DEFAULT NULL,
`MEM_ZIP` int(11) DEFAULT NULL,
`MEM_TELEPHONE` varchar(25) DEFAULT NULL,
`MEM_AP` tinyint(1) NOT NULL,
`MEM_HONORS` tinyint(1) NOT NULL,
`MEM_ESOL` tinyint(1) NOT NULL,
`MEM_HISP` tinyint(1) NOT NULL,
`MEM_WHITE` tinyint(1) NOT NULL,
`MEM_MULTI` tinyint(1) NOT NULL,
`MEM_NATIVE` tinyint(1) NOT NULL,
`MEM_BLACK` tinyint(1) NOT NULL,
`MEM_ASIAN` tinyint(1) NOT NULL,
`MEM_EXTRA` varchar(10000) DEFAULT NULL,
`MEM_ARCHIVE` tinyint(1) NOT NULL DEFAULT '0',
UNIQUE KEY `MEM_ID` (`MEM_ID`,`CLUB_ID`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
CREATE TABLE IF NOT EXISTS `c_positions` (
`POS_ID` int(11) NOT NULL AUTO_INCREMENT,
`CLUB_ID` int(11) NOT NULL,
`POS_NAME` varchar(20) NOT NULL,
`POS_DESC` varchar(500) NOT NULL,
`POS_ADMIN` tinyint(1) NOT NULL,
`POS_ATN_VIEW` tinyint(1) NOT NULL,
`POS_ATN_CHKIN` tinyint(1) NOT NULL,
`POS_ATN_FINALIZE` tinyint(1) NOT NULL,
`POS_MEM_VIEW` tinyint(1) NOT NULL,
`POS_MEM_ADD` tinyint(1) NOT NULL,
`POS_MEM_EDIT` tinyint(1) NOT NULL,
`POS_POS_VIEW` tinyint(1) NOT NULL,
`POS_POS_ADD` tinyint(1) NOT NULL,
`POS_POS_EDIT` tinyint(1) NOT NULL,
`POS_MEET_VIEW` tinyint(1) NOT NULL,
`POS_MEET_ADD` tinyint(1) NOT NULL,
`POS_MEET_EDIT` tinyint(1) NOT NULL,
`POS_EVENT_VIEW` tinyint(1) NOT NULL,
`POS_EVENT_ADD` tinyint(1) NOT NULL,
`POS_EVENT_EDIT` tinyint(1) NOT NULL,
`POS_EVENT_UPDATE` tinyint(1) NOT NULL,
`POS_REPORT_VIEW` tinyint(1) NOT NULL,
`POS_ARCHIVE_VIEW` tinyint(1) NOT NULL,
`POS_ANNOUNCEMENTS` tinyint(1) NOT NULL,
`POS_WEB_CUSTOM` tinyint(1) NOT NULL,
PRIMARY KEY (`POS_ID`),
UNIQUE KEY `UNIQUE_NAME` (`CLUB_ID`,`POS_NAME`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=14 ;
CREATE TABLE IF NOT EXISTS `c_users` (
`MEM_ID` int(11) NOT NULL,
`CLUB_ID` int(11) NOT NULL,
`USER_PIN` int(11) NOT NULL,
`USER_POINTS` double NOT NULL,
`POS_ID` int(11) NOT NULL,
`USER_ARCHIVE` tinyint(1) NOT NULL DEFAULT '0',
UNIQUE KEY `MEM_ID` (`MEM_ID`,`CLUB_ID`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Edit 2:
Yes all ids are indexed, the SUM(...) >= # calculates the number of missed meetings. # is a parameter set by the user (I just hard coded a 1 for testing)

You need to have indexes an all fields used in the WHERE clause, on all fields on which the tables are joined (you need to explicitly state your join conditions as right now, you are getting a Cartesian join), on all fields used for grouping and all fields used for sorting.
The last problem is the HAVING clause there. You are not going to be able to use an index at all for that since it is a calculated value. If this is a query you will use often in the system (i.e. not just for reporting), you might consider adding a field you can use as a flag for this filtering purpose. Whenever you set c_done_meetings.MEDONE_ATTEND = 'u' in any of your queries, you could also set this flag for the member or user or whatever this is associated with so that you have an easy field to filter on in a WHERE clause.
Outside of that, you might actually gain better performance by getting a reduced list of users or members with that value of u in a subselect and then join using that subselect as a table.
EDIT:
After seeing your actual table structure, I can clearly see where you need to add indexes. I am also wondering why you have tables c_users and c_members with the same exact primary key. Why would these not just be a single table?

Some things that pop out at me:
You use like quite a lot. Try to do something with your php code so that this isn't necessary. For example, mem_sex has a limited number of choices. Make the front end a radio button or dropdown so you send a value where you can use = instead of like.
Two, if you add a sum() to the select clause, and the appropriate group by clause, it should run faster. It's worth a shot.

You can denormalise you tables and add field to c_members that represent your
SUM(c_done_meetings.MEDONE_ATTEND = 'u') >= 1
But you need to update that field always, when updating c_done_meetings (can be done with trigger)
Also try to avoid LIKE conditions. Use = insead (it is possible at least for SEX)

Related

MySQL indexes stop working

I am working on News website, the table (news) contain about 200,000 rows.
I use Zend framework 1.
the website was working very good, but from a week ago, I have found some errors in data retrieve from the queries.
I use zend paginator, like this:
$paginator = Zend_Paginator::factory($select);
$paginator->setItemCountPerPage(10);
$pageCounter = $paginator->count();
so $pageCounter return 0,
when I try to debug the error, I have tried to drop the indexes from the database table and create the index again, then the problem was resolved the website back to work again.
but every couple days this problem back again.
the tables I use is:
CREATE TABLE `news` (
`news_id` int(11) NOT NULL AUTO_INCREMENT,
`category_id` int(11) DEFAULT NULL,
`news_type` enum('text','photo','video') DEFAULT 'text',
`news_title` varchar(255) NOT NULL,
`news_url` varchar(255) NOT NULL,
`news_date` datetime NOT NULL,
`news_image` varchar(255) DEFAULT NULL,
`video` varchar(255) DEFAULT NULL,
`news_source` int(11) NOT NULL DEFAULT '0',
`author_id` int(11) NOT NULL DEFAULT '0',
`news_brief` text,
`news_content` mediumtext,
`meta_title` varchar(255) NOT NULL,
`meta_description` varchar(255) DEFAULT NULL,
`meta_keywords` varchar(255) DEFAULT NULL,
`status` tinyint(2) NOT NULL DEFAULT '1',
`news_read` int(11) DEFAULT '0',
`old_id` int(11) DEFAULT NULL,
`old_section_id` int(11) DEFAULT NULL,
`old_section` varchar(50) DEFAULT NULL,
`lang` varchar(10) DEFAULT NULL,
`featured` int(11) NOT NULL DEFAULT '0',
`in_timeline` tinyint(2) NOT NULL DEFAULT '0',
`is_highlight` tinyint(2) NOT NULL DEFAULT '0',
`home_exclusive` tinyint(2) NOT NULL DEFAULT '0',
`home_articles` tinyint(4) NOT NULL DEFAULT '0',
`home_articles_selected` tinyint(4) NOT NULL DEFAULT '0',
`home_mostread` tinyint(4) NOT NULL DEFAULT '0',
`video_featured` tinyint(4) NOT NULL DEFAULT '0',
`photos_featured` tinyint(4) NOT NULL DEFAULT '0',
`party_featured` tinyint(4) NOT NULL DEFAULT '0',
`party_activity` tinyint(4) NOT NULL DEFAULT '0',
`sport_featured` tinyint(4) NOT NULL DEFAULT '0',
`fnoun_featured` tinyint(4) NOT NULL DEFAULT '0',
`special_reports` tinyint(4) NOT NULL DEFAULT '0',
`mainnav` tinyint(4) NOT NULL DEFAULT '0',
PRIMARY KEY (`news_id`),
KEY `News_Category_idx` (`category_id`),
KEY `News_Type` (`news_type`),
KEY `News_Date` (`news_date`),
KEY `News_Read` (`news_read`),
KEY `old_id` (`old_id`),
KEY `Sources` (`news_source`),
KEY `Authors` (`author_id`),
KEY `Featured` (`featured`),
KEY `Articles` (`home_articles`),
KEY `Mostread` (`home_mostread`),
KEY `video_featured` (`video_featured`,`photos_featured`),
KEY `party_activity` (`party_activity`),
KEY `sport_featured` (`sport_featured`,`fnoun_featured`),
KEY `old_section_id` (`old_section_id`),
KEY `old_section` (`old_section`),
KEY `old section id` (`old_section_id`),
KEY `old section` (`old_section`),
KEY `articles_selected` (`home_articles_selected`),
KEY `URL` (`news_url`),
KEY `special_reports` (`special_reports`),
KEY `mainnav` (`mainnav`),
KEY `status` (`status`),
KEY `in_timeline` (`in_timeline`),
KEY `is_highlight` (`is_highlight`),
KEY `home_exclusive` (`home_exclusive`),
KEY `party_featured` (`party_featured`),
FULLTEXT KEY `news_title` (`news_title`)
) ENGINE=InnoDB AUTO_INCREMENT=167614 DEFAULT CHARSET=utf8
the columns (status - featured - in_timeline - is_highlight) type was (ENUM) but I have changed to (tinyint)
It works for a few days, but the problem come back again.
so I have to drop the indexed and create it again.
I don't know what is the problem?
can anyone help, please.
The Explain Query, it's the same when the problem happen and after I drop the indexes and create it again.
SQL query: Explain SELECT `N`.`news_id`, `N`.`news_title`, `N`.`news_url`, `N`.`news_image`, `N`.`news_type`, `N`.`news_date`, `N`.`is_highlight`, `N`.`news_brief`, `C`.`category_name`, `C`.`category_url`, `C`.`category_color`, `NT`.*, GROUP_CONCAT(T.tag_name separator ",") AS tags, GROUP_CONCAT(T.tag_url separator ",") AS `tags_urls` FROM `news` AS `N` INNER JOIN `news_categories` AS `C` ON C.category_id = N.category_id AND C.status = "1" LEFT JOIN `news_tags_relation` AS `NT` ON NT.news_id = N.news_id LEFT JOIN `news_tags` AS `T` ON T.tag_id = NT.tag_id WHERE (N.status = "1" AND N.category_id = "22") GROUP BY `N`.`news_id` ORDER BY `N`.`news_date` DESC;
Rows: 4
This table does not contain a unique column. Grid edit, checkbox, Edit, Copy and Delete features are not available.
the table in the attached images

How to speed up slow MySQL UPDATE queries with InnoDB tables

I have a very simple MySQL update query on my InnoDB table.
UPDATE `players_teams` SET t_last_active=NOW() WHERE t_player_id=11225 AND t_team_id=6912 AND t_season_id=2002 LIMIT 1
My table is structured as so:
CREATE TABLE `players_teams` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`t_player_id` int(11) DEFAULT NULL,
`t_team_id` int(11) DEFAULT NULL,
`t_league_id` int(11) DEFAULT NULL,
`t_season_id` int(11) DEFAULT NULL,
`t_div` varchar(64) DEFAULT NULL,
`t_player_number` varchar(3) DEFAULT NULL,
`t_player_jersey_size` enum('UNKNOWN','XS','S','M','L','XL','XXL','XXXL') DEFAULT 'UNKNOWN',
`t_player_registration_number` varchar(64) DEFAULT NULL,
`t_player_class` enum('ROSTER','SPARE','COACH','INJURED','HOLIDAY','SUSPENDED','SCOREKEEPER') DEFAULT 'ROSTER',
`t_access_level` enum('PLAYER','MANAGER','ASSISTANT') DEFAULT 'PLAYER',
`t_player_position` enum('ANY','FORWARD','DEFENCE','GOALIE','PITCHER','CATCHER','FIRST BASE','SECOND BASE','THIRD BASE','SHORTSTOP','LEFT FIELD','CENTER FIELD','RIGHT FIELD') DEFAULT 'ANY',
`t_spare_status` enum('INVITED','IN','OUT') DEFAULT NULL,
`t_drink_next` int(1) DEFAULT '0',
`t_no_fees` tinyint(1) DEFAULT '0',
`t_no_drink` tinyint(1) DEFAULT '0',
`t_auto_check_in` tinyint(1) DEFAULT '0',
`t_print_reminder` tinyint(1) DEFAULT '0',
`t_notes` text,
`t_last_chatter_id` int(11) DEFAULT NULL,
`t_last_history_id` int(11) DEFAULT NULL,
`t_last_active` timestamp NULL DEFAULT NULL,
`t_status` enum('ACTIVE','INACTIVE','ARCHIVED') DEFAULT 'ACTIVE',
`t_added` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
KEY `t_player_id` (`t_player_id`),
KEY `t_team_id` (`t_team_id`),
KEY `t_season_id` (`t_season_id`),
KEY `t_player_id_2` (`t_player_id`,`t_team_id`,`t_season_id`),
KEY `Team/Player ID` (`t_team_id`,`t_player_id`),
KEY `UpdatePlayersDiv` (`t_team_id`,`t_season_id`)
) ENGINE=InnoDB AUTO_INCREMENT=23454 DEFAULT CHARSET=latin1;
This simple update query takes on average, 3.5 seconds? I'm running this on a MediaTemple MySQL GRID container.
Here is the result of an EXPLAIN when switching the UPDATE to a SELECT.
Can someone provide some insight on how I'm not doing this correctly?
[EDIT: Added list of Indexes]
So is this a big mess of indexes? Do I have a bunch of redundant indexes in here?
Cheers,
Jon
It's using the wrong key instead of the index on the 3 columns. You can hint at indexes with USE KEY ... syntax.
https://dev.mysql.com/doc/refman/5.1/en/index-hints.html
You may also want to try reordering your key on the 3 columns. Generally, you want the most restricting column to be first in the index, then the next most restricting and so on.

Making join on 2 tables with millions record to be faster

I have two tables
security_stat => having 4 millions record
security_trade => having 10 millions record
I have this query running successfully but how can i OPTIMIZE this to be able to at least query 100,000 record within 10 seconds ( is it possible? ).. Currently it is very very slow.
SELECT `sec_stat_sec_name`, `sec_stat_date`, `sec_stat_market`, `sec_trade_close`, `sec_stat_date`
FROM security_stat` LEFT JOIN `security_trade`
ON `security_trade`.`sec_trade_sec_name` = `security_stat`.`sec_stat_sec_name`
and `security_trade`.`sec_trade_date` = `security_stat`.`sec_stat_date`
limit 100,000
I have INDEX on sec_trade_sec_name, sec_stat_sec_name, sec_trade_date , sec_stat_date
I tried limiting result with WHERE sec_stat_date >= 2005-01-01 but that doesn't help much. (my records range from 1975 - 2014)
EDIT
security_stat schema
CREATE TABLE `security_stat` (
`sec_stat_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`sec_stat_date` date NOT NULL,
`sec_stat_sec_name` varchar(255) NOT NULL,
`sec_stat_sec_id` int(11) NOT NULL,
`sec_stat_market` varchar(255) NOT NULL,
`sec_stat_industry` int(11) NOT NULL,
`sec_stat_sector` int(11) NOT NULL,
`sec_stat_subsector` int(11) NOT NULL,
`sec_stat_sec_type` varchar(1) NOT NULL,
`sec_stat_status` varchar(2) NOT NULL,
`sec_stat_benefit` varchar(2) NOT NULL,
`sec_stat_listed_share` bigint(20) NOT NULL,
`sec_stat_earn_p_share` decimal(12,5) NOT NULL,
`sec_stat_value` decimal(9,2) NOT NULL,
`sec_stat_p_of_earn` int(11) NOT NULL,
`sec_stat_as_date` date NOT NULL,
`sec_stat_div_p_share` decimal(16,12) NOT NULL,
`sec_stat_p_of_div` int(11) NOT NULL,
`sec_stat_end_date_div` date NOT NULL,
`sec_stat_pe` decimal(8,2) NOT NULL,
`sec_stat_pbv` decimal(8,2) NOT NULL,
`sec_stat_div_yield` decimal(8,2) NOT NULL,
`sec_stat_par_value` decimal(16,5) NOT NULL,
`sec_stat_market_cap` decimal(20,2) NOT NULL,
`sec_stat_turn_ratio` decimal(8,2) NOT NULL,
`sec_stat_npg_flag` varchar(1) NOT NULL,
`sec_stat_acc_div` decimal(16,12) NOT NULL,
`sec_stat_acc_no_of_pay` int(11) NOT NULL,
`sec_stat_div_pay_ratio` decimal(6,2) NOT NULL,
`sec_stat_earn_date` date NOT NULL,
`created_at` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
`updated_at` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
`sec_stat_ev` decimal(20,2) DEFAULT NULL,
`sec_stat_ev_revenue` decimal(20,2) DEFAULT NULL,
`sec_stat_ev_ebit` decimal(20,2) DEFAULT NULL,
`sec_stat_ev_ebitda` decimal(20,2) DEFAULT NULL,
`sec_stat_earning_yield` decimal(10,5) DEFAULT NULL,
`sec_stat_ps_ratio` decimal(10,5) DEFAULT NULL,
PRIMARY KEY (`sec_stat_id`),
UNIQUE KEY `sec_stat_date_name_id_cap` (`sec_stat_date`,`sec_stat_market`,`sec_stat_sec_id`,`sec_stat_sector`),
KEY `sec_stat_date` (`sec_stat_date`),
KEY `sec_stat_sec_name` (`sec_stat_sec_name`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 AUTO_INCREMENT=3598612 ;
security_trade schema
CREATE TABLE `security_trade` (
`sec_trade_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`sec_trade_date` date NOT NULL,
`sec_trade_sec_name` varchar(20) NOT NULL,
`sec_trade_sec_id` int(11) NOT NULL,
`sec_trade_market` varchar(1) NOT NULL,
`sec_trade_trading_method` varchar(1) NOT NULL,
`sec_trade_trade_report` varchar(1) NOT NULL,
`sec_trade_prior_date` date NOT NULL,
`sec_trade_prior` decimal(8,2) NOT NULL,
`sec_trade_open` decimal(8,2) NOT NULL,
`sec_trade_high` decimal(8,2) NOT NULL,
`sec_trade_low` decimal(8,2) NOT NULL,
`sec_trade_close` decimal(8,2) NOT NULL,
`sec_trade_last_bid` decimal(8,2) NOT NULL,
`sec_trade_last_offer` decimal(8,2) NOT NULL,
`sec_trade_transaction` int(11) NOT NULL,
`sec_trade_volume` bigint(20) NOT NULL,
`sec_trade_value` decimal(20,2) NOT NULL,
`sec_trade_avg_price` decimal(8,2) NOT NULL,
`created_at` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
`updated_at` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
PRIMARY KEY (`sec_trade_id`),
UNIQUE KEY `sec_trade_close` (`sec_trade_date`,`sec_trade_sec_name`,`sec_trade_market`,`sec_trade_trade_report`,`sec_trade_trading_method`),
KEY `security_trade_sec_trade_sec_name_index` (`sec_trade_sec_name`),
KEY `security_trade_sec_trade_date_index` (`sec_trade_date`),
KEY `security_trade_sec_trade_prior_date_index` (`sec_trade_prior_date`),
KEY `security_trade_sec_trade_close_index` (`sec_trade_close`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 AUTO_INCREMENT=10019817 ;
My Final query will actually have more
WHERE sec_stat_earning_yield IS NULL
ORDER BY updated_at ASC
but because when i add this two statement into the query with LIMIT of 1,000 records it make my query even slower ( may be because i didn't have index on this two columns? )
Thanks in Advance
Taking the following as your actual query:
SELECT `sec_stat_sec_name`, `sec_stat_date`, `sec_stat_market`, `sec_trade_close`, `sec_stat_date`
FROM `security_stat` LEFT JOIN `security_trade`
ON `security_trade`.`sec_trade_sec_name` = `security_stat`.`sec_stat_sec_name`
and `security_trade`.`sec_trade_date` = `security_stat`.`sec_stat_date`
WHERE sec_stat_earning_yield IS NULL
ORDER BY updated_at ASC
limit 100,000
You filter the security_stat table in two ways:
1. Only where sec_stat_earning_yield IS NULL
2. First 100k records when ordered by updated_at
Note: I've assume you mean security_stat.updated_at, but you don't make that clear.
In order to make that as cheap as possible add an index that covers both of those fields (sec_stat_earning_yield, updated_at).
Note: Adding indexes that change a lot, especially when the order of the records changes within the index, can make a INSERTs slower. You will need to balance INSERT performance against SELECT performance.
Then you join the trades table on, and so you want that lookup to be as fast as possible, which can be achieved with an index on that table covering (sec_trade_sec_name, sec_trade_date, sec_trade_close).
- The first two fields in the index make the lookup simpler
- The last field in the index means the DBMS can avoid having to look in the table
Once done you may also be well served looking at the EXPLAIN plan, although relatively complicated it will give you key information to understand the best places to target your optimisation.
First, try createing indexes to match the join so:
security_trade (sec_trade_sec_name, sec_trade_date)
security_stat (sec_stat_sec_name, sec_stat_date)
or possibly
security_stat (sec_stat_earning_yield, sec_stat_sec_name, sec_stat_date)
And as pointed out in the comments above, your "Limit" clause may cause the result set to be not clearly defined.

Mysql LEFT Join on big data table

I have a table called "tables_data_info" where is stored some of the data relative to different tables. Data like "created time", "editing time", "editing user", "create by user id" etc. I'm using this table because there was a dynamic php script that generate it automatically.
But, when i have a huge number of record ( 15k in this case ) the query getting very very very slow, and take "minutes" to do his job! But i'm not selecting all 15k records, i'm limiting to select 10 records at all!
A simple query:
SELECT pd.id, pd.title, pd.sell_price, pd.available_qt, tdi.createtime, tdi.lastupdatetime, tdi.create_member_id, tdi.create_group_id, tdi.last_update_member_id, tdi.last_update_group_id FROM zd_products AS pd LEFT JOIN zd_tables_data_info AS tdi ON ( tdi.targetid = pd.id and tdi.table_name = 'products' ) ORDER by pd.title ASC LIMIT 0, 10
How can i run this query differently but more efficiently ?
Here the table structure:
zd_products
CREATE TABLE `zd_products` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`title` varchar(256) NOT NULL DEFAULT '',
`internalcode` varchar(256) DEFAULT NULL,
`ean13_jan_code` varchar(256) DEFAULT NULL,
`upc_code` varchar(11) DEFAULT NULL,
`active` tinyint(1) NOT NULL DEFAULT '0',
`status` varchar(12) NOT NULL DEFAULT 'new',
`product_tags` longtext,
`buy_price` double NOT NULL DEFAULT '0',
`sell_price` double NOT NULL DEFAULT '0',
`fiscal_tax_id` int(11) NOT NULL DEFAULT '0',
`box_width` double NOT NULL DEFAULT '0',
`box_height` double NOT NULL DEFAULT '0',
`box_depth` double NOT NULL DEFAULT '0',
`box_weight` double NOT NULL DEFAULT '0',
`shipment_extra_price` double NOT NULL DEFAULT '0',
`available_qt` int(11) NOT NULL DEFAULT '0',
`allow_purchase_out_stock` tinyint(1) NOT NULL DEFAULT '0',
`meta_title` varchar(256) DEFAULT NULL,
`meta_description` varchar(256) DEFAULT NULL,
`meta_keywords` longtext,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=15730 DEFAULT CHARSET=utf8;
zd_tables_data_info
CREATE TABLE `zd_tables_data_info` (
`table_name` varchar(256) NOT NULL DEFAULT '',
`targetid` int(11) DEFAULT NULL,
`create_member_id` int(11) unsigned DEFAULT NULL,
`create_group_id` int(11) unsigned DEFAULT NULL,
`last_update_member_id` int(11) unsigned DEFAULT NULL,
`last_update_group_id` int(11) unsigned DEFAULT NULL,
`createtime` int(11) unsigned DEFAULT NULL,
`lastupdatetime` int(11) unsigned DEFAULT NULL,
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
PRIMARY KEY (`id`),
KEY `INDEX` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=19692 DEFAULT CHARSET=utf8;
Your data is not particularly big. Here is the query:
SELECT pd.id, pd.title, pd.sell_price, pd.available_qt,
tdi.createtime, tdi.lastupdatetime, tdi.create_member_id, tdi.create_group_id,
tdi.last_update_member_id, tdi.last_update_group_id
FROM zd_products pd LEFT JOIN
zd_tables_data_info tdi
ON tdi.targetid = pd.id and tdi.table_name = 'products'
ORDER by pd.title ASC
LIMIT 0, 10;
You can improve performance of this query with indexes. The two that come to mind are zd_products(title, id) and zd_tables_data_info(targetid, table_name). Try these and see if they help. You can create these indexes either in the create table statement (or alter table) or by using:
create index zd_products_title_id on zd_products(title, id);
create index zd_tables_data_info_targetid_table_name on zd_tables_data_info(targetid, table_name);
If not, put explain in front of your query and then edit your question with the resulting plan.

MySQL Stored Procedure [Copy table 1 -> table 2]

My knowledge of MySQL/ SQL apart from the simple CRUD operations is basic.
If I had to use a stored procedure to move certain attributes (not in a specific order) to another table how could it be done?
These are the following tables. I want to move from the 1st to the 2nd table.
As you can see the datatype sizes are different for certain columns.
CREATE TABLE IF NOT EXISTS `source_cdr` (
`callstart` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
`src` varchar(80) NOT NULL DEFAULT '',
`dst` varchar(80) NOT NULL DEFAULT '',
`accountcode` varchar(50) NOT NULL,
`uniqueid` varchar(100) NOT NULL,
`ID` int(11) NOT NULL AUTO_INCREMENT,
`callanswer` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
`callend` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
`disposition` varchar(50) NOT NULL,
`cdr_id` int(11) unsigned NOT NULL DEFAULT '0',
`pin_code` varchar(4) NOT NULL,
`provider` int(11) NOT NULL,
PRIMARY KEY (`ID`),
KEY `calldate_idx` (`callstart`) USING BTREE,
KEY `idx_acc_code_calldate` (`accountcode`,`callstart`) USING BTREE,
KEY `uniqueid` (`uniqueid`),
KEY `cdr_id` (`cdr_id`),
KEY `idx_uniqueid_cdr_id` (`uniqueid`,`cdr_id`)
) ENGINE=MyISAM;
--
CREATE TABLE IF NOT EXISTS `destination_cdr` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`calldate` datetime NOT NULL,
`source` varchar(80) NOT NULL,
`destination` varchar(80) NOT NULL,
`account_code` varchar(30) DEFAULT NULL,
`pincode` varchar(45) NOT NULL,
`duration_call` bigint(20) NOT NULL DEFAULT '0',
`duration_talk` bigint(20) NOT NULL,
`disposition` varchar(255) NOT NULL,
`clid` varchar(80) DEFAULT NULL,
`cdr_id` bigint(20) DEFAULT NULL,
`vxcdr_id` bigint(20) DEFAULT NULL,
`provider` int(11) NOT NULL DEFAULT '0'
PRIMARY KEY (`id`)
) ENGINE=MyISAM;
EDIT 1
An example of a row
('2012-03-18 20:54:49', '5796', '0761100866', '103f0124ad510516f33cab132c0a695b', 'call-F1884808-6753-2F10-181C-3A#10.217.164.33', 308006367, '2012-03-18 20:55:05', '2012-03-18 20:55:51', '200 OK', 2, '', 0),
Thanks
You can use MySQL: INSERT ... SELECT Syntax to copy data from one table to the other.
Decide common fields in both and copy the same.
Example:
INSERT INTO TABLE2( COL1, COLx, ... ) SELECT colM, colY FROM TABLE1;
If the column sizes mismatch, data truncation takes place, and you can't overcome that but redefine the destination table columns.

Categories