I am working on News website, the table (news) contain about 200,000 rows.
I use Zend framework 1.
the website was working very good, but from a week ago, I have found some errors in data retrieve from the queries.
I use zend paginator, like this:
$paginator = Zend_Paginator::factory($select);
$paginator->setItemCountPerPage(10);
$pageCounter = $paginator->count();
so $pageCounter return 0,
when I try to debug the error, I have tried to drop the indexes from the database table and create the index again, then the problem was resolved the website back to work again.
but every couple days this problem back again.
the tables I use is:
CREATE TABLE `news` (
`news_id` int(11) NOT NULL AUTO_INCREMENT,
`category_id` int(11) DEFAULT NULL,
`news_type` enum('text','photo','video') DEFAULT 'text',
`news_title` varchar(255) NOT NULL,
`news_url` varchar(255) NOT NULL,
`news_date` datetime NOT NULL,
`news_image` varchar(255) DEFAULT NULL,
`video` varchar(255) DEFAULT NULL,
`news_source` int(11) NOT NULL DEFAULT '0',
`author_id` int(11) NOT NULL DEFAULT '0',
`news_brief` text,
`news_content` mediumtext,
`meta_title` varchar(255) NOT NULL,
`meta_description` varchar(255) DEFAULT NULL,
`meta_keywords` varchar(255) DEFAULT NULL,
`status` tinyint(2) NOT NULL DEFAULT '1',
`news_read` int(11) DEFAULT '0',
`old_id` int(11) DEFAULT NULL,
`old_section_id` int(11) DEFAULT NULL,
`old_section` varchar(50) DEFAULT NULL,
`lang` varchar(10) DEFAULT NULL,
`featured` int(11) NOT NULL DEFAULT '0',
`in_timeline` tinyint(2) NOT NULL DEFAULT '0',
`is_highlight` tinyint(2) NOT NULL DEFAULT '0',
`home_exclusive` tinyint(2) NOT NULL DEFAULT '0',
`home_articles` tinyint(4) NOT NULL DEFAULT '0',
`home_articles_selected` tinyint(4) NOT NULL DEFAULT '0',
`home_mostread` tinyint(4) NOT NULL DEFAULT '0',
`video_featured` tinyint(4) NOT NULL DEFAULT '0',
`photos_featured` tinyint(4) NOT NULL DEFAULT '0',
`party_featured` tinyint(4) NOT NULL DEFAULT '0',
`party_activity` tinyint(4) NOT NULL DEFAULT '0',
`sport_featured` tinyint(4) NOT NULL DEFAULT '0',
`fnoun_featured` tinyint(4) NOT NULL DEFAULT '0',
`special_reports` tinyint(4) NOT NULL DEFAULT '0',
`mainnav` tinyint(4) NOT NULL DEFAULT '0',
PRIMARY KEY (`news_id`),
KEY `News_Category_idx` (`category_id`),
KEY `News_Type` (`news_type`),
KEY `News_Date` (`news_date`),
KEY `News_Read` (`news_read`),
KEY `old_id` (`old_id`),
KEY `Sources` (`news_source`),
KEY `Authors` (`author_id`),
KEY `Featured` (`featured`),
KEY `Articles` (`home_articles`),
KEY `Mostread` (`home_mostread`),
KEY `video_featured` (`video_featured`,`photos_featured`),
KEY `party_activity` (`party_activity`),
KEY `sport_featured` (`sport_featured`,`fnoun_featured`),
KEY `old_section_id` (`old_section_id`),
KEY `old_section` (`old_section`),
KEY `old section id` (`old_section_id`),
KEY `old section` (`old_section`),
KEY `articles_selected` (`home_articles_selected`),
KEY `URL` (`news_url`),
KEY `special_reports` (`special_reports`),
KEY `mainnav` (`mainnav`),
KEY `status` (`status`),
KEY `in_timeline` (`in_timeline`),
KEY `is_highlight` (`is_highlight`),
KEY `home_exclusive` (`home_exclusive`),
KEY `party_featured` (`party_featured`),
FULLTEXT KEY `news_title` (`news_title`)
) ENGINE=InnoDB AUTO_INCREMENT=167614 DEFAULT CHARSET=utf8
the columns (status - featured - in_timeline - is_highlight) type was (ENUM) but I have changed to (tinyint)
It works for a few days, but the problem come back again.
so I have to drop the indexed and create it again.
I don't know what is the problem?
can anyone help, please.
The Explain Query, it's the same when the problem happen and after I drop the indexes and create it again.
SQL query: Explain SELECT `N`.`news_id`, `N`.`news_title`, `N`.`news_url`, `N`.`news_image`, `N`.`news_type`, `N`.`news_date`, `N`.`is_highlight`, `N`.`news_brief`, `C`.`category_name`, `C`.`category_url`, `C`.`category_color`, `NT`.*, GROUP_CONCAT(T.tag_name separator ",") AS tags, GROUP_CONCAT(T.tag_url separator ",") AS `tags_urls` FROM `news` AS `N` INNER JOIN `news_categories` AS `C` ON C.category_id = N.category_id AND C.status = "1" LEFT JOIN `news_tags_relation` AS `NT` ON NT.news_id = N.news_id LEFT JOIN `news_tags` AS `T` ON T.tag_id = NT.tag_id WHERE (N.status = "1" AND N.category_id = "22") GROUP BY `N`.`news_id` ORDER BY `N`.`news_date` DESC;
Rows: 4
This table does not contain a unique column. Grid edit, checkbox, Edit, Copy and Delete features are not available.
the table in the attached images
Related
I have a invoices, invoices_items, order, order_items. Invoices and Orders tables contains around 1 Millions records. Invoices_items and Orders_items tables contains more than 2 Millions records. Items table contains 2 Hundred Thousands records. Now I want to generate a report based on my filter like customers, item categories and more....
Please refer queries.
Running on PHP 5.6. MySql 5.7 and Apache2.
SELECT
`si_items`.`item_id`
, SUM(qty) AS `qty`
, IFNULL(SUM(selling_price * (qty)), 0) AS `salestotal`
, GROUP_CONCAT(si.id) AS `siso_id`
, MAX(si.date_transaction) AS `date_transaction`
FROM
`invoice_items` AS `si_items`
LEFT JOIN `invoice` AS `si`
ON si.id = si_items.parent_id
LEFT JOIN `items`
ON si_items.item_id = items.id
WHERE (
DATE_FORMAT(si.date_transaction, '%Y-%m-%d') BETWEEN '2019-01-01'
AND '2019-02-15'
)
AND (si.approved = 1)
AND (si.deleted = 0)
AND (items.deleted = 0)
GROUP BY `item_id`
UNION
SELECT
`so_items`.`item_id`
, SUM(qty) AS `qty`
, IFNULL(SUM(selling_price * (qty)), 0) AS `salestotal`
, GROUP_CONCAT(so.id) AS `soso_id`
, MAX(so.date_transaction) AS `date_transaction`
FROM
`order_items` AS `so_items`
LEFT JOIN `order` AS `so`
ON so.id = so_items.parent_id
LEFT JOIN `items`
ON so_items.item_id = items.id
WHERE (
DATE_FORMAT(so.date_transaction, '%Y-%m-%d') BETWEEN '2019-01-01'
AND '2019-02-15'
)
AND (so.approved = 1)
AND (so.deleted = 0)
AND (items.deleted = 0)
GROUP BY `item_id`
When I executed this query for 50 days. It took 1 minute 20 seconds to execute this query.
INDEXES are added in tables
Invoice & Order Tables
PRIMARY KEY (`id`),
KEY `account_id` (`account_id`),
KEY `approved` (`approved`),
KEY `deleted` (`deleted`),
KEY `finalised` (`finalised`),
KEY `rp_status` (`rp_status`),
KEY `sales_types_id` (`sales_types_id`),
KEY `account_type_id` (`account_type_id`),
KEY `company_id` (`company_id`),
KEY `date_transaction` (`date_transaction`)
Invoices_items & Order_items
PRIMARY KEY (`id`),
KEY `deleted` (`deleted`),
KEY `item_id` (`item_id`),
KEY `parent_id` (`parent_id`),
KEY `vat_id` (`vat_id`),
KEY `qty` (`qty`),
Explain Query
Explain Query
I need to increase performance of this query. Could you please guide me how to proceed?
Show Create Tables
CREATE TABLE `invoice` (
`id` char(36) NOT NULL,
`reference` varchar(25) DEFAULT NULL,
`company_id` char(36) DEFAULT NULL,
`branch_id` char(36) DEFAULT NULL,
`account_id` char(36) DEFAULT NULL,
`contact_id` char(36) DEFAULT NULL,
`transaction_type` varchar(10) DEFAULT NULL,
`sales_types_id` int(11) DEFAULT '0',
`quote_validity` int(11) DEFAULT '0',
`delivery_method_id` int(11) DEFAULT '0',
`sales_representative_id` int(11) DEFAULT '0',
`account_type_id` char(36) DEFAULT NULL,
`vat_exempted` tinyint(1) DEFAULT '0',
`description` text,
`finalised` tinyint(1) DEFAULT '0' COMMENT 'Not Yet finalised - status=1; Need Approval - status = 2; Approved - status = 3',
`approved` tinyint(1) DEFAULT '0',
`approved_user_id` int(11) DEFAULT '0',
`default_sales_location_id` char(36) DEFAULT NULL COMMENT '0-Yes; 1-No',
`generate_do` tinyint(1) DEFAULT '1',
`generate_dn` tinyint(4) DEFAULT '1',
`do_status` tinyint(1) DEFAULT '0',
`cn_status` tinyint(1) DEFAULT '0',
`rp_status` tinyint(1) DEFAULT '0',
`dm_status` tinyint(1) DEFAULT '0',
`currency_id` char(36),
`exchange_rate_id` tinyint(1) DEFAULT '0',
`exchange_rate` double DEFAULT '1',
`date_transaction` datetime DEFAULT NULL,
`date_created` datetime DEFAULT NULL,
`date_modified` datetime DEFAULT NULL,
`created_user_id` int(11) DEFAULT '0',
`modified_user_id` int(11) DEFAULT '0',
`deleted` tinyint(1) DEFAULT '0',
PRIMARY KEY (`id`),
KEY `account_id` (`account_id`),
KEY `approved` (`approved`),
KEY `branch_id` (`branch_id`),
KEY `cn_status` (`cn_status`),
KEY `created_user_id` (`created_user_id`),
KEY `date_created` (`date_created`),
KEY `deleted` (`deleted`),
KEY `do_status` (`do_status`),
KEY `finalised` (`finalised`),
KEY `reference` (`reference`),
KEY `rp_status` (`rp_status`),
KEY `sales_types_id` (`sales_types_id`),
KEY `account_type_id` (`account_type_id`),
KEY `company_id` (`company_id`),
KEY `date_transaction` (`date_transaction`),
KEY `default_sales_location_id` (`default_sales_location_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
CREATE TABLE `invoice_items` (
`id` char(36) NOT NULL,
`parent_id` char(36) DEFAULT NULL,
`item_id` char(36) DEFAULT NULL,
`qty` double DEFAULT '0',
`cost_price` double DEFAULT '0',
`list_price` double DEFAULT '0',
`selling_price` double DEFAULT '0',
`unit_price` double DEFAULT '0',
`vat` double DEFAULT '0',
`amount` double DEFAULT '0',
`special_discount` double DEFAULT '0',
`price_change_status` tinyint(1) DEFAULT '0',
`remarks` text,
`vat_id` int(11) DEFAULT '1',
`stock_category_id` tinyint(2) DEFAULT '0' COMMENT '1: Stockable 2: Service',
`is_giftitem` tinyint(1) DEFAULT '0' COMMENT '1: Gift Item 0: NO Gift',
`item_type_status` tinyint(1) DEFAULT '0',
`date_created` datetime DEFAULT NULL,
`date_modified` datetime DEFAULT NULL,
`created_user_id` int(11) DEFAULT '0',
`modified_user_id` int(11) DEFAULT '0',
`deleted` tinyint(1) DEFAULT '0',
PRIMARY KEY (`id`),
KEY `deleted` (`deleted`),
KEY `item_id` (`item_id`),
KEY `parent_id` (`parent_id`),
KEY `stock_category_id` (`stock_category_id`),
KEY `item_type_status` (`item_type_status`),
KEY `vat_id` (`vat_id`),
KEY `amount` (`amount`),
KEY `qty` (`qty`),
KEY `unit_price` (`unit_price`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
Don't use LEFT JOIN when you mean JOIN. In particular, for joining to si.
WHERE (
DATE_FORMAT(si.date_transaction, '%Y-%m-%d') BETWEEN '2019-01-01'
AND '2019-02-15'
)
-->
WHERE si.date_transaction >= '2019-01-01'
AND si.date_transaction < '2019-01-15'
so that an index (see below) can use that column
WHERE si.date_transaction ...
AND (si.approved = 1)
AND (si.deleted = 0)
Add a composite index:
INDEX(deleted, approved, -- in either order
date_transaction) -- last
Make similar changes to so. Then let's hear how the performance is and see what the EXPLAIN has changed to.
UUIDs
Beware of UUIDs, they are bulky and slow. They are especially slow if the entire table cannot be cached.
I suspect you have uuids because I see CHAR(36).
By having CHARACTER SET utf8, that means 108 bytes is being used!. A UUID can be packed into a 16-byte BINARY(16). This would help with space (and hence speed).
But the real problem with UUIDs is with the randomness. Once the table becomes huge, the system becomes I/O-bound since the 'next' UUID is unlikely to be cached.
Consider switching to AUTO_INCREMENT ids. This is much preferred for single-server systems. If you need to generate ids from multiple locations, you may still need UUIDs.
More on UUIDs.
i have a problem in my examination system. In the admin panel, in the results page displays the correct score but the firstname and the last name not. I dont know what to do.
Here is the code:
$result1=executeQuery("select s.nume_student,s.prenume_student,s.stdname,s.stdid, IFNULL((select sum(q.marks) from studentquestion as sq,question as q where q.qnid=sq.qnid and sq.testid=".$_REQUEST['testid']." and sq.stdid=st.stdid and sq.stdanswer=q.correctanswer),0) as om from studenttest as st, student as s where s.stdid=st.stdid and st.testid=".$_REQUEST['testid'].";" );
$result3=executeQuery("SELECT sq.stdid, sq.testid, COUNT(*) AS correctAnswers, SUM(q.marks) AS studentScore,(SELECT SUM(marks) FROM question WHERE testid=".$_REQUEST['testid'].") AS totalScore FROM question q, studentquestion sq WHERE sq.testid=".$_REQUEST['testid']." AND q.testid = sq.testid AND q.qnid = sq.qnid AND sq.answered = 'answered' AND q.correctanswer = sq.stdanswer GROUP BY sq.stdid");
without $result1 doesen't display anything
And here are the tables from database:
question table:
CREATE TABLE IF NOT EXISTS `question` (
`testid` bigint(20) NOT NULL DEFAULT '0',
`qnid` int(11) NOT NULL DEFAULT '0',
`question` varchar(500) DEFAULT NULL,
`optiona` varchar(100) DEFAULT NULL,
`optionb` varchar(100) DEFAULT NULL,
`optionc` varchar(100) DEFAULT NULL,
`optiond` varchar(100) DEFAULT NULL,
`correctanswer` enum('optiona','optionb','optionc','optiond') DEFAULT NULL,
`marks` float(3,2) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
studentquestion table:
CREATE TABLE IF NOT EXISTS `studentquestion` (
`stdid` bigint(20) NOT NULL DEFAULT '0',
`testid` bigint(20) NOT NULL DEFAULT '0',
`qnid` int(11) NOT NULL DEFAULT '0',
`answered` enum('answered','unanswered','review') DEFAULT NULL,
`stdanswer` enum('optiona','optionb','optionc','optiond') DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
student table:
CREATE TABLE IF NOT EXISTS `student` (
`stdid` bigint(20) NOT NULL,
`stdname` varchar(40) DEFAULT NULL,
`stdpassword` varchar(40) DEFAULT NULL,
`emailid` varchar(40) DEFAULT NULL,
`contactno` varchar(20) DEFAULT NULL,
`fname` varchar(40) DEFAULT NULL,
`lname` varchar(40) DEFAULT NULL,
`pincode` varchar(20) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
student test table:
CREATE TABLE IF NOT EXISTS `studenttest` (
`stdid` bigint(20) NOT NULL DEFAULT '0',
`testid` bigint(20) NOT NULL DEFAULT '0',
`starttime` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`endtime` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
`correctlyanswered` int(11) DEFAULT NULL,
`status` enum('over','inprogress') DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
and test table:
CREATE TABLE IF NOT EXISTS `test` (
`testid` bigint(20) NOT NULL,
`testname` varchar(30) NOT NULL,
`testdesc` varchar(100) DEFAULT NULL,
`testdate` date DEFAULT NULL,
`testtime` time DEFAULT NULL,
`subid` int(11) DEFAULT NULL,
`testfrom` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`testto` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
`duration` int(11) DEFAULT NULL,
`totalquestions` int(11) DEFAULT NULL,
`attemptedstudents` bigint(20) DEFAULT NULL,
`testcode` varchar(40) NOT NULL,
`tcid` bigint(20) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
I resolved the same problem of here:
How to get sql WHERE statement auto update from localhost link
but i'm stucked here with fname and last name.
If your second query returns the correct student ids, you can just change it to add student names:
SELECT s.lastname, s.firstname, e.*
from
students s join (
... your second query ... ) e
on s.stdid=e.stdid
you can change your join type depends on what you want.
Thanks, but i resolved it like this:
SELECT ***s.fname, s.lname***, sq.stdid, sq.testid, COUNT(*) AS correctAnswers,
SUM(q.marks) AS studentScore,(SELECT SUM(marks) FROM question
WHERE testid=1) AS totalScore
FROM question q, studentquestion sq, ***student s***
WHERE sq.testid=1 AND q.testid = sq.testid
AND q.qnid = sq.qnid AND sq.answered = 'answered' ***and sq.stdid=s.stdid***
AND q.correctanswer = sq.stdanswer GROUP BY sq.stdid
I have a table called "tables_data_info" where is stored some of the data relative to different tables. Data like "created time", "editing time", "editing user", "create by user id" etc. I'm using this table because there was a dynamic php script that generate it automatically.
But, when i have a huge number of record ( 15k in this case ) the query getting very very very slow, and take "minutes" to do his job! But i'm not selecting all 15k records, i'm limiting to select 10 records at all!
A simple query:
SELECT pd.id, pd.title, pd.sell_price, pd.available_qt, tdi.createtime, tdi.lastupdatetime, tdi.create_member_id, tdi.create_group_id, tdi.last_update_member_id, tdi.last_update_group_id FROM zd_products AS pd LEFT JOIN zd_tables_data_info AS tdi ON ( tdi.targetid = pd.id and tdi.table_name = 'products' ) ORDER by pd.title ASC LIMIT 0, 10
How can i run this query differently but more efficiently ?
Here the table structure:
zd_products
CREATE TABLE `zd_products` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`title` varchar(256) NOT NULL DEFAULT '',
`internalcode` varchar(256) DEFAULT NULL,
`ean13_jan_code` varchar(256) DEFAULT NULL,
`upc_code` varchar(11) DEFAULT NULL,
`active` tinyint(1) NOT NULL DEFAULT '0',
`status` varchar(12) NOT NULL DEFAULT 'new',
`product_tags` longtext,
`buy_price` double NOT NULL DEFAULT '0',
`sell_price` double NOT NULL DEFAULT '0',
`fiscal_tax_id` int(11) NOT NULL DEFAULT '0',
`box_width` double NOT NULL DEFAULT '0',
`box_height` double NOT NULL DEFAULT '0',
`box_depth` double NOT NULL DEFAULT '0',
`box_weight` double NOT NULL DEFAULT '0',
`shipment_extra_price` double NOT NULL DEFAULT '0',
`available_qt` int(11) NOT NULL DEFAULT '0',
`allow_purchase_out_stock` tinyint(1) NOT NULL DEFAULT '0',
`meta_title` varchar(256) DEFAULT NULL,
`meta_description` varchar(256) DEFAULT NULL,
`meta_keywords` longtext,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=15730 DEFAULT CHARSET=utf8;
zd_tables_data_info
CREATE TABLE `zd_tables_data_info` (
`table_name` varchar(256) NOT NULL DEFAULT '',
`targetid` int(11) DEFAULT NULL,
`create_member_id` int(11) unsigned DEFAULT NULL,
`create_group_id` int(11) unsigned DEFAULT NULL,
`last_update_member_id` int(11) unsigned DEFAULT NULL,
`last_update_group_id` int(11) unsigned DEFAULT NULL,
`createtime` int(11) unsigned DEFAULT NULL,
`lastupdatetime` int(11) unsigned DEFAULT NULL,
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
PRIMARY KEY (`id`),
KEY `INDEX` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=19692 DEFAULT CHARSET=utf8;
Your data is not particularly big. Here is the query:
SELECT pd.id, pd.title, pd.sell_price, pd.available_qt,
tdi.createtime, tdi.lastupdatetime, tdi.create_member_id, tdi.create_group_id,
tdi.last_update_member_id, tdi.last_update_group_id
FROM zd_products pd LEFT JOIN
zd_tables_data_info tdi
ON tdi.targetid = pd.id and tdi.table_name = 'products'
ORDER by pd.title ASC
LIMIT 0, 10;
You can improve performance of this query with indexes. The two that come to mind are zd_products(title, id) and zd_tables_data_info(targetid, table_name). Try these and see if they help. You can create these indexes either in the create table statement (or alter table) or by using:
create index zd_products_title_id on zd_products(title, id);
create index zd_tables_data_info_targetid_table_name on zd_tables_data_info(targetid, table_name);
If not, put explain in front of your query and then edit your question with the resulting plan.
My knowledge of MySQL/ SQL apart from the simple CRUD operations is basic.
If I had to use a stored procedure to move certain attributes (not in a specific order) to another table how could it be done?
These are the following tables. I want to move from the 1st to the 2nd table.
As you can see the datatype sizes are different for certain columns.
CREATE TABLE IF NOT EXISTS `source_cdr` (
`callstart` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
`src` varchar(80) NOT NULL DEFAULT '',
`dst` varchar(80) NOT NULL DEFAULT '',
`accountcode` varchar(50) NOT NULL,
`uniqueid` varchar(100) NOT NULL,
`ID` int(11) NOT NULL AUTO_INCREMENT,
`callanswer` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
`callend` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
`disposition` varchar(50) NOT NULL,
`cdr_id` int(11) unsigned NOT NULL DEFAULT '0',
`pin_code` varchar(4) NOT NULL,
`provider` int(11) NOT NULL,
PRIMARY KEY (`ID`),
KEY `calldate_idx` (`callstart`) USING BTREE,
KEY `idx_acc_code_calldate` (`accountcode`,`callstart`) USING BTREE,
KEY `uniqueid` (`uniqueid`),
KEY `cdr_id` (`cdr_id`),
KEY `idx_uniqueid_cdr_id` (`uniqueid`,`cdr_id`)
) ENGINE=MyISAM;
--
CREATE TABLE IF NOT EXISTS `destination_cdr` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`calldate` datetime NOT NULL,
`source` varchar(80) NOT NULL,
`destination` varchar(80) NOT NULL,
`account_code` varchar(30) DEFAULT NULL,
`pincode` varchar(45) NOT NULL,
`duration_call` bigint(20) NOT NULL DEFAULT '0',
`duration_talk` bigint(20) NOT NULL,
`disposition` varchar(255) NOT NULL,
`clid` varchar(80) DEFAULT NULL,
`cdr_id` bigint(20) DEFAULT NULL,
`vxcdr_id` bigint(20) DEFAULT NULL,
`provider` int(11) NOT NULL DEFAULT '0'
PRIMARY KEY (`id`)
) ENGINE=MyISAM;
EDIT 1
An example of a row
('2012-03-18 20:54:49', '5796', '0761100866', '103f0124ad510516f33cab132c0a695b', 'call-F1884808-6753-2F10-181C-3A#10.217.164.33', 308006367, '2012-03-18 20:55:05', '2012-03-18 20:55:51', '200 OK', 2, '', 0),
Thanks
You can use MySQL: INSERT ... SELECT Syntax to copy data from one table to the other.
Decide common fields in both and copy the same.
Example:
INSERT INTO TABLE2( COL1, COLx, ... ) SELECT colM, colY FROM TABLE1;
If the column sizes mismatch, data truncation takes place, and you can't overcome that but redefine the destination table columns.
I use the following prepared statement:
SELECT *
FROM
c_members,c_users,c_positions,c_done_meetings
WHERE
c_positions.POS_ID=c_users.POS_ID
AND c_members.CLUB_ID = ?
AND USER_POINTS >= ?
AND USER_POINTS <= ?
AND c_users.POS_ID LIKE ?
AND MEM_ACADEMY LIKE ?
AND MEM_SEX LIKE ?
AND MEM_GRADELVL LIKE ?
AND MEM_GPA >= ?
AND MEM_GPA <= ?
AND MEM_ARCHIVE = 0
GROUP BY
c_members.MEM_ID, c_members.CLUB_ID
HAVING
SUM(c_done_meetings.MEDONE_ATTEND = 'u') >= 1
ORDER BY
USER_POINTS DESC
However this query takes 21.971405982971 seconds to load 111 records. When I remove "Having SUM(...)" clause, the performance is 100% better. Is there a way I could optimize it better?
Edit: (Table structures)
CREATE TABLE IF NOT EXISTS `c_done_meetings` (
`MEM_ID` int(11) NOT NULL,
`CLUB_ID` int(11) NOT NULL,
`MEETING_ID` int(11) NOT NULL,
`MEDONE_ATTEND` varchar(1) NOT NULL COMMENT 'E=excused, U=unexcused, P=present',
UNIQUE KEY `unique` (`MEM_ID`,`CLUB_ID`,`MEETING_ID`),
KEY `MEETING_ID` (`MEETING_ID`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
CREATE TABLE IF NOT EXISTS `c_members` (
`MEM_ID` int(11) NOT NULL,
`CLUB_ID` int(11) NOT NULL,
`MEM_FIRST` varchar(50) NOT NULL,
`MEM_MIDDLE` varchar(50) DEFAULT NULL,
`MEM_LAST` varchar(50) NOT NULL,
`MEM_SEX` tinyint(1) NOT NULL COMMENT '0-Male 1-Female',
`MEM_EMAIL` varchar(100) DEFAULT NULL,
`MEM_GRADELVL` int(11) NOT NULL,
`MEM_ACADEMY` varchar(50) DEFAULT '',
`MEM_GPA` double DEFAULT '0',
`MEM_ADDRESS` varchar(500) DEFAULT NULL,
`MEM_CITY` varchar(100) DEFAULT NULL,
`MEM_STATE` varchar(100) DEFAULT NULL,
`MEM_ZIP` int(11) DEFAULT NULL,
`MEM_TELEPHONE` varchar(25) DEFAULT NULL,
`MEM_AP` tinyint(1) NOT NULL,
`MEM_HONORS` tinyint(1) NOT NULL,
`MEM_ESOL` tinyint(1) NOT NULL,
`MEM_HISP` tinyint(1) NOT NULL,
`MEM_WHITE` tinyint(1) NOT NULL,
`MEM_MULTI` tinyint(1) NOT NULL,
`MEM_NATIVE` tinyint(1) NOT NULL,
`MEM_BLACK` tinyint(1) NOT NULL,
`MEM_ASIAN` tinyint(1) NOT NULL,
`MEM_EXTRA` varchar(10000) DEFAULT NULL,
`MEM_ARCHIVE` tinyint(1) NOT NULL DEFAULT '0',
UNIQUE KEY `MEM_ID` (`MEM_ID`,`CLUB_ID`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
CREATE TABLE IF NOT EXISTS `c_positions` (
`POS_ID` int(11) NOT NULL AUTO_INCREMENT,
`CLUB_ID` int(11) NOT NULL,
`POS_NAME` varchar(20) NOT NULL,
`POS_DESC` varchar(500) NOT NULL,
`POS_ADMIN` tinyint(1) NOT NULL,
`POS_ATN_VIEW` tinyint(1) NOT NULL,
`POS_ATN_CHKIN` tinyint(1) NOT NULL,
`POS_ATN_FINALIZE` tinyint(1) NOT NULL,
`POS_MEM_VIEW` tinyint(1) NOT NULL,
`POS_MEM_ADD` tinyint(1) NOT NULL,
`POS_MEM_EDIT` tinyint(1) NOT NULL,
`POS_POS_VIEW` tinyint(1) NOT NULL,
`POS_POS_ADD` tinyint(1) NOT NULL,
`POS_POS_EDIT` tinyint(1) NOT NULL,
`POS_MEET_VIEW` tinyint(1) NOT NULL,
`POS_MEET_ADD` tinyint(1) NOT NULL,
`POS_MEET_EDIT` tinyint(1) NOT NULL,
`POS_EVENT_VIEW` tinyint(1) NOT NULL,
`POS_EVENT_ADD` tinyint(1) NOT NULL,
`POS_EVENT_EDIT` tinyint(1) NOT NULL,
`POS_EVENT_UPDATE` tinyint(1) NOT NULL,
`POS_REPORT_VIEW` tinyint(1) NOT NULL,
`POS_ARCHIVE_VIEW` tinyint(1) NOT NULL,
`POS_ANNOUNCEMENTS` tinyint(1) NOT NULL,
`POS_WEB_CUSTOM` tinyint(1) NOT NULL,
PRIMARY KEY (`POS_ID`),
UNIQUE KEY `UNIQUE_NAME` (`CLUB_ID`,`POS_NAME`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=14 ;
CREATE TABLE IF NOT EXISTS `c_users` (
`MEM_ID` int(11) NOT NULL,
`CLUB_ID` int(11) NOT NULL,
`USER_PIN` int(11) NOT NULL,
`USER_POINTS` double NOT NULL,
`POS_ID` int(11) NOT NULL,
`USER_ARCHIVE` tinyint(1) NOT NULL DEFAULT '0',
UNIQUE KEY `MEM_ID` (`MEM_ID`,`CLUB_ID`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Edit 2:
Yes all ids are indexed, the SUM(...) >= # calculates the number of missed meetings. # is a parameter set by the user (I just hard coded a 1 for testing)
You need to have indexes an all fields used in the WHERE clause, on all fields on which the tables are joined (you need to explicitly state your join conditions as right now, you are getting a Cartesian join), on all fields used for grouping and all fields used for sorting.
The last problem is the HAVING clause there. You are not going to be able to use an index at all for that since it is a calculated value. If this is a query you will use often in the system (i.e. not just for reporting), you might consider adding a field you can use as a flag for this filtering purpose. Whenever you set c_done_meetings.MEDONE_ATTEND = 'u' in any of your queries, you could also set this flag for the member or user or whatever this is associated with so that you have an easy field to filter on in a WHERE clause.
Outside of that, you might actually gain better performance by getting a reduced list of users or members with that value of u in a subselect and then join using that subselect as a table.
EDIT:
After seeing your actual table structure, I can clearly see where you need to add indexes. I am also wondering why you have tables c_users and c_members with the same exact primary key. Why would these not just be a single table?
Some things that pop out at me:
You use like quite a lot. Try to do something with your php code so that this isn't necessary. For example, mem_sex has a limited number of choices. Make the front end a radio button or dropdown so you send a value where you can use = instead of like.
Two, if you add a sum() to the select clause, and the appropriate group by clause, it should run faster. It's worth a shot.
You can denormalise you tables and add field to c_members that represent your
SUM(c_done_meetings.MEDONE_ATTEND = 'u') >= 1
But you need to update that field always, when updating c_done_meetings (can be done with trigger)
Also try to avoid LIKE conditions. Use = insead (it is possible at least for SEX)