I turned on slow query monitor in MySQL config.
Below is the query and time:
Time: 160330 20:54:11
User#Host: user[user] # [xx.xx.xxx.xxx]
Query_time: 8.794170 Lock_time: 0.000141 Rows_sent: 3942 Rows_examined: 4742825
SET timestamp=1459371251;
SELECT (SELECT (CASE WHEN ce_type = 'IN' then SUM(payment_amount)
END) as debit
FROM customer_payment_options cpo
WHERE wallet_id=cw.id
AND (cpo.real_account_type='HQ')
AND cpo.source_country_id='40'
GROUP BY cpo.wallet_id)
as debit,
(SELECT SUM(payment_amount)
as credit
FROM customer_payment_options cpo
WHERE wallet_id=cw.id
AND (cpo.real_account_type='HQ')
AND cpo.tran_id IS NOT NULL
AND cpo.source_country_id='40'
GROUP BY cpo.wallet_id)
as credit
FROM customer_wallet cw
WHERE cw.company_id='1'
AND cw.currency='40'
AND cw.is_approved = '1'
AND DATE(cw.date_added) < '2016-03-30';
Indexes on customer_payment_options:
company_id
tran_id
ce_id
wallet_id
What should I do to improve it's performance?
EXPLAIN:
http://i.stack.imgur.com/iH8rt.png
SCHEMA
CREATE TABLE `customer_payment_options` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`company_id` int(11) NOT NULL,
`local_branch_id` int(11) NOT NULL,
`tran_id` bigint(11) DEFAULT NULL,
`ce_id` int(11) DEFAULT NULL,
`wallet_id` int(11) DEFAULT NULL,
`reward_credit_id` int(11) DEFAULT NULL,
`ce_invoice_id` varchar(32) DEFAULT NULL,
`ce_type` enum('IN','OUT') DEFAULT NULL,
`payment_type` enum('CASH','DEBIT','CREDIT','CHEQUE','DRAFT','BANK_DEPOSIT','EWIRE','WALLET','LOAN','REWARD_CREDIT') NOT NULL,
`payment_amount` varchar(20) NOT NULL,
`payment_type_number` varchar(100) DEFAULT NULL,
`source_country_id` int(11) NOT NULL,
`real_account_id` int(11) DEFAULT NULL,
`real_account_type` enum('LOCAL','HQ') DEFAULT NULL,
`date_added` datetime NOT NULL,
`event_type` enum('MONEY_TRANSFER','CURRENCY_EXCHANGE','WALLET') DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `company_id` (`company_id`),
KEY `real_account_type` (`real_account_type`),
KEY `tran_id` (`tran_id`),
KEY `ce_id` (`ce_id`),
KEY `wallet_id` (`wallet_id`),
CONSTRAINT `customer_payment_options_ibfk_4` FOREIGN KEY (`wallet_id`) REFERENCES `customer_wallet` (`id`),
CONSTRAINT `customer_payment_options_ibfk_1` FOREIGN KEY (`company_id`) REFERENCES `company` (`id`),
CONSTRAINT `customer_payment_options_ibfk_2` FOREIGN KEY (`tran_id`) REFERENCES `transaction` (`id`),
CONSTRAINT `customer_payment_options_ibfk_3` FOREIGN KEY (`ce_id`) REFERENCES `currency_exchange` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=412 DEFAULT CHARSET=utf8
CREATE TABLE `customer_wallet` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`wallet_unique_id` varchar(100) DEFAULT NULL,
`company_id` int(11) NOT NULL,
`branch_admin_id` int(11) DEFAULT NULL,
`emp_id` int(11) DEFAULT NULL,
`emp_type` enum('SUPER_ADMIN','ADMIN','AGENT_ADMIN','AGENT','OVER_AGENT_ADMIN','OVER_AGENT') DEFAULT NULL,
`cus_id` bigint(11) NOT NULL,
`tran_id` bigint(11) DEFAULT NULL,
`beehive_id` int(11) DEFAULT NULL,
`type` enum('DEPOSIT','WITHDRAW','TRANSACTION') NOT NULL,
`sub_type` enum('MONEY_TRANSFER','BEEHIVE_DEPOSIT') DEFAULT NULL,
`credit_in` varchar(20) DEFAULT NULL,
`credit_out` varchar(20) DEFAULT NULL,
`currency` varchar(20) NOT NULL,
`date_added` datetime NOT NULL,
`note` varchar(255) DEFAULT NULL,
`location` enum('DIRECT') DEFAULT NULL,
`is_approved` enum('0','1') NOT NULL DEFAULT '1',
`idebit_issconf` varchar(50) DEFAULT NULL,
`idebit_issname` varchar(50) DEFAULT NULL,
`idebit_isstrack2` varchar(100) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `cus_id` (`cus_id`),
KEY `company_id` (`company_id`),
KEY `branch_admin_id` (`branch_admin_id`),
KEY `emp_id` (`emp_id`),
KEY `tran_id` (`tran_id`),
KEY `beehive_id` (`beehive_id`),
CONSTRAINT `customer_wallet_ibfk_1` FOREIGN KEY (`cus_id`) REFERENCES `customers` (`id`),
CONSTRAINT `customer_wallet_ibfk_2` FOREIGN KEY (`company_id`) REFERENCES `company` (`id`),
CONSTRAINT `customer_wallet_ibfk_3` FOREIGN KEY (`tran_id`) REFERENCES `transaction` (`id`),
CONSTRAINT `customer_wallet_ibfk_4` FOREIGN KEY (`emp_id`) REFERENCES `employees` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=152 DEFAULT CHARSET=utf8
What you are doing as a correlated query on every wallet ID to get the corresponding debits and credits. It appears you are getting one record per wallet id. This is very busy. Having a join to the customer payments table on your criteria that is common (including the join per wallet id). Then, simplify the CASE as a SUM( case/when ) as respective debit / credit.
I don't know your underlying criteria of table columns, but I would even hedge to (and did) include NOT the CE_TYPE = 'IN' as that appears basis of a debit and you would not want to falsely count as part of a credit too. Again, dont know correlation of fields, trans_id, types.
Now, as stated, having individual indexes on individual fields will not help optimize this query. I would suggest the following indexes.
table index
customer_wallet ( company_id, is_approved, currency, id, date_added )
customer_payment_options ( wallet_id, account_type, country_id )
SELECT
cw.wallet_id,
SUM( case when cpo.ce_type = 'IN'
then cpo.payment_amount
ELSE 0 end ) as Debit,
SUM( case when NOT cpo.ce_type = 'IN'
AND cpo.tran_id IS NOT NULL
then cpo.payment_amount
ELSE 0 end ) as Credit
FROM
customer_wallet cw
JOIN customer_payment_options cpo
ON cw.id = cpo.wallet_id
AND cpo.real_account_type = 'HQ'
AND cpo.source_country_id = '40'
WHERE
cw.company_id = '1'
AND cw.currency = '40'
AND cw.is_approved = '1'
AND cw.date_added < '2016-03-30'
GROUP BY
cw.id
One additional comment. if your ID columns, Currency flag, country ID, approved are actually numeric values in the table structure, remove the quotes and let compare directly on the numeric value. Also, for your date_added. You had that based on DATE( date_added ). Doing a function on a column can not fully utilize the index. Since date() strips off any time portion of a date/time stamp column, and you are asking for all added less than Mar 30, then date added of March 29 # 11:59:59pm is still less than Mar 30 at 12:00:00am, so no date conversion is required.
As commented by Ivan (below), if you want ALL Wallet IDs regardless of having any payments (debit or credit), then change from a join to a LEFT JOIN.
You need to add indexes and multi-column indexes to make it fast.
Please keep in mind, that if you have large table, extra indexes will slow-down the insertions , since index file update will take more time.
If a multiple-column index exists on col1 and col2, the appropriate
rows can be fetched directly. If separate single-column indexes exist
on col1 and col2, the optimizer attempts to use the Index Merge
optimization (see Section 8.2.1.4, “Index Merge Optimization”), or
attempts to find the most restrictive index by deciding which index
excludes more rows and using that index to fetch the rows.
If the table has a multiple-column index, any leftmost prefix of the
index can be used by the optimizer to look up rows. For example, if
you have a three-column index on (col1, col2, col3), you have indexed
search capabilities on (col1), (col1, col2), and (col1, col2, col3).
Read more
We have the following two tables:
CREATE TABLE IF NOT EXISTS `gp` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`date` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`amount` decimal(15,2) NOT NULL,
`user` varchar(100) NOT NULL DEFAULT '',
`status` tinyint(2) NOT NULL DEFAULT '1',
`ip` varchar(20) NOT NULL DEFAULT 'N/A',
`token` varchar(100) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=1;
CREATE TABLE IF NOT EXISTS `gp_logs` (
`id` int(11) NOT NULL,
`log` varchar(1000) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
We JOIN them, for statistics, but we do this rarely, since the data from the 2nd table is not used too often except when we need to verify things.
Considering that we have many queries per second, how can our query be optimized to use 1 INSERT query instead of two and to insert the correct id in the 2nd table (gp_logs) that was generated by the INSERT into table gp?
Right now, we do a combination of MYSQL with PHP:
mysqli_query($con,"INSERT INTO `gp` (amount,user) VALUES ('1234','1')");
$id = mysqli_insert_id($con);
mysqli_query($con,"INSERT INTO gp_logs(id,log) VALUES ('$id','some_data')");
We want to eliminate the requirement of PHP for getting the last inserted ID and to insert both entries by running a single INSERT query (with a JOIN).
Trying to track outbound clicks on advertisements, but im having troubles constructing the query to compile all the statistics for the user to view and track.
I have two tables, one to hold all of the advertisements, the other to track clicks and basic details on the user. ip address, timestamp, user agent.
I need to pull all of map_advertisements information along with Unique Clicks based on IP Address, and Total Clicks based on map_advertisements.id to be showin in a table with rows. 1 row per advertisement and two of its columns will be totalClicks and totalUniqueClicks
Aside from running three seperate queries for each advertisement is there a better way to go about this?
I am using MySQL5 PHP 5.3 and CodeIgniter 2.1
#example of an advertisements id
$aid = 13;
SELECT
*
count(acl.aid)
count(acl.DISTINCT(ip_address))
FROM
map_advertisements a
LEFT JOIN map_advertisements_click_log acl ON a.id = acl.aid
WHERE
a.id = $aid;
map_advertisements
-- ----------------------------
-- Table structure for `map_advertisements`
-- ----------------------------
DROP TABLE IF EXISTS `map_advertisements`;
CREATE TABLE `map_advertisements` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`youtube_id` varchar(255) NOT NULL,
`status` int(11) NOT NULL DEFAULT '1',
`timestamp` int(11) NOT NULL,
`type` enum('video','picture') NOT NULL DEFAULT 'video',
`filename` varchar(255) NOT NULL,
`url` varchar(255) NOT NULL,
`description` varchar(64) NOT NULL,
`title` varchar(64) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=8 DEFAULT CHARSET=utf8 ROW_FORMAT=COMPACT;
map_advertisements_click_log
-- ----------------------------
-- Table structure for `map_advertisements_click_log`
-- ----------------------------
DROP TABLE IF EXISTS `map_advertisements_click_log`;
CREATE TABLE `map_advertisements_click_log` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`aid` int(11) NOT NULL,
`ip_address` varchar(15) NOT NULL DEFAULT '',
`browser` varchar(255) NOT NULL,
`timestamp` int(11) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=26 DEFAULT CHARSET=latin1;
A problem seems to be in your query there is no column with the name totalClicks in your table and distinct keyword is also used incorrectly. Try this:
SELECT *, count(acl.id) as totalClicks, count(DISTINCT acl.ip_address) as uniqueClicks
FROM map_advertisements a
LEFT JOIN map_advertisements_click_log acl ON a.id = acl.aid
WHERE a.id = $aid;
i am building a real estate application where in it will store the properties and search it. the property will have different categories like (residential, commercial, industrial or agricultural). based upon the category i want to serailize each and every property listing . for example the property with id 1 belongs to resedential will have the serial code rs_SOMERANDOMUNIQUENUMBER. and for commercial it can be cm_SOMERANDOMUNIQUENUMBER and so on. for this my database table looks like this.
CREATE TABLE IF NOT EXISTS `propSerials` (
`id` bigint(20) NOT NULL auto_increment,
`serial` varchar(50) NOT NULL,
`property_id` int(10) UNIQUE NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
what would be the best possible format to store the serial with the prefix according to category?
thank you
Why dont you add another column that holds category_id and in category table add column with prefixes for that category.
CREATE TABLE IF NOT EXISTS `propSerials` (
`id` bigint(20) NOT NULL auto_increment,
`serial` varchar(50) NOT NULL,
`property_id` int(10) UNIQUE NOT NULL,
`category_id` int(10) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
CREATE TABLE IF NOT EXISTS `propCategories` (
`id` bigint(20) NOT NULL auto_increment,
`category` varchar(50) NOT NULL,
`property_prefix` char(3) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
In query you can:
SELECT CONCAT('prefix_', 'serial');
I am a bit new to sql and php so I need some help here. I created two tables and I want to establish a 'one-to-many relationship' but I can't figure out how I can give the data through the php file. Here is some code:
CREATE TABLE `details` (
`details_id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(30) NOT NULL,
`latitude` double NOT NULL,
`longitude` double NOT NULL,
`mytext` varchar(60) NOT NULL,
`description` varchar(600) NOT NULL,
`city_details` int(11) NOT NULL,
PRIMARY KEY (`details_id`),
FOREIGN KEY (`city_details`) REFERENCES city(`city_id`)
on update cascade
);
CREATE TABLE `city` (
`city_id` int(11) NOT NULL AUTO_INCREMENT,
`cityName` varchar(30) NOT NULL,
PRIMARY KEY (`city_id`)
);
So I want to write a cityName and then be able to give some data for various places of the city. How can I create a php file so that I will only have to give the name of the city (to table city) and then write things for multiple locations inside the city (to table details).
$sql="SELECT * FROM `details` as d INNER JOIN `city` as c ON d.city_details=c.city_id;";
if you want to look for a city name you can use this
$sql="SELECT * FROM `details` as d INNER JOIN `city` as c ON d.city_details=c.city_id WHERE c.cityName='The name you're looking for'";
then to fetch everyting from the table use this code
while($row=mysql_fetch_assoc(mysql_query($sql))){
echo $row['name']; //for name
echo $row['mytext']; //etc.
}
for more info see
http://dev.mysql.com/doc/refman/5.1/en/join.html
and
http://php.net/manual/en/function.mysql-fetch-assoc.php