Mysql 5.6 doesn't want to set a column to NULL - php

I am about 98% sure this query has been working properly on mysql 5.1. We have upgraded one of our machines to mysql 5.6 and I am running an update and its getting a database error with this query:
UPDATE diagnostic
LEFT JOIN contact ON diagnosticdata_suppliercontact = contact_id
SET diagnosticdata_suppliercontact = NULL
WHERE (!contactdata_issupplier) OR (contact.contact_id IS NULL);
However the table schema allows it to be NULL. Does anyone know of any problems or changes that happened in mysql 5.6 that could cause us grief?
Here is a description of the table:
mysql> describe diagnostic;
+--------------------------------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------------------------+-------------+------+-----+---------+----------------+
| diagnostic_id | int(11) | NO | PRI | NULL | auto_increment |
| diagnostic_time | int(15) | NO | | NULL | |
| diagnostic_user | int(15) | NO | | NULL | |
| diagnosticdata_time | int(15) | NO | | NULL | |
| diagnosticdata_user | int(15) | NO | | NULL | |
| diagnosticdata_name | varchar(50) | NO | | NULL | |
| diagnosticdata_suppliercontact | int(11) | YES | MUL | NULL | |
+--------------------------------+-------------+------+-----+---------+----------------+
7 rows in set (0.00 sec)
ERROR 1048 (23000): Column 'diagnosticdata_suppliercontact' cannot be null
CREATE STATEMENT IS HERE TO there are triggers:
delimiter $$
CREATE TABLE `diagnostic` (
`diagnostic_id` int(11) NOT NULL AUTO_INCREMENT,
`diagnostic_time` int(15) NOT NULL,
`diagnostic_user` int(15) NOT NULL,
`diagnosticdata_time` int(15) NOT NULL,
`diagnosticdata_user` int(15) NOT NULL,
`diagnosticdata_name` varchar(50) NOT NULL,
`diagnosticdata_suppliercontact` int(11) DEFAULT NULL,
PRIMARY KEY (`diagnostic_id`),
KEY `diagnostic_suppliercontact` (`diagnosticdata_suppliercontact`) USING BTREE,
CONSTRAINT `fk_diagnosticdata_suppliercontact_contact` FOREIGN KEY (`diagnosticdata_suppliercontact`) REFERENCES `contact` (`contact_id`)
) ENGINE=InnoDB AUTO_INCREMENT=188 DEFAULT CHARSET=utf8$$
CREATE
DEFINER=`dotuser`#`localhost`
TRIGGER `ezymerged`.`diagnostic_insert`
AFTER INSERT ON `ezymerged`.`diagnostic`
FOR EACH ROW
BEGIN
REPLACE INTO ezymerged_history.diagnostic
SELECT diagnostic.* FROM ezymerged.diagnostic
WHERE diagnostic_id=NEW.diagnostic_id;
END
$$
CREATE
DEFINER=`dotuser`#`localhost`
TRIGGER `ezymerged`.`diagnostic_update`
AFTER UPDATE ON `ezymerged`.`diagnostic`
FOR EACH ROW
BEGIN
REPLACE INTO ezymerged_history.diagnostic
SELECT diagnostic.* FROM ezymerged.diagnostic
WHERE diagnostic_id=NEW.diagnostic_id;
END
$$

While the diagnosticdata_suppliercontact allows for NULLs the constraint:
CONSTRAINT `fk_diagnosticdata_suppliercontact_contact`
FOREIGN KEY (`diagnosticdata_suppliercontact`)
REFERENCES `contact` (`contact_id`)
will take precedence, and requires the value in diagnosticdata_suppliercontact to have a matching value in the contact_id field in the contact table.

Related

MySql LEFT JOIN returns wrong first in PHP page, but correct results in MYSQL command line

We added two columns to our main MySQL table (named objects), a column for quantity and a column for id of the units. If the quantity and/or units are unknown, those columns are set to NULL.
We wanted to join this table with the unittype table to display the name of the units (pounds, pages, grams, whatever).
When I run the SELECT inside MYSQL's command line client, it returns the expected the results. When my PHP page runs the query, the first row shows values from the wrong side of the join.
Example:
The tables being joined are objects and unittypes with these definitions
mysql> show create table objects;
CREATE TABLE `objects` (
`objId` int NOT NULL AUTO_INCREMENT,
`name` varchar(100) DEFAULT NULL,
`description` varchar(250) DEFAULT NULL,
`quantity` int DEFAULT NULL,
`unitid` int DEFAULT NULL,
`create_date` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`modify_date` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`objId`),
KEY `unitid` (`unitid`),
CONSTRAINT `objects_ibfk_1` FOREIGN KEY (`unitid`) REFERENCES `unittypes` (`unitId`) ON DELETE SET NULL ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=4735 DEFAULT CHARSET=utf8
and:
mysql> show create table unittypes;
CREATE TABLE `unittypes` (
`unitId` int NOT NULL AUTO_INCREMENT,
`name` varchar(50) DEFAULT NULL,
`description` varchar(50) DEFAULT NULL,
`create_date` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`modify_date` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`unitId`),
UNIQUE KEY `name` (`name`)
) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8
Sample rows:
Unittypes:
+--------+------+--------------+-
| unitId | name | description |
+--------+------+--------------+-
| 1 | Each | Each |
Objects:
+-------+---------------------------------------+-------------+----------+--------+
| objId | NAME | description | quantity | unitid |
+-------+---------------------------------------+-------------+----------+--------+
| 1018 | UNKNOWN cables | NULL | NULL | NULL |
| 3466 | UNKNOWN replies | NULL | NULL | NULL |
+-------+---------------------------------------+-------------+----------+--------+
| 722 | Soundgarden-Loudest Love | NULL | 1 | 1 |
| 4703 | Soundgarden-Live FROM The Artists Den | NULL | 1 | 1 |
+-------+---------------------------------------+-------------+----------+--------+
Sample join query:
SELECT o.objid, o.name, o.description,
o.quantity, u.name, u.description, o.create_date, o.modify_date
FROM objects AS o
LEFT JOIN unittypes u USING(unitid)
WHERE o.NAME LIKE '%Soundgarden-L%'
In MYSql command line client it returns:
+-------+---------------------------------------+-------------+----------+------+-------------+---------------------+---------------------+
| objid | name | description | quantity | name | description | create_date | modify_date |
+-------+---------------------------------------+-------------+----------+------+-------------+---------------------+---------------------+
| 722 | Soundgarden-Loudest Love | NULL | 1 | Each | Each | 2017-04-01 05:44:56 | 2020-01-15 08:59:45 |
| 4703 | Soundgarden-Live From The Artists Den | NULL | 1 | Each | Each | 2019-09-17 21:52:26 | 2020-01-15 08:59:45 |
+-------+---------------------------------------+-------------+----------+------+-------------+---------------------+---------------------+
In PHP page, it returns (PHP page returned the expected results without a join:
objid name description quantity create_date modify_date
722 Each Each 1 2017-04-01 05:44:56 2020-01-15 08:59:45
4703 Soundgarden-Live From The Artists Den NULL 1 Each Each 2019-09-17 21:52:26 2020-01-15 08:59:45
Result set has 3 rows.
Example when unittypes doesn't have a matching row:
In command line client:
SELECT o.objid, o.name, o.description,
o.quantity, u.name, u.description, o.create_date, o.modify_date
FROM objects AS o
LEFT JOIN unittypes u USING(unitid)
WHERE o.NAME LIKE '%unknown %';
result:
+-------+--------------------------------------------------+-------------+----------+------+-------------+---------------------+---------------------+
| objid | name | description | quantity | name | description | create_date | modify_date |
+-------+--------------------------------------------------+-------------+----------+------+-------------+---------------------+---------------------+
| 1018 | Unknown cables | NULL | NULL | NULL | NULL | 2017-06-27 02:22:12 | 2017-06-27 02:22:12 |
| 3466 | Unknown replies | NULL | NULL | NULL | NULL | 2018-11-05 01:45:17 | 2018-11-05 01:45:17 |
|
In PHP page:
objid name description quantity create_date modify_date
1018 NULL NULL NULL 2017-06-27 02:22:12 2017-06-27 02:22:12
3466 Unknown replies NULL NULL NULL NULL 2018-11-05 01:45:17 2018-11-05 01:45:17
I get the same results using the join syntax
left join unittypes u on o.unitid = u.unitid
PHP code that performs query:
if ($result = $GLOBALS['DB']->query($selObj)) {
if ($row = $result->fetch_assoc()) {
$row_cnt = $result->num_rows;
echo '<table><tr>';
foreach(array_keys($row) as $heading) echo "<th>$heading</th>";
echo '</tr>';
/* process result set */
do {
echo '<tr>';
foreach($row as $item) echo '<td>'.($item==NULL?'NULL':$item).'</td>';
echo '</tr>';
} while ($row = $result->fetch_row()); // get next result set
echo '</table>';
printf("Result set has %d row%s.<br><br>", $row_cnt,($row_cnt==1?"":"s"));
$result->close();
} else echo 'Empty result set<br><br>';
} else echo $DB->error;
I would expect the same query to behave the same way in both examples.
I would have guessed that the command-line and the PHP simply hand off a string to mySQL to process, but there is obviously some difference I'm not seeing.
Is there something weird about how the PHP passes the query in; does MYsql not handle joins through that interface properly; is the code wrong (it works without a join)?
(I also asked this at https://dba.stackexchange.com/q/258940/200179)
You have two columns called name. The second will overwrite the first.
You should alias them to have distinct names. select `o`.`name` as `o_name` ...
In your case, the reason why only the first row is affected is because you use fetch_assoc to get that, but then fetch_row to get the others (and fetch_row works because it ignores column names entirely)

Confusion between PK and FK table design

I have a person table and a score table. The Person table basically stores a person's information while score table stores what kind of score a person has. I set the FK constraint in score table to ON DELETE: CASCADE
person
- id
- name
- scored_id (FK)
score
- id (PK)
- bmi
- weight
So, in the table setting score.id is linked with person's scored_id. That being said when I delete a record in score, a person will get deleted as well. But why when I delete a record in person, the record of his in score is not deleted?
Just an idea how you might structure the tables and use a foreign key which will delete records from the score table if/when a user from the person table is deleted. The score table should have a reference to the user - pid which is used as the foreign key dependancy. It makes sense to me that the score is dependant upon the user so no user, no score.
create table `person` (
`id` int(10) unsigned not null auto_increment,
`name` varchar(50) null default null,
primary key (`id`)
)
collate='latin1_swedish_ci'
engine=innodb
auto_increment=4;
mysql> describe person;
+-------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------+------------------+------+-----+---------+----------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| name | varchar(50) | YES | | NULL | |
+-------+------------------+------+-----+---------+----------------+
create table `score` (
`id` int(10) unsigned not null auto_increment,
`bmi` int(10) unsigned not null default '0',
`weight` int(10) unsigned not null default '0',
`pid` int(10) unsigned not null default '0',
primary key (`id`),
index `pid` (`pid`),
constraint `fk_sc_pid` foreign key (`pid`) references `person` (`id`) on update cascade on delete cascade
)
collate='latin1_swedish_ci'
engine=innodb
auto_increment=4;
mysql> describe score;
+--------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------+------------------+------+-----+---------+----------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| bmi | int(10) unsigned | NO | | 0 | |
| weight | int(10) unsigned | NO | | 0 | |
| pid | int(10) unsigned | NO | MUL | 0 | |
+--------+------------------+------+-----+---------+----------------+
mysql> select * from person;
+----+------+
| id | name |
+----+------+
| 1 | bob |
| 2 | rita |
| 3 | sue |
+----+------+
mysql> select * from score;
+----+-----+--------+-----+
| id | bmi | weight | pid |
+----+-----+--------+-----+
| 1 | 34 | 34 | 1 |
| 2 | 56 | 41 | 2 |
| 3 | 56 | 77 | 3 |
+----+-----+--------+-----+
mysql> delete from person where id=3;
Query OK, 1 row affected (0.00 sec)
/* delete a user, the score disappears too which makes sense */
mysql> select * from score;
+----+-----+--------+-----+
| id | bmi | weight | pid |
+----+-----+--------+-----+
| 1 | 34 | 34 | 1 |
| 2 | 56 | 41 | 2 |
+----+-----+--------+-----+
Your issue is semantic understanding of the task, rather than syntax. Intuitively your relation looks wrong. It is unlikely, that a particular score, say 75kg and bmi of 20 will need to have a many relations link to people with the same score. This would be arbitary. More likely, your want, a person to have different scores over time, then when you delete a person, you want their associated values deleted. So table relation should be:
person
- id (Primary Key)
- name
score
- id (Primary Key)
- bmi
- weight
- scoreDate
- personID (Foreign Key to person)
A score date would be a helpful addition.
This structure will allow a person to have a history of many score and see the fluctuation of their weight and body mass index over time. A semantically helpful task that resonates with reality, and therefore follows the notions of entity analysis and table structures following the real world application.
Helpful discussion of ERD and table structure levels and relations
In you tables, "person" table is having reference(FK) of "score" table so when you delete a record in "score" table mysql search related record in "users" table to delete.
but "score" table dose not have any reference(FK) of "person" table.
You can try below table structure if you want to delete score record when person record will be delete but person record will be still safe if score record will be delete
person
- id (PK)
- name
score
- id (PK)
- person_id (FK)
- bmi
- weight

Query execution time issue

I have some trouble with the execution time of a script. The query takes a lot of time. This is the query:
select avg(price) from voiture where duration<30 AND make="Audi" AND model="A4"
+---+---+---+---+---+---+---+---+---+---+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+---+---+---+---+---+---+---+---+---+---+
| 1 | SIMPLE | voiture | ALL | NULL | NULL | NULL | NULL | 1376949 | Using where |
+---+---+---+---+---+---+---+---+---+---+
select price from voiture where duration<30 AND make="Audi" AND model="A4"
+---+---+---+---+---+---+---+---+---+---+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+---+---+---+---+---+---+---+---+---+---+
| 1 | SIMPLE | voiture | ALL | NULL | NULL | NULL | NULL | 1376949 | Using where |
+---+---+---+---+---+---+---+---+---+---+
This query take around 2 seconds to be executed on the phpMyAdmin interface. I tried to see what the issue was and removing the avg function makes the query lasts around 0.0080 seconds.
I asked myself how long it would take to make the calculate the avg in the php script, but the query with or withouth avg takes around 2 seconds both.
So I decided to take all the values of my table and make the process in the script, so I use this query:
select * from voiture where duration<30 AND make="Audi" AND model="A4"
+---+---+---+---+---+---+---+---+---+---+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+---+---+---+---+---+---+---+---+---+---+
| 1 | SIMPLE | voiture | ALL | NULL | NULL | NULL | NULL | 1376949 | Using where |
+---+---+---+---+---+---+---+---+---+---+
On the phpMyAdmin interface, it takes 0.0112 seconds. But in my php script, it takes 25 seconds !
$timestart2=microtime(true);
$querysec='select * from voiture where duration<30 AND make="Audi" AND model="A4"';
$requestsec=mysql_query($querysec) or die(mysql_error());
$timeend2=microtime(true);
$time2=$timeend2-$timestart2;
$page_load_time2 = number_format($time2, 9);
Here is the table structure:
CREATE TABLE `voiture` (
`carid` bigint(9) NOT NULL,
`serviceid` bigint(9) NOT NULL,
`service` varchar(256) NOT NULL,
`model` varchar(256) DEFAULT NULL,
`gearingType` varchar(256) DEFAULT NULL,
`displacement` int(5) DEFAULT NULL,
`cylinders` int(2) DEFAULT NULL,
`fuel` varchar(32) DEFAULT NULL,
`mileage` int(7) DEFAULT NULL,
`existFlag` tinyint(1) DEFAULT NULL,
`lastUpdate` date DEFAULT NULL,
`version` varchar(256) DEFAULT NULL,
`bodyType` varchar(256) DEFAULT NULL,
`firstRegistration` date DEFAULT NULL,
`powerHp` int(4) DEFAULT NULL,
`powerKw` int(4) DEFAULT NULL,
`vat` varchar(256) DEFAULT NULL,
`price` decimal(12,2) DEFAULT NULL,
`duration` int(3) DEFAULT NULL,
`pageUrl` varchar(256) DEFAULT NULL,
`carImg` varchar(256) DEFAULT NULL,
`color` varchar(256) DEFAULT NULL,
`doors` int(1) DEFAULT NULL,
`seats` int(1) DEFAULT NULL,
`prevOwner` int(1) DEFAULT NULL,
`co2` varchar(256) DEFAULT NULL,
`consumption` varchar(256) DEFAULT NULL,
`gears` int(1) DEFAULT NULL,
`equipment` varchar(1024) NOT NULL,
`make` varchar(256) NOT NULL,
`country` varchar(3) NOT NULL
)
There's an index on carid and serviceid
Why does my query takes so long to be executed ? Is there a way it can be improved ?
Why is the execution time different from phpMyAdmin and my php script ?
On the phpMyAdmin interface, it takes 0.0112 seconds. But in my php
script, it takes 25 seconds!
phpMyAdmin interface adds LIMIT to each query. By default it's LIMIT 30.
To decrease time of your aggregate query you need to create indexes for each condition you use(or one composite index, may be).
So, try to create indexes for your model, make and duration fields.
Also, your table is too denormalized. You can to create pair of table to normalize it for a bit.
Ex: Vendors(id, name), Models(id, name) and modify your voiture to have vendor_id/model_id fields instead of text make/model.
Then your initial query will look like:
select avg(t.price) from voiture t
INNER JOIN Models m ON m.id = t.model_id
INNER JOIN Vendors v ON v.id = t.vendor_id
where t.duration<30 AND v.name="Audi" AND m.name="A4"
It will scan light lookup tables for text matches and operate with your heavy table with indexed ids.
There are many possible solution, first thing you can do is create index at DB level this can also improve your execution time.
second this check server, there may be some processes which would occupying your server resources, and making your server slow.
You can make it faster using GROUP BY like
select AVG(price) from voiture where duration<30 AND make="Audi" AND model="A4" GROUP BY make
Also you can add an index for make column
ALTER TABLE `voiture` ADD INDEX `make` (`make`)

"Integrity constraint violation: 1062 Duplicate entry" - but no duplicate rows

I'm converting an app from native mysqli calls to PDO. Running into an error when attempting to insert a row into a table with a foreign key constraint.
Note: this is a simplified test case and should not be copy/pasted into a production environment.
Info PHP 5.3, MySQL 5.4
First, here are the tables:
CREATE TABLE `z_one` (
`customer_id` int(10) unsigned NOT NULL DEFAULT '0',
`name_last` varchar(255) DEFAULT NULL,
`name_first` varchar(255) DEFAULT NULL,
`dateadded` datetime DEFAULT NULL,
PRIMARY KEY (`customer_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
INSERT INTO `z_one` VALUES (1,'Khan','Ghengis','2014-12-17 10:43:01');
CREATE TABLE `z_many` (
`order_id` varchar(15) NOT NULL DEFAULT '',
`customer_id` int(10) unsigned DEFAULT NULL,
`dateadded` datetime DEFAULT NULL,
PRIMARY KEY (`order_id`),
KEY `order_index` (`customer_id`,`order_id`),
CONSTRAINT `z_many_ibfk_1` FOREIGN KEY (`customer_id`) REFERENCES `z_one` (`customer_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Or if you prefer,
mysql> describe z_one;
+-------------+------------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+------------------+------+-----+---------+-------+
| customer_id | int(10) unsigned | NO | PRI | 0 | |
| name_last | varchar(255) | YES | | NULL | |
| name_first | varchar(255) | YES | | NULL | |
| dateadded | datetime | YES | | NULL | |
+-------------+------------------+------+-----+---------+-------+
4 rows in set (0.00 sec)
mysql> describe z_many;
+-------------+------------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+------------------+------+-----+---------+-------+
| order_id | varchar(15) | NO | PRI | | |
| customer_id | int(10) unsigned | YES | MUL | NULL | |
| dateadded | datetime | YES | | NULL | |
+-------------+------------------+------+-----+---------+-------+
3 rows in set (0.00 sec)
Next, here is the query:
$order_id = '22BD24';
$customer_id = 1;
try
{
$q = "
INSERT INTO
z_many
(
order_id,
customer_id,
dateadded
)
VALUES
(
:order_id,
:customer_id,
NOW()
)
";
$stmt = $dbx_pdo->prepare($q);
$stmt->bindValue(':order_id', $order_id, PDO::PARAM_STR);
$stmt->bindValue(':customer_id', $customer_id, PDO::PARAM_INT);
$stmt->execute();
} catch(PDOException $err) {
// test case only. do not echo sql errors to end users.
echo $err->getMessage();
}
This results in the following PDO error:
SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry
'22BD24' for key 'PRIMARY'
The same query works fine when handled by mysqli. Why is PDO rejecting the INSERT with a 'duplicate entry' message when there aren't any duplicates found?
Since not all code is available (from php side) just in case your query is in some sort of loop the quickest (and perhaps partly) solution to this is the following:
$order_id = '22BD24';
$customer_id = 1;
try {
$q = "INSERT INTO `z_many` (`order_id`,`customer_id`,`dateadded`)
VALUES (:order_id,:customer_id,NOW())
ON DUPLICATE KEY UPDATE `dateadded`=NOW()";
$stmt = $dbx_pdo->prepare($q);
$stmt->bindValue(':order_id', $order_id, PDO::PARAM_STR);
$stmt->bindValue(':customer_id', $customer_id, PDO::PARAM_INT);
$stmt->execute();
} catch(PDOException $err) {
// test case only. do not echo sql errors to end users.
echo $err->getMessage();
}
I've copied the SQL schema you provided on my mysql DB and added script code, but with PDO initialization at the start:
$dbx_pdo = new PDO('mysql:host=127.0.0.1;dbname=test12;charset=utf8','root','');
.. and it worked fine, but my setup is php 5.5.9 and mysql 5.6.16
So I think your code executes twice and maybe its inside of a transaction, so you get rollback. Need to know more context
Please REMOVE default for primary key column. Also use the construction
INSERT INTO (`field1`, `field2`, `...`) values (val1, val2, val3);
If you tell what default value is for inserts - some mysql versions can see errors when inserting. Thats why you should use auto-increment or dont use default value at all.
Just a shot in the dark. I use PDO with ORACLE PL/SQL only with bindParam(). And have a look at the forth parameter ,15 if it is a PARAM_STR value. So try this, hope it helps.
$stmt->bindParam(':order_id', $order_id, PDO::PARAM_STR, 15);
$stmt->bindParam(':customer_id', $customer_id, PDO::PARAM_INT);
This 15 descibes the (bufffer-)length from order_id in your table.

Should the following queries take 1 minute or more to complete?

The following queries use 80% or more CPU and can take more than 1 minute to complete.
My question: Is there anything wrong with my queries that would cause CPU usage like that? Can I decrease CPU usage and query time by optimizing the MySQL server conf?
Query 1 (loan_history contains 2.6 million records)
SELECT officer, SUM(balance) as balance
FROM loan_history
WHERE bank_id = '1'
AND date ='2013-07-04'
AND officer IS NOT NULL
AND officer <> ''
GROUP BY officer
ORDER BY officer;
Query 2 (loan_history contains 2.6 million records)
SELECT SUM(weighted_interest_rate) as total
FROM (SELECT balance, tmp1.balance_sum,
(balance / tmp1.balance_sum * interest_rate) as weighted_interest_rate
FROM loan_history,
(SELECT SUM(balance) balance_sum FROM loan_history
WHERE date = '2013-07-04'
AND bank_id = '1') as tmp1
WHERE date = '2013-07-04'
AND bank_id = '1') tmp2
Table information:
CREATE TABLE `loan_history` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`bank_id` int(11) DEFAULT NULL,
`loan_purpose_id` int(11) DEFAULT NULL,
`date` date NOT NULL,
`credit_grade` varchar(5) COLLATE utf8_unicode_ci DEFAULT NULL,
`interest_rate` decimal(5,2) NOT NULL,
`officer` varchar(5) COLLATE utf8_unicode_ci DEFAULT NULL,
`balance` decimal(10,2) NOT NULL,
`start_date` date DEFAULT NULL,
`days_delinquent` int(11) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `IDX_9F5FE3F11C8FB41` (`bank_id`),
KEY `IDX_9F5FE3F6F593857` (`loan_purpose_id`),
KEY `date` (`date`),
KEY `credit_grade` (`credit_grade`),
KEY `officer` (`officer`),
KEY `start_date` (`start_date`),
KEY `days_delinquent` (`days_delinquent`),
KEY `interest_rate` (`interest_rate`),
KEY `balance` (`balance`),
CONSTRAINT `FK_9F5FE3F11C8FB41` FOREIGN KEY (`bank_id`) REFERENCES `bank` (`id`),
CONSTRAINT `FK_9F5FE3F6F593857` FOREIGN KEY (`loan_purpose_id`) REFERENCES `loan_purpose` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=2630634 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
Query 1 EXPLAIN:
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
| 1 | SIMPLE | loan_history | index_merge | IDX_9F5FE3F11C8FB41,date,officer | date,IDX_9F5FE3F11C8FB41 | 3,5 | NULL | 4829 | Using intersect(date,IDX_9F5FE3F11C8FB41); Using where; Using temporary; Using filesort |
Query 2 EXPLAIN:
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra
| 1 | PRIMARY | <derived2> | ALL | NULL | NULL | NULL | NULL | 8236 |
| 2 | DERIVED | <derived3> | system | NULL | NULL | NULL | NULL | 1 |
| 2 | DERIVED | loan_history | index_merge | IDX_9F5FE3F11C8FB41,date | date,IDX_9F5FE3F11C8FB41 | 3,5 | NULL | 4829 | Using intersect(date,IDX_9F5FE3F11C8FB41); Using where; Using index |
| 3 | DERIVED | loan_history | index_merge | IDX_9F5FE3F11C8FB41,date | date,IDX_9F5FE3F11C8FB41 | 3,5 | NULL | 4829 | Using intersect(date,IDX_9F5FE3F11C8FB41); Using where; Using index |
My.cnf file:
default-storage-engine=MyISAM
interactive_timeout=300
key_buffer_size=256M
key_cache_block_size=4096
max_heap_table_size=128M
max_join_size=1000000000
max_allowed_packet=32M
open_files_limit=4096
query_cache_size=256M
query_cache_limit=10240M
query_cache_type=1
table_cache=256
thread_cache_size=100
tmp_table_size=128M
wait_timeout=7800
max_user_connections=50
join_buffer_size=256K
sort_buffer_size=4M
read_rnd_buffer_size=1M
innodb_open_files=300
innodb_log_file_size=256M
innodb_log_buffer_size=8M
innodb_file_per_table=1
innodb_additional_mem_pool_size=20M
innodb_flush_log_at_trx_commit=0
innodb_flush_method=O_DIRECT
innodb_support_xa=0
innodb_thread_concurrency=0
innodb_buffer_pool_size=3000M
The sum() and GROUP BY from the first query could be taking some time but I don't think there is much you can do there.
In the second query your FROM (SELECT.... is probably hitting the system pretty hard, I would recommend turning
(SELECT balance, tmp1.balance_sum,
(balance / tmp1.balance_sum * interest_rate) as weighted_interest_rate
FROM loan_history,
(SELECT SUM(balance) balance_sum FROM loan_history
WHERE date = '2013-07-04'
AND bank_id = '1') as tmp1
WHERE date = '2013-07-04'
AND bank_id = '1')
into a view or figuring out how to do it with JOINs
Please tell us how much does exactly each one take. More than 1 minute each isn't that much indicative.
Any how, From MySQL manual
When tuning a MySQL server, the two most important variables to configure are key_buffer_size and table_cache. You should first feel confident that you have these set appropriately before trying to change any other variables.
Also, take a look here.
As for optimization, first try a composite index.

Categories