Partitioning of table in mysql for MYISAM eninge based on year - php

I am trying to partition the table based on the year range.
but i'm getting the following error "#1503 - A PRIMARY KEY must include all columns in the table's partitioning function".
Following is my query
CREATE TABLE IF NOT EXISTS `t2123` (
`j_id` int(11) NOT NULL AUTO_INCREMENT,
`u_id` int(11) NOT NULL,
`w_id` int(5) NOT NULL,
`p_id` int(5) NOT NULL,
`s_p_id` int(5) NOT NULL,
`p_t` tinyint(3) unsigned NOT NULL,
`a_n` int(5) NOT NULL,
`type` enum('GP','NGP','PGP','PAGP') NOT NULL,
`p_p` int(11) NOT NULL,
`s_p` int(11) NOT NULL,
`p_c` float NOT NULL,
`s_c` float NOT NULL,
`s_p_p_c` float NOT NULL DEFAULT '0',
`g_d` date NOT NULL,
`datetimes` TIMESTAMP NOT NULL DEFAULT '0000-00-00 00:00:00',
`c_c` double DEFAULT '0',
`c_m` double DEFAULT '0',
`c_y` double DEFAULT '0',
`c_k` double DEFAULT '0' ,
`c_total` double DEFAULT '0' ,
`p_a` float DEFAULT '0',
PRIMARY KEY (`j_id`),
UNIQUE KEY(j_id, g_d),
KEY `u_id` (`u_id`),
KEY `w_id` (`w_id`),
KEY `p_id` (`p_id`),
KEY `s_p_id` (`s_p_id`),
KEY `a_n` (`a_n`),
KEY `g_d` (`g_d`)
) engine=myisam PARTITION BY RANGE (j_id + year(g_d)) (
PARTITION t2_2012 VALUES LESS THAN (2012),
PARTITION t2_2013 VALUES LESS THAN (2013)
);
Please suggest me how can I create the partition?

As the error message advises:
A PRIMARY KEY must include all columns in the table's partitioning function
Your partitioning function refers to j_id and g_d while your primary key only covers j_id.
By the way, your UNIQUE constraint is useless (UNIQUE KEY(j_id, g_d)) because j_id is already unique by definition (it is primary key).
In fact you probably want your primary key on (j_id, g_d)

Related

how to create Duplicatable entry at foreign key column in mysql

I created a site that gives you info about universities, every university has registration and exams, etc
so in database I have two columns
THE FIRST TABLE IS THE UNIVERSITIES:
CREATE TABLE `universities` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`title` varchar(255) NOT NULL,
`slug` varchar(255) NOT NULL,
`views` int(11) NOT NULL DEFAULT 0,
`image` varchar(255) NOT NULL,
`banner` varchar(255) NOT NULL,
`aboutUni` text NOT NULL,
`aboutCity` text NOT NULL,
`localRank` int(5) DEFAULT NULL,
`globalRank` int(3) DEFAULT NULL,
`foundedY` int(4) NOT NULL,
`lang` varchar(10) DEFAULT NULL,
`published` tinyint(1) NOT NULL DEFAULT 1,
`created_at` timestamp NOT NULL DEFAULT current_timestamp() ON UPDATE current_timestamp(),
`user_id` int(11) DEFAULT NULL,
`video` varchar(255) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `slug` (`slug`),
UNIQUE KEY `title` (`title`),
UNIQUE KEY `slug_2` (`slug`),
KEY `universities_ibfk_1` (`user_id`),
FULLTEXT KEY `title_2` (`title`),
CONSTRAINT `universities_ibfk_1` FOREIGN KEY (`user_id`) REFERENCES `users` (`id`) ON DELETE NO ACTION ON UPDATE NO ACTION
) ENGINE=InnoDB AUTO_INCREMENT=40 DEFAULT CHARSET=utf8
THE SECOND TABLE IS THE EXAMS
CREATE TABLE `bsc` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`university_id` int(11) DEFAULT NULL,
`dates` text NOT NULL,
`date_end` date DEFAULT NULL,
`date` date DEFAULT NULL,
`papers` text NOT NULL,
`info` text NOT NULL,
`method` text NOT NULL,
`link` varchar(255) DEFAULT NULL,
`register_link` varchar(255) NOT NULL,
`links` text NOT NULL,
`sort` int(2) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `university_id` (`university_id`),
UNIQUE KEY `university_id_2` (`university_id`),
CONSTRAINT `bsc_ibfk_1` FOREIGN KEY (`university_id`) REFERENCES `universities` (`id`) ON DELETE CASCADE ON UPDATE NO ACTION
) ENGINE=InnoDB AUTO_INCREMENT=27 DEFAULT CHARSET=utf8mb4
NOTICE!: the tables have other columns but I think it's not important
university_id has a foreign key relation with universities.id
the problem is when I want to add tow exam to a university it appears me this error:
Duplicate entry '39' for key 'university_id'
The problem you faced Duplicate entry '39' for key 'university_id' results from trying to store in university_id a value that already existed somewhere, which is not allowed according to some constraints.
as shown in the DDL part
UNIQUE KEY `university_id` (`university_id`)
This constrain is the main cause for that error, so you need to remove that constrain by
ALTER TABLE `bsc` DROP INDEX `university_id`;
Thanks for #Tangentially, who was the first one who expected that error in his comment.
It seems that these keys are unnecesary:
UNIQUE KEY university_id (university_id),
UNIQUE KEY university_id_2 (university_id),
given that keys, you should only have distinct values in that fields.

Improving/ Optimizing Performance Query in PHP, MySQL,Zend

I have a invoices, invoices_items, order, order_items. Invoices and Orders tables contains around 1 Millions records. Invoices_items and Orders_items tables contains more than 2 Millions records. Items table contains 2 Hundred Thousands records. Now I want to generate a report based on my filter like customers, item categories and more....
Please refer queries.
Running on PHP 5.6. MySql 5.7 and Apache2.
SELECT
`si_items`.`item_id`
, SUM(qty) AS `qty`
, IFNULL(SUM(selling_price * (qty)), 0) AS `salestotal`
, GROUP_CONCAT(si.id) AS `siso_id`
, MAX(si.date_transaction) AS `date_transaction`
FROM
`invoice_items` AS `si_items`
LEFT JOIN `invoice` AS `si`
ON si.id = si_items.parent_id
LEFT JOIN `items`
ON si_items.item_id = items.id
WHERE (
DATE_FORMAT(si.date_transaction, '%Y-%m-%d') BETWEEN '2019-01-01'
AND '2019-02-15'
)
AND (si.approved = 1)
AND (si.deleted = 0)
AND (items.deleted = 0)
GROUP BY `item_id`
UNION
SELECT
`so_items`.`item_id`
, SUM(qty) AS `qty`
, IFNULL(SUM(selling_price * (qty)), 0) AS `salestotal`
, GROUP_CONCAT(so.id) AS `soso_id`
, MAX(so.date_transaction) AS `date_transaction`
FROM
`order_items` AS `so_items`
LEFT JOIN `order` AS `so`
ON so.id = so_items.parent_id
LEFT JOIN `items`
ON so_items.item_id = items.id
WHERE (
DATE_FORMAT(so.date_transaction, '%Y-%m-%d') BETWEEN '2019-01-01'
AND '2019-02-15'
)
AND (so.approved = 1)
AND (so.deleted = 0)
AND (items.deleted = 0)
GROUP BY `item_id`
When I executed this query for 50 days. It took 1 minute 20 seconds to execute this query.
INDEXES are added in tables
Invoice & Order Tables
PRIMARY KEY (`id`),
KEY `account_id` (`account_id`),
KEY `approved` (`approved`),
KEY `deleted` (`deleted`),
KEY `finalised` (`finalised`),
KEY `rp_status` (`rp_status`),
KEY `sales_types_id` (`sales_types_id`),
KEY `account_type_id` (`account_type_id`),
KEY `company_id` (`company_id`),
KEY `date_transaction` (`date_transaction`)
Invoices_items & Order_items
PRIMARY KEY (`id`),
KEY `deleted` (`deleted`),
KEY `item_id` (`item_id`),
KEY `parent_id` (`parent_id`),
KEY `vat_id` (`vat_id`),
KEY `qty` (`qty`),
Explain Query
Explain Query
I need to increase performance of this query. Could you please guide me how to proceed?
Show Create Tables
CREATE TABLE `invoice` (
`id` char(36) NOT NULL,
`reference` varchar(25) DEFAULT NULL,
`company_id` char(36) DEFAULT NULL,
`branch_id` char(36) DEFAULT NULL,
`account_id` char(36) DEFAULT NULL,
`contact_id` char(36) DEFAULT NULL,
`transaction_type` varchar(10) DEFAULT NULL,
`sales_types_id` int(11) DEFAULT '0',
`quote_validity` int(11) DEFAULT '0',
`delivery_method_id` int(11) DEFAULT '0',
`sales_representative_id` int(11) DEFAULT '0',
`account_type_id` char(36) DEFAULT NULL,
`vat_exempted` tinyint(1) DEFAULT '0',
`description` text,
`finalised` tinyint(1) DEFAULT '0' COMMENT 'Not Yet finalised - status=1; Need Approval - status = 2; Approved - status = 3',
`approved` tinyint(1) DEFAULT '0',
`approved_user_id` int(11) DEFAULT '0',
`default_sales_location_id` char(36) DEFAULT NULL COMMENT '0-Yes; 1-No',
`generate_do` tinyint(1) DEFAULT '1',
`generate_dn` tinyint(4) DEFAULT '1',
`do_status` tinyint(1) DEFAULT '0',
`cn_status` tinyint(1) DEFAULT '0',
`rp_status` tinyint(1) DEFAULT '0',
`dm_status` tinyint(1) DEFAULT '0',
`currency_id` char(36),
`exchange_rate_id` tinyint(1) DEFAULT '0',
`exchange_rate` double DEFAULT '1',
`date_transaction` datetime DEFAULT NULL,
`date_created` datetime DEFAULT NULL,
`date_modified` datetime DEFAULT NULL,
`created_user_id` int(11) DEFAULT '0',
`modified_user_id` int(11) DEFAULT '0',
`deleted` tinyint(1) DEFAULT '0',
PRIMARY KEY (`id`),
KEY `account_id` (`account_id`),
KEY `approved` (`approved`),
KEY `branch_id` (`branch_id`),
KEY `cn_status` (`cn_status`),
KEY `created_user_id` (`created_user_id`),
KEY `date_created` (`date_created`),
KEY `deleted` (`deleted`),
KEY `do_status` (`do_status`),
KEY `finalised` (`finalised`),
KEY `reference` (`reference`),
KEY `rp_status` (`rp_status`),
KEY `sales_types_id` (`sales_types_id`),
KEY `account_type_id` (`account_type_id`),
KEY `company_id` (`company_id`),
KEY `date_transaction` (`date_transaction`),
KEY `default_sales_location_id` (`default_sales_location_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
CREATE TABLE `invoice_items` (
`id` char(36) NOT NULL,
`parent_id` char(36) DEFAULT NULL,
`item_id` char(36) DEFAULT NULL,
`qty` double DEFAULT '0',
`cost_price` double DEFAULT '0',
`list_price` double DEFAULT '0',
`selling_price` double DEFAULT '0',
`unit_price` double DEFAULT '0',
`vat` double DEFAULT '0',
`amount` double DEFAULT '0',
`special_discount` double DEFAULT '0',
`price_change_status` tinyint(1) DEFAULT '0',
`remarks` text,
`vat_id` int(11) DEFAULT '1',
`stock_category_id` tinyint(2) DEFAULT '0' COMMENT '1: Stockable 2: Service',
`is_giftitem` tinyint(1) DEFAULT '0' COMMENT '1: Gift Item 0: NO Gift',
`item_type_status` tinyint(1) DEFAULT '0',
`date_created` datetime DEFAULT NULL,
`date_modified` datetime DEFAULT NULL,
`created_user_id` int(11) DEFAULT '0',
`modified_user_id` int(11) DEFAULT '0',
`deleted` tinyint(1) DEFAULT '0',
PRIMARY KEY (`id`),
KEY `deleted` (`deleted`),
KEY `item_id` (`item_id`),
KEY `parent_id` (`parent_id`),
KEY `stock_category_id` (`stock_category_id`),
KEY `item_type_status` (`item_type_status`),
KEY `vat_id` (`vat_id`),
KEY `amount` (`amount`),
KEY `qty` (`qty`),
KEY `unit_price` (`unit_price`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
Don't use LEFT JOIN when you mean JOIN. In particular, for joining to si.
WHERE (
DATE_FORMAT(si.date_transaction, '%Y-%m-%d') BETWEEN '2019-01-01'
AND '2019-02-15'
)
-->
WHERE si.date_transaction >= '2019-01-01'
AND si.date_transaction < '2019-01-15'
so that an index (see below) can use that column
WHERE si.date_transaction ...
AND (si.approved = 1)
AND (si.deleted = 0)
Add a composite index:
INDEX(deleted, approved, -- in either order
date_transaction) -- last
Make similar changes to so. Then let's hear how the performance is and see what the EXPLAIN has changed to.
UUIDs
Beware of UUIDs, they are bulky and slow. They are especially slow if the entire table cannot be cached.
I suspect you have uuids because I see CHAR(36).
By having CHARACTER SET utf8, that means 108 bytes is being used!. A UUID can be packed into a 16-byte BINARY(16). This would help with space (and hence speed).
But the real problem with UUIDs is with the randomness. Once the table becomes huge, the system becomes I/O-bound since the 'next' UUID is unlikely to be cached.
Consider switching to AUTO_INCREMENT ids. This is much preferred for single-server systems. If you need to generate ids from multiple locations, you may still need UUIDs.
More on UUIDs.

mysql unique for multiple columns

When payment happen, sometimes its captured double entry in table.
I want to ignore double entry capture so i want to insert records when these created, user_id, amount fields should be unique.
How do i make it ? Below is my table.
CREATE TABLE `transactions` (
`id` int(20) NOT NULL AUTO_INCREMENT,
`created` datetime NOT NULL,
`modified` datetime NOT NULL,
`user_id` int(20) NOT NULL,
`project_id` int(20) DEFAULT NULL,
`foreign_id` int(20) NOT NULL,
`class` varchar(25) COLLATE utf8_unicode_ci NOT NULL,
`transaction_type_id` int(20) DEFAULT NULL,
`amount` float(10,2) NOT NULL,
`description` text COLLATE utf8_unicode_ci,
`payment_gateway_id` int(20) DEFAULT NULL,
`gateway_fees` float(10,2) NOT NULL,
`is_old` tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=266 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci
To strictly answer your question, you create a unique composite key on the combination of those 3 columns. That way no two rows can exist with a combination of the 3 of them in the composite index.
CREATE TABLE `transactions2` (
`id` int(20) NOT NULL AUTO_INCREMENT,
`created` datetime NOT NULL,
`modified` datetime NOT NULL,
`user_id` int(20) NOT NULL,
`project_id` int(20) DEFAULT NULL,
`foreign_id` int(20) NOT NULL,
`class` varchar(25) COLLATE utf8_unicode_ci NOT NULL,
`transaction_type_id` int(20) DEFAULT NULL,
`amount` float(10,2) NOT NULL,
`description` text COLLATE utf8_unicode_ci,
`payment_gateway_id` int(20) DEFAULT NULL,
`gateway_fees` float(10,2) NOT NULL,
`is_old` tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`),
unique key(created,user_id,amount) -- <------------------- right here
);
insert some data to test it:
insert transactions2 (created,modified,user_id,project_id,foreign_id,class,
transaction_type_id,amount,description,payment_gateway_id,gateway_fees,is_old) values
('2009-01-01 12:00:00','2009-01-01 12:00:00',666,1,1,'a',1,100,'desc',1,12,1);
-- inserts fine (above)
Try it again with exactly the same data:
insert transactions2 (created,modified,user_id,project_id,foreign_id,class,
transaction_type_id,amount,description,payment_gateway_id,gateway_fees,is_old) values
('2009-01-01 12:00:00','2009-01-01 12:00:00',666,1,1,'a',1,100,'desc',1,12,1);
-- error 1062: Duplicate entry
-- change it barely:
insert transactions2 (created,modified,user_id,project_id,foreign_id,class,
transaction_type_id,amount,description,payment_gateway_id,gateway_fees,is_old) values
('2009-01-01 13:00:00','2009-01-01 12:00:00',666,1,1,'a',1,100,'desc',1,12,1);
-- inserts fine
Also, use ENGINE=INNODB. Read about that Here.
Please read Mysql multi column indexes a.k.a. composite indexes.
Lastly, the concept of what you are talking about is not far off from Insert on Duplicate Key Update. Just throwing that reference out there for you.
You can achieve the same using ,handling at the time of inserting,
You can try WHERE NOT EXISTS with INSERT.
Something like this.(You need to specify column name who has NOT NULL constraint,i missed all those columns)
INSERT INTO table_name(`created`,`user_id`,`amount`) VALUES =
'$created','$user_id','$amount'
WHERE NOT EXISTS
(SELECT *
FROM table_name
WHERE created ='$created' AND user_id='$user_id' AND amount='$amount')
Hope this helps.

How to speed up slow MySQL UPDATE queries with InnoDB tables

I have a very simple MySQL update query on my InnoDB table.
UPDATE `players_teams` SET t_last_active=NOW() WHERE t_player_id=11225 AND t_team_id=6912 AND t_season_id=2002 LIMIT 1
My table is structured as so:
CREATE TABLE `players_teams` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`t_player_id` int(11) DEFAULT NULL,
`t_team_id` int(11) DEFAULT NULL,
`t_league_id` int(11) DEFAULT NULL,
`t_season_id` int(11) DEFAULT NULL,
`t_div` varchar(64) DEFAULT NULL,
`t_player_number` varchar(3) DEFAULT NULL,
`t_player_jersey_size` enum('UNKNOWN','XS','S','M','L','XL','XXL','XXXL') DEFAULT 'UNKNOWN',
`t_player_registration_number` varchar(64) DEFAULT NULL,
`t_player_class` enum('ROSTER','SPARE','COACH','INJURED','HOLIDAY','SUSPENDED','SCOREKEEPER') DEFAULT 'ROSTER',
`t_access_level` enum('PLAYER','MANAGER','ASSISTANT') DEFAULT 'PLAYER',
`t_player_position` enum('ANY','FORWARD','DEFENCE','GOALIE','PITCHER','CATCHER','FIRST BASE','SECOND BASE','THIRD BASE','SHORTSTOP','LEFT FIELD','CENTER FIELD','RIGHT FIELD') DEFAULT 'ANY',
`t_spare_status` enum('INVITED','IN','OUT') DEFAULT NULL,
`t_drink_next` int(1) DEFAULT '0',
`t_no_fees` tinyint(1) DEFAULT '0',
`t_no_drink` tinyint(1) DEFAULT '0',
`t_auto_check_in` tinyint(1) DEFAULT '0',
`t_print_reminder` tinyint(1) DEFAULT '0',
`t_notes` text,
`t_last_chatter_id` int(11) DEFAULT NULL,
`t_last_history_id` int(11) DEFAULT NULL,
`t_last_active` timestamp NULL DEFAULT NULL,
`t_status` enum('ACTIVE','INACTIVE','ARCHIVED') DEFAULT 'ACTIVE',
`t_added` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
KEY `t_player_id` (`t_player_id`),
KEY `t_team_id` (`t_team_id`),
KEY `t_season_id` (`t_season_id`),
KEY `t_player_id_2` (`t_player_id`,`t_team_id`,`t_season_id`),
KEY `Team/Player ID` (`t_team_id`,`t_player_id`),
KEY `UpdatePlayersDiv` (`t_team_id`,`t_season_id`)
) ENGINE=InnoDB AUTO_INCREMENT=23454 DEFAULT CHARSET=latin1;
This simple update query takes on average, 3.5 seconds? I'm running this on a MediaTemple MySQL GRID container.
Here is the result of an EXPLAIN when switching the UPDATE to a SELECT.
Can someone provide some insight on how I'm not doing this correctly?
[EDIT: Added list of Indexes]
So is this a big mess of indexes? Do I have a bunch of redundant indexes in here?
Cheers,
Jon
It's using the wrong key instead of the index on the 3 columns. You can hint at indexes with USE KEY ... syntax.
https://dev.mysql.com/doc/refman/5.1/en/index-hints.html
You may also want to try reordering your key on the 3 columns. Generally, you want the most restricting column to be first in the index, then the next most restricting and so on.

MySQL Stored Procedure [Copy table 1 -> table 2]

My knowledge of MySQL/ SQL apart from the simple CRUD operations is basic.
If I had to use a stored procedure to move certain attributes (not in a specific order) to another table how could it be done?
These are the following tables. I want to move from the 1st to the 2nd table.
As you can see the datatype sizes are different for certain columns.
CREATE TABLE IF NOT EXISTS `source_cdr` (
`callstart` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
`src` varchar(80) NOT NULL DEFAULT '',
`dst` varchar(80) NOT NULL DEFAULT '',
`accountcode` varchar(50) NOT NULL,
`uniqueid` varchar(100) NOT NULL,
`ID` int(11) NOT NULL AUTO_INCREMENT,
`callanswer` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
`callend` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
`disposition` varchar(50) NOT NULL,
`cdr_id` int(11) unsigned NOT NULL DEFAULT '0',
`pin_code` varchar(4) NOT NULL,
`provider` int(11) NOT NULL,
PRIMARY KEY (`ID`),
KEY `calldate_idx` (`callstart`) USING BTREE,
KEY `idx_acc_code_calldate` (`accountcode`,`callstart`) USING BTREE,
KEY `uniqueid` (`uniqueid`),
KEY `cdr_id` (`cdr_id`),
KEY `idx_uniqueid_cdr_id` (`uniqueid`,`cdr_id`)
) ENGINE=MyISAM;
--
CREATE TABLE IF NOT EXISTS `destination_cdr` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`calldate` datetime NOT NULL,
`source` varchar(80) NOT NULL,
`destination` varchar(80) NOT NULL,
`account_code` varchar(30) DEFAULT NULL,
`pincode` varchar(45) NOT NULL,
`duration_call` bigint(20) NOT NULL DEFAULT '0',
`duration_talk` bigint(20) NOT NULL,
`disposition` varchar(255) NOT NULL,
`clid` varchar(80) DEFAULT NULL,
`cdr_id` bigint(20) DEFAULT NULL,
`vxcdr_id` bigint(20) DEFAULT NULL,
`provider` int(11) NOT NULL DEFAULT '0'
PRIMARY KEY (`id`)
) ENGINE=MyISAM;
EDIT 1
An example of a row
('2012-03-18 20:54:49', '5796', '0761100866', '103f0124ad510516f33cab132c0a695b', 'call-F1884808-6753-2F10-181C-3A#10.217.164.33', 308006367, '2012-03-18 20:55:05', '2012-03-18 20:55:51', '200 OK', 2, '', 0),
Thanks
You can use MySQL: INSERT ... SELECT Syntax to copy data from one table to the other.
Decide common fields in both and copy the same.
Example:
INSERT INTO TABLE2( COL1, COLx, ... ) SELECT colM, colY FROM TABLE1;
If the column sizes mismatch, data truncation takes place, and you can't overcome that but redefine the destination table columns.

Categories