MySQL Insert query becomes slow on progress - php

Question

Im using Yii createCommand QueryBuilder and constructing below insert query, below code works like charm (inserts 6000 records in 3 secs) but after sometime its speed becomes very slow.
My code:
$taskmodel = TaskModel::model()->findAll($cri); // filtering using some criteria
$now = new DateTime();
foreach( $taskmodel as $tsk ) {
$recall_date = $tsk['recall_date'] == "" ? "null" : '"'.$tsk['recall_date'].'"';
$recall_agent = $tsk['recall_agent_id'] == "" ? "null" : $tsk['recall_agent_id'];
$next_date = $tsk['next_action_date'] == "" ? "null" : '"'.$tsk['next_action_date'].'"';
$pc_color = $tsk['postcode_color'] == "" ? "null" : '"'.$tsk['postcode_color'].'"';
$cpc_id = $tsk['crm_campaign_post_code_id'] == "" ? "null" : $tsk['crm_campaign_post_code_id'];
$priority = $tsk['priority'] == "" ? 10 : $tsk['priority'];
$field1 = "null" ;
$field2 = "null"
$field3 = "null"
$field4 = "null" ;
$contact_timezone = $tsk['contact_timezone'] == "" ? "''" : '"'.$tsk['contact_timezone'].'"';
$sql[] = '('.$tsk['crm_task_id'].', '.$tsk['crm_campaign_id'].', "'.$tsk['crm_contact_id'].'", "'.$now->format('Y-m-d H:i:s').'", "'.$now->format('Y-m-d H:i:s').'
", '.$tsk['crm_filter_id'].', '.$tsk['current_status'].', '.$priority.', '.$recall_date.', '.$recall_agent.', '.$next_date.
', '.$pc_color.', '.$cpc_id.', '.$sort.', '.$field1.', '.$field2.', '.$field3.', '.$field4.', '.$contact_timezone.', '.$tsk['is_active'].')';
}
if(sizeof($sql) > 0){
$ins ='INSERT INTO crm_pending_task (crm_task_id, crm_campaign_id,crm_contact_id,created,updated,crm_filter_id,
current_status,priority,recall_date,recall_agent_id,next_action_date,postcode_color,crm_campaign_post_code_id,sort,
field1,field2,field3,field4,contact_timezone,is_active) VALUES '.implode(',', $sql);
Yii::app()->db->createCommand($ins)->execute();
}
i found something interesting, after inserting 50000 records it becomes slow !!! why is it so ???
How to improve insert query speed till it finishes all insertion ?

Answer 1

I think is because your request is taking too much memory and you are swapping.
To free some memory you shouldt try to enable Gc and launch it in your foreach (maybe not at each iteration)
gc_enable();
gc_collect_cycles();

Answer 2

Perhaps trying inserting smaller chunks into the database.
foreach( $taskmodel as $tsk ) {
$recall_date = $tsk['recall_date'] == "" ? "null" : '"'.$tsk['recall_date'].'"';
$recall_agent = $tsk['recall_agent_id'] == "" ? "null" : $tsk['recall_agent_id'];
$next_date = $tsk['next_action_date'] == "" ? "null" : '"'.$tsk['next_action_date'].'"';
$pc_color = $tsk['postcode_color'] == "" ? "null" : '"'.$tsk['postcode_color'].'"';
$cpc_id = $tsk['crm_campaign_post_code_id'] == "" ? "null" : $tsk['crm_campaign_post_code_id'];
$priority = $tsk['priority'] == "" ? 10 : $tsk['priority'];
$field1 = "null" ;
$field2 = "null"
$field3 = "null"
$field4 = "null" ;
$contact_timezone = $tsk['contact_timezone'] == "" ? "''" : '"'.$tsk['contact_timezone'].'"';
$sql[] = '('.$tsk['crm_task_id'].', '.$tsk['crm_campaign_id'].', "'.$tsk['crm_contact_id'].'", "'.$now->format('Y-m-d H:i:s').'", "'.$now->format('Y-m-d H:i:s').'
", '.$tsk['crm_filter_id'].', '.$tsk['current_status'].', '.$priority.', '.$recall_date.', '.$recall_agent.', '.$next_date.
', '.$pc_color.', '.$cpc_id.', '.$sort.', '.$field1.', '.$field2.', '.$field3.', '.$field4.', '.$contact_timezone.', '.$tsk['is_active'].')';
}
$recordCount = sizeof($sql);
$sliceSize = 250;
for($x = 0; $x < $recordCount; $x += $sliceSize) {
$ins = sprintf('
INSERT INTO
crm_pending_task
(
crm_task_id, crm_campaign_id, crm_contact_id, created,updated,
crm_filter_id, current_status, priority, recall_date, recall_agent_id,
next_action_date, postcode_color, crm_campaign_post_code_id, sort,
field1, field2, field3, field4, contact_timezone, is_active
)
VALUES
%s; ', implode(',', array_slice($sql, $x, $sliceSize));
Yii::app()->db->createCommand($ins)->execute();
}
}