I got some issues with my current code. Example, on a PHP page, there's a table that displays all tools that borrowed the users. In that table, each data rows contain a checkbox. Users can select any tools that they want to return first by tick the checkbox and press "return" button.
At Server-side, after clicking the "return" button, it will go to the page named return_selected.php. At this page, it will update Table A and Table B. This one is successful.
Now, I want to update Table C if all there's a condition, for example, ALL tools returned. This one I still do but failed. Below is the code
return_selected.php
<?php
include("../../../../config/configPDO.php");
include("../../../../config/check.php");
$tools_id = $_POST['tools_id'];
$borrow_id = $_POST['borrow_id'];
$checkbox=explode( ',', $_POST['ids'][0] );
for($i=0;$i < count($checkbox); $i++){
$tools_id=$checkbox[$i];
$sql="UPDATE ets_tools SET borrow_id = NULL WHERE tools_id=:tools_id";
$query=$conn->prepare($sql);
$query->execute(array(':tools_id' => $tools_id));
$sql2="UPDATE ets_tools_borrow SET time_to = GETDATE() WHERE tools_id=:tools_id";
$query3=$conn->prepare($sql2);
$query3->execute(array(':tools_id' => $tools_id));
// want to update table if all tools returned.
$query2 = "
SELECT
ets_tools.tools_id, ets_tools.tools_name, ets_tools.borrow_id,
ets_borrow.time_from, ets_borrow.time_to, ets_borrow.status_id
FROM ets_tools
INNER JOIN ets_borrow ON ets_tools.borrow_id = ets_borrow.borrow_id
WHERE ets_tools.borrow_id IS NOT NULL AND ets_borrow.borrow_id = :borrow_id
";
$sql2=$conn->prepare($query2);
$sql2->execute(array(':borrow_id' => $borrow_id));
if($sql2->rowCount() > 0)
{
header("Location: return.php");
}else{
$sql3="UPDATE ets_borrow SET time_to = GETDATE(), status_id = 2 WHERE borrow_id=:borrow_id";
$query3=$conn->prepare($sql3);
$query3->execute(array(':borrow_id' => $borrow_id));
header("Location: return.php");
}
}
?>
Can anyone know how to solve this? Thank you.
You need to consider the following:
As is mentioned in the documentation, PDOStatement::rowCount() returns the number of rows affected by the last DELETE, INSERT, or UPDATE statement executed by the corresponding PDOStatement object and for most databases the number of rows returned from a SELECT statement is not returned correctly. So, you may use a SELECT COUNT(*) ... approach to get this count.
For your specific case you may try to use an UPDATE ... WHERE NOT EXISTS ... statement to update the rows in the ets_borrow table and skip this SELECT ... FROM ... INNER JOIN ... statement.
I'm not sure if I understand corectly the use of header("Location: return.php"); in a for loop, probably you need to redirect after this loop.
Example, based on your code:
<?php
include("../../../../config/configPDO.php");
include("../../../../config/check.php");
$tools_id = $_POST['tools_id'];
$borrow_id = $_POST['borrow_id'];
$checkbox=explode( ',', $_POST['ids'][0] );
for($i=0; $i < count($checkbox); $i++) {
$tools_id=$checkbox[$i];
$sql = "UPDATE ets_tools SET borrow_id = NULL WHERE tools_id = :tools_id";
$query = $conn->prepare($sql);
$query->execute(array(':tools_id' => $tools_id));
$sql2 = "UPDATE ets_tools_borrow SET time_to = GETDATE() WHERE tools_id = :tools_id";
$query2 = $conn->prepare($sql2);
$query2->execute(array(':tools_id' => $tools_id));
$sql3 = "
UPDATE ets_borrow
SET time_to = GETDATE(), status_id = 2
WHERE
(borrow_id = :borrow_id1) AND
NOT EXISTS (
SELECT 1
FROM ets_tools
INNER JOIN ets_borrow ON ets_tools.borrow_id = ets_borrow.borrow_id
WHERE ets_tools.borrow_id IS NOT NULL AND ets_borrow.borrow_id = :borrow_id2
)
";
$borrow_id1 = $borrow_id;
$borrow_id2 = $borrow_id;
$query3 = $conn->prepare($sql3);
$query3->execute(array(':borrow_id1' => $borrow_id1, ':borrow_id2' => $borrow_id2));
}
header("Location: return.php");
?>
I have a php API where i am checking different conditions and based on that, i am fetching the count of the row and updating the count in the same table.
Below are the Conditions i am writing the query: (Note: 1 user can have multiple campaigns)
1) For a given user(Uid), for a particular campaign whose type = Impression, i am getting the Impression count and updating the same in table
2) For a given user(Uid), for a particular campaign whose type = Action i am getting the Action count and updating the same in table
//To get the Impression/Action/Lead count
$Impressionarr = [];
$Actionarr = [];
//IMPRESSION COUNT
$imp_qry = "select count(*) as ImpressionCount from ClicksAndImpressions where Uid = 101642 and CampaignID =100 and Type='Impression' ;";
$impData = $this->getClicksAndImpressionsTable()->CustomQuery($imp_qry);
if($impData[0]['ImpressionCount'] != '' || $impData[0]['ImpressionCount'] !=NULL ){
$impr_update = "UPDATE ClicksAndImpressions SET ImpressionCount = ". $impData[0]['ImpressionCount'] ." where Uid = 101642 and CampaignID =100 ;";
}else{
echo '----not Present';
}
//Impression count
foreach($impData as $key=>$data)
{
$data['ImpressionCount'];
$Impressionarr[] = $data;
}
//ACTION COUNT
$action_qry = "select count(*) as ActionCount from ClicksAndImpressions where Uid = 101617 and CampaignID =81 and Type = 'Action';";
$actionData = $this->getClicksAndImpressionsTable()->CustomQuery($action_qry);
if($actionData[0]['ActionCount'] != '' || $actionData[0]['ActionCount'] !=NULL ){
$action_update = "UPDATE ClicksAndImpressions SET ActionCount = ". $actionData[0]['ActionCount'] ." where Uid = 101617 and CampaignID =81 ;";
$actionData = $this->getClicksAndImpressionsTable()->CustomUpdate($action_update); //print_r($actionData);exit;
}else{
echo '----not Present';
}
//Action count
foreach($actionData as $key=>$data)
{
$data['ActionCount'];
$Actionarr[] = $data;
}
//Trying to combine the 3 arrays- but 2nd and 3rd not merging into 1st
$combine = array_merge($CampaignDetailsarr,$Impressionarr,$Actionarr);
1) Can the above duplicates - either in Update query or multiple for loops be avoided. if so how to optimize the above as per my conditions
ie) For
if( Uid = 123 for some campaignId = 11 and if type is 'Impression')
-> Update query for impression
else if(Uid = 123 for campaignId= 22 and if type = 'something else')
-> Update query for something else
2) I already have one array 1. I need to merge my 2 new arrays (Impressionarr[], Actionarr[]) into the 1st one, but below is how im getting.
Expected:
Coming as:
You could probably speed it up by creating a stored procedure in your database and calling that with the required data. From what I can see, most of the logic you have to do can be done in the stored procedure.
I have a method, where about 20 rows must changed. Im passing per ajax a string like "14-33-61-10 ..." for saving the order of some rows.
Here is my actual (working) code:
$explode = explode('--',$_POST['data']); // -> array('0'=>'8', '1'=>'4', '2'=> ... )
$i = 1;
foreach( $explode as $task ) {
if( !is_int($task) ) continue;
$exec = $project->exec("UPDATE tasks SET pos=$i WHERE rowid=". $task );
$i++;
}
My problem is, that it takes about 1 second.
My question: Is there a way to edit multiple rows with just one query?
Sure, send a SQL string like:
update tasks
set pos =
case rowid
when 13 then 1
when 33 then 2
when 61 then 3
when 10 then 4
end
where rowid in (13,33,61,10)
The translation from 13-33-61-10 to (13,33,61,10) is best done in PHP.
If the range is short then we can use the ID IN (ids) argument.
UPDATE `profile` SET `app_status`=3 WHERE `id` IN (3,37,95,136,1087,1795,1817)
If it's long and we know that range, let's use the BETWEEN value AND value argument.
UPDATE `profile` SET `app_status`=3 WHERE `id` BETWEEN 3904 AND 3918
$explode = explode('--',$_POST['data']); // -> array('0'=>'8', '1'=>'4', ... )
$i = 1;
$sql = "";
foreach($explode as $task)
{
if(!is_int($task)) continue;
$sql .= "UPDATE tasks SET pos=$i WHERE rowid=$task;";
$i++;
}
$project->exec($sql);
Right, this code goes through a rather large multidimensional array (has about 28,000 rows and 16 parts).
Order of events:
Check if the data exists in the database
if it exists - Update it with the new data
if it doesn't exist - Insert it
Simple.
But right now to go through this it has taken over 30min i think and Still going.
$counter = 0;
$events = sizeof($feed_array['1'])-1;
while($counter <= $events ) {
$num_rows = mysql_num_rows(mysql_query("SELECT * FROM it_raw WHERE perfID = '".addslashes($feed_array['16'][$counter])."'"));
if($num_rows) {
$eventDate=explode("/", $feed_array['1'][$counter]); //print_r($eventDate);
$eventTime=explode(":", $feed_array['2'][$counter]); //print_r($eventTime);
$eventUnixTime=mktime($eventTime[0], $eventTime[1], "00", $eventDate[1], $eventDate[0], $eventDate[2]);
mysql_query("UPDATE `it_raw` SET
`eventtime` = '".$eventUnixTime."',
`eventname` = '".addslashes($feed_array['3'][$counter])."',
`venuename` = '".addslashes($feed_array['4'][$counter])."',
`venueregion` = '".addslashes($feed_array['5'][$counter])."',
`venuepostcode` = '".addslashes($feed_array['6'][$counter])."',
`country` = '".addslashes($feed_array['7'][$counter])."',
`minprice` = '".addslashes($feed_array['8'][$counter])."',
`available` = '".addslashes($feed_array['9'][$counter])."',
`link` = '".addslashes($feed_array['10'][$counter])."',
`eventtype` = '".addslashes($feed_array['11'][$counter])."',
`seaOnSaleDate` = '".addslashes($feed_array['12'][$counter])."',
`perOnSaleDate` = '".addslashes($feed_array['13'][$counter])."',
`soldOut` = '".addslashes($feed_array['14'][$counter])."',
`eventImageURL` = '".addslashes($feed_array['15'][$counter])."',
`perfID`= '".addslashes($feed_array['16'][$counter])."'
WHERE `perfID` = ".$feed_array['16'][$counter]." LIMIT 1 ;");
echo "UPDATE ".$feed_array['16'][$counter].": ".addslashes($feed_array['3'][$counter])."\n";
} else {
$eventDate=explode("/", $feed_array['1'][$counter]); //print_r($eventDate);
$eventTime=explode(":", $feed_array['2'][$counter]); //print_r($eventTime);
$eventUnixTime=mktime($eventTime[0], $eventTime[1], "00", $eventDate[1], $eventDate[0], $eventDate[2]);
$sql = "INSERT INTO `dante_tickets`.`it_raw` (
`id` ,
`eventtime` ,
`eventname` ,
`venuename` ,
`venueregion` ,
`venuepostcode` ,
`country` ,
`minprice` ,
`available` ,
`link` ,
`eventtype` ,
`seaOnSaleDate` ,
`perOnSaleDate` ,
`soldOut` ,
`eventImageURL` ,
`perfID`
)
VALUES (
NULL ,
'".$eventUnixTime."',
'".addslashes($feed_array['3'][$counter])."',
'".addslashes($feed_array['4'][$counter])."',
'".addslashes($feed_array['5'][$counter])."',
'".addslashes($feed_array['6'][$counter])."',
'".addslashes($feed_array['7'][$counter])."',
'".addslashes($feed_array['8'][$counter])."',
'".addslashes($feed_array['9'][$counter])."',
'".addslashes($feed_array['10'][$counter])."',
'".addslashes($feed_array['11'][$counter])."',
'".addslashes($feed_array['12'][$counter])."',
'".addslashes($feed_array['13'][$counter])."',
'".addslashes($feed_array['14'][$counter])."',
'".addslashes($feed_array['15'][$counter])."',
'".addslashes($feed_array['16'][$counter])."'
);";
mysql_query($sql) or die(mysql_error().":".$sql);
echo "Inserted ".$feed_array['16'][$counter].": ".addslashes($feed_array['3'][$counter])."\n";
}
unset($sql);
$counter++;
}
UPDATE
I just carried out profiling one one of the rows:
mysql> EXPLAIN SELECT * FROM it_raw WHERE perfID = 210968;
+----+-------------+--------+------+---------------+--------+---------+-------+------+-------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+--------+------+---------------+--------+---------+-------+------+-------+
| 1 | SIMPLE | it_raw | ref | perfID | perfID | 4 | const | 1 | |
+----+-------------+--------+------+---------------+--------+---------+-------+------+-------+
1 row in set (0.07 sec)
UPDATE 2
To try and "speed" things up, instead of carrying out the UPDATE and INSERT statements straight away, i've now placed them in a variable (so only the initial select runs - to check a duplicate - then stores the action [insert or update]). At the end of the loop it executes all the statments.
Except now, it's coming up with MySQL error that the syntax is incorrect. (when initially there was nothing wrong).
I've simply replaced the mysql_query with:
$sql_exec .= "SELECT.... ;";
is there something i'm missing here for the formatting?
UPDATE 3
OK finally fixed it
Lessons Learned:
Do logic search first on database
Carry out insert/updates in bulk.
Here is the final code which now takes about 60 seconds to run (from over 30min+)
while($counter <= $events ) {
$num_rows = mysql_num_rows(mysql_query("SELECT * FROM it_raw WHERE perfID = '".addslashes($feed_array['16'][$counter])."'"));
if($num_rows) {
$eventDate=explode("/", $feed_array['1'][$counter]); //print_r($eventDate);
$eventTime=explode(":", $feed_array['2'][$counter]); //print_r($eventTime);
$eventUnixTime=mktime($eventTime[0], $eventTime[1], "00", $eventDate[1], $eventDate[0], $eventDate[2]);
$sql_exec[] = "UPDATE `it_raw` SET `eventtime` = '".$eventUnixTime."',`eventname` = '".addslashes($feed_array['3'][$counter])."',`venuename` = '".addslashes($feed_array['4'][$counter])."',`venueregion` = '".addslashes($feed_array['5'][$counter])."',`venuepostcode` = '".addslashes($feed_array['6'][$counter])."',`country` = '".addslashes($feed_array['7'][$counter])."',`minprice` = '".addslashes($feed_array['8'][$counter])."',`available` = '".addslashes($feed_array['9'][$counter])."',`link` = '".addslashes($feed_array['10'][$counter])."',`eventtype` = '".addslashes($feed_array['11'][$counter])."',`seaOnSaleDate` = '".addslashes($feed_array['12'][$counter])."',`perOnSaleDate` = '".addslashes($feed_array['13'][$counter])."',`soldOut` = '".addslashes($feed_array['14'][$counter])."',`eventImageURL` = '".addslashes($feed_array['15'][$counter])."',`perfID`='".addslashes($feed_array['16'][$counter])."' WHERE `perfID` = ".$feed_array['16'][$counter]." LIMIT 1;";
echo "UPDATE ".$feed_array['16'][$counter].": ".addslashes($feed_array['3'][$counter])."\n";
} else {
$eventDate=explode("/", $feed_array['1'][$counter]); //print_r($eventDate);
$eventTime=explode(":", $feed_array['2'][$counter]); //print_r($eventTime);
$eventUnixTime=mktime($eventTime[0], $eventTime[1], "00", $eventDate[1], $eventDate[0], $eventDate[2]);
$sql_exec[] = "INSERT INTO `it_raw` (`id` ,`eventtime` ,`eventname` ,`venuename` ,`venueregion` ,`venuepostcode` ,`country` ,`minprice` ,`available` ,`link` ,`eventtype` ,`seaOnSaleDate` ,
`perOnSaleDate` ,`soldOut` ,`eventImageURL` ,`perfID`) VALUES ( NULL ,'".$eventUnixTime."','".addslashes($feed_array['3'][$counter])."','".addslashes($feed_array['4'][$counter])."','".addslashes($feed_array['5'][$counter])."','".addslashes($feed_array['6'][$counter])."','".addslashes($feed_array['7'][$counter])."','".addslashes($feed_array['8'][$counter])."','".addslashes($feed_array['9'][$counter])."','".addslashes($feed_array['10'][$counter])."','".addslashes($feed_array['11'][$counter])."','".addslashes($feed_array['12'][$counter])."','".addslashes($feed_array['13'][$counter])."','".addslashes($feed_array['14'][$counter])."','".addslashes($feed_array['15'][$counter])."','".addslashes($feed_array['16'][$counter])."');";
//mysql_query($sql) or die(mysql_error().":".$sql);
echo "Inserted ".$feed_array['16'][$counter].": ".addslashes($feed_array['3'][$counter])."\n";
}
unset($sql);
$counter++;
}
foreach($sql_exec as $value) {
mysql_query($value) or die (mysql_error().": ".$value);
}
You could try grouping inserts and updates into groups so the code runs less queries.
For example, you could group all of the inserts into one very large insert, or maybe group every 100 inserts.
Also using prepared statements as gradbot suggested may help.
Other than that, it's not very easy to say which part of it is the major contributor to slowness. You should use a profiler to determine that, for example by using a smaller dataset so the profiled script runs in a reasonable time.
You can do a number of things.
on duplicate key instead of select
prepared statement if you have access to PDO MySQL.
stored procedure in MySQL
Try This. I'm unable to test it but the syntax should be right.
$counter = 0;
$events = sizeof($feed_array['1']) - 1;
while($counter <= $events )
{
$eventDate = explode("/", $feed_array['1'][$counter]); //print_r($eventDate);
$eventTime = explode(":", $feed_array['2'][$counter]); //print_r($eventTime);
$eventUnixTime = mktime($eventTime[0], $eventTime[1], "00", $eventDate[1], $eventDate[0], $eventDate[2]);
$data = array(
'eventtime' => $eventUnixTime,
'eventname' => addslashes($feed_array['3'][$counter]),
'venuename' => addslashes($feed_array['4'][$counter]),
'venueregion' => addslashes($feed_array['5'][$counter]),
'venuepostcode' => addslashes($feed_array['6'][$counter]),
'country' => addslashes($feed_array['7'][$counter]),
'minprice' => addslashes($feed_array['8'][$counter]),
'available' => addslashes($feed_array['9'][$counter]),
'link' => addslashes($feed_array['10'][$counter]),
'eventtype' => addslashes($feed_array['11'][$counter]),
'seaOnSaleDate' => addslashes($feed_array['12'][$counter]),
'perOnSaleDate' => addslashes($feed_array['13'][$counter]),
'soldOut' => addslashes($feed_array['14'][$counter]),
'eventImageURL' => addslashes($feed_array['15'][$counter]),
'perfID' => addslashes($feed_array['16'][$counter]),
);
$update = array();
foreach ($data as $key => $value)
$update[] = "`$key` = '$value'";
$sql = "INSERT INTO `dante_tickets`.`it_raw`" .
'(`id`, `'. implode ('`,`', array_keys($data)) . '`) VALUES ' .
'(NULL, ' . implode (',', $data) . ') ON DUPLICATE KEY UPDATE ' .
implode (',', $update);
mysql_query($sql) or die(mysql_error().":".$sql);
echo "Inserted or Updated".$feed_array['16'][$counter].": ".addslashes($feed_array['3'][$counter])."\n";
unset($sql);
$counter++;
}
I forgot to mention this requires that perfID is a unique key.
Have you profiled this query?
"SELECT * FROM it_raw WHERE perfID = '".addslashes($feed_array['16'][$counter])."'"
Because you run it 28000 times.. so unless it is REALLY quick, it will cause you a head ache. Use the EXPLAIN syntax and add an appropriate index if needed.
EDIT: With profile, I mean that you should try to use EXPLAIN on the mysql-prompt to see what execution plan the MySQL Query Optimizer suggests for this query. I.e, on the prompt, run:
EXPLAIN SELECT * FROM it_raw WHERE perfID = 426;
# Change this id to something existing and valid
What you want to see is that it is using an index, and ONLY an index. If you don't understand the output, copy and paste it here so can I go through it with you.
UPDATE: As you can see, it takes 0.07 seconds for EVERY row in your data array, plus the time to actually query the database, transfer the result etc. This is roughly 28000 * 0.07 = 1960 seconds, or 32 minutes, just to check if the data exists or not. You need to come up with another way of checking if the data already exists... One, very simple optimization might be:
EXPLAIN SELECT perfId FROM it_raw WHERE perfID = 210968;
This way, you can use the index on perfId and don't need to visit the table
If it is possible, try to avoid quering the database for each run in the loop. Perhaps it is possible to fetch the ids from the database into a big array of ids that will fit in PHP memory? This will be MUCH quicker than quering the database for every row in your big data array.
This is exactly the scenario for which prepared statements were made:
$prepared_statement =
$DB->prepare('INSERT INTO table(column, column) VALUES(?, ?)');
loop {
$prepared_statement->execute(array('value1', 'value2');
}
It's implemented in the MySQLi and PDO wrappers. It only compiles the query once and automagically sanitizes the given data, saving time (both developmental and executional) and headache.