PHP Update Query While Loop - php

So I have 2 tables named accounts and tblDataNumbers. The accounts table has a sponsorID field while the tblDataNumbers table has xsponsorID field. So first, this is the code:
for ($x = 501; $x <= 1061; $x++) {
$data = $conn->query("SELECT * FROM tblDataNumbers WHERE tableID = $x");
foreach ($data as $d) {
$xsponsorID = $d['xsponsorID'];
}
$checkSponsor = $conn->query("SELECT * FROM accounts WHERE sponsorID = '".$xsponsorID."' ");
if(count($checkSponsor) > 0) {
$xsid = $xsponsorID;
} else {
$xsid = "";
}
$stmt = $conn->prepare('UPDATE tblDataNumbers SET xsponsorID=:xsponsorID WHERE tableID=:x');
$stmt->execute(array(
':xsponsorID' => $xsid,
':x' => $x
));
}
The code above will check if the $xsponsorID is existing on the sponsorID field of the accounts table. If yes, it will set $xsid to the same value which is $xsponsorID and if not, it will be set to blank or "". When I run the code, nothing happens. There are lots of numbers in the xsponsorID field of the tblDataNumbersthat table that is not present in the sponsorID field of the accounts table so basically something should be set to blank but not even one.
What I wanted to happen with this is to repeat the code inside from row 501 to 1061 of table tblDataNumbers but nothing happens. Is there something wrong with my code? Thank you.

I don't know if this is what you are attempting but that could potentially kill the server for a while as there will be 560 x records (from 1st query ) = lots!
I think there must be a neater solution that does the work in one query - I shall look at the queries in detail to see if there is a solution.
for( $x = 501; $x <= 1061; $x++ ) {
$data = $conn->query("SELECT * FROM tblDataNumbers WHERE tableID = $x");
/* This loop will only ever get one result..
foreach( $data as $d ){
$xsponsorID = $d['xsponsorID'];
}
*/
foreach( $data as $d ){
$xsponsorID = $d['xsponsorID'];
$checkSponsor = $conn->query("SELECT * FROM accounts WHERE sponsorID = '".$xsponsorID."' ");
$xsid=( count( $checkSponsor ) > 0 ) ? $xsponsorID : '';
$stmt = $conn->prepare('UPDATE `tblDataNumbers` SET `xsponsorID`=:xsponsorID WHERE `tableID`=:x;');
$stmt->execute(array(
':xsponsorID' => $xsid,
':x' => $x
));
}
}
After looking at this again and reading your comments I wondered if something along the lines of the following might do the trick. I created a new database with the same two named tables and populated some test data ( though I do not know if it resembles your own data ) and used the following as a test within heidi ( mysql gui app )
set #tblid=1;
update `tblDataNumbers` t set t.`xsponsorID`=(
case
when ( t.`xsponsorID` not in ( select `sponsorID` from `accounts` ) ) then
null
else
t.`xsponsorID`
end
)
where t.`tableID`=#tblid;
Clearly this updates a record where the tableID is equal to the #tblid var only. However, to update all the records that match the criteria, remove the where clause and it will apply the NULL value to any record in the tblDataNumbers table that does not have a corresponding sponsorID in the accounts table. Before running this I would strongly advise making a backup of your data though!!!!
Without the mysql variable the query becomes the following and means you would not need the loops.
update `tblDataNumbers` t set t.`xsponsorID`=(
case
when ( t.`xsponsorID` not in ( select `sponsorID` from `accounts` ) ) then
null
else
t.`xsponsorID`
end
)
If it is important to update the records only between 501 and 1061 then you could add a where clause such as:-
where t.`tableID` between 501 and 1061

Thanks to #RamRaider, managed to fix it by adding a single line and removing the count() function.
for( $x = 501; $x <= 1061; $x++ ) {
$data = $conn->query("SELECT * FROM tblDataNumbers WHERE tableID = $x");
foreach( $data as $d ){
$xsponsorID = $d['xsponsorID'];
$checkSponsor = $conn->query("SELECT * FROM accounts WHERE sponsorID = '".$xsponsorID."' ");
$row = $checkSponsor->fetch(PDO::FETCH_ASSOC); //I forgot this line
$xsid=( $row > 0 ) ? $xsponsorID : '';
$stmt = $conn->prepare('UPDATE `tblDataNumbers` SET `xsponsorID`=:xsponsorID WHERE `tableID`=:x;');
$stmt->execute(array(
':xsponsorID' => $xsid,
':x' => $x
));
}
}

Related

PHP / SQL: Update Table C if meet condition after updated in Table A and Table B

I got some issues with my current code. Example, on a PHP page, there's a table that displays all tools that borrowed the users. In that table, each data rows contain a checkbox. Users can select any tools that they want to return first by tick the checkbox and press "return" button.
At Server-side, after clicking the "return" button, it will go to the page named return_selected.php. At this page, it will update Table A and Table B. This one is successful.
Now, I want to update Table C if all there's a condition, for example, ALL tools returned. This one I still do but failed. Below is the code
return_selected.php
<?php
include("../../../../config/configPDO.php");
include("../../../../config/check.php");
$tools_id = $_POST['tools_id'];
$borrow_id = $_POST['borrow_id'];
$checkbox=explode( ',', $_POST['ids'][0] );
for($i=0;$i < count($checkbox); $i++){
$tools_id=$checkbox[$i];
$sql="UPDATE ets_tools SET borrow_id = NULL WHERE tools_id=:tools_id";
$query=$conn->prepare($sql);
$query->execute(array(':tools_id' => $tools_id));
$sql2="UPDATE ets_tools_borrow SET time_to = GETDATE() WHERE tools_id=:tools_id";
$query3=$conn->prepare($sql2);
$query3->execute(array(':tools_id' => $tools_id));
// want to update table if all tools returned.
$query2 = "
SELECT
ets_tools.tools_id, ets_tools.tools_name, ets_tools.borrow_id,
ets_borrow.time_from, ets_borrow.time_to, ets_borrow.status_id
FROM ets_tools
INNER JOIN ets_borrow ON ets_tools.borrow_id = ets_borrow.borrow_id
WHERE ets_tools.borrow_id IS NOT NULL AND ets_borrow.borrow_id = :borrow_id
";
$sql2=$conn->prepare($query2);
$sql2->execute(array(':borrow_id' => $borrow_id));
if($sql2->rowCount() > 0)
{
header("Location: return.php");
}else{
$sql3="UPDATE ets_borrow SET time_to = GETDATE(), status_id = 2 WHERE borrow_id=:borrow_id";
$query3=$conn->prepare($sql3);
$query3->execute(array(':borrow_id' => $borrow_id));
header("Location: return.php");
}
}
?>
Can anyone know how to solve this? Thank you.
You need to consider the following:
As is mentioned in the documentation, PDOStatement::rowCount() returns the number of rows affected by the last DELETE, INSERT, or UPDATE statement executed by the corresponding PDOStatement object and for most databases the number of rows returned from a SELECT statement is not returned correctly. So, you may use a SELECT COUNT(*) ... approach to get this count.
For your specific case you may try to use an UPDATE ... WHERE NOT EXISTS ... statement to update the rows in the ets_borrow table and skip this SELECT ... FROM ... INNER JOIN ... statement.
I'm not sure if I understand corectly the use of header("Location: return.php"); in a for loop, probably you need to redirect after this loop.
Example, based on your code:
<?php
include("../../../../config/configPDO.php");
include("../../../../config/check.php");
$tools_id = $_POST['tools_id'];
$borrow_id = $_POST['borrow_id'];
$checkbox=explode( ',', $_POST['ids'][0] );
for($i=0; $i < count($checkbox); $i++) {
$tools_id=$checkbox[$i];
$sql = "UPDATE ets_tools SET borrow_id = NULL WHERE tools_id = :tools_id";
$query = $conn->prepare($sql);
$query->execute(array(':tools_id' => $tools_id));
$sql2 = "UPDATE ets_tools_borrow SET time_to = GETDATE() WHERE tools_id = :tools_id";
$query2 = $conn->prepare($sql2);
$query2->execute(array(':tools_id' => $tools_id));
$sql3 = "
UPDATE ets_borrow
SET time_to = GETDATE(), status_id = 2
WHERE
(borrow_id = :borrow_id1) AND
NOT EXISTS (
SELECT 1
FROM ets_tools
INNER JOIN ets_borrow ON ets_tools.borrow_id = ets_borrow.borrow_id
WHERE ets_tools.borrow_id IS NOT NULL AND ets_borrow.borrow_id = :borrow_id2
)
";
$borrow_id1 = $borrow_id;
$borrow_id2 = $borrow_id;
$query3 = $conn->prepare($sql3);
$query3->execute(array(':borrow_id1' => $borrow_id1, ':borrow_id2' => $borrow_id2));
}
header("Location: return.php");
?>

PHP optimize multiple update and select query

I have a php API where i am checking different conditions and based on that, i am fetching the count of the row and updating the count in the same table.
Below are the Conditions i am writing the query: (Note: 1 user can have multiple campaigns)
1) For a given user(Uid), for a particular campaign whose type = Impression, i am getting the Impression count and updating the same in table
2) For a given user(Uid), for a particular campaign whose type = Action i am getting the Action count and updating the same in table
//To get the Impression/Action/Lead count
$Impressionarr = [];
$Actionarr = [];
//IMPRESSION COUNT
$imp_qry = "select count(*) as ImpressionCount from ClicksAndImpressions where Uid = 101642 and CampaignID =100 and Type='Impression' ;";
$impData = $this->getClicksAndImpressionsTable()->CustomQuery($imp_qry);
if($impData[0]['ImpressionCount'] != '' || $impData[0]['ImpressionCount'] !=NULL ){
$impr_update = "UPDATE ClicksAndImpressions SET ImpressionCount = ". $impData[0]['ImpressionCount'] ." where Uid = 101642 and CampaignID =100 ;";
}else{
echo '----not Present';
}
//Impression count
foreach($impData as $key=>$data)
{
$data['ImpressionCount'];
$Impressionarr[] = $data;
}
//ACTION COUNT
$action_qry = "select count(*) as ActionCount from ClicksAndImpressions where Uid = 101617 and CampaignID =81 and Type = 'Action';";
$actionData = $this->getClicksAndImpressionsTable()->CustomQuery($action_qry);
if($actionData[0]['ActionCount'] != '' || $actionData[0]['ActionCount'] !=NULL ){
$action_update = "UPDATE ClicksAndImpressions SET ActionCount = ". $actionData[0]['ActionCount'] ." where Uid = 101617 and CampaignID =81 ;";
$actionData = $this->getClicksAndImpressionsTable()->CustomUpdate($action_update); //print_r($actionData);exit;
}else{
echo '----not Present';
}
//Action count
foreach($actionData as $key=>$data)
{
$data['ActionCount'];
$Actionarr[] = $data;
}
//Trying to combine the 3 arrays- but 2nd and 3rd not merging into 1st
$combine = array_merge($CampaignDetailsarr,$Impressionarr,$Actionarr);
1) Can the above duplicates - either in Update query or multiple for loops be avoided. if so how to optimize the above as per my conditions
ie) For
if( Uid = 123 for some campaignId = 11 and if type is 'Impression')
-> Update query for impression
else if(Uid = 123 for campaignId= 22 and if type = 'something else')
-> Update query for something else
2) I already have one array 1. I need to merge my 2 new arrays (Impressionarr[], Actionarr[]) into the 1st one, but below is how im getting.
Expected:
Coming as:
You could probably speed it up by creating a stored procedure in your database and calling that with the required data. From what I can see, most of the logic you have to do can be done in the stored procedure.

SQL | Update multiple rows with one query

I have a method, where about 20 rows must changed. Im passing per ajax a string like "14-33-61-10 ..." for saving the order of some rows.
Here is my actual (working) code:
$explode = explode('--',$_POST['data']); // -> array('0'=>'8', '1'=>'4', '2'=> ... )
$i = 1;
foreach( $explode as $task ) {
if( !is_int($task) ) continue;
$exec = $project->exec("UPDATE tasks SET pos=$i WHERE rowid=". $task );
$i++;
}
My problem is, that it takes about 1 second.
My question: Is there a way to edit multiple rows with just one query?
Sure, send a SQL string like:
update tasks
set pos =
case rowid
when 13 then 1
when 33 then 2
when 61 then 3
when 10 then 4
end
where rowid in (13,33,61,10)
The translation from 13-33-61-10 to (13,33,61,10) is best done in PHP.
If the range is short then we can use the ID IN (ids) argument.
UPDATE `profile` SET `app_status`=3 WHERE `id` IN (3,37,95,136,1087,1795,1817)
If it's long and we know that range, let's use the BETWEEN value AND value argument.
UPDATE `profile` SET `app_status`=3 WHERE `id` BETWEEN 3904 AND 3918
$explode = explode('--',$_POST['data']); // -> array('0'=>'8', '1'=>'4', ... )
$i = 1;
$sql = "";
foreach($explode as $task)
{
if(!is_int($task)) continue;
$sql .= "UPDATE tasks SET pos=$i WHERE rowid=$task;";
$i++;
}
$project->exec($sql);

If ( $currentCount <= $allowedCount ) .. Returns 1 extra?

For some reason this code still returns 2 instead of one?
$allowedCount = 1; ( This figure is set inside the database )
$sql_count = mysql_query("SELECT usrid FROM mytable WHERE usrid=".$uid) or die(mysql_error());
$currentCount = mysql_num_rows( $sql_count );
$currentCount = 1; ( In this example current count returns a 1 )
if ( $currentCount <= $allowedCount ) {
// do this OK
} else {
// do this Not OK
}
What im trying to do is check if the user has reached their limit, if so then disallow them further INSERT queries etc....
Would this be a simple approach?
check your db, you may have duplicate record. if your using a form to insert the data it may be reinserting data upon page refresh. if not check your db for duplication.

Speed up this code - PHP/SQL - Short Code but takes incredibly long at the moment

Right, this code goes through a rather large multidimensional array (has about 28,000 rows and 16 parts).
Order of events:
Check if the data exists in the database
if it exists - Update it with the new data
if it doesn't exist - Insert it
Simple.
But right now to go through this it has taken over 30min i think and Still going.
$counter = 0;
$events = sizeof($feed_array['1'])-1;
while($counter <= $events ) {
$num_rows = mysql_num_rows(mysql_query("SELECT * FROM it_raw WHERE perfID = '".addslashes($feed_array['16'][$counter])."'"));
if($num_rows) {
$eventDate=explode("/", $feed_array['1'][$counter]); //print_r($eventDate);
$eventTime=explode(":", $feed_array['2'][$counter]); //print_r($eventTime);
$eventUnixTime=mktime($eventTime[0], $eventTime[1], "00", $eventDate[1], $eventDate[0], $eventDate[2]);
mysql_query("UPDATE `it_raw` SET
`eventtime` = '".$eventUnixTime."',
`eventname` = '".addslashes($feed_array['3'][$counter])."',
`venuename` = '".addslashes($feed_array['4'][$counter])."',
`venueregion` = '".addslashes($feed_array['5'][$counter])."',
`venuepostcode` = '".addslashes($feed_array['6'][$counter])."',
`country` = '".addslashes($feed_array['7'][$counter])."',
`minprice` = '".addslashes($feed_array['8'][$counter])."',
`available` = '".addslashes($feed_array['9'][$counter])."',
`link` = '".addslashes($feed_array['10'][$counter])."',
`eventtype` = '".addslashes($feed_array['11'][$counter])."',
`seaOnSaleDate` = '".addslashes($feed_array['12'][$counter])."',
`perOnSaleDate` = '".addslashes($feed_array['13'][$counter])."',
`soldOut` = '".addslashes($feed_array['14'][$counter])."',
`eventImageURL` = '".addslashes($feed_array['15'][$counter])."',
`perfID`= '".addslashes($feed_array['16'][$counter])."'
WHERE `perfID` = ".$feed_array['16'][$counter]." LIMIT 1 ;");
echo "UPDATE ".$feed_array['16'][$counter].": ".addslashes($feed_array['3'][$counter])."\n";
} else {
$eventDate=explode("/", $feed_array['1'][$counter]); //print_r($eventDate);
$eventTime=explode(":", $feed_array['2'][$counter]); //print_r($eventTime);
$eventUnixTime=mktime($eventTime[0], $eventTime[1], "00", $eventDate[1], $eventDate[0], $eventDate[2]);
$sql = "INSERT INTO `dante_tickets`.`it_raw` (
`id` ,
`eventtime` ,
`eventname` ,
`venuename` ,
`venueregion` ,
`venuepostcode` ,
`country` ,
`minprice` ,
`available` ,
`link` ,
`eventtype` ,
`seaOnSaleDate` ,
`perOnSaleDate` ,
`soldOut` ,
`eventImageURL` ,
`perfID`
)
VALUES (
NULL ,
'".$eventUnixTime."',
'".addslashes($feed_array['3'][$counter])."',
'".addslashes($feed_array['4'][$counter])."',
'".addslashes($feed_array['5'][$counter])."',
'".addslashes($feed_array['6'][$counter])."',
'".addslashes($feed_array['7'][$counter])."',
'".addslashes($feed_array['8'][$counter])."',
'".addslashes($feed_array['9'][$counter])."',
'".addslashes($feed_array['10'][$counter])."',
'".addslashes($feed_array['11'][$counter])."',
'".addslashes($feed_array['12'][$counter])."',
'".addslashes($feed_array['13'][$counter])."',
'".addslashes($feed_array['14'][$counter])."',
'".addslashes($feed_array['15'][$counter])."',
'".addslashes($feed_array['16'][$counter])."'
);";
mysql_query($sql) or die(mysql_error().":".$sql);
echo "Inserted ".$feed_array['16'][$counter].": ".addslashes($feed_array['3'][$counter])."\n";
}
unset($sql);
$counter++;
}
UPDATE
I just carried out profiling one one of the rows:
mysql> EXPLAIN SELECT * FROM it_raw WHERE perfID = 210968;
+----+-------------+--------+------+---------------+--------+---------+-------+------+-------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+--------+------+---------------+--------+---------+-------+------+-------+
| 1 | SIMPLE | it_raw | ref | perfID | perfID | 4 | const | 1 | |
+----+-------------+--------+------+---------------+--------+---------+-------+------+-------+
1 row in set (0.07 sec)
UPDATE 2
To try and "speed" things up, instead of carrying out the UPDATE and INSERT statements straight away, i've now placed them in a variable (so only the initial select runs - to check a duplicate - then stores the action [insert or update]). At the end of the loop it executes all the statments.
Except now, it's coming up with MySQL error that the syntax is incorrect. (when initially there was nothing wrong).
I've simply replaced the mysql_query with:
$sql_exec .= "SELECT.... ;";
is there something i'm missing here for the formatting?
UPDATE 3
OK finally fixed it
Lessons Learned:
Do logic search first on database
Carry out insert/updates in bulk.
Here is the final code which now takes about 60 seconds to run (from over 30min+)
while($counter <= $events ) {
$num_rows = mysql_num_rows(mysql_query("SELECT * FROM it_raw WHERE perfID = '".addslashes($feed_array['16'][$counter])."'"));
if($num_rows) {
$eventDate=explode("/", $feed_array['1'][$counter]); //print_r($eventDate);
$eventTime=explode(":", $feed_array['2'][$counter]); //print_r($eventTime);
$eventUnixTime=mktime($eventTime[0], $eventTime[1], "00", $eventDate[1], $eventDate[0], $eventDate[2]);
$sql_exec[] = "UPDATE `it_raw` SET `eventtime` = '".$eventUnixTime."',`eventname` = '".addslashes($feed_array['3'][$counter])."',`venuename` = '".addslashes($feed_array['4'][$counter])."',`venueregion` = '".addslashes($feed_array['5'][$counter])."',`venuepostcode` = '".addslashes($feed_array['6'][$counter])."',`country` = '".addslashes($feed_array['7'][$counter])."',`minprice` = '".addslashes($feed_array['8'][$counter])."',`available` = '".addslashes($feed_array['9'][$counter])."',`link` = '".addslashes($feed_array['10'][$counter])."',`eventtype` = '".addslashes($feed_array['11'][$counter])."',`seaOnSaleDate` = '".addslashes($feed_array['12'][$counter])."',`perOnSaleDate` = '".addslashes($feed_array['13'][$counter])."',`soldOut` = '".addslashes($feed_array['14'][$counter])."',`eventImageURL` = '".addslashes($feed_array['15'][$counter])."',`perfID`='".addslashes($feed_array['16'][$counter])."' WHERE `perfID` = ".$feed_array['16'][$counter]." LIMIT 1;";
echo "UPDATE ".$feed_array['16'][$counter].": ".addslashes($feed_array['3'][$counter])."\n";
} else {
$eventDate=explode("/", $feed_array['1'][$counter]); //print_r($eventDate);
$eventTime=explode(":", $feed_array['2'][$counter]); //print_r($eventTime);
$eventUnixTime=mktime($eventTime[0], $eventTime[1], "00", $eventDate[1], $eventDate[0], $eventDate[2]);
$sql_exec[] = "INSERT INTO `it_raw` (`id` ,`eventtime` ,`eventname` ,`venuename` ,`venueregion` ,`venuepostcode` ,`country` ,`minprice` ,`available` ,`link` ,`eventtype` ,`seaOnSaleDate` ,
`perOnSaleDate` ,`soldOut` ,`eventImageURL` ,`perfID`) VALUES ( NULL ,'".$eventUnixTime."','".addslashes($feed_array['3'][$counter])."','".addslashes($feed_array['4'][$counter])."','".addslashes($feed_array['5'][$counter])."','".addslashes($feed_array['6'][$counter])."','".addslashes($feed_array['7'][$counter])."','".addslashes($feed_array['8'][$counter])."','".addslashes($feed_array['9'][$counter])."','".addslashes($feed_array['10'][$counter])."','".addslashes($feed_array['11'][$counter])."','".addslashes($feed_array['12'][$counter])."','".addslashes($feed_array['13'][$counter])."','".addslashes($feed_array['14'][$counter])."','".addslashes($feed_array['15'][$counter])."','".addslashes($feed_array['16'][$counter])."');";
//mysql_query($sql) or die(mysql_error().":".$sql);
echo "Inserted ".$feed_array['16'][$counter].": ".addslashes($feed_array['3'][$counter])."\n";
}
unset($sql);
$counter++;
}
foreach($sql_exec as $value) {
mysql_query($value) or die (mysql_error().": ".$value);
}
You could try grouping inserts and updates into groups so the code runs less queries.
For example, you could group all of the inserts into one very large insert, or maybe group every 100 inserts.
Also using prepared statements as gradbot suggested may help.
Other than that, it's not very easy to say which part of it is the major contributor to slowness. You should use a profiler to determine that, for example by using a smaller dataset so the profiled script runs in a reasonable time.
You can do a number of things.
on duplicate key instead of select
prepared statement if you have access to PDO MySQL.
stored procedure in MySQL
Try This. I'm unable to test it but the syntax should be right.
$counter = 0;
$events = sizeof($feed_array['1']) - 1;
while($counter <= $events )
{
$eventDate = explode("/", $feed_array['1'][$counter]); //print_r($eventDate);
$eventTime = explode(":", $feed_array['2'][$counter]); //print_r($eventTime);
$eventUnixTime = mktime($eventTime[0], $eventTime[1], "00", $eventDate[1], $eventDate[0], $eventDate[2]);
$data = array(
'eventtime' => $eventUnixTime,
'eventname' => addslashes($feed_array['3'][$counter]),
'venuename' => addslashes($feed_array['4'][$counter]),
'venueregion' => addslashes($feed_array['5'][$counter]),
'venuepostcode' => addslashes($feed_array['6'][$counter]),
'country' => addslashes($feed_array['7'][$counter]),
'minprice' => addslashes($feed_array['8'][$counter]),
'available' => addslashes($feed_array['9'][$counter]),
'link' => addslashes($feed_array['10'][$counter]),
'eventtype' => addslashes($feed_array['11'][$counter]),
'seaOnSaleDate' => addslashes($feed_array['12'][$counter]),
'perOnSaleDate' => addslashes($feed_array['13'][$counter]),
'soldOut' => addslashes($feed_array['14'][$counter]),
'eventImageURL' => addslashes($feed_array['15'][$counter]),
'perfID' => addslashes($feed_array['16'][$counter]),
);
$update = array();
foreach ($data as $key => $value)
$update[] = "`$key` = '$value'";
$sql = "INSERT INTO `dante_tickets`.`it_raw`" .
'(`id`, `'. implode ('`,`', array_keys($data)) . '`) VALUES ' .
'(NULL, ' . implode (',', $data) . ') ON DUPLICATE KEY UPDATE ' .
implode (',', $update);
mysql_query($sql) or die(mysql_error().":".$sql);
echo "Inserted or Updated".$feed_array['16'][$counter].": ".addslashes($feed_array['3'][$counter])."\n";
unset($sql);
$counter++;
}
I forgot to mention this requires that perfID is a unique key.
Have you profiled this query?
"SELECT * FROM it_raw WHERE perfID = '".addslashes($feed_array['16'][$counter])."'"
Because you run it 28000 times.. so unless it is REALLY quick, it will cause you a head ache. Use the EXPLAIN syntax and add an appropriate index if needed.
EDIT: With profile, I mean that you should try to use EXPLAIN on the mysql-prompt to see what execution plan the MySQL Query Optimizer suggests for this query. I.e, on the prompt, run:
EXPLAIN SELECT * FROM it_raw WHERE perfID = 426;
# Change this id to something existing and valid
What you want to see is that it is using an index, and ONLY an index. If you don't understand the output, copy and paste it here so can I go through it with you.
UPDATE: As you can see, it takes 0.07 seconds for EVERY row in your data array, plus the time to actually query the database, transfer the result etc. This is roughly 28000 * 0.07 = 1960 seconds, or 32 minutes, just to check if the data exists or not. You need to come up with another way of checking if the data already exists... One, very simple optimization might be:
EXPLAIN SELECT perfId FROM it_raw WHERE perfID = 210968;
This way, you can use the index on perfId and don't need to visit the table
If it is possible, try to avoid quering the database for each run in the loop. Perhaps it is possible to fetch the ids from the database into a big array of ids that will fit in PHP memory? This will be MUCH quicker than quering the database for every row in your big data array.
This is exactly the scenario for which prepared statements were made:
$prepared_statement =
$DB->prepare('INSERT INTO table(column, column) VALUES(?, ?)');
loop {
$prepared_statement->execute(array('value1', 'value2');
}
It's implemented in the MySQLi and PDO wrappers. It only compiles the query once and automagically sanitizes the given data, saving time (both developmental and executional) and headache.

Categories