Query stopped running instead of inserting zero values - php

Question

We have this MySQL query where we want to find the date_time and price_open from price table and update these two values into the price_datetime and price_open columns in comment table:
UPDATE comment AS fc
INNER JOIN price AS p
ON p.ticker_id = fc.ticker_id
AND p.date_time =
( SELECT pi.date_time
FROM price AS pi
WHERE pi.ticker_id = fc.ticker_id
AND pi.date_time >= fc.date_time
ORDER BY pi.date_time ASC
LIMIT 1
) SET
fc.price_datetime = p.date_time,
fc.price_open = p.price_open;
Which we converted to PHP+MySQL hoping for a more efficiency and much faster process:
<?php
ob_flush();
flush();
usleep(160);
$tickers = array();
$stmt = $mysqli->prepare("SELECT ticker_id, date_time FROM flaggedcomment order by ticker_id, date_time");
$stmt->execute(); //Execute prepared Query
$stmt->bind_result($tid, $dt);
$arr_index = 0;
while ($stmt->fetch() ) {
$tickers[$arr_index] = array();
$tickers[$arr_index]["id"] = $tid;
$tickers[$arr_index]["dt"] = $dt;
$arr_index++;
}
/* free result set */
$stmt->free_result();
$record_index = 0;
$flaggedcomment_index = 0;
$sql = "";
// get total tickers
$total_tickers = count($tickers);
echo "Total records: " . $total_tickers . "<br />";
foreach ($tickers as $ticker) { //fetch values
$stmt = $mysqli->prepare("SELECT price_open, date_time FROM price WHERE ticker_id =? AND date_time >=? ORDER BY date_time ASC LIMIT 1;");
$stmt->bind_param("is",$ticker["id"], $ticker["dt"]); // two params: one is integer, and other one is string
$stmt->execute(); //Execute prepared Query
$results = $stmt->get_result();
$myrow = $results->fetch_row();
$set_string = "SET";
// bind values
$price_open = $myrow[0];
$date_time = $myrow[1];
// set initial insert query value
$set_string .= " price_datetime='". $date_time ."'";
$set_string .= ", price_open=". $price_open;
$set_string .= " WHERE ticker_id=". $ticker["id"] ." AND date_time='" . $ticker["dt"] ."'";
if($set_string != ""){
$sql .= "UPDATE flaggedcomment ". $set_string . ";";
}
$idx = $record_index + 1;
if(($record_index + 1) % 100 == 0){
?>
<script>
$('#page-wrap > h1').html("Processing Ticker id #" + <?= $ticker["id"]; ?> + " - Record #" + <?= $idx; ?>);
</script>
<?php
ob_flush();
flush();
usleep(160);
}
$record_index++;
/* free result set */
$stmt->free_result();
} // end while fetch ticker id
$update_flaggedcomment_qry = "LOCK TABLES flaggedcomment WRITE; ". $sql . "UNLOCK TABLES; ";
echo $update_flaggedcomment_qry;
//echo "<br />";
if ($mysqli->multi_query($update_flaggedcomment_qry)) {
// nothing
} else {
echo "Error updating record: " . $mysqli->error . "<br />";
$mysqli->close();
exit;
}
echo "<span style='color:blue;'> <b> Done. </b> </span>";
ob_end_flush();
exit;
?>
Using the MySQL query, if there's no match of ticker_id and date_time from both tables, the fc.price_datetime and fc.price_open columns will show 0000-00-00 00:00:00 and 0.00 values. However, when executing the PHP code, instead of inserting the zero values, it will stop when it encounters its very first "no matching" of ticker_id and date_time and cannot continue. We've spent a long time figuring out on how to fix it, unfortunately, none of the ways we use can fix this.
Any help from the community is definitely much appreciated.
Thank you. :)

Answer 1

You do not check if there is a record returned from price . Hence your code just tries picking the first and second elements of the result array when that array is null. You then likely land up with a statement trying to assign blank to each of the price_datetime and price_open fields. With the price_datetime you have quotes around the empty value and mysql will probably cope with this, but for price_open you do not have quotes around the expected numeric value. Hence you will land up with an invalid update statement (some like the following):-
UPDATE flaggedcomment price_datetime='', price_open= WHERE ticker_id=123 AND date_time='2016-01-01 00:00:00';
As you are executing multiple SQL statements at once to do the updates, I expect that it won't execute any after the invalid statement.
A quick play with your code and the following should work. This checks the returned row and if one isn't found it just uses the default (zero like) values for the 2 fields you want to update.
<?php
ob_flush();
flush();
usleep(160);
$tickers = array();
$stmt = $mysqli->prepare("SELECT ticker_id, date_time FROM flaggedcomment order by ticker_id, date_time");
$stmt->execute(); //Execute prepared Query
$stmt->bind_result($tid, $dt);
$arr_index = 0;
while ($stmt->fetch() )
{
$tickers[$arr_index] = array();
$tickers[$arr_index]["id"] = $tid;
$tickers[$arr_index]["dt"] = $dt;
$arr_index++;
}
/* free result set */
$stmt->free_result();
$record_index = 0;
$flaggedcomment_index = 0;
$sql = "";
// get total tickers
$total_tickers = count($tickers);
echo "Total records: " . $total_tickers . "<br />";
foreach ($tickers as $ticker)
{ //fetch values
$stmt = $mysqli->prepare("SELECT price_open, date_time FROM price WHERE ticker_id =? AND date_time >=? ORDER BY date_time ASC LIMIT 1;");
$stmt->bind_param("is",$ticker["id"], $ticker["dt"]); // two params: one is integer, and other one is string
$stmt->execute(); //Execute prepared Query
$results = $stmt->get_result();
if ($myrow = $results->fetch_assoc())
{
$price_open = $myrow['price_open'];
$date_time = $myrow['date_time'];
}
else
{
$price_open = 0.00;
$date_time = "0000-00-00 00:00:00";
}
$sql .= "UPDATE flaggedcomment SET";
$sql .= " price_datetime='". $date_time ."'";
$sql .= ", price_open=".$price_open;
$sql .= " WHERE ticker_id=". $ticker["id"] ." AND date_time='" . $ticker["dt"] ."';";
$idx = $record_index++;
if(($record_index + 1) % 100 == 0)
{
?>
<script>
$('#page-wrap > h1').html("Processing Ticker id #" + <?= $ticker["id"]; ?> + " - Record #" + <?= $idx; ?>);
</script>
<?php
ob_flush();
flush();
usleep(160);
}
$record_index++;
/* free result set */
$stmt->free_result();
} // end while fetch ticker id
$update_flaggedcomment_qry = "LOCK TABLES flaggedcomment WRITE; ". $sql . "UNLOCK TABLES; ";
echo $update_flaggedcomment_qry;
//echo "<br />";
if ($mysqli->multi_query($update_flaggedcomment_qry)) {
// nothing
}
else
{
echo "Error updating record: " . $mysqli->error . "<br />";
$mysqli->close();
exit;
}
echo "<span style='color:blue;'> <b> Done. </b> </span>";
ob_end_flush();
exit;
?>
However I suspect that looping around the result of one query and doing another query for each row in php will be slower than a well constructed single update query.
If you do want to loop around the results like you are doing now it may be quicker to create a tmp table and insert the rows to that (as you can insert hundreds of rows with a single statement) and then update your flaggedcomment table with a single update statement that joins it to the tmp table
EDIT - If you can post the table declares and a bit of sample data I will have an attempt at doing it in a single SQL statement.
A first attempt (untested) would be:-
UPDATE comment AS fc
INNER JOIN price AS p
ON p.ticker_id = fc.ticker_id
INNER JOIN
(
SELECT *
FROM
(
SELECT fc.ticker_id,
MIN(pi.date_time) AS date_time
FROM comment AS fc
INNER JOIN price AS pi
ON pi.ticker_id = fc.ticker_id
AND pi.date_time >= fc.date_time
GROUP BY fc.ticker_id
) sub1
) sub0
ON p.ticker_id = sub0.ticker_id
AND p.date_time = sub0.date_time
SET fc.price_datetime = p.date_time,
fc.price_open = p.price_open;
This is using the extra seemingly redundant sub query to hopefully bypass a MySQL restriction on updating a table that is also used in a subquery.