Is there a faster way to delete orphaned table entries - php

I currently have 20,000 rows in the listing table, and already this query is running quite slow.
What it does is delete all listing rows where the current time is 60 seconds or more since the listing endDate and where the listing is not associated with any task.
DELETE
listing
FROM
listing
LEFT JOIN
task
ON
task.listingId = listing.listingId
WHERE
task.listingId IS NULL
AND listing.endDate < DATE_SUB(NOW(), INTERVAL 1 MINUTE)
Is there a faster way to delete the orphaned listing rows?
UPDATE:
The original (above) query takes 11.6 secs.
This equivalent bunch of queries takes 0.09 secs, but it's a lot messier and requires a bunch of PHP:
$q = "
SELECT
listingId
FROM
task
";
$result = mysql_query($q) or die(mysql_error());
$taskListingIds = array();
while ($task = mysql_fetch_array($result)) {
$taskListingIds[$task['listingId']] = NULL;
}
$q = "
SELECT
listingId
FROM
listing
WHERE
endDate < DATE_SUB(NOW(), INTERVAL 1 MINUTE)
";
$result = mysql_query($q) or die(mysql_error());
while ($listing = mysql_fetch_array($result)) {
if (!array_key_exists($listing['listingId'], $taskListingIds)) {
$q = "
DELETE
FROM
listings
WHERE
listingId = " . $listing['listingId'] . "
";
mysql_query($q) or die(mysql_error());
}
}
I'll stick with this for now, but any suggestions for a fast way to do this purely with SQL would be great.

Related

show same id one time but in column count how many times php

I am displaying data from database, but I want to display one time same id but in other column how many times its stored in MySQL database.
Thanks!
if (!isset($_REQUEST['completed_consu_id'])) {
$query = "SELECT * FROM completed_consumers";
} else {
$query = "SELECT * FROM completed_consumers WHERE consu_id=consu_id";
}
$result = mysql_query($query) or die('Query failed: ' . mysql_error());
$numberofrow=mysql_num_rows($result);
while ($row = #mysql_fetch_array($result)) {
$consu_id = $row['consu_id'];
$consu_first_name = $row['consu_first_name'];
$consu_last_name = $row['consu_last_name'];
$consu_phone = $row['consu_phone'];
$consu_email = $row['consu_email'];
$consu_address = $row['consu_address'];
$consu_city = $row['consu_city'];
$consu_state = $row['consu_state'];
$consu_zip = $row['consu_zip'];
$consu_IP = $row['consu_IP'];
$status = $row['status'];
$query2 ="SELECT consu_id, COUNT(*) FROM completed_consumers WHERE consu_id=$consu_id GROUP BY consu_id";
$result2 = mysql_query($query2) or die('Query failed: ' . mysql_error());
$numberofrow2=mysql_num_rows($result2);
$times= $numberofrow2;
echo $times;
This is code i have written, Its displaying all the consumers details and how many they have entered data or details, but I want to show one time consumer details/Name but how many time they submitted data in "times" column.
Like David consumer added 2 times, Monika added 1 time, Arshi added 3 times data/details, but when i retrieve data its showing 6 rows in table, i want to show in 3 row, 1 for David, 1 for Monika, 1 for Arshi but with how many times they added in "times" column when i retrieve it details.
Take a look at the GROUP BY clause.
SELECT id, COUNT(*) FROM completed_consumers WHERE consu_id=$consu_id GROUP BY consu_id

Having trouble with MySQL double to date conversion

I have searched and searched for ways to do this but have found very limited information.
I have a MySQL table 'msgdb' that contains a field 'ttime' that is in the format double(25,8) (example row = 1352899856.95249200).
I need to routinely cleanup the table by removing any rows where the field 'ttime' is <= today's date -5 days.
These are the only two lines of code I could find related to double to time conversion but cannot get either to work.
SELECT ADDDATE(ADDDATE(ADDDATE('1899-12-31 00:00:00',FLOOR(ttime)), INTERVAL -1 DAY),INTERVAL(MOD(ttime,1)*86400)SECOND) AS TrueDate FROM msgdb
select date('1899-12-31 00:00:00'+ INTERVAL ttime * 24*3600 SECOND) as date from msgdb
I have tried first to display any rows that match the criteria using the code below, before I started using DELETE FROM to make sure I'm getting the correct results.
$query = "select date('1899-12-31 00:00:00'+ INTERVAL ttime * 24*3600 SECOND) as date from msgdb";
$result = mysql_db_query ($dbname, $query, $link);
while($row = mysql_fetch_array($result)) {
echo $row['date'];
echo '<br>';
}
and also
$query = "SELECT ADDDATE(ADDDATE(ADDDATE('1899-12-31 00:00:00',FLOOR(ttime)), INTERVAL -1 DAY),INTERVAL(MOD(ttime,1)*86400)SECOND) AS TrueDate FROM msgdb";
$result = mysql_db_query ($dbname, $query, $link);
while($row = mysql_fetch_array($result)) {
echo $row['TrueDate'];
echo '<br>';
}
but both are returning nothing.
UPDATE: Ok so by using this code:
$query = "select ttime from msgdb";
$result = mysql_db_query ($dbname, $query, $link);
while($row = mysql_fetch_array($result)) {
echo date('m-j-Y, H:i:s', $row[0]);
echo '<br>';
}
I am able to see it convert 'ttime' field from the stored value of 1352899856.95249200 to 11-14-2012, 07:30:56.
So how would I DELETE from the table all rows where ttime is <=now - 5 days?
Figuring out which records have a date before a point in time should be easy:
DELETE FROM table WHERE ttime <= DATE_SUB(NOW(), INTERVAL 5 DAY);
It might also be better to use UTC_TIMESTAMP() if you store all your times in UTC, which is the only sane way to do it.

Rolling Average Efficiency, php mysql

This script uses php and mysql to compute a one minute rolling average to reduce the impact of outliers on the my data (one minute = 6 10-second rows). It computes everything correctly, but is not efficient enough to do more than 150 rows at a time. I'd like to do as many rows as I can at a time, possibly between 5-10,000 as my table is over 150,000 and I input approximately 8,000 rows per day.
Does anyone have any suggestions as to how I can make this script run more efficiently?
Thanks!
<?php
//connect to database
mysql_connect("localhost","user","password");//database connection
mysql_select_db("database");
$result = mysql_query("SELECT Timestamp FROM table");
if (!$result) {
die('Could not query:' . mysql_error());
}
//get number of rows in table
$resultA = mysql_query("SELECT * FROM table");
$num_rows = mysql_num_rows($result);
echo "There are $num_rows rows.</br>";
//select column to be averaged
$resultB = mysql_query("SELECT PortRPMSignal FROM table");
if (!$resultB) {
die('Could not query:' . mysql_error());
}
//set start equal to the first row you want to calculate the averages from, likely the first null row
$start = 5;
//calculate 1 minute average, the average is correct
for($i = $start; $i<$num_rows; $i++){
$output = mysql_result($result,$i);
$test = mysql_result($resultB,$i)+mysql_result($resultB,$i-1)+mysql_result($resultB,$i-2)+mysql_result($resultB,$i-3)+mysql_result($resultB,$i-4)+mysql_result($resultB,$i-5);
$test2 = $test/6;
$round = round($test2,4);
$temp = mysql_query("SELECT Timestamp FROM table");
if(!$temp){
die('Could not query:' . mysql_error());
}
//gets timestamp at row $i, and inserts new average value into that row in RPMAve column
$time = mysql_result($result,$i);
mysql_query("UPDATE table SET PortMinuteAveRPM = $round WHERE Timestamp = '$time'");
}
For starters, the initial "count" block here can be cleaned up by adding the COUNT() aggregate:
$resultA = mysql_query("SELECT * FROM table");
$num_rows = mysql_num_rows($result);
echo "There are $num_rows rows.</br>";
Change to:
$resultA = mysql_query("SELECT COUNT(*) FROM table");
$row = mysql_fetch_array($result);
$num_rows = $row[0];
echo "There are $num_rows rows.</br>";
That should speed things up considerably on its own. Without it, you're selecting all of the data from the table - a query that will only grow slower the more you put into the table.
For the averages you're computing, is there any logic required that can't be accomplished directly in a MySQL query? Something such as:
UPDATE table SET PortMinuteAveRPM=(SELECT AVG(PortRPMSignal) FROM table WHERE Timestamp BETWEEN '$startTime' AND '$endTime') WHERE TimeStamp='$endTime'
This may save you from looping through results, if it's plausible.
It sounds like you're trying to calculate an autoregressive moving average (ARMA) but there's numerous issues with your interpretation of your data and how you are capturing it.
If you've got a complete set of data (though your question implies that you don't), then work out what time interval contains the required amount of records and get it direct from the database, e.g.
SELECT a.timestamp as base, AVG(b.PortRPMSignal)
FROM table a, table b
WHERE b.timestamp BETWEEN a.timestamp AND a.timestamp+INTERVAL 6 HOUR
GROUP BY a.timestamp
If you want to thin out the datapoints, then try something like....
SELECT a.timestamp as base, AVG(b.PortRPMSignal)
FROM table a, table b
WHERE b.timestamp BETWEEN a.timestamp AND a.timestamp+INTERVAL 6 HOUR
AND DATE_FORMAT(a.timestamp, '%i%s')='0000'
GROUP BY a.timestamp
Although a better solution if you've not got a complete dataset but there's only a small amount of jitter would be to use the modulus of an auto-increment id to pick out fewer rows from 'a'
It's only a start, but you can bin this bit
//get number of rows in table
$resultA = mysql_query("SELECT * FROM table");
$num_rows = mysql_num_rows($result);
echo "There are $num_rows rows.</br>";
Because the following line
$resultB = mysql_query("SELECT PortRPMSignal FROM table");
...will give you a result set that you can use mysql_num_rows on.
Using the * in a query increases the load on the database.
In your for loop you then have this
$temp = mysql_query("SELECT Timestamp FROM table");
if(!$temp){
die('Could not query:' . mysql_error());
}
which means this query runs every time you loop and you're not even using the results.
I don't know if mysqli will give you better performance, but you should use it.

Can these queries be turned into a REPLACE INTO?

$sql2 = "SELECT `id` FROM `saa_game` WHERE `domain_id` = '".$row['id']."' AND `unique_id` = '".s($oGame->unique_id)."' AND `year` = '".$iYear."' AND `month` = '".$iMonth."' LIMIT 1";
$result2 = mysql_query($sql2) or die(mail('rpaiva#golevel.com','SAA Gather Error',mysql_error()));
if(mysql_num_rows($result2) == 1)
{
$row = mysql_fetch_assoc($result2);
$sql3 = "UPDATE `saa_game` SET `plays` = '".s($oGame->plays)."' WHERE `id` = '".$row['id']."' LIMIT 1";
$result3 = mysql_query($sql3) or die(mail('email#sample.com','SAA Gather Error',mysql_error()));
}
else
{
$sql3 = "INSERT INTO `saa_game` (`domain_id`,`type`,`source`,`unique_id`,`plays`,`year`,`month`) VALUES ('".$row['id']."','".s($oGame->type)."','".s($oGame->source)."','".s($oGame->unique_id)."','".s($oGame->plays)."','".$iYear."','".$iMonth."')";
$result3 = mysql_query($sql3) or die(mail('email#sample.com','SAA Gather Error',mysql_error()));
}
I've got this set of queries running 40,000 times on a single page load on a cron job every 10 minutes. This takes so long that it almost runs into the next cron job. If I could reduce this into one query instead of two, that'd be great. (as long as there will actually be a performance difference)
If the select hits all of the right indexes, there won't be a performance increase. In fact, it's likely to be worse! MySQL's REPLACE INTO is implemented as a delete, then an insert!
Consider MySQL's INSERT INTO ... ON DUPLICATE KEY UPDATE instead.
Keep in mind that both of these options are exclusive to MySQL.

Php/MySQL help - random daily pick?

I'm trying to get a pick from my DB that would last for a day (daily pick). I use the following code:
$query = 'SELECT * FROM table ORDER BY rand() LIMIT 1
But as you can see it only gives me a random pick from the table, and every time I refresh the page it gets me a new random pick. How can I make the pick to last for a whole day?
Thanks in advance <3
I'm trying this:
$query = "SELECT * FROM table ORDER BY rand(" . date("Ymd") . ") LIMIT 1";
But I get the following error: mysql_fetch_assoc(): supplied argument is not a valid MySQL result resource. This is the part that gets broken:
$results = mysql_query($query);
while($line = mysql_fetch_assoc($results))
So... it should look like this, right? (I mean, choosing the daily random pick?)
$dailyPick = 'SELECT * FROM table ORDER BY rand() LIMIT 1';
$cacheKey = 'dailyPick'. date('dmY');
if($cache->has($cacheKey)) {
$dailyPick = $cache->get($cacheKey);
} else {
// hit database
$dailyPick = $cache->save($cacheKey);
}
I'm trying this now:
$dailyPick = 'SELECT * FROM table ORDER BY rand() LIMIT 1';
$cacheKey = 'dailyPick'. date('dmY');
if($cache->has($cacheKey)) {
$dailyPick = $cache->get($cacheKey);
} else {
// hit database
$dailyPick = $cache->save($cacheKey);
}
However, it gets me a mistake that I'm using the 'has' function on a non-object.
If you set the SEED for the rand to an integer value that changes daily, that would solve your problem
$query = "SELECT * FROM table ORDER BY rand(" . date("Ymd") . ") LIMIT 1";
Would do the trick.
A sane means of doing this would be to automatically generate the pick of the day content via a cron job that was setup to run once a day.
As such, the cron job would execute the SQL you provided and store the appropriate content in a flat file/database table, etc. (or perhaps even just store the choosen id in another table for future lookup purposes).
You can try something like this:
$total = 'SELECT COUNT(*) FROM table;';
$query = 'SELECT * FROM table ORDER BY id ASC LIMIT 1 OFFSET ' . (date('Ymd') % $total) . ';';
I think you'll need to update the random picked record with "today" field = 1..
Something like this:
// ------------
// Run this 3 commands once a day
// Reset all records
mysql_query("UPDATE `table` SET `today` = 0");
// Pick one
$sql = mysql_query("SELECT `id` FROM `table` ORDER BY RAND() LIMIT 1");
$id = mysql_result($sql, 0, 'id');
// Update the record
mysql_query("UPDATE `table` SET `today` = 1 WHERE `id` = {$id}");
// ------------
// Now you can find again your "random found record":
$query = mysql_query("SELECT * FROM `table` WHERE `today` = 1");

Categories