Here is the scenario.
I have a schedule job running every minute which inserts data into a MYSQL table "demo". The total number of records per day are 60*24 = 1440.
Table demo already has 55000 records.
I want to clean records less than today's date. Therefore I am using below code to do the work daily at 10.00 AM.
$demo = Demo::whereDate('created_at','<', Carbon::today());
if(count($demo) > 0)
{
$demo->delete();
}
Now a point will come where at the same time I am inserting to the same table and deleting from the same table.
I want to know that it will be safe? Or there will be an error or any other impact.
I don't think this will be an issue, since Carbon::today() returns 00:00:00 as time and you are executing the deletion job at 10:00:00. Only records that are inserted more than 10 hours ago will be deleted.
According to me, there should not be any problem. If there are two requests at the same time MySQL server will handle them by itself. The same thing happens when a website is loaded. There is a lot of call at the same time.
There is no problem. Imagine you made 1000 requests per minutes, no one of them will overwrites an other one.
Related
Not sure the best way to describe what I'm trying to do, so bear with me. I'm working in PHP with a mysql database.
I have a database of 10,000 records. Suppose I want to update 10000 / 365 each day throughout the year. Each record gets updated 1x per year, and nicely spread throughout the year.
One easy way to do this is to select all records, then for each, if ID % 365 = $day_of_year, update that record. I'm not worried about leap year.
Is there a way I can select only the records from the database that I need (around 27), rather than selecting all 10,000 and looping through each? This is a cron job that I will run in the middle of the night, so maybe this is a moot point. Still, it bugs me that I have to brute force my way through all 10,000. Would love to find a more elegant solution by only pulling the tiny fraction needed.
Add a column to your table indicating the day of the year the record should be updated.
Then add an event that runs once a year that resets that column values and calculates it new - 10000 records spread on the days of the year.
Then add another event that runs every night updating the records for the day.
Thanks for the response, especially Juergen and Mike Pomax. Here's the solution.
I added a new column and populated each row with a random value between 0 and 364. I could have populated it some other way but this was easy and close enough.
Here is my pseudocode:
$day_of_year = date('z');
if($day_of_year < 365){ //don't do on Dec 31 of leap year
$qry = "Select ... WHERE ID = $day_of_year";
... //do my thing
}
Works great. Thanks again.
I am trying to set up a database to record the last 30 days of information for each user. The data will be recorded once a day (i.e. by a cron job) and will be the value of an item (i.e. constantly changes).
What would be the best way to structure this? I was thinking of setting a table and then just storing the 30 days in the table and deleting the 31st day as I add the new day with the cron job (and shifting all of the others up one day) but this doesn't seem very efficient..
Thanks for the help.
What you can do is store the current date with each entry, then in your cron job, delete all entries that are greater than thirty days old.
For example (with MySQL),
DELETE FROM user_statistics WHERE DATEDIFF(NOW(), date_of_record) > 30;
Store the user data with its own date and delete the oldest when you exceed your limit. No need to shift anything.
I'd log by actual date using a DATE column. You can query up "last 30 days" pretty easily in MySQL.
As for purging, the cron job can delete anything older than 30 days pretty easily as well. Or, since it's so easy to ignore anything older than 30 days, you might even choose to not delete older records (at least not every day).
I want to make a table where the entries expire 24 hours after they have been inserted in PHP and MySQL.
Ideally I want to run a "deleting process" every time a user interacts with my server, that deletes old entries. Since this is more frequent you should it will not have large amounts of data to delete so it should only take a few milliseconds.
I have given each entry a date/time added value.
How would I do this?
You could use MySQL's event scheduler either:
to automatically delete such records when they expire:
CREATE EVENT delete_expired_101
ON SCHEDULE AT CURRENT_TIMESTAMP + INTERVAL 24 HOUR DO
DELETE FROM my_table WHERE id = 101;
to run an automatic purge of all expired records on a regular basis:
CREATE EVENT delete_all_expired
ON SCHEDULE EVERY HOUR DO
DELETE FROM my_table WHERE expiry < NOW();
you shouldn't do a delete process when a user interacts. it slows down things, you should use a cronjob (every minute / hour)
you'll want to index the added timestamp value and then run DELETE FROM table WHERE added < FROM_UNIXTIME(UNIX_TIMESTAMP()-24*60*60)
maybe you'll want to checkout Partitions, which divide the table into different tables, but it behaves as one table. The advantage is that you don't need to delete the entries and you'll have seperate tables for each day.
i think that YOU think that much data slows down tables. Maybe you should use EXPLAIN (MySQL Manual) and optimize your SELECT queries using indexes (MySQL Manual)
UPDATE Check out eggyal's answer - This is another approach worth taking a look.
You can look into using Cron Job, http://en.wikipedia.org/wiki/Cron Make it run once every 24 hours when it matches your requirement.
This will help
Delete MySQL row after time passes
Database Values For User Return After Prolonged Amount Of Time?
So I have values in my database e.g. stamina, max_stamina, ...
What I want to do is this:
If stamina is < max_stamina then
after 60 seconds stamina=stamina+1
else
nothing
I will need to loop this untill stamina equals max_stamina
I can easily create up the if statement, the problem I need help with is coding the 60 seconds, how could i go about it?
All so this will need to run when the player is and isnt logged in.
take a look at mysql events and schedules think that is what you are looking for. Also if you are having more than 1 users it would be smart to update them all in that 60 seconds interval you need.
Without knowing more about your code architecture, you most likely will need a cronjob to run every minute and to update all stamina records according to your criteria.
I'm building a toplist for my site and instead of recording in and out hits and emptying the database every 24 hours I would like to record in and out hits "within the last 24 hours", this way the toplist will always look busy no matter what time you look at it, while still acurately ranking sites.
So basically I'll have to add a record to the database logging an 'in' hit (for example) and the remove it around 24 hours later, or simply not consider it when ranking the sites.
I want to know the most efficient way of doing this, I have some ideas but they all seem really resource heavy. I hope you understand what I'm trying to achieve :) Any help would be appreciated.
You don't necessarily have to delete "old" data all the time. You can also limit the data set to records within the last 24 hours when querying data.
WHERE
site='xyz'
AND hit_datetime > Now()-Interval 24 hour
see http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html
I would delete the data that is older than 24 hours with a simple
DELETE...WHERE hit_time < now() - interval 24 hour
The other question is - when to call it? The tradeoff is between performance and stale data. The more often you call it, the less "stale" data there will be, but the server load will grow.
I see several approaches, pick one that suits your needs most:
Call it at the start of every script. This can be optmized by calling it only if the script will do something with the hit data. That way every script will always run with "correct" data. However this will have the maximum load.
Schedule a cron job and call it once every 1h/2h/24h/etc. This way there will be a little bit of "stale" data, but the overhead will be reduced to a minimum.
Do it like PHP does it with sessions - on every script startup give it a x% (x is configurable) chance of being run. That is, take a value from 0 to 100, and if it is smaller than x, execute the DELETE.
You can also invent some other schemes - for example, run it once per user session; or run it only if the execution time is evenly divisable by, say, 7; or something else. Either way you trade off performance for correctness.
Write a Stored Procedure that deletes records older than 24 hours. Then write a trigger that runs on every INSERT statement and calls the SP.
you could store the timestamp with each "hit" and then call a query like
$time = time()-86400;
mysql_query("DELETE FROM xxx WHERE timestamp < $time");
or you could same thing within the SELECT statement, depends on if you still need the hits afterwards, etc
If the time-constraint is not really hard (e.g. you'll loose money or are really annoying your users if the data is kept in the the db longer than 24 hours), I'd use use PHP's register_shutdown_function like this:
function cleanup() {
// open db-connection etc.
$query = 'DELETE FROM <yourtable> ' .
'WHERE UNIX_TIMESTAMP(<timstampfield>) < ' . (time() - 86400);
mysql_query($query);
// close connection
}
register_shutdown_function('cleanup');
The above code assumes, <timestampfield> is of one of the the MYSQL-date-datatypes (TIMESTAMP, DATE, DATETIME).