I'm writing a realtime wep application, something similar to auction site. The problem is that I need a daemon script, preferrably php, that runs in background and constantly launches queries to mysql db and basing on some of criterias (time and conditions from resultsets) updates other tables. Performance of the daemon is crucial. Sample use case: we have a deal that is going to expire in 2:37 minutes. Even if nobody is watching/bidding it we need to expire it exactly in 2:37 since the time it started.
Can anybody advise a programming technology/software that performs this kind of task the best?
Thanks in advance
UPDATED: need to perform a query when a deal expires, no matter if it has ever been accessed by a user or not.
Why do you need to fire queries at time intervals? Can't you just change how your frontend works?
For example, in the "Deals" page, just show only deals that haven't expired - simplified example:
SELECT * FROM Deal WHERE NOW() <= DateTimeToExpire
Accordingly for the "Orders" page, a deal can become a placed order only if time hasn't expired yet.
Does your daemon need to trigger actions instantaneously? If you need a table containing the expired state as a column you could just compute the expire value on the fly or define a view? You could then use a daemon/cron job querying the view every 10 minutes or so if you have to send out emails or do some cleanup work etc.
Related
I'm working on a basic lamp(willing to change) website , and I currently need a way to run some function on the server that runs for several hours per user, and every X hours it needs to query the mysql database to see if the value for that user has been updated, if it hasn't it need it to insert a new record in the database...I also should mention that the 'every X hours' can change per user too, and the total runtime of the function per user can also vary.
So basically I need a function that runs continuously on the server for few hours per user. What is the best way to do this? I want the site to be able to support many users (like 10000 +).
I'm willing to try new technologies for every aspect of the site, I'm still in the design phase and I was looking for some input.
I've looked at cron but not really sure how well it would work when dealing with so many users...
edit: Here is a typical scenario of events;
User presses button on the website and closes the browser.
Server starts a timer from when they pressed the button, now
the server will check if that user has pressed a different button within a given time frame (time frame can change per user), say within 30 minutes. If they didn't press the other button then the server needs to automatically insert a new record in the database.
The script will need to continue running, checking every 30 mins for say the next 5 hours.
Thank you!
Cron would work as well as you can code the page it will run. It's not a cron limitation.
The question is ambiguous btw. Maybe explaining your full scenario would help.
Meanwhile, my suggestion would be to set up a scrip that allows you to manually check what you need to check.
You definitely need the DB to be InnoDB optimized with proper indexes to be able to support 1000 plus users.
To alleviate the number of calls to the database, a common practice is to run scripts only on what you are interested (so in the case of users you would only select those who have logged on in say the past 3 hours)
That's achievable in 2 ways, a simple select statement, or by adding entries to a specific table on the login page, and remove them after the automated script has finished running.
All of this is pure theory without understanding exactly what you need to do though.
You are telling what/how you want to do, but not why you want to do it. Maybe letting us know why could lead to a different how ;)
However, what you can do is still use cron (or anything similar). The trick is to have
a last_interaction timestamp column
a maximum_interval column
a daily_runtime column
in your users database. Not optimized but you are in the design phase so you shouldn't pay too much attention to the performance aspect (except is explicitly required).
I'm developing a web game (js php mysql) in which one clicks a button to start an action that takes time to complete (let's say 10 hours) and when it finishes some points are added to that player's total.. The problem is that I need those points to be added even if the player is not online at the time the action finishes.. for example I need to have the rankings updated, or an email sent to the player..
I thought about a cron job checking constantly for ending actions, but I think that would kill the resources (contantly checking against actions of thousands of players..).
Is there a better solution to this problem?
Thanks for your attention!!
You can just write into your database when it's finished and when the user logs in you add the earned points to his account. You can also check with a cronjob. Even if you have millions of user this will not kill your server.
Cron is perfect for this. You could write your tasks in stored procedures, then have cron run an SQL script to call the stored procedure that would update the records of your players.
Databases are designed to work with thousands and millions of pieces of information efficiently, so I don't think the idea that it will kill system resources is a valid one unless you hosting system is really constrained already.
If you want to be safe against cheating you need to do the checking on the server anyway. If the "waiting" will happen within a Javascript on the client, one could easily decrease the remaing time.
So you need to send the job to the server (which is assumed to be safe against clock modifications) and the server will determine the end timestamp. You could store your jobs in a queue.
If you only need this information for the user himself you can just look at the queue when the user logs in. Otherwise run a cron job every minute (or so). This job will mark all jobs finished when their timestamp is in the past (and remove them from the database).
If you need more precise checking you will need to come up with an alternative server side solution that is doing this more often (e.g. a simple program polling the database every few seconds).
I've search on the web and apparently there is no way to launch a php script without user interaction.
Few advisors recommend me Cron but I am not sure this is the right way to go.
I am building a website where auctions are possible just like ebay. And after an amount of time the objects are not available anymore and the auction is considered as finished.
I would like to know a way to interact with the database automatically.
When do you need to know if an object is available? -> Only if someone asks.
And then you have the user interaction you are searching for.
It's something different if you want to, let's say, send an email to the winner of an auction. In this case you'd need some timer set to the ending time of the auction. The easiest way to do this would be a cron job...
There are several ways to do this. Cron is a valid one of them and the one I would recommend if its available.
Another is to check before handling each request related to an object whether it is still valid. If it is not, you can delete it from the database on-the-fly (or do whatever you need to) and display a different page.
Also you could store the time at which your time-based script was run last in the database and compare that time with the current time. If the delay is large enough, you can run your time based code. However, this is prone to race conditions if multiple users hit the page at the same time, so the script may run multiple times (maybe this can be avoided using locks or anything though).
To edit cronjobs from the shell: crontab -e
A job to run every 10 minutes: */10 * * * * curl "http://example.com/finished.php"
TheGeekStuff.com cron Examples
Use heartbeat/bot implement
ation
Cron job that runs pretty frequently or a program that starts on boot and runs continuously (maybe sleeping periodically) is the way to go. With a cron job you'll need to make sure that you don't have two running at any given time or write it such that it doesn't matter if you have more than one working at any given time. With "resident" program you'll need to figure out how to handle the case when it crashes unexpectedly.
I wouldn't rely on this mechanism to actually close the auction, though. That should be handled in your database/web site. That is, the auction has a close time and either the database constraints or your code makes it impossible to bid on a closed auction. Notifying the winner and seller, setting up the payment process, etc. are things your service/scheduled task could do.
I am developing a web application (PHP/MySQL), in which I need to implement timers (record id expires at expiration_date), where expiration involves the record's state being updated and arbitrary code being executed as required.
I basically need a BPMN Timer event.
Options I have considered, none of which I am thrilled with:
Cronjob calling a function in the application that just queries for and updates expired records.
At a commonly called point in the code, call this function at a fixed interval.
How is this commonly done in PHP applications?
The following is an option which you have not considered:
You could use MySQL Event Scheduler to run a stored expiration routine within the database periodically.
Some good things about this approach:
Platform independent, works the same way on UNIX and Windows.
Easy to set-up. No need to explain to the user how to setup cron jobs. You just create the event together with your database schema.
Downsides:
Not enabled by default in MySQL. Need to put event_scheduler=on in my.cnf or otherwise enable it.
Requires MySQL 5.1 or newer if I remember correctly.
Example:
DROP EVENT IF EXISTS expire_event;
DELIMITER //
CREATE EVENT expire_event
ON SCHEDULE EVERY 1 MINUTE
DO BEGIN
DELETE FROM data WHERE time < UNIX_TIMESTAMP(NOW() - INTERVAL 1 HOUR);
END //
DELIMITER ;
The above will every minute delete any rows from data table which have a UNIX time stamp older than 1 hour.
I would also choose the cronjob option.
I would also, however, have logic in my app that knows how to deal (or ignore) expired records. Services like cron can break, so it's nice to NOT have to rely upon it.
Unfortunately, DBMSs don't give us quite this sort of flexibility, and you won't want to go off and do database maintenance in the event you come across one of these "expired" records.
You're best excluding them through queries, the use of views, or simple application logic.
I would choose the cronjob option.
I need to show some basic stats on the front page of our site like the number of blogs, members, and some counts - all of which are basic queries.
Id prefer to find a method to run these queries say every 30 mins and store the output but im not sure of the best approach and I don't really want to use a cron. Basically, I don't want to make thousands of queries per day just to display these results.
Any ideas on the best method for this type of function?
Thanks in advance
Unfortunately, cron is better and reliable solution.
Cron is a time-based job scheduler in Unix-like computer operating systems. The name cron comes from the word "chronos", Greek for "time". Cron enables users to schedule jobs (commands or shell scripts) to run periodically at certain times or dates. It is commonly used to automate system maintenance or administration, though its general-purpose nature means that it can be used for other purposes, such as connecting to the Internet and downloading email.
If you are to store the output into disk file,
you can always check the filemtime is lesser than 30 minutes,
before proceed to re-run the expensive queries.
There is nothing at all wrong with using a cron to store this kind of stuff somewhere.
If you're looking for a bit more sophisticated caching methods, I suggest reading into memcached or APC, which could both provide a solution for your problem.
Cron Job is best approach nothing else i seen feasible.
You have many to do this, I think the good not the best, you can store your data on table and display it every 30 min. using the function sleep()
I recommend you to take a look at wordpress blog system, and specially at the plugin BuddyPress..
I did the same some time ago, and every time someone load the page, the query do the job and retrieve the information from database, I remenber It was something like
SELECT COUNT(*) FROM my_table
and I got the number of posts in my case.
Anyway, there are so many approach. Good Luck.
Dont forget The cron is always your best friend.
Using cron is the simplest way to solve the problem.
One good reason for not using cron - you'll be generating the stats even if nobody will request them.
Depending on the length of time it takes to generate the data (you might want to keep track of the previous counts and just add counts where the timestamp is greater than the previous run - with appropriate indexes!) then you could trigger this when a request comes in and the data looks as if it is stale.
Note that you should keep the stats in the database and think about how to implement a mutex to avoid multiple requests trying to update the cache at the same time.
However the right solution would be to update the stats every time a record is added. Unless you've got very large traffic volumes, the overhead would be minimal. While 'SELECT count(*) FROM some_table' will run very quickly you'll obviously run into problems if you don't simply want to count all the rows in a table (e.g. if blogs and replies are held in the same table). Indeed, if you were to implement the stats update as a trigger on the relevant tables, then you wouldn't need to make any changes to your PHP code.