I've been looking for the answer to this question for some time now and I find mixed answers.
Are cron jobs heavy and expensive processes that consume a lot of resources? Or are they basically hits to that page (regardless of the script that executes when cron triggers it)
I intend to use several cron jobs for several sites. Let's say I have 3 different cron jobs that'll hit certain pages every minute for 10 sites.
Does anyone have 10's or 100's of cron jobs like this in a triggering manner much like a browser hit (-wget...>/dev/null 2>&1)? If you do, do you experience additional load?
Further explanation; as you might know, WP-cron events do not take place if noone visits the WordPress site at the time of the scheduled event, until after someone comes along and triggers it.
I have some not very active WordPress sites that I plan to carry out scheduled events and I am trying to get it right.
What do you think of the online cron services ? Do they exist just because most shared users are not allowed to create cron jobs? Or is it because cron jobs slow the server down and this way you can take some load off your server?
Are cron jobs heavy and expensive processes that consume a lot of resources?
Not unless you make them like that. The cron process itself is very lightweight. All it's going to do is invoke your script. If your script is a heavy and expensive process, that has nothing to do with cron.
are they basically hits to that page
Kind of an odd metaphor, but I suppose so. The cron job executes the script. If that script is also used as a web page in some sense, then yes the two scenarios are comparable. (In fact, cron invoking the script is probably less resource-intensive than a web server invoking the script.) Though I recommend separating your webpage code from command-line code. (Unless your cron task is invoking an HTTP request to a page, such as using wget or something of that nature. In which case it has nothing to do with the "page" and is just invoking a command-line utility.)
What do you think of the online cron services? Do they exists just because most shared users are not allowed to create cron jobs? Or is it because cron jobs slow the server down and this way you can take some load off your server?
The former sounds more plausible. cron isn't resource-intensive. But it does require access that some shared hosting providers don't provide.
Using a cron job to hit a web page to trigger application processes is not heavy.
It is a bit of a long winded approach but the act of fetching the page itelf (wget etc) is not heavy.
The heaviness of your application process is another matter entirely of course.
Related
I have created a script on PHP that creates cache files from API and it takes around 30 minutes to load the page completely means when it creates all cache files.
I have a concern that my hostinger's customer support is telling me that it won't run for 30 minutes but in some answers, I found that it can run in the background and nothing to worry about until it's loaded.
So is that possible that the cronjob will run up to 30 minutes?
If not what is the best solution to run that cache making script at a specific time in the background like the cronjob does? Please Explain in brief so I can get a way.
Thanks for the great answer.
Ideally, for long running tasks, the task should be hosted in a platform that allows extended operations and defined in a way that it can be externally triggered, this might be in the form of an endpoint in a web API.
Then you can use the cronjob to trigger that process.
Without creating a whole API, you could make this a single endpoint on your website, a hidden page that only the cronjob knows how to call, then run your script from there.
There are lots of ways around this but the methodology is similar just use the cronjob as the trigger to a different process. Move the core logic of your script to a platform that allows the long execution time.
This is a similar post: Run a “long” php-script via Cronjob with an answer that suggests you can try to execute the script without waiting for the response, that is the same expectation with calling an external web process or API, the cronjob should not wait for a response.
It's good practice to limit resources on web server, especially in the shared hosting account. Because, in most cases, it may cause the web server to slow down and Denial of Services situation.
It's recommended to run the script using php-cli and cron.
php-cli offer much more relaxation, such as time and resource limitation. Please also read
Events in MariaDB VS Cron in php - which is better
I need to write a cron (php) script to get the html result from several websites.
Let's say my database has 50 websites records (ie. http://www.somewebsite.com/page.php). So the cron job will be set to run every x mins. When it's running, it will load the records from database, check the status of each websites then get the HTML result back from it then analysis it.
My concern is, if the website from n'th record is not responding, or it takes some time to load (ie. oversea website), then n+1's record won't be run, not until n'th record is finished, then this cron job will take a while to finish.
If I execute the script on a browser, then it can be easily handled by using ajax async, however it's a cron job, so I have no idea how to handle this situation.
Here is what you can do,
Run the sh script from crontab and in the script invoke .php program that takes care of async tasks.
I think its better to move to other language if there is need of 'Async'. It's a major upgrade to how you would create the overall system architecture. So this is a major upgrade to any PHP developer as myself spending a lot in PHP and looking into NodeJS for better solutions. And PHP does not support such need internally or from the core yet, although the term 'Aync' is introduced.
I'm putting together my first commercial PHP application, it's nothing really huge as I'm still eagerly learning PHP :)
Right now I'm still in the conceptual stage of planning my application but I run into one problem all the time, the application is supposed to be self-hosted by my customers, on their own servers and will include some very long running scripts, depending on how much data every customer enters in his application.
Now I think I have two options, either use cronjobs, like for example let one or multiple cronjobs run at a time that every customer can set himself, OR make the whole processing of data as daemons that run in the background...
My question is, since it's a self-hosted application (and every server is different)... is it even recommended to try to write php that starts background processes on a customers server, or is this more something that you can do reliably only on your own server...?
Or should I use cronjobs for these long running processes?
(depending on the amount of data my customers will enter in the application, a process could run 3+ hours)
Is that even a problem that can be solved, reliably, with PHP...? Excuse me if this should be a weird question, I'm really not experienced with PHP daemons and/or long running cronjobs created by php.
So to recap everything:
Commercial self-hosted application, including long running processes, cronjobs or daemons? And is either or maybe both also a reliable solution for a paid application that you can give to your customers with a clear conscience because you know it will work reliable on all kinds of different servers...?
EDIT*
PS: Sorry, I forgot to mention that the application targets only Linux servers, so everything like Debian, Ubuntu etc etc.
Short answer, no, don't go for background process if this will be a client hosted solution. If you go towards the ASP concept (Application Service Provider... not Active Server Pages ;)) then you can do some wacky stuff with background processes and external apps connecting to your sql servers and processing stuff for you.
What i suggest is to create a strong task management backbone and link that to a solid task processing infrastructure. I'll recommend you read an old post i did quite some time ago regarding background processes and a strategy i had adopted to fix long running processes:
Start & Stop PHP Script from Backend Administrative Webpage
Happy reading...
UPDATE
I realize that my old post is far from easy to understand so here goes:
You need 2 models: Job and JobQueue, 2 controller: JobProcessor, XYZProcessor
JobProcessor is called either by a user when a page triggers or using a cronjob as you wish. JobProcessor::process() is the key that starts the whole processing or continues it. It loads the JobQueues and asks the job queues if there is work to do. If there is work to do, it asks the jobqueue to start/continue it's job.
JobQueue Model: Used to queue several JOBS one behind each other and controls what job is currently current by keep some kind of ID and STATE about which job is running.
Job Model: Represents exactly what needs to be done, it contains for example the name of the controller that will process the data, the function to call to process the data and a serialized configuration property that describe what must be done.
XYZController: Is the one that contains the processing method. When the processing method is called, the controller must load everything it needs to memory and then process each individual unit of work as fast as possible.
Example:
Call of index.php
Index.php creates a jobprocessor controller
Index.php calls the jobprocessor's process()
JobProcessor::Process() loads all the queues and processes them
For each JobQueue::Process(), the job queue loads it's possible Jobs and detects if one is currently running or not. If none is running, it starts the next one by calling Job::Process();
Job::Process() creates the XYZController that will work the task at hand. For example, my old system had an InvoicingController and a MassmailingController that worked hand in hand.
Job::Process() calls XYZController::Prepare() so that it loads it's information to process. (For example, load a batch of emails to process, load a batch of invoices to create)
Job::Process() calls XYZController::RunWorkUnit() so that it processes a single unit of work (For example, create one invoice, send one email)
Job::Process() asks JobProcessingController::DoIStillHaveTimeToProcess() and if so, continues processing the next element.
Job::Process() runs out of time and calls XYZController::Cleanup() so that all resources are released
JobQueue::Process() ends and returns to JobController
JobController::Process() is about to end? Open a socket, call myself back so i can start another round of processing until i don't have anything to do anymore
Handle the request from the user that start in position #1.
Ultimately, you can instead open a socket each time and ask the processor to do something, or you can queue a CronJob to call your processor. This way your users won't get stuck waiting for the 3/4 work units to complete each time.
Its worth noting that, in addition to running daemons or cron jobs, you can kick off long running processes from a web request (but note that it must run outside of the webserver process group) and of course asynchronous message processing (which is essentially a variant on the batch approach).
All four of these approaches are very different in terms of how they behave, how concurrency and timing are managed. The factors which make them all different are the same ones you omitted from your question - so it's not really possible to answer.
Unfortunately all rely on facilities which are very different between MSWindows and POSIX systems - so although PHP will run on both, if you want to sell your app on both platforms it's going to need 2 versions.
Maybe you should talk to your potential customer base and ask them what they want?
I would like to add an intensive task (lets say 5 minutes execution time) into Wordpress using cron job.
I been using this code to add new cron task inside the Wordpress system.
wp_schedule_event(time(), "interval-name", "hook-name");
I read somewhere in the net that cron task will be executed when there is request hit the Wordpress (either in the public site or the admin). Can anybody acknowledge that is true?
If that the case then I should not put my intensive task into cron task, because it will make user wait for long time after the task finished. What should I do now?
Anybody experienced this situation? Any suggestion?
I think to create a new page to be executed by crontab (for example http://example.com/wp-content/plugins/plugin-example/intensive-task.php)
The wordpress documentation says that it will be run when someone visits your site, so yes, you're correct. It will only be one user that gets a slow page load, so it's up to you if you want to avoid that.
If you are running it from a regular con job, there's no need to make it a page on your site though; especially if it's an intensive job, as you say, then this could easily be exploited place a large load on your server. You can easily run php from the command line to execute your job safely and without causing any slow load times on your page.
If you would use regular cronjob that wouldn't be the case
but i suspect that wp does what you said, since that would make it versatile working in different hosts with different setups as long as they have php and mysql running independent from real cronjobs which must be installed by the web host separately
i wonder how can i schedule and automate tasks in PHP? can i? or is web server features like cron jobs needed.
i am wondering if there is a way i can say delete files after say 3 days when the file are likely outdated or not needed
PHP natively doesn't support automating tasks, you have to build a solution yourself or search google for available solutions. If you have a frequently visited site/page, you could add a timestamp to the database linking to the file, when visiting your site in a chosen time (e.g. 8 in the morning) the script (e.g. deleteOlderDocuments.php) runs and deletes the files that are older.
Just an idea. Hope it helps.
PHP operates under the request-response model, so it won't be the responsibility of PHP to initiate and perform the scheduled job. Use cron, or make your PHP site to register the cron jobs.
(Note: the script that the job executes can be written in PHP of course)
In most shared hosting environments, a PHP interpreter is started for each page request. This means that for each PHP script in said environment, all that script will know about is the fact that it's handling a request, and the information that request gave it. Technically you could check the current time in PHP and see if a task needs to be performed, but that is relying on a user requesting that script near a given time.
It is better to use cron for such tasks. especially if the tasks you need performed can be slow -- then, every once in a while, around a certain time, a user would have a particularly slow response, because them accessing a script caused the server to do a whole bunch of scheduled stuff.