I have created a script on PHP that creates cache files from API and it takes around 30 minutes to load the page completely means when it creates all cache files.
I have a concern that my hostinger's customer support is telling me that it won't run for 30 minutes but in some answers, I found that it can run in the background and nothing to worry about until it's loaded.
So is that possible that the cronjob will run up to 30 minutes?
If not what is the best solution to run that cache making script at a specific time in the background like the cronjob does? Please Explain in brief so I can get a way.
Thanks for the great answer.
Ideally, for long running tasks, the task should be hosted in a platform that allows extended operations and defined in a way that it can be externally triggered, this might be in the form of an endpoint in a web API.
Then you can use the cronjob to trigger that process.
Without creating a whole API, you could make this a single endpoint on your website, a hidden page that only the cronjob knows how to call, then run your script from there.
There are lots of ways around this but the methodology is similar just use the cronjob as the trigger to a different process. Move the core logic of your script to a platform that allows the long execution time.
This is a similar post: Run a “long” php-script via Cronjob with an answer that suggests you can try to execute the script without waiting for the response, that is the same expectation with calling an external web process or API, the cronjob should not wait for a response.
It's good practice to limit resources on web server, especially in the shared hosting account. Because, in most cases, it may cause the web server to slow down and Denial of Services situation.
It's recommended to run the script using php-cli and cron.
php-cli offer much more relaxation, such as time and resource limitation. Please also read
Events in MariaDB VS Cron in php - which is better
Related
I need to write a cron (php) script to get the html result from several websites.
Let's say my database has 50 websites records (ie. http://www.somewebsite.com/page.php). So the cron job will be set to run every x mins. When it's running, it will load the records from database, check the status of each websites then get the HTML result back from it then analysis it.
My concern is, if the website from n'th record is not responding, or it takes some time to load (ie. oversea website), then n+1's record won't be run, not until n'th record is finished, then this cron job will take a while to finish.
If I execute the script on a browser, then it can be easily handled by using ajax async, however it's a cron job, so I have no idea how to handle this situation.
Here is what you can do,
Run the sh script from crontab and in the script invoke .php program that takes care of async tasks.
I think its better to move to other language if there is need of 'Async'. It's a major upgrade to how you would create the overall system architecture. So this is a major upgrade to any PHP developer as myself spending a lot in PHP and looking into NodeJS for better solutions. And PHP does not support such need internally or from the core yet, although the term 'Aync' is introduced.
I have a web-app, written in PHP, and I am using Symfony2 as the core framework.
I need to regularly run thousands of small network requests every 10 minutes or so and I am trying to find the best solution for asynchronously running these jobs, without conflicting or doubling up.
Currently I have a very basic and inelegant solution where a cron job executes a PHP command script. The command synchronously works through each entry in the database and sends a network request. When that request completes (or fails), it moves on to the next one. When it has iterated over all entries, it exists, to be executed again from the cron job.
For the rewrite, I have looked at php-resque and pcntl_fork as solutions for running jobs in parallel, thereby speeding up the execution significantly. I have also looked at running multiple non-blocking socket requests from PHP but, so far, have preferred the simplicity of isolated jobs.
PHP can't do threading in the traditional sense, so what you're trying to do isn't really possible in the way that you're thinking. You've looked at the best solution (probably) in pcntl_fork, but that still won't be truly asynchronous. Have you considered using cron to accomplish this instead?
http://php.net/pthreads
http://github.com/krakjoe/pthreads
Examples on github... and included in distribution, use code in github, it contains fixes not yet released.
I am developing a website that requires a lot background processes for the site to run. For example, a queue, a video encoder and a few other types of background processes. Currently I have these running as a PHP cli script that contains:
while (true) {
// some code
sleep($someAmountOfSeconds);
}
Ok these work fine and everything but I was thinking of setting these up as a deamon which will give them an actual process id that I can monitor, also I can run them int he background and not have a terminal open all the time.
I would like to know if there is a better way of handling these? I was also thinking about cron jobs but some of these processes need to loop every few seconds.
Any suggestions?
Creating a daemon which you can make calls to and ask questions would seem the sensible option. Depends on wether your hoster permits such things, especially if you're requiring it to do work every few seconds, then definately an OS based service/daemon would seem far more sensible than anything else.
You could create a daemon in PHP, but in my experience this is a lot of hard work and the result is unreliable due to PHP's memory management and error handling.
I had the same problem, I wanted to write my logic in PHP but have it daemonised by a stable program that could restart the PHP script if it failed and so I wrote The Fat Controller.
It's written in C, runs as a daemon and can run PHP scripts, or indeed anything. If the PHP script ends for whatever reason, The Fat Controller will restart it. This means you don't have to take care of daemonising or error recovery - it's all handled for you.
The Fat Controller can also do lots of other things such as parallel processing which is ideal for queue processing, you can read about some potential use cases here:
http://fat-controller.sourceforge.net/use-cases.html
I've done this for 5 years using PHP to run background tasks and its no different to doing in any other language. Just use CRON and lock files. The lock file will prevent multiple instances of your script running.
Also its important to monitor your code and one check I always do to prevent stale lock files from preventing scripts to run is to have second CRON job to check if if the lock file is older than a few minutes and if an instance of the PHP script is running, if not it then removes the lock file.
Using this technique allows you to set your CRON to run the script every minute without issues.
Use the System::Daemon module from PEAR.
One solution (that I really need to try myself, as I may need it) is to use cron, but get the process to loop for five mins or so. Then, get cron to kick it off every five minutes. As one dies, the next one should be finishing (or close to finishing).
Bear in mind that the two may overlap a bit, and so you need to ensure that this doesn't cause a clash (e.g. writing to the same video file). Some simple inter-process communication may be useful, even if it is just writing to a PID file in the temp directory.
This approach is a bit low-tech but helps avoid PHP hanging onto memory over the longer term - sort of in-built task restarts!
I would like to add an intensive task (lets say 5 minutes execution time) into Wordpress using cron job.
I been using this code to add new cron task inside the Wordpress system.
wp_schedule_event(time(), "interval-name", "hook-name");
I read somewhere in the net that cron task will be executed when there is request hit the Wordpress (either in the public site or the admin). Can anybody acknowledge that is true?
If that the case then I should not put my intensive task into cron task, because it will make user wait for long time after the task finished. What should I do now?
Anybody experienced this situation? Any suggestion?
I think to create a new page to be executed by crontab (for example http://example.com/wp-content/plugins/plugin-example/intensive-task.php)
The wordpress documentation says that it will be run when someone visits your site, so yes, you're correct. It will only be one user that gets a slow page load, so it's up to you if you want to avoid that.
If you are running it from a regular con job, there's no need to make it a page on your site though; especially if it's an intensive job, as you say, then this could easily be exploited place a large load on your server. You can easily run php from the command line to execute your job safely and without causing any slow load times on your page.
If you would use regular cronjob that wouldn't be the case
but i suspect that wp does what you said, since that would make it versatile working in different hosts with different setups as long as they have php and mysql running independent from real cronjobs which must be installed by the web host separately
i wonder how can i schedule and automate tasks in PHP? can i? or is web server features like cron jobs needed.
i am wondering if there is a way i can say delete files after say 3 days when the file are likely outdated or not needed
PHP natively doesn't support automating tasks, you have to build a solution yourself or search google for available solutions. If you have a frequently visited site/page, you could add a timestamp to the database linking to the file, when visiting your site in a chosen time (e.g. 8 in the morning) the script (e.g. deleteOlderDocuments.php) runs and deletes the files that are older.
Just an idea. Hope it helps.
PHP operates under the request-response model, so it won't be the responsibility of PHP to initiate and perform the scheduled job. Use cron, or make your PHP site to register the cron jobs.
(Note: the script that the job executes can be written in PHP of course)
In most shared hosting environments, a PHP interpreter is started for each page request. This means that for each PHP script in said environment, all that script will know about is the fact that it's handling a request, and the information that request gave it. Technically you could check the current time in PHP and see if a task needs to be performed, but that is relying on a user requesting that script near a given time.
It is better to use cron for such tasks. especially if the tasks you need performed can be slow -- then, every once in a while, around a certain time, a user would have a particularly slow response, because them accessing a script caused the server to do a whole bunch of scheduled stuff.