I need to write a cron (php) script to get the html result from several websites.
Let's say my database has 50 websites records (ie. http://www.somewebsite.com/page.php). So the cron job will be set to run every x mins. When it's running, it will load the records from database, check the status of each websites then get the HTML result back from it then analysis it.
My concern is, if the website from n'th record is not responding, or it takes some time to load (ie. oversea website), then n+1's record won't be run, not until n'th record is finished, then this cron job will take a while to finish.
If I execute the script on a browser, then it can be easily handled by using ajax async, however it's a cron job, so I have no idea how to handle this situation.
Here is what you can do,
Run the sh script from crontab and in the script invoke .php program that takes care of async tasks.
I think its better to move to other language if there is need of 'Async'. It's a major upgrade to how you would create the overall system architecture. So this is a major upgrade to any PHP developer as myself spending a lot in PHP and looking into NodeJS for better solutions. And PHP does not support such need internally or from the core yet, although the term 'Aync' is introduced.
Related
I have created a script on PHP that creates cache files from API and it takes around 30 minutes to load the page completely means when it creates all cache files.
I have a concern that my hostinger's customer support is telling me that it won't run for 30 minutes but in some answers, I found that it can run in the background and nothing to worry about until it's loaded.
So is that possible that the cronjob will run up to 30 minutes?
If not what is the best solution to run that cache making script at a specific time in the background like the cronjob does? Please Explain in brief so I can get a way.
Thanks for the great answer.
Ideally, for long running tasks, the task should be hosted in a platform that allows extended operations and defined in a way that it can be externally triggered, this might be in the form of an endpoint in a web API.
Then you can use the cronjob to trigger that process.
Without creating a whole API, you could make this a single endpoint on your website, a hidden page that only the cronjob knows how to call, then run your script from there.
There are lots of ways around this but the methodology is similar just use the cronjob as the trigger to a different process. Move the core logic of your script to a platform that allows the long execution time.
This is a similar post: Run a “long” php-script via Cronjob with an answer that suggests you can try to execute the script without waiting for the response, that is the same expectation with calling an external web process or API, the cronjob should not wait for a response.
It's good practice to limit resources on web server, especially in the shared hosting account. Because, in most cases, it may cause the web server to slow down and Denial of Services situation.
It's recommended to run the script using php-cli and cron.
php-cli offer much more relaxation, such as time and resource limitation. Please also read
Events in MariaDB VS Cron in php - which is better
I have a PHP script that needs to be executed by the end of every month. How would I do that without using cron on Unix-like systems or Task Scheduler on Windows systems? Isn't there some built-in feature in php for this? Because using cron/scheduler is not that much portable way IMO.
The only other way is by triggering it when the website is loaded. For example, you'd store the time the script was last run. Then every time a page is loaded, you'd check if it's time to run it. If so, then you would run it as part of the page load. Of course, this depends on (1) the task being a very quick one, and (2) that the task doesn't have to be run exactly on schedule, down to the second. This is basically how wp_cron() works in Wordpress.
Php is just a scripting language, not a process that is aware of time and may trigger events at desired moments.
Once you settle in a server, you don't have to worry about portability: migrating to another place won't be an automatic process because the environment could be very different.
So stick with cron, or write yourself a OS-agnostic layer.
In the system i worked on, i had a little module that had the last running of the script saved, and on every running of the main script, it checked, if the script shouldn't have been run. In case it should have, it did before anything else executed, so the system had the right data anyway, even if the periodical script wasn't run "on time". It also checked if the script shouldn't have been run more than once, and ran it several times, if needed.
A bit crude, but produces the right results without anything but PHP/mysql.
I would like to add an intensive task (lets say 5 minutes execution time) into Wordpress using cron job.
I been using this code to add new cron task inside the Wordpress system.
wp_schedule_event(time(), "interval-name", "hook-name");
I read somewhere in the net that cron task will be executed when there is request hit the Wordpress (either in the public site or the admin). Can anybody acknowledge that is true?
If that the case then I should not put my intensive task into cron task, because it will make user wait for long time after the task finished. What should I do now?
Anybody experienced this situation? Any suggestion?
I think to create a new page to be executed by crontab (for example http://example.com/wp-content/plugins/plugin-example/intensive-task.php)
The wordpress documentation says that it will be run when someone visits your site, so yes, you're correct. It will only be one user that gets a slow page load, so it's up to you if you want to avoid that.
If you are running it from a regular con job, there's no need to make it a page on your site though; especially if it's an intensive job, as you say, then this could easily be exploited place a large load on your server. You can easily run php from the command line to execute your job safely and without causing any slow load times on your page.
If you would use regular cronjob that wouldn't be the case
but i suspect that wp does what you said, since that would make it versatile working in different hosts with different setups as long as they have php and mysql running independent from real cronjobs which must be installed by the web host separately
i wonder how can i schedule and automate tasks in PHP? can i? or is web server features like cron jobs needed.
i am wondering if there is a way i can say delete files after say 3 days when the file are likely outdated or not needed
PHP natively doesn't support automating tasks, you have to build a solution yourself or search google for available solutions. If you have a frequently visited site/page, you could add a timestamp to the database linking to the file, when visiting your site in a chosen time (e.g. 8 in the morning) the script (e.g. deleteOlderDocuments.php) runs and deletes the files that are older.
Just an idea. Hope it helps.
PHP operates under the request-response model, so it won't be the responsibility of PHP to initiate and perform the scheduled job. Use cron, or make your PHP site to register the cron jobs.
(Note: the script that the job executes can be written in PHP of course)
In most shared hosting environments, a PHP interpreter is started for each page request. This means that for each PHP script in said environment, all that script will know about is the fact that it's handling a request, and the information that request gave it. Technically you could check the current time in PHP and see if a task needs to be performed, but that is relying on a user requesting that script near a given time.
It is better to use cron for such tasks. especially if the tasks you need performed can be slow -- then, every once in a while, around a certain time, a user would have a particularly slow response, because them accessing a script caused the server to do a whole bunch of scheduled stuff.
I have a PHP script that needs to be run at certain times every weekday. Is cron or Windows task scheduler the only way to do this?
Is there a way to set this up from within another PHP script?
Depends how exact the timing needs to be. A technique I've seen used (dubbed "poor man's cron") is to have a frequently accessed script (a site's home page, for example) fire off a task on the first page load after a certain time (kept track of in the database).
If you need any sort of guaranteed accuracy, though, cron or a Windows scheduled task is really the best option. If your host doesn't support them, it's time to get a better one.
Apart from cron or Scheduled Tasks, you can have your PHP script to always run it. The process should sleep (within a loop) until the time has reached. Then, the process could execute the remaining functions/methods. Or, you can set up another script (which will act as a daemon) to check for the time and execute the other script.
Well since the web is a pull mechanism you have to have some sort of action that will trigger a PHP script to execute. cron is an option on *nix and task scheduler on windows. You could also write your own service that has a timer but only if needed, this is common on windows services for updaters, jobs etc.
One way you could do it is in the cron task just call a php script for each action needed. Or one php script that executes other tasks. The problem with web based tasks though such as PHP is timeouts. Make sure your tasks are under 60-90 seconds. If not you might look at using python , perl or ruby or even bash scripts to do the work rather than the PHP script.
cron seems like the best option for you though. You will have to call your script with wget. There are examples here: http://www.thesitewizard.com/general/set-cron-job.shtml
For instance this runs the script everyday at 11:
30 11 * * * /usr/bin/wget http://www.example.com/cron.php
Cron, of course, is by far the best way to schedule anything on *nix.
If this is in a remote server you do not have cron access to, you can setup cron/windows scheduler on your computer, to open a web browser to the page that contains the script you wish to run
You probably want to use cron (or windows scheduled tasks).
If you really wanted, you could set up another php script to run continuously with an infinite loop (with a sleep command inside the loop, say for 30 seconds or so) and then when you reach your desired day/time execute the other script via a shell command call. While possible, I can't think if a single good reason to use this method rather than cron/scheduled tasks
You can write a long running script that runs your main script in predefined times but it will be very unnecessary, error prone, and it will basically be a "cron rewrite in phph".
Using the real cron itself will be easier and a more robust solution. If you are packaging an application, you can put a file in /etc/cron.d which contains a single cron line running your application.
You'll need to use a cron job (under Linux/Unix) or a scheduled task under Windows. You could have another script running on a continuous basis which checks the time and executes a script at a specified interval, but using the OS-supplied mechanism is easier to manage and resilient to restarts, etc.
The Uniform Server project has some good suggestions on mimicking cron in environments where cron is unacceptable. Still though, if cron is at all an option, use it.