I have a Drupal 7 site with some none Drupal web services that use some Drupal API.
I need to hold a global array with some values that I need to update each minute, that is available to each call to the web service.
I'm new to Drupal and PHP and I'm wondering if should I use pure PHP like:
while(true){
doSomething();
sleep(60);
}
or Drupal cron or something else?
Yes, you should use Drupal's Cron. There's a link to a comprehensive setup video for Drupal's Cron in the link you provided. Using sleep() in an infinite loop is a bad idea because if you are on a shared hosting server, such as GoDaddy, the number of concurrent processes that can be running is limited. So if 20 users are sending requests to your server and 20 PHP processes are sleeping it can cause your server to crash (i.e. HTTP 50x error).
With Cron, you can just save the data you need in a file that is updated by Cron and access the file concurrently (multiple PHP processes) within your PHP script.
Related
I have created a script on PHP that creates cache files from API and it takes around 30 minutes to load the page completely means when it creates all cache files.
I have a concern that my hostinger's customer support is telling me that it won't run for 30 minutes but in some answers, I found that it can run in the background and nothing to worry about until it's loaded.
So is that possible that the cronjob will run up to 30 minutes?
If not what is the best solution to run that cache making script at a specific time in the background like the cronjob does? Please Explain in brief so I can get a way.
Thanks for the great answer.
Ideally, for long running tasks, the task should be hosted in a platform that allows extended operations and defined in a way that it can be externally triggered, this might be in the form of an endpoint in a web API.
Then you can use the cronjob to trigger that process.
Without creating a whole API, you could make this a single endpoint on your website, a hidden page that only the cronjob knows how to call, then run your script from there.
There are lots of ways around this but the methodology is similar just use the cronjob as the trigger to a different process. Move the core logic of your script to a platform that allows the long execution time.
This is a similar post: Run a “long” php-script via Cronjob with an answer that suggests you can try to execute the script without waiting for the response, that is the same expectation with calling an external web process or API, the cronjob should not wait for a response.
It's good practice to limit resources on web server, especially in the shared hosting account. Because, in most cases, it may cause the web server to slow down and Denial of Services situation.
It's recommended to run the script using php-cli and cron.
php-cli offer much more relaxation, such as time and resource limitation. Please also read
Events in MariaDB VS Cron in php - which is better
I have a PHP script that processes my email subscriptions.
It does something like:
foreach email to be sent:
mailer->send-email
print "Email sent to whoever."
I'm now encountering rate-limiting by my web host. The mailing library has a built in throttler that will sleep to ensure I stay under the rate. However, this could result in the web page taken multiple hours to actually load.
Will the client side browser ever give up on the page loading? Any suggested better solutions to this?
Why is this being done on a webpage load? This should be an off-line back-end process which is scheduled to run. (Look into cron for scheduling tasks.)
Any long running process should be delegated to a back-end service to handle that process. Application interfaces (such as a web page) should respond back to the user as quickly as possible instead of forcing the user to wait (for upwards of an hour?) for a response.
The application can track progress, usually by means of some shared data source (a simple database, for example), of the back-end process and present that progress to the user. That's fine. But the process itself should happen outside of the application.
For example, at a high level...
Have a PHP script scheduled to run to process the emails.
When the script starts, save a record to a database indicating that it's started.
Each time the script reaches a milestone of some kind, update the database record to indicate this.
When the script finishes, update the database record to indicate this.
Have a web application which checks for that database record and shows the user the current status of the back-end process.
You may not care, but even if you coerce this script into staying alive, you shouldn't purposely run a long running script through the webserver. Webserver's use resource heavy threads or processes to run your script, and they have a finite amount of them available to server web requests. A long running script basically takes one of them out of the pool of processes that can be used to server web visitors.
Instead, use a cron job which executes the php binary directly. Specifically, do not use wget or lynx or any other web browser like program as part of the cron job, because those methods run the script through the webserver. The cron command should include something like
php /full/path/to/the/script.php
I'm making a transcoding server which uses FFMPEG to convert videos to flv. After user uploads a video it's queued for processing in amazon Simple Queue Service. System is linux ubuntu.
Instead of running CRON each 1min I wonder if it would be possible to continously run several PHP scripts (dowload queued files, process downloaded etc). Each of them would have its own queue which would be read every 10s or so looking for new tasks.
My question is:
How to detect if the script is already running? I'd run CRON each 1min and if one of the programs would not be running I'd load it again. How stuff like that is done on linux? PID files?
thanks for help,
ian
Instead of doing this with only pure-PHP, I would probably go with a solution based on gearman (quoting wikipedia) :
Gearman is an open source application
framework [...]. Gearman is
designed to distribute appropriate
computer tasks to multiple computers,
so large tasks can be done more
quickly.
It works well with PHP, thanks to the gearman extension, and will deal with most of the queuing stuff for you.
Note that it'll also facilitate things when you have more videos to transcode, making scaling to several servers easier.
Yes, you can use PID files
Or temporary table, or memcache, e.t.c.
But I do like this:
By cron runs script that execute convert video, and it check if process is terminated or not
This cron script get movie what needs to convert from database or file
PHP's PEAR repository has a System_Daemon class for creating daemons out of PHP. I've used it for a couple systems with good results.
I've created a similar script specifically for this issue
Check: https://github.com/SirNarsh/EasyCron
The idea is to save PID of the script to a file, then check if the process is running by checking /proc/PID existence
I would like to add an intensive task (lets say 5 minutes execution time) into Wordpress using cron job.
I been using this code to add new cron task inside the Wordpress system.
wp_schedule_event(time(), "interval-name", "hook-name");
I read somewhere in the net that cron task will be executed when there is request hit the Wordpress (either in the public site or the admin). Can anybody acknowledge that is true?
If that the case then I should not put my intensive task into cron task, because it will make user wait for long time after the task finished. What should I do now?
Anybody experienced this situation? Any suggestion?
I think to create a new page to be executed by crontab (for example http://example.com/wp-content/plugins/plugin-example/intensive-task.php)
The wordpress documentation says that it will be run when someone visits your site, so yes, you're correct. It will only be one user that gets a slow page load, so it's up to you if you want to avoid that.
If you are running it from a regular con job, there's no need to make it a page on your site though; especially if it's an intensive job, as you say, then this could easily be exploited place a large load on your server. You can easily run php from the command line to execute your job safely and without causing any slow load times on your page.
If you would use regular cronjob that wouldn't be the case
but i suspect that wp does what you said, since that would make it versatile working in different hosts with different setups as long as they have php and mysql running independent from real cronjobs which must be installed by the web host separately
i wonder how can i schedule and automate tasks in PHP? can i? or is web server features like cron jobs needed.
i am wondering if there is a way i can say delete files after say 3 days when the file are likely outdated or not needed
PHP natively doesn't support automating tasks, you have to build a solution yourself or search google for available solutions. If you have a frequently visited site/page, you could add a timestamp to the database linking to the file, when visiting your site in a chosen time (e.g. 8 in the morning) the script (e.g. deleteOlderDocuments.php) runs and deletes the files that are older.
Just an idea. Hope it helps.
PHP operates under the request-response model, so it won't be the responsibility of PHP to initiate and perform the scheduled job. Use cron, or make your PHP site to register the cron jobs.
(Note: the script that the job executes can be written in PHP of course)
In most shared hosting environments, a PHP interpreter is started for each page request. This means that for each PHP script in said environment, all that script will know about is the fact that it's handling a request, and the information that request gave it. Technically you could check the current time in PHP and see if a task needs to be performed, but that is relying on a user requesting that script near a given time.
It is better to use cron for such tasks. especially if the tasks you need performed can be slow -- then, every once in a while, around a certain time, a user would have a particularly slow response, because them accessing a script caused the server to do a whole bunch of scheduled stuff.