I have a php-cli script that is run by cron every 5 minutes. Because this interval is short, multiple processes are run at the same time. That's not what I want, since this script has to write inside a text file a numeric id that is incremented each time. It happens that writers are writing at the same time on this text file, and the value written is incorrect.
I have tried to use php's flock function to block writing in the file, when another process is writing on it but it doesnt work.
$fw = fopen($path, 'r+');
if (flock($fw, LOCK_EX)) {
ftruncate($fw, 0);
fwrite($fw, $latestid);
fflush($fw);
flock($fw, LOCK_UN);
}
fclose($fw);
So I suppose that the solution to this is create a bash script that verifies if there is an instance of this php script that is running, if so it should wait until it finished. But I dont know how to do it, any ideas?
The solution I'm using with a bash script is this:
exec 9>/path/to/lock/file
if ! flock -n 9 ; then
echo "another instance is running";
exit 1
fi
# this now runs under the lock until 9 is closed (it will be closed automatically when the script ends)
A file descriptor 9> is created in /var/lock/file, and flock will exit a new process that's trying to run, unless there is no other instance of the script that is running.
How can I ensure that only one instance of a script is running at a time (mutual exclusion)?
I don't really understand how incrementing a counter every 5 minutes will result in multiple processes trying to write the counter file at the same time, but...
A much simpler approach is to use a simple locking mechanism similar to the below:
<?php
$lock_filename = 'nobodyshouldincrementthecounterwhenthisfileishere';
if(file_exists($lock_filename)) {
return;
}
touch($lock_filename);
// your stuff...
unlink($lock_filename);
This as a simple approach will not deal with a situation when the script breaks before it could remove the lock file, in which case it would never run again until it is removed.
More sophisticated approaches are also possible as you suggest, e.g. fork the job in its own process, write the PID into a file, then before running the job it could be checked whether that PID is still running.
To prevent start of a next session of any program until the previous session still running, such as next cron job, I recommend to use either built into your program or external check of running process of this program. Just execute before starting of your program
ps -ef|grep <process_name>|grep -v grep|wc -l
and check, if its result will be 0. Only in this case your program could be started.
I suppose, that you must guarantee an absence of 3rd party process having similar name. (For this purpose give your program a longer and unique name). And a name of your program must not contain pattern "grep".
This work good in combination with normal regular starting of your program, that is configured in a cron table, by cron daemon.
For the case if your check is written as an external script, an entry in the crontab might look like
<time_specification> <your_starter_script> <your_program> ...
2 important remarks: Exit code of your_starter_script must be 0 in case of not starting of your program and it would be better to completely prohibit writing to stdout or stderr by this script.
Such starter is very short and a simple programming exercise. Therefore I don't feel a need to provide its complete code.
Instead of using cron to run your script every 5 minutes, how about using at to schedule your script to run again, 5 minutes after it finishes. Near the end of your script, you can use shell_exec() to run an at command to schedule your script to run again in 5 minutes, like so:
at now + 5 minutes /path/to/script
Or, perhaps even simpler than my previous answer (using at to schedule the script to run again in 5 minutes) is make your script a daemon, by using a non-terminating loop, like so:
while(1) {
// whatever your script does here....
sleep(300) //wait 5 minutes
}
Then, you can do away with scheduling by way of cron or at altogether. Just simply run your script in the background from the command line, like so:
/path/to/your/script &
Or, add /path/to/your/script in /etc/rc.local to make your script start automatically when the machine boots.
Related
2 part question,
I am running a script that executes a second script.
I have it setup this way because I read if I put a script in the /etc/init.d directory it will run it at start up.(true or false?)
I have tried adding >> LoopTriggerLogging.log at the end of each line but nothing comes out in the log file
So I have a first script as follows
#!/bin/bash
/var/www/Dev/LoopTrigger.sh
exit
This triggers the following script to run
#!/bin/bash
while true; do
# do some work
php /var/www/Dev/FindNearestDriverAndSendPush.php
php /var/www/Dev/ProcessCustomerPayment.php
php /var/www/Dev/ProcessDriversPayment.php
# write to LoopTriggerLogging.log
sleep 2 # sleep and repeat
done
What I would like is to have the commands logged along with any errors. I have tried to read a little on this but get lost in the answers and what they are trying to tell the user. I am still new at this and learning, kindly give definition to any commands or options. I am open to a best practice scenario.
Also, with putting in the etc/init.d directory will this tell the script to run at start up?
Is there a way to run this script without it taking up the command line because its an endless script?
My ultimate goal is to get the 3 php files to execute every 2 seconds with some sort of logging.
I did some reading on Cron but seems it is not meant for this type of use case.
Ive also seen this:
exec > logfile 2>&1 (dont know what this does)
set -x makes bash print every command before executing it
FOO=BAR (dont know what this means)
echo $FOO (dont know what this means)
if I put a script in the /etc/init.d directory it will run it at start
up.(true or false)
True. If you put a script in init.d then that script will run for every startup.
My ultimate goal is to get the 3 php files to execute every 2 seconds
You are using the correct way of running it approx every 2sec depending upon the time your php script takes to run. Crontab runs after a minimum of one minute so that would not an option.
I have tried adding >> LoopTriggerLogging.log at the end of each line
but nothing comes out in the log file
You can use /var/www/Dev/LoopTrigger.sh >> LoopTriggerLogging.log in your first script so that whenever it runs it will
Create a file for the first time and append the content from the next time.
Push all the logs of the second script into the file.
Note: As logs will keep on appending to the single file, this file will become very huge at some point of time so make sure your handle it well.
I have a PHP script that runs on a shared hosting environment server. This PHP script takes a long time to run. It may take 20 to 30 minutes to finish a run. It is a recurring background process. However I do not have control over when the process starts (it could be triggered every five minutes, or every three hours, no one knows).
Anyway, at the beginnin of this script I would like to detect if the previous process is still running, if the earlier run is still running and has not finished, then I would not run the script again. If it is not running, then I run the new process.
In other words, here is a pseudo code. Let's call the script abc.php
1. Start script abc.php
2. Check if an older version of abc.phh is still running. If it is running, then terminate
3. If it is not running, then continue with abc.php and do your work which might take 30 minutes or more
How can I do that? Please keep in mind this is shared hosting.
UPDATE: I was thinking of using a DB detection mechanism. So, when the script starts, it will set a value in a DB as 'STARTED=TRUE', when done, it will set 'STARTED=FALSE'. However this solution is not proper, because there is no garantee that the script will terminate properly. It might get interrupted, and therefore may not update the STARTED value to FALSE. So the DB solution is out of the question. It has to be a process detection of some sort, or maybe a different solution that I did not think off. Thanks.
If this is a CGI process, I would try using exec + ps, if the latter is available in your environment. A quick SO search turns up this answer: https://stackoverflow.com/a/7182595/177920
You'll need to have a script that is responsible for (and separate from) checking to see if your target script is running, of course, otherwise you'll always see that your target script is running based on the order of ops in your "psuedo code".
You can implement a simple locking mechanism: Create a tmp lock file when script starts and check before if the lock file already exists. If it does, dont run the script, it it doesnt create a lock file and run the script. At then end of successful run, delete the lock file so that it will run properly next time.
if(!locked()) {
lock();
// your code here
unlock();
} else {
echo "script already running";
}
function lock() { file_put_contents("write.lock", 'running'); }
function locked() { return file_exists("write.lock"); }
function unlock() { return unlink("write.lock"); }
I have scheduled a CRON which calls/executes a PHP script every five minutes. PHP script perform following tasks
Checks for the flag value in database to identify if the previous run is still executing. Value of 1 in the DB tells that process is still running while a value of 0 means it is not.
If the flag value is 1, then exit the PHP else continue to next step.
Update the flag value in database from 0 to 1.
Execute the business logic.
Update the flag value back from 1 to 0, so that next CRON can executes if the data is available in user tables.
All works fine so far, depending on the size of user uploaded data the process on an average takes 35 to 40 minutes to complete.
Question, Is there anyway to kill or stop the execution of PHP script once started by cron. May be a button to let users stop the execution, upload new data and wait for CRON run. I can take care of reseting all the flags and data it's just the kill of PHP script is what i am trying to figure out.
I did some google and figured i can use some commands like:
Killall -9 PHP
to kill all php processes running on server, but not sure how to do this through PHP.
Try this:
ps aux |grep 'part_of_the_name_of_your_script'|awk '{print $2}' |xargs kill -9 {}
Or in your crontab file use crun and variable CRUN_TIME
see crun -h
A lock file would be very appropriate for this tasks.
The PHP script can attempt to create a new file, and if none is created already you can safely know that the script is the only one running at the present time. If a file exists, you can simply exit the script.
Example:
<?php
if (file_exists('/var/run/my-script')) {
exit(1); // already running
}
file_put_contents('/var/run/my-script', getmypid());
/** Business Logic **/
unlink('/var/run/my-script');
exit(0);
?>
You can try system() or exec(), but it might not work (or return permission denied errors) as cron processes are executed by either the current user or root, and the web server user doesn't usually have access to these processes.
Currently I have a parser.php which loads an xml file and inserts new data from the xml file into a mysql database. How would I go about refreshing this php file every 30 seconds so my mysql table always has fresh data? I think I could use short-polling to do this, but I'm guessing this is not the most efficient of options.
Thanks in advance
This is a non-PHP solution which will require you to have shell (SSH) access in order to run the script, however you can also run it through PHP with exec() if you want to. Shared hosting environments might present a challenge for this approach but as long as you can execute scripts under your user credentials you should have no problems running it.
First you will need to create a bash script with the following content and save it (I'll use the name parser.sh for the purpose of this example). You can then adjust the timeout in the sleep 30 line if you want to.
#!/bin/sh
while true
do
php parser.php
sleep 30
done
In order to run the script you'll need to give it execute permissions.
chmod +x parser.sh
Now you can use the nohup command with the ampersand (&) argument to ensure that the script will run in the background even when a termination signal is sent after, lets say, closing the shell (SSH). The ampersand is important!
nohup ./parser.sh &
Now you can use top or ps aux | grep parser to ensure that the script is running. As I already said before you can also use PHP exec() to start the process but shell is still the preferred and most reliable way to do this.
If you want to stop the background process which executes your script then you'll simply have to kill it. Just use ps aux | grep parser to find out the PID of the parser process (its in the second column to the left) and use it with the kill command.
kill 4183
You need to use a cronjob, but crons jobs runs every 1 minute or more.
Another way is to make a "daemon".
Very basic example:
<?php
while(true) {
// check if 30 seconds has passed.
// then execute some function...
};
?>
Then you need to execute this in your terminal:
$ php script.php &
This link should help.
Greatings!
I have 3 scripts that do some stuff.
I want to run them continously and concurrently.
Let's say for example:
First script took 30 minutes to finish.
Second - 20 mins.
Third - 5 mins.
So I need everyone of them to run immediately after it's finished.
The 3 scripts make UPDATE in a same DB, but they need to work separately.
They can run together at once, but not couple of times(my english sucks, sorry about that).
Let's explain what I mean with example:
firstScript.php is running
secondScript.php is running
thirdScript.php is running
firstScript.php trying to start but it still running. Wait.(till finish)
May be some shell script will do the job, but how?
Make a bash script that takes one argument, and have it do something like this:
if [ -f /tmp/$1 ]
then
echo "Script already running"
else
touch /tmp/$1
php $1
rm /tmp/$1
fi
Set up a cron to run this script and pass it the name of the php script you want to run.
You could execute a shell command just before the php script dies. Example :
while($i < 1000)
{
$i++;
}
shell_exec("bin/php.exe some_script.php");
If you are working on a shared hosting account this might not work do to security issues.
note the "bin/php.exe" need to be edited for your server, point to where ever your php is installed.