I've got a bunch of php scripts scheduled to run every couple of minutes in cron on a CentOS machine. I would like for every script to self check if the previous instance of it is still running when it starts and stop if it is.
I do this to manage tasks and making sure they only run one at a time
public function startTask($taskname)
{
$running = "running";
$lockfile = $taskname . "RUNNING";
file_put_contents($lockfile, $running);
}
public function endTask($taskname)
{
$lockfile = $taskname . "RUNNING";
unlink($lockfile);
}
public function isTaskRunning($taskname)
{
$lockfile = $taskname . "RUNNING";
if (file_exists($lockfile))
{
return true;
}
}
You call startTask('name') when you start the task and then endTask('name') when you finish. And on the first line of the task you use
if (isTaskRunning('name')) {
die('already running');
}
Put these in a config class or something thats included in all task files and your away
Use a lock file:
<?php
$lockfile = "/tmp/lock.txt";
$fp = fopen($lockfile, "r+");
if (flock($fp, LOCK_EX)) { // acquire an exclusive lock
ftruncate($fp, 0); // truncate file
fwrite($fp, sprintf("Started: %s\nPID: %s", date(), getmypid()));
// perform your tasks here.
fflush($fp); // flush output before releasing the lock
flock($fp, LOCK_UN); // release the lock
} else {
echo "Couldn't get the lock!\nCheck $lockfile for more info.";
}
fclose($fp);
Alternatively, if you are using a database you can do a database Named Lock like this:
<?php
$process = "myProcess";
$sql = mysql_query("select get_lock('$process', 0)");
$got_lock = (bool)mysql_fetch_array($sql)[0];
// If process is already running exit
if(!$got_lock){
echo "Process running";
exit;
}
// Run my process
for($i=0;$i<100000000;$i++){
echo $i;
}
// Release the lock
mysql_query("select release_lock('$process')");
This form of locking is called a named lock, it doesn't "Lock" the database, it just creates a "Named Lock" and when you call it it checks to see if the name exists. It is nothing like a table lock or row lock It was built into mysql for other applications, such as this.
You can have as many locks as you need, and they are automatically released once the application ends, such as your client disconnecting from mysql: process finishes, php breaks/crashes, mysql crashes (not 100% sure on this one), etc.
You can add to your PHP script a simple echo at the end (maybe with date()) which will be visible in your cron logs and shows you if the script reaches the end (and then finish its task)
I'm trying to write a PHP script that I want to ensure only has a single instance of it running at any given time. All of this talk about different ways of locking, and race conditions, and etc. etc. etc. is giving me the willies.
I'm confused as to whether lock files are the way to go, or semaphores, or using MySQL locks, or etc. etc. etc.
Can anyone tell me:
a) What is the correct way to implement this?
AND
b) Point me to a PHP implementation (or something easy to port to PHP?)
One way is to use the php function flock with a dummy file, that will act as a watchdog.
On the beginning of our job, if the file raise a LOCK_EX flag, exit, or wait, can be done.
Php flock documentation: http://php.net/manual/en/function.flock.php
For this examples, a file called lock.txt must be created first.
Example 1, if another twin process is running, it will properly quit, without retrying, giving a state message.
It will throw the error state, if the file lock.txt isn't reachable.
<?php
$fp = fopen("lock.txt", "r+");
if (!flock($fp, LOCK_EX|LOCK_NB, $blocked)) {
if ($blocked) {
// another process holds the lock
echo "Couldn't get the lock! Other script in run!\n";
}
else {
// couldn't lock for another reason, e.g. no such file
echo "Error! Nothing done.";
}
}
else {
// lock obtained
ftruncate($fp, 0); // truncate file
// Your job here
echo "Job running!\n";
// Leave a breathe
sleep(3);
fflush($fp); // flush output before releasing the lock
flock($fp, LOCK_UN); // release the lock
}
fclose($fp); // Empty memory
Example 2, FIFO (First in, first out): we wants the process to wait, for an execution after the queue, if any:
<?php
$fp = fopen("lock.txt", "r+");
if (flock($fp, LOCK_EX)) { // acquire an exclusive lock
ftruncate($fp, 0); // truncate file
// Your job here
echo "Job running!\n";
// Leave a breathe
sleep(3);
fflush($fp); // flush output before releasing the lock
flock($fp, LOCK_UN); // release the lock
}
fclose($fp);
It is also doable with fopen into x mode, by creating and erasing a file when the script ends.
Create and open for writing only; place the file pointer at the
beginning of the file. If the file already exists, the fopen() call
will fail by returning FALSE
http://php.net/manual/en/function.fopen.php
However, into a Unix environment, for fine tuning, I found easier to list the PID's of every background scripts with getmypid() into a DB, or a separate JSON file.
When one task ends, the script is responsible to declare his state in this file (eq: success/failure/debug infos, etc), and then remove his PID. This allows from my view to create admins tools and daemons in a simpler way. And use posix_kill() to kill a PID from PHP if necessary.
Micro-Services are composed using Unix-like pipelines.
Services can call services.
https://en.wikipedia.org/wiki/Microservices
See also: Prevent PHP script using up all resources while it runs?
// borrow from 2 anwsers on stackoverflow
function IsProcessRunning($pid) {
return shell_exec("ps aux | grep " . $pid . " | wc -l") > 2;
}
function AmIRunning($process_file) {
// Check I am running from the command line
if (PHP_SAPI != 'cli') {
error('Run me from the command line');
exit;
}
// Check if I'm already running and kill myself off if I am
$pid_running = false;
$pid = 0;
if (file_exists($process_file)) {
$data = file($process_file);
foreach ($data as $pid) {
$pid = (int)$pid;
if ($pid > 0 && IsProcessRunning($pid)) {
$pid_running = $pid;
break;
}
}
}
if ($pid_running && $pid_running != getmypid()) {
if (file_exists($process_file)) {
file_put_contents($process_file, $pid);
}
info('I am already running as pid ' . $pid . ' so stopping now');
return true;
} else {
// Make sure file has just me in it
file_put_contents($process_file, getmypid());
info('Written pid with id '.getmypid());
return false;
}
}
/*
* Make sure there is only one instance running at a time
*/
$lockdir = '/data/lock';
$script_name = basename(__FILE__, '.php');
// The file to store our process file
$process_file = $lockdir . DS . $script_name . '.pid';
$am_i_running = AmIRunning($process_file);
if ($am_i_running) {
exit;
}
Use semaphores:
$key = 156478953; //this should be unique for each script
$maxAcquire = 1;
$permissions =0666;
$autoRelease = 1; //releases semaphore when request is shut down (you dont have to worry about die(), exit() or return
$non_blocking = false; //if true, fails instantly if semaphore is not free
$semaphore = sem_get($key, $maxAcquire, $permissions, $autoRelease);
if (sem_acquire($semaphore, $non_blocking )) //blocking (prevent simultaneous multiple executions)
{
processLongCalculation();
}
sem_release($semaphore);
See:
https://www.php.net/manual/en/function.sem-get.php
https://www.php.net/manual/en/function.sem-acquire.php
https://www.php.net/manual/en/function.sem-release.php
You can go for the solution that fits best your project, the two simple ways to achieve that are file locking or database locking.
For implementations of file locking, check http://us2.php.net/flock
If you already use a database, create a table, generate known token for that script, put it there, and just remove it after the end of the script. To avoid problems on errors, you can use expiry times.
Perhaps this could work for you,
http://www.electrictoolbox.com/check-php-script-already-running/
In case you are using php on linux and I think the most practical way is:
<?php
if(shell_exec('ps aux | grep '.__FILE__.' | wc -l')>3){
exit('already running...');
}
?>
Another way to do it is with file flag and exit callback,
the exit callback will ensures that the file flag will be reset to 0 in any case of php execution end also fatal errors.
<?php
function exitProcess(){
if(file_get_contents('inprocess.txt')!='0'){
file_put_contents('inprocess.txt','0');
}
}
if(file_get_contents('inprocess.txt')=='1'){
exit();
}
file_put_contents('inprocess.txt','1');
register_shutdown_function('exitProcess');
/**
execute stuff
**/
?>
I have a script i use that checks an IP address stored within my hosts.allow file against what IP is mapped to my dyndns hostname so i can log into my servers once i've synced my current IP to that hostname. For some reason though the script seems to cause really intermittent issues.
within my hosts.allow file i have a section like this:
#SOme.gotdns.com
sshd : 192.168.0.1
#EOme.gotdns.com
#SOme2.gotdns.com
sshd : 192.168.0.2
#EOme2.gotdns.com
I have a script running on a cron (every minute) that looks like this:
#!/usr/bin/php
<?php
$hosts = array('me.gotdns.com','me2.gotdns.com');
foreach($hosts as $host)
{
$ip = gethostbyname($host);
$replaceWith = "#SO".$host."\nsshd : ".$ip."\n#EO".$host;
$filename = '/etc/hosts.allow';
$handle = fopen($filename,'r');
$contents = fread($handle, filesize($filename));
fclose($handle);
if (preg_match('/#SO'.$host.'(.*?)#EO'.$host.'/si', $contents, $regs))
{
$result = $regs[0];
}
if($result != $replaceWith)
{
$newcontents = str_replace($result,$replaceWith,$contents);
$handle = fopen($filename,'w');
if (fwrite($handle, $newcontents) === FALSE) {
}
fclose($handle);
}
}
?>
The problem i have is that intermittently characters are being dropped (i assume during the replace) that causes future updates to fail as it inserts something like:
#SOme.gotdns.com
sshd : 192.168.0.1
#EOme.gotdn
note the missing "s.com"
This of course means i lose access to the server, any ideas why this would be happening?
Thanks.
that might be because of script execution time - can be too short- OR 1 min interval is too short. While cron is doing the job, another process of script starts and it may effect the first one.
This is almost certainly because the script hasn't finished executing within the one minute time period before it's started again via cron. You need to implement some sort of locking, or use a tool that only allows once instance of the script to be run. There are several tools available out there that can do this, for example lockrun.
I would say that in order to do this safely, you should acquire an exclusive lock on the file at the beginning of the script, read it all into memory once, modify it in memory, then write it back to the file at the end. This would also be considerably more efficient in terms of disk I/O.
You should also alter the cron job to run less frequently. It is likely that the reason you currently have this problem is because two processes are running at the same time - by locking the file, if this is the case, you risk having the processes stack up waiting to acquire a lock. Setting it for every 5 minutes should be good enough - your IP shouldn't change that often!
So do this (FIXED):
#!/usr/bin/php
<?php
// Settings
$hosts = array(
'me.gotdns.com',
'me2.gotdns.com'
);
$filename = '/etc/hosts.allow';
// No time limit (shouldn't be necessary with CLI, but just in case)
set_time_limit(0);
// Open the file in read/write mode and lock it
// flock() should block until it gets a lock
if ((!$handle = fopen($filename, 'r+')) || !flock($handle, LOCK_EX)) exit(1);
// Read the file
if (($contents = fread($handle, filesize($filename)) === FALSE) exit(1);
// Will be set to true if we actually make any changes to the file
$changed = FALSE;
// Loop hosts list
foreach ($hosts as $host) {
// Get current IP address of host
if (($ip = gethostbyname($host)) == $host) continue;
// Find the entry in the file
$replaceWith = "#SO{$host}\nsshd : {$ip}\n#EO{$host}";
if (preg_match("/#SO{$host}(.*?)#EO{$host}/si", $contents, $regs)) {
// Only do this if there was a match - otherise risk overwriting previous
// entries because you didn't reset the value of $result
if ($regs[0] != $replaceWith) {
$changed = TRUE;
$contents = str_replace($regs[0], $replaceWith, $contents);
}
}
}
// We'll only change the contents of the file if the data changed
if ($changed) {
ftruncate($handle, 0); // Zero the length of the file
rewind($handle); // start writing from the beginning
fwrite($handle, $contents); // write the new data
}
flock($handle, LOCK_UN); // Unlock
fclose($handle); // close
I need to execute a directory copy upon a user action, but the directories are quite large, so I would like to be able to perform such an action without the user being aware of the time it takes for the copy to complete.
Any suggestions would be much appreciated.
Assuming this is running on a Linux machine, I've always handled it like this:
exec(sprintf("%s > %s 2>&1 & echo $! >> %s", $cmd, $outputfile, $pidfile));
This launches the command $cmd, redirects the command output to $outputfile, and writes the process id to $pidfile.
That lets you easily monitor what the process is doing and if it's still running.
function isRunning($pid){
try{
$result = shell_exec(sprintf("ps %d", $pid));
if( count(preg_split("/\n/", $result)) > 2){
return true;
}
}catch(Exception $e){}
return false;
}
Write the process as a server-side script in whatever language (php/bash/perl/etc) is handy and then call it from the process control functions in your php script.
The function probably detects if standard io is used as the output stream and if it is then that will set the return value..if not then it ends
proc_close( proc_open( "./command --foo=1 &", array(), $foo ) );
I tested this quickly from the command line using "sleep 25s" as the command and it worked like a charm.
(Answer found here)
You might want to try to append this to your command
>/dev/null 2>/dev/null &
eg.
shell_exec('service named reload >/dev/null 2>/dev/null &');
I'd just like to add a very simple example for testing this functionality on Windows:
Create the following two files and save them to a web directory:
foreground.php:
<?php
ini_set("display_errors",1);
error_reporting(E_ALL);
echo "<pre>loading page</pre>";
function run_background_process()
{
file_put_contents("testprocesses.php","foreground start time = " . time() . "\n");
echo "<pre> foreground start time = " . time() . "</pre>";
// output from the command must be redirected to a file or another output stream
// http://ca.php.net/manual/en/function.exec.php
exec("php background.php > testoutput.php 2>&1 & echo $!", $output);
echo "<pre> foreground end time = " . time() . "</pre>";
file_put_contents("testprocesses.php","foreground end time = " . time() . "\n", FILE_APPEND);
return $output;
}
echo "<pre>calling run_background_process</pre>";
$output = run_background_process();
echo "<pre>output = "; print_r($output); echo "</pre>";
echo "<pre>end of page</pre>";
?>
background.php:
<?
file_put_contents("testprocesses.php","background start time = " . time() . "\n", FILE_APPEND);
sleep(10);
file_put_contents("testprocesses.php","background end time = " . time() . "\n", FILE_APPEND);
?>
Give IUSR permission to write to the directory in which you created the above files
Give IUSR permission to READ and EXECUTE C:\Windows\System32\cmd.exe
Hit foreground.php from a web browser
The following should be rendered to the browser w/the current timestamps and local resource # in the output array:
loading page
calling run_background_process
foreground start time = 1266003600
foreground end time = 1266003600
output = Array
(
[0] => 15010
)
end of page
You should see testoutput.php in the same directory as the above files were saved, and it should be empty
You should see testprocesses.php in the same directory as the above files were saved, and it should contain the following text w/the current timestamps:
foreground start time = 1266003600
foreground end time = 1266003600
background start time = 1266003600
background end time = 1266003610
If you need to just do something in background without the PHP page waiting for it to complete, you could use another (background) PHP script that is "invoked" with wget command. This background PHP script will be executed with privileges, of course, as any other PHP script on your system.
Here is an example on Windows using wget from gnuwin32 packages.
The background code (file test-proc-bg.php) as an exmple ...
sleep(5); // some delay
file_put_contents('test.txt', date('Y-m-d/H:i:s.u')); // writes time in a file
The foreground script, the one invoking ...
$proc_command = "wget.exe http://localhost/test-proc-bg.php -q -O - -b";
$proc = popen($proc_command, "r");
pclose($proc);
You must use the popen/pclose for this to work properly.
The wget options:
-q keeps wget quiet.
-O - outputs to stdout.
-b works on background
Well i found a bit faster and easier version to use
shell_exec('screen -dmS $name_of_screen $command');
and it works.
Here is a function to launch a background process in PHP. Finally created one that actually works on Windows too, after a lot of reading and testing different approaches and parameters.
function LaunchBackgroundProcess($command){
// Run command Asynchroniously (in a separate thread)
if(PHP_OS=='WINNT' || PHP_OS=='WIN32' || PHP_OS=='Windows'){
// Windows
$command = 'start "" '. $command;
} else {
// Linux/UNIX
$command = $command .' /dev/null &';
}
$handle = popen($command, 'r');
if($handle!==false){
pclose($handle);
return true;
} else {
return false;
}
}
Note 1: On windows, do not use /B parameter as suggested elsewhere. It forces process to run the same console window as start command itself, resulting in the process being processed synchronously. To run the process in a separate thread (asynchronously), do not use /B.
Note 2: The empty double quotes after start "" are required if the command is a quoted path. start command interprets the first quoted parameter as window title.
Can you arrange to fork off a separate process, and then run your copy in the background? It's been a while since I did any PHP, but the function pcntl-fork looks promising.
Use this function to run your program in background. It cross-platform and fully customizable.
<?php
function startBackgroundProcess(
$command,
$stdin = null,
$redirectStdout = null,
$redirectStderr = null,
$cwd = null,
$env = null,
$other_options = null
) {
$descriptorspec = array(
1 => is_string($redirectStdout) ? array('file', $redirectStdout, 'w') : array('pipe', 'w'),
2 => is_string($redirectStderr) ? array('file', $redirectStderr, 'w') : array('pipe', 'w'),
);
if (is_string($stdin)) {
$descriptorspec[0] = array('pipe', 'r');
}
$proc = proc_open($command, $descriptorspec, $pipes, $cwd, $env, $other_options);
if (!is_resource($proc)) {
throw new \Exception("Failed to start background process by command: $command");
}
if (is_string($stdin)) {
fwrite($pipes[0], $stdin);
fclose($pipes[0]);
}
if (!is_string($redirectStdout)) {
fclose($pipes[1]);
}
if (!is_string($redirectStderr)) {
fclose($pipes[2]);
}
return $proc;
}
Note that after command started, by default this function closes the stdin and stdout of running process. You can redirect process output into some file via $redirectStdout and $redirectStderr arguments.
Note for windows users:
You cannot redirect stdout/stderr to nul in the following manner:
startBackgroundProcess('ping yandex.com', null, 'nul', 'nul');
However, you can do this:
startBackgroundProcess('ping yandex.com >nul 2>&1');
Notes for *nix users:
1) Use exec shell command if you want get actual PID:
$proc = startBackgroundProcess('exec ping yandex.com -c 15', null, '/dev/null', '/dev/null');
print_r(proc_get_status($proc));
2) Use $stdin argument if you want to pass some data to the input of your program:
startBackgroundProcess('cat > input.txt', "Hello world!\n");
You might try a queuing system like Resque. You then can generate a job, that processes the information and quite fast return with the "processing" image. With this approach you won't know when it is finished though.
This solution is intended for larger scale applications, where you don't want your front machines to do the heavy lifting, so they can process user requests.
Therefore it might or might not work with physical data like files and folders, but for processing more complicated logic or other asynchronous tasks (ie new registrations mails) it is nice to have and very scalable.
A working solution for both Windows and Linux. Find more on My github page.
function run_process($cmd,$outputFile = '/dev/null', $append = false){
$pid=0;
if (strtoupper(substr(PHP_OS, 0, 3)) === 'WIN') {//'This is a server using Windows!';
$cmd = 'wmic process call create "'.$cmd.'" | find "ProcessId"';
$handle = popen("start /B ". $cmd, "r");
$read = fread($handle, 200); //Read the output
$pid=substr($read,strpos($read,'=')+1);
$pid=substr($pid,0,strpos($pid,';') );
$pid = (int)$pid;
pclose($handle); //Close
}else{
$pid = (int)shell_exec(sprintf('%s %s %s 2>&1 & echo $!', $cmd, ($append) ? '>>' : '>', $outputFile));
}
return $pid;
}
function is_process_running($pid){
if (strtoupper(substr(PHP_OS, 0, 3)) === 'WIN') {//'This is a server using Windows!';
//tasklist /FI "PID eq 6480"
$result = shell_exec('tasklist /FI "PID eq '.$pid.'"' );
if (count(preg_split("/\n/", $result)) > 0 && !preg_match('/No tasks/', $result)) {
return true;
}
}else{
$result = shell_exec(sprintf('ps %d 2>&1', $pid));
if (count(preg_split("/\n/", $result)) > 2 && !preg_match('/ERROR: Process ID out of range/', $result)) {
return true;
}
}
return false;
}
function stop_process($pid){
if (strtoupper(substr(PHP_OS, 0, 3)) === 'WIN') {//'This is a server using Windows!';
$result = shell_exec('taskkill /PID '.$pid );
if (count(preg_split("/\n/", $result)) > 0 && !preg_match('/No tasks/', $result)) {
return true;
}
}else{
$result = shell_exec(sprintf('kill %d 2>&1', $pid));
if (!preg_match('/No such process/', $result)) {
return true;
}
}
}
Thanks to this answer: A perfect tool to run a background process would be Symfony Process Component, which is based on proc_* functions, but it's much easier to use. See its documentation for more information.
Instead of initiating a background process, what about creating a trigger file and having a scheduler like cron or autosys periodically execute a script that looks for and acts on the trigger files? The triggers could contain instructions or even raw commands (better yet, just make it a shell script).
If using PHP there is a much easier way to do this using pcntl_fork:
http://www.php.net/manual/en/function.pcntl-fork.php
I am heavily using fast_cgi_finish_request()
In combination with a closure and register_shutdown_function()
$message ='job executed';
$backgroundJob = function() use ($message) {
//do some work here
echo $message;
}
Then register this closure to be executed before shutdown.
register_shutdown_function($backgroundJob);
Finally when the response was sent to the client you can close the connection to the client and continue working with the PHP process:
fast_cgi_finish_request();
The closure will be executed after fast_cgi_finish_request.
The $message will not be visible at any time. And you can register as much closures as you want, but take care about script execution time.
This will only work if PHP is running as a Fast CGI module (was that right?!)
If you are looking to execute a background process via PHP, pipe the command's output to /dev/null and add & to the end of the command.
exec("bg_process > /dev/null &");
Note that you can not utilize the $output parameter of exec() or else PHP will hang (probably until the process completes).
PHP scripting is not like other desktop application developing language. In desktop application languages we can set daemon threads to run a background process but in PHP a process is occuring when user request for a page. However It is possible to set a background job using server's cron job functionality which php script runs.
For those of us using Windows, look at this:
Reference: http://php.net/manual/en/function.exec.php#43917
I too wrestled with getting a program to run in the background in
Windows while the script continues to execute. This method unlike the
other solutions allows you to start any program minimized, maximized,
or with no window at all. llbra#phpbrasil's solution does work but it
sometimes produces an unwanted window on the desktop when you really
want the task to run hidden.
start Notepad.exe minimized in the background:
<?php
$WshShell = new COM("WScript.Shell");
$oExec = $WshShell->Run("notepad.exe", 7, false);
?>
start a shell command invisible in the background:
<?php
$WshShell = new COM("WScript.Shell");
$oExec = $WshShell->Run("cmd /C dir /S %windir%", 0, false);
?>
start MSPaint maximized and wait for you to close it before continuing the script:
<?php
$WshShell = new COM("WScript.Shell");
$oExec = $WshShell->Run("mspaint.exe", 3, true);
?>
For more info on the Run() method go to:
http://msdn.microsoft.com/library/en-us/script56/html/wsMthRun.asp
Edited URL:
Go to https://technet.microsoft.com/en-us/library/ee156605.aspx instead as the link above no longer exists.
New answer to an old question. Using this library, the following code would spawn an asynchronous/parallel PHPThread to do background work.
Must have pcntl, posix, and socket extensions
Designed for/tested in CLI mode.
EZ code sample:
function threadproc($thread, $param) {
echo "\tI'm a PHPThread. In this example, I was given only one parameter: \"". print_r($param, true) ."\" to work with, but I can accept as many as you'd like!\n";
for ($i = 0; $i < 10; $i++) {
usleep(1000000);
echo "\tPHPThread working, very busy...\n";
}
return "I'm a return value!";
}
$thread_id = phpthread_create($thread, array(), "threadproc", null, array("123456"));
echo "I'm the main thread doing very important work!\n";
for ($n = 0; $n < 5; $n++) {
usleep(1000000);
echo "Main thread...working!\n";
}
echo "\nMain thread done working. Waiting on our PHPThread...\n";
phpthread_join($thread_id, $retval);
echo "\n\nOur PHPThread returned: " . print_r($retval, true) . "!\n";
From PHP official documentation(php.net)
<?php
function execInBackground($cmd) {
if (substr(php_uname(), 0, 7) == "Windows"){
pclose(popen("start /B ". $cmd, "r"));
}
else {
exec($cmd . " > /dev/null &");
}
}
?>
I know it is a 100 year old post, but anyway, thought it might be useful to someone. You can put an invisible image somewhere on the page pointing to the url that needs to run in the background, like this:
<img src="run-in-background.php" border="0" alt="" width="1" height="1" />