Related
To prevent multiple instances of a PHP-based daemon I wrote from ever running simultaneously, I wrote a simple function to acquire a lock with flock when the process starts, and called it at the start of my daemon. A simplified version of the code looks like this:
#!/usr/bin/php
<?php
function acquire_lock () {
$file_handle = fopen('mylock.lock', 'w');
$got_lock_successfully = flock($file_handle, LOCK_EX);
if (!$got_lock_successfully) {
throw new Exception("Unexpected failure to acquire lock.");
}
}
acquire_lock(); // Block until all other instances of the script are done...
// ... then do some stuff, for example:
for ($i=1; $i<=10; $i++) {
echo "Sleeping... $i\n";
sleep(1);
}
?>
When I execute the script above multiple times in parallel, the behaviour I expect to see - since the lock is never explicitly released throughout the duration of the script - is that the second instance of the script will wait until the first has completed before it proceeds past the acquire_lock() call. In other words, if I run this particular script in two parallel terminals, I expect to see one terminal count to 10 while the other waits, and then see the other count to 10.
This is not what happens. Instead, I see both scripts happily executing in parallel - the second script does not block and wait for the lock to be available.
As you can see, I'm checking the return value from flock and it is true, indicating that the (exclusive) lock has been acquired successfully. Yet this evidently isn't preventing another process from acquiring another 'exclusive' lock on the same file.
Why, and how can I fix this?
Simply store the file pointer resource returned from fopen in a global variable. In the example given in the question, $file_handle is automatically destroyed upon going out of scope when acquire_lock() returns, and this releases the lock taken out by flock.
For example, here is a modified version of the script from the question which exhibits the desired behaviour (note that the only change is storing the file handle returned by fopen in a global):
#!/usr/bin/php
<?php
function acquire_lock () {
global $lock_handle;
$lock_handle = fopen('mylock.lock', 'w');
$got_lock_successfully = flock($lock_handle, LOCK_EX);
if (!$got_lock_successfully) {
throw new Exception("Unexpected failure to acquire lock.");
}
}
acquire_lock(); // Block until all other instances of the script are done...
// ... then do some stuff, for example:
for ($i=1; $i<=10; $i++) {
echo "Sleeping... $i\n";
sleep(1);
}
?>
Note that this seems to be a bug in PHP. The changelog from the flock documentation states that in version 5.3.2:
The automatic unlocking when the file's resource handle is closed was removed. Unlocking now always has to be done manually.
but at least for PHP 5.5, this is false; flock locks are released both by explicit calls to fclose and by the resource handle going out of scope.
I reported this as a bug in November 2014 and may update this question and answer pair if it is ever resolved. In case I get eaten by piranhas before then, you can check the bug report yourself to see if this behaviour has been fixed: https://bugs.php.net/bug.php?id=68509
I'm using cronjob to run php script that will be executed every 1 minute
I need also to make sure only of copy is running so if this php script is still running after 2 minutes, cronjob should not run another version.
currently I have 2 options and I would like to see your feedback and if you have any more options
Option 1: create a tmp file when the php script start and remove it when php script finish (and check if the file exists) ---> the problem for me with this option is that if I have my php script crash for any reason, it will not run again (the tmp file will not be deleted)
Option 2: run a bash script like the one below to control the php script execution ---> good but looking for something that can be done within php
#!/bin/bash
function rerun {
BASEDIR=$(dirname $0)
echo $BASEDIR/$1
if ps -ef | grep -v grep | grep $1; then
echo "Running"
exit 0
else
echo "NOT running";
/usr/local/bin/php $BASEDIR/$1 &
exit $?
fi
}
rerun myphpscript.php
PS: I just saw "Mutex class" at http://www.php.net/manual/en/class.mutex.php but not sure if it's stable and anyone tried it.
You might want to use my library ninja-mutex which provides simple interface for handling mutex. Currently it can use flock, memcache, redis or mysql to handle lock.
Below is an example which uses memcache:
<?php
require 'vendor/autoload.php';
use NinjaMutex\Lock\MemcacheLock;
use NinjaMutex\Mutex;
$memcache = new Memcache();
$memcache->connect('127.0.0.1', 11211);
$lock = new MemcacheLock($memcache);
$mutex = new Mutex('very-critical-stuff', $lock);
if ($mutex->acquireLock(1000)) {
// Do some very critical stuff
// and release lock after you finish
$mutex->releaseLock();
} else {
throw new Exception('Unable to gain lock!');
}
I often use the program flock that comes with many linux distributions directly in my crontabs like:
* * * * * flock -n /var/run/mylock.LCK /usr/local/bin/myprogram
Of cause it is still possible to actually start two simultaneously instances of myprogram if you do it by hand, but crond will only make one.
Flock being a small compiled binary, makes it super fast to launch compared to a eventually larger chunk of php code. This is especially a benefit if you have many longer running executions, which it is not perfectly clear that you actually have.
If you're not on a NFS mount, you can use flock() (http://php.net/manual/en/function.flock.php):
$fh = fopen('guestbook.txt','a') or die($php_errormsg);
$tries = 3;
while ($tries > 0) {
$locked = flock($fh,LOCK_EX | LOCK_NB);
if (! $locked) {
sleep(5);
$tries--;
} else {
// don't go through the loop again
$tries = 0;
}
}
if ($locked) {
fwrite($fh,$_REQUEST['guestbook_entry']) or die($php_errormsg);
fflush($fh) or die($php_errormsg);
flock($fh,LOCK_UN) or die($php_errormsg);
fclose($fh) or die($php_errormsg);
} else {
print "Can't get lock.";
}
From: http://docstore.mik.ua/orelly/webprog/pcook/ch18_25.htm
I found the best solution for me is creating a separate database user for your Script and limit the concurent connection to 1 for that user.
I'm trying to test a race condition in PHP. I'd like to have N PHP processes get ready to do something, then block. When I say "go", they should all execute the action at the same time. Hopefully this will demonstrate the race.
In Java, I would use Object.wait() and Object.notifyAll(). What can I use in PHP?
(Either Windows or linux native answers are acceptable)
Create a file "wait.txt"
Start N processes, each with the code shown below
Delete the "wait.txt" file.
...
<?php
while (file_exists('wait.txt')) {}
runRaceTest();
Usually with PHP file lock approach is used. One create a RUN_LOCK or similar file and asks for file_exists("RUN_LOCK"). This system is also used to secure potential endless loops in recursive threads.
I decided to require the file for the execution. Other approach may be, that existence of the file invokes the blocking algorithm. That depends on your situation. Always the safer situation should be the easier to achieve.
Wait code:
/*prepare the program*/
/* ... */
/*Block until its time to go*/
define("LOCK_FILE", "RUN_UNLOCK"); //I'd define this in some config.php
while(!file_exists(LOCK_FILE)) {
usleep(1); //No sleep will eat lots of CPU
}
/*Execute the main code*/
/* ... */
/*Delete the "run" file, so that no further executions should be allowed*/
usleep(1); //Just for sure - we want other processes to start execution phase too
if(file_exists(LOCK_FILE))
unlink(LOCK_FILE);
I guess it would be nice to have a blocking function for that, like this one:
function wait_for_file($filename, $timeout = -1) {
if($timeout>=0) {
$start = microtime(true)*1000; //Remember the start time
}
while(!file_exists($filename)) { //Check the file existence
if($timeout>=0) { //Only calculate when timeout is set
if((microtime(true)*1000-$start)>$timeout) //Compare current time with start time (current always will be > than start)
return false; //Return failure
}
usleep(1); //Save some CPU
}
return true; //Return success
}
It implements timeout. You don't need them but maybe someone else will.
Usage:
header("Content-Type: text/plain; charset=utf-8");
ob_implicit_flush(true);while (#ob_end_clean()); //Flush buffers so the output will be live stream
define("LOCK_FILE","RUN_FOREST_RUN"); //Define lock file name again
echo "Starting the blocking algorithm. Waiting for file: ".LOCK_FILE."\n";
if(wait_for_file(LOCK_FILE, 10000)) { //Wait for 10 secconds
echo "File found and deleted!\n";
if(file_exists(LOCK_FILE)) //May have been deleted by other proceses
unlink(LOCK_FILE);
}
else {
echo "Wait failed!\n";
}
This will output:
Starting the blocking algorithm. Waiting for file: RUN_FOREST_RUN
Wait failed!
~or~
Starting the blocking algorithm. Waiting for file: RUN_FOREST_RUN
File found and deleted!
PHP doesn't have multithreading. And its not planned to be implemented either.
You can try hacks with sockets though or 0MQ to communicate between multiple processes
See Why does PHP not support multithreading?
Php multithread
I have a cron script that executes a PHP script every 10 minutes. The script checks a queue and processes the data in the queue. Sometimes the queue has enough data to last over 10 minutes of processing, creating the potential of two scripts trying to access the same data. I want to be able to detect whether the script is already running to prevent launching multiple copies of the script. I thought about creating a database flag that says that a script is processing, but if the script were ever to crash it would leave it in the positive state. Is there an easy way to tell if the PHP script is already running from withing a PHP or shell script?
You can just use a lock file. PHP's flock() function provides a simple wrapper for Unix's flock function, which provides advisory locks on files.
If you don't explicitly release them, the OS will automatically release these locks for you when the process holding them terminates, even if it terminates abnormally.
You can also follow the loose Unix convention of making your lock file a 'PID file' - that is, upon obtaining a lock on the file, have your script write its PID to it. Even if you never read this from within your script, it will be convenient for you if your script ever hangs or goes crazy and you want to find its PID in order to manually kill it.
Here's a copy/paste-ready implementation:
#!/usr/bin/php
<?php
$lock_file = fopen('path/to/yourlock.pid', 'c');
$got_lock = flock($lock_file, LOCK_EX | LOCK_NB, $wouldblock);
if ($lock_file === false || (!$got_lock && !$wouldblock)) {
throw new Exception(
"Unexpected error opening or locking lock file. Perhaps you " .
"don't have permission to write to the lock file or its " .
"containing directory?"
);
}
else if (!$got_lock && $wouldblock) {
exit("Another instance is already running; terminating.\n");
}
// Lock acquired; let's write our PID to the lock file for the convenience
// of humans who may wish to terminate the script.
ftruncate($lock_file, 0);
fwrite($lock_file, getmypid() . "\n");
/*
The main body of your script goes here.
*/
echo "Hello, world!";
// All done; we blank the PID file and explicitly release the lock
// (although this should be unnecessary) before terminating.
ftruncate($lock_file, 0);
flock($lock_file, LOCK_UN);
Just set the path of your lock file to wherever you like and you're set.
If you need it to be absolutely crash-proof, you should use semaphores, which are released automatically when php ends the specific request handling.
A simpler approach would be to create a DB record or a file at the beginning of the execution, and remove it at the end. You could always check the "age" of that record/file, and if it's older than say 3 times the normal script execution, suppose it crashed and remove it.
There's no "silver bullet", it just depends on your needs.
If you are running Linux, this should work at the top of your script:
$running = exec("ps aux|grep ". basename(__FILE__) ."|grep -v grep|wc -l");
if($running > 1) {
exit;
}
A common way for *nix daemons (though not necessarily PHP scripts, but it will work) is to use a .pid file.
When the script starts check for the existence of a .pid file named for the script (generally stored in /var/run/). If it doesn't exist, create it setting its contents to the PID of the process running the script (using getmypid) then continue with normal execution. If it does exist read the PID from it and see if that process is still running, probably by running ps $pid. If it is running, exit. Otherwise, overwrite its contents with your PID (as above) and continue normal execution.
When execution finished, delete the file.
I know this is an old question but in case someone else is looking here I'll post some code. This is what I have done recently in a similar situation and it works well. Put put this code at the top of your file and if the same script is already running it will leave it be and end the new one.
I use it to keep a monitoring system running at all times. A cron job starts the script every 5 minutes but unless the other has stopped from some reason (usually if it has crashed, which is very rare!) the new one will just exit itself.
// The file to store our process file
define('PROCESS_FILE', 'process.pid');
// Check I am running from the command line
if (PHP_SAPI != 'cli') {
log_message('Run me from the command line');
exit;
}
// Check if I'm already running and kill myself off if I am
$pid_running = false;
if (file_exists(PROCESS_FILE)) {
$data = file(PROCESS_FILE);
foreach ($data as $pid) {
$pid = (int)$pid;
if ($pid > 0 && file_exists('/proc/' . $pid)) {
$pid_running = $pid;
break;
}
}
}
if ($pid_running && $pid_running != getmypid()) {
if (file_exists(PROCESS_FILE)) {
file_put_contents(PROCESS_FILE, $pid);
}
log_message('I am already running as pid ' . $pid . ' so stopping now');
exit;
} else {
// Make sure file has just me in it
file_put_contents(PROCESS_FILE, getmypid());
log_message('Written pid with id '.getmypid());
}
It will NOT work without modification on Windows, but should be fine in unix based systems.
You can use new Symfony 2.6 LockHandler.
Source
$lock = new LockHandler('update:contents');
if (!$lock->lock()) {
echo 'The command is already running in another process.';
}
This worked for me. Set a database record with a lock flag and a time stamp. My script should complete well within 15min so added that as a last locked feild to check:
$lockresult = mysql_query("
SELECT *
FROM queue_locks
WHERE `lastlocked` > DATE_SUB(NOW() , INTERVAL 15 MINUTE)
AND `locked` = 'yes'
AND `queid` = '1'
LIMIT 1
");
$LockedRowCount = mysql_num_rows($lockresult);
if($LockedRowCount>0){
echo "this script is locked, try again later";
exit;
}else{
//Set the DB record to locked and carry on son
$result = mysql_query("
UPDATE `queue_locks` SET `locked` = 'yes', `lastlocked` = CURRENT_TIMESTAMP WHERE `queid` = 1;
");
}
Then unlock it at the end of the script:
$result = mysql_query("UPDATE `queue_locks` SET `locked` = 'no' WHERE `queid` = 1;");
I know this is an old question, but there's an approach which hasn't been mentioned before that I think is worth considering.
One of the problems with a lockfile or database flag solution, as already mentioned, is that if the script fails for some reason other than normal completion it won't release the lock. And therefore the next instance won't start until the lock is either manually cleared or cleared by a clean-up function.
If, though, you are certain that the script should only ever be running once, then it's relatively easy to check from within the script whether it is already running when you start it. Here's some code:
function checkrun() {
exec("ps auxww",$ps);
$r = 0;
foreach ($ps as $p) {
if (strpos($p,basename(__FILE__))) {
$r++;
if ($r > 1) {
echo "too many instances, exiting\n";
exit();
}
}
}
}
Simply call this function at the start of the script, before you do anything else (such as open a database handler or process an import file), and if the same script is already running then it will appear twice in the process list - once for the previous instance, and once for this one. So, if it appears more than once, just exit.
A potential gotcha here: I'm assuming that you will never have two scripts with the same basename that may legitimately run simultaneously (eg, the same script running under two different users). If that is a possibility, then you'd need to extend the checking to something more sophisticated than a simple substring on the file's basename. But this works well enough if you have unique filenames for your scripts.
Assuming this is a linux server and you have cronjobs available
///Check for running script and run if non-exist///
#! /bin/bash
check=$(ps -fea | grep -v grep | grep script.php | wc -l)
date=$(date +%Y-%m%d" "%H:%M:%S)
if [ "$check" -lt 1 ]; then
echo "["$date"] Starting script" >> /path/to/script/log/
/sbin/script ///Call the script here - see below///
fi
script file
#/usr/bin/php /path/to/your/php/script.php
Home / Check if a PHP script is already running
Check if a PHP script is already running
If you have long running batch processes with PHP that are run by cron and you want to ensure there’s only ever one running copy of the script, you can use the functions getmypid() and posix_kill() to check to see if you already have a copy of the process running. This post has a PHP class for checking if the script is already running.
Each process running on a Linux/Unix computer has a pid, or process identifier. In PHP this can be retrieved using getmypid() which will return an integer number. This pid number can be saved to a file and each time the script is run a check made to see if the file exists. If it is the posix_kill() function can be used to see if a process is running with that pid number.
My PHP class for doing this is below. Please feel free to use this and modify to suit your individual requirements.
class pid {
protected $filename;
public $already_running = false;
function __construct($directory) {
$this->filename = $directory . '/' . basename($_SERVER['PHP_SELF']) . '.pid';
if(is_writable($this->filename) || is_writable($directory)) {
if(file_exists($this->filename)) {
$pid = (int)trim(file_get_contents($this->filename));
if(posix_kill($pid, 0)) {
$this->already_running = true;
}
}
}
else {
die("Cannot write to pid file '$this->filename'. Program execution halted.n");
}
if(!$this->already_running) {
$pid = getmypid();
file_put_contents($this->filename, $pid);
}
}
public function __destruct() {
if(!$this->already_running && file_exists($this->filename) && is_writeable($this->filename)) {
unlink($this->filename);
}
}
}
Use Class below
$pid = new pid('/tmp');
if($pid->already_running) {
echo "Already running.n";
exit;
}
else {
echo "Running...n";
}
Inspired by Mark Amery's answer I created this class. This might help someone. Simply change the "temp/lockFile.pid" to where you want the file placed.
class ProcessLocker
{
private $lockFile;
private $gotLock;
private $wouldBlock;
function __construct()
{
$this->lockFile = fopen('temp/lockFile.pid', 'c');
if ($this->lockFile === false) {
throw new Exception("Unable to open the file.");
}
$this->gotLock = flock($this->lockFile, LOCK_EX | LOCK_NB, $this->wouldBlock);
}
function __destruct()
{
$this->unlockProcess();
}
public function isLocked()
{
if (!$this->gotLock && $this->wouldBlock) {
return true;
}
return false;
}
public function lockProcess()
{
if (!$this->gotLock && !$this->wouldBlock) {
throw new Exception("Unable to lock the file.");
}
ftruncate($this->lockFile, 0);
fwrite($this->lockFile, getmypid() . "\n");
}
public function unlockProcess()
{
ftruncate($this->lockFile, 0);
flock($this->lockFile, LOCK_UN);
}
}
Simply use the class as such in the beginning of your script:
$locker = new ProcessLocker();
if(!$locker->isLocked()){
$locker->lockProcess();
} else{
// The process is locked
exit();
}
Read some texts about locking in PHP.
They all, mainly, direct to http://php.net/manual/en/function.flock.php .
This page talks about opening a file on the hard-disk!!
Is it really so? I mean, this makes locking really expensive - it means each time I want to lock I'll have to access the hard-disk )=
Can anymore comfort me with a delightful news?
Edit:
Due to some replies I've got here, I want to ask this;
My script will run only by one thread, or several? Because if it's by one then I obviously don't need a mutex. Is there a concise answer?
What exactly I'm trying to do
Asked by ircmaxell.
This is the story:
I have two ftp servers. I want to be able to show at my website how many online users are online.
So, I thought that these ftp servers will "POST" their stats to a certain PHP script page. Let's assume that the URL of this page is "http://mydomain.com/update.php".
On the website's main page ("http://mydomain.com/index.php") I will display the cumulative statistics (online users).
That's it.
My problem is that I'm not sure if, when one ftp server updates his stats while another does it too, the info will get mixed.
Like when multi-threading; Two threads increase some "int" variable at the same time. It will not happen as expected unless you sync between them.
So, will I have a problem? Yes, no, maybe?
Possible solution
Thinking hard about it all day long, I have an idea here and I want you to give your opinion.
As said these ftp servers will post their stats, once every 60sec.
I'm thinking about having this file "stats.php".
It will be included at the updating script that the ftp servers go to ("update.php") and at the "index.php" page where visitors see how many users are online.
Now, when an ftp server updates, the script at "update.php" will modify "stats.php" with the new cumulative statistics.
First it will read the stats included at "stats.php", then accumulate, and then rewrite that file.
If I'm not mistaken PHP will detect that the file ("stats.php") is changed and load the new one. Correct?
Well, most of PHP runs in a different process space (there are few threading implementations). The easy one is flock. It's guaranteed to work on all platforms.
However, if you compile in support, you can use a few other things such as the Semaphore extension. (Compile PHP with --enable-sysvsem). Then, you can do something like (note, sem_acquire() should block. But if it can't for some reason, it will return false):
$sem = sem_get(1234, 1);
if (sem_acquire($sem)) {
//successful lock, go ahead
sem_release($sem);
} else {
//Something went wrong...
}
The other options that you have, are MySQL user level locks GET_LOCK('name', 'timeout'), or creating your own using something like APC or XCache (Note, this wouldn't be a true lock, since race conditions could be created where someone else gets a lock between your check and acceptance of the lock).
Edit: To answer your edited question:
It all depends on your server configuration. PHP May be run multi-threaded (where each request is served by a different thread), or it may be run multi-process (where each request is served by a different process). It all depends on your server configuration...
It's VERY rare that PHP will serve all requests serially, with only one process (and one thread) serving all requests. If you're using CGI, then it's multi-process by default. If you're using FastCGI, it's likely multi-process and multi-thread. If you're using mod_php with Apache, then it depends on the worker type:
mpm_worker will be both multi-process and multi-thread, with the number of processes dictated by the ServerLimit variable.
prefork will be multi-process
perchild will be multi-process as well
Edit: To answer your second edited question:
It's quite easy. Store it in a file:
function readStatus() {
$f = fopen('/path/to/myfile', 'r');
if (!$f) return false;
if (flock($f, LOCK_SH)) {
$ret = fread($f, 8192);
flock($f, LOCK_UN);
fclose($f);
return $ret;
}
fclose($f);
return false;
}
function updateStatus($new) {
$f = fopen('/path/to/myfile', 'w');
if (!$f) return false;
if (flock($f, LOCK_EX)) {
ftruncate($f, 0);
fwrite($f, $new);
flock($f, LOCK_UN);
fclose($f);
return true;
}
fclose($f);
return false;
}
function incrementStatus() {
$f = fopen('/path/to/myfile', 'rw');
if (!$f) return false;
if (flock($f, LOCK_EX)) {
$current = fread($f, 8192);
$current++;
ftruncate($f, 0);
fwrite($f, $current);
flock($f, LOCK_UN);
fclose($f);
return true;
}
fclose($f);
return false;
}
The question is: Where will you store the stats that the FTP servers are pushing with POST to your update.php file? If it's a local file, than ircmaxell in the second post has answered you. You can do this with a mutex as well - the semaphore functions. Another solution is to use MySQL MyISAM table to store the stats and use something like update info_table set value = value + 1. It should lock the table, and serialize your requests, and you will have no problems.
I recently created my own simple implementation of a mutex-like mechanism using the flock function of PHP. Of course the code below can be improved, but it is working for most use cases.
function mutex_lock($id, $wait=10)
{
$resource = fopen(storage_path("app/".$id.".lck"),"w");
$lock = false;
for($i = 0; $i < $wait && !($lock = flock($resource,LOCK_EX|LOCK_NB)); $i++)
{
sleep(1);
}
if(!$lock)
{
trigger_error("Not able to create a lock in $wait seconds");
}
return $resource;
}
function mutex_unlock($id, $resource)
{
$result = flock($resource,LOCK_UN);
fclose($resource);
#unlink(storage_path("app/".$id.".lck"));
return $result;
}
Yes that's true, as PHP is run by Apache, and Apache can organize the threads of execution as it deems the best (see the various worker model). So if you want to access a resource one at a time, you either lock to a file (which is good if you are dealing with cron jobs for example), or you rely on database transaction mechanism, ACID features, and database resources locking, if you are dealing with data.
PHP doesn't support multithreading, every request (and therefore every PHP script) will be executed in only one thread (or even process, depending on the way you run PHP).