how to synchronise multiple processes in PHP to simulate wait()/notifyAll() - php

I'm trying to test a race condition in PHP. I'd like to have N PHP processes get ready to do something, then block. When I say "go", they should all execute the action at the same time. Hopefully this will demonstrate the race.
In Java, I would use Object.wait() and Object.notifyAll(). What can I use in PHP?
(Either Windows or linux native answers are acceptable)

Create a file "wait.txt"
Start N processes, each with the code shown below
Delete the "wait.txt" file.
...
<?php
while (file_exists('wait.txt')) {}
runRaceTest();

Usually with PHP file lock approach is used. One create a RUN_LOCK or similar file and asks for file_exists("RUN_LOCK"). This system is also used to secure potential endless loops in recursive threads.
I decided to require the file for the execution. Other approach may be, that existence of the file invokes the blocking algorithm. That depends on your situation. Always the safer situation should be the easier to achieve.
Wait code:
/*prepare the program*/
/* ... */
/*Block until its time to go*/
define("LOCK_FILE", "RUN_UNLOCK"); //I'd define this in some config.php
while(!file_exists(LOCK_FILE)) {
usleep(1); //No sleep will eat lots of CPU
}
/*Execute the main code*/
/* ... */
/*Delete the "run" file, so that no further executions should be allowed*/
usleep(1); //Just for sure - we want other processes to start execution phase too
if(file_exists(LOCK_FILE))
unlink(LOCK_FILE);
I guess it would be nice to have a blocking function for that, like this one:
function wait_for_file($filename, $timeout = -1) {
if($timeout>=0) {
$start = microtime(true)*1000; //Remember the start time
}
while(!file_exists($filename)) { //Check the file existence
if($timeout>=0) { //Only calculate when timeout is set
if((microtime(true)*1000-$start)>$timeout) //Compare current time with start time (current always will be > than start)
return false; //Return failure
}
usleep(1); //Save some CPU
}
return true; //Return success
}
It implements timeout. You don't need them but maybe someone else will.
Usage:
header("Content-Type: text/plain; charset=utf-8");
ob_implicit_flush(true);while (#ob_end_clean()); //Flush buffers so the output will be live stream
define("LOCK_FILE","RUN_FOREST_RUN"); //Define lock file name again
echo "Starting the blocking algorithm. Waiting for file: ".LOCK_FILE."\n";
if(wait_for_file(LOCK_FILE, 10000)) { //Wait for 10 secconds
echo "File found and deleted!\n";
if(file_exists(LOCK_FILE)) //May have been deleted by other proceses
unlink(LOCK_FILE);
}
else {
echo "Wait failed!\n";
}
This will output:
Starting the blocking algorithm. Waiting for file: RUN_FOREST_RUN
Wait failed!
~or~
Starting the blocking algorithm. Waiting for file: RUN_FOREST_RUN
File found and deleted!

PHP doesn't have multithreading. And its not planned to be implemented either.
You can try hacks with sockets though or 0MQ to communicate between multiple processes
See Why does PHP not support multithreading?
Php multithread

Related

flock call within function always 'succeeds', ignoring previous lock

To prevent multiple instances of a PHP-based daemon I wrote from ever running simultaneously, I wrote a simple function to acquire a lock with flock when the process starts, and called it at the start of my daemon. A simplified version of the code looks like this:
#!/usr/bin/php
<?php
function acquire_lock () {
$file_handle = fopen('mylock.lock', 'w');
$got_lock_successfully = flock($file_handle, LOCK_EX);
if (!$got_lock_successfully) {
throw new Exception("Unexpected failure to acquire lock.");
}
}
acquire_lock(); // Block until all other instances of the script are done...
// ... then do some stuff, for example:
for ($i=1; $i<=10; $i++) {
echo "Sleeping... $i\n";
sleep(1);
}
?>
When I execute the script above multiple times in parallel, the behaviour I expect to see - since the lock is never explicitly released throughout the duration of the script - is that the second instance of the script will wait until the first has completed before it proceeds past the acquire_lock() call. In other words, if I run this particular script in two parallel terminals, I expect to see one terminal count to 10 while the other waits, and then see the other count to 10.
This is not what happens. Instead, I see both scripts happily executing in parallel - the second script does not block and wait for the lock to be available.
As you can see, I'm checking the return value from flock and it is true, indicating that the (exclusive) lock has been acquired successfully. Yet this evidently isn't preventing another process from acquiring another 'exclusive' lock on the same file.
Why, and how can I fix this?
Simply store the file pointer resource returned from fopen in a global variable. In the example given in the question, $file_handle is automatically destroyed upon going out of scope when acquire_lock() returns, and this releases the lock taken out by flock.
For example, here is a modified version of the script from the question which exhibits the desired behaviour (note that the only change is storing the file handle returned by fopen in a global):
#!/usr/bin/php
<?php
function acquire_lock () {
global $lock_handle;
$lock_handle = fopen('mylock.lock', 'w');
$got_lock_successfully = flock($lock_handle, LOCK_EX);
if (!$got_lock_successfully) {
throw new Exception("Unexpected failure to acquire lock.");
}
}
acquire_lock(); // Block until all other instances of the script are done...
// ... then do some stuff, for example:
for ($i=1; $i<=10; $i++) {
echo "Sleeping... $i\n";
sleep(1);
}
?>
Note that this seems to be a bug in PHP. The changelog from the flock documentation states that in version 5.3.2:
The automatic unlocking when the file's resource handle is closed was removed. Unlocking now always has to be done manually.
but at least for PHP 5.5, this is false; flock locks are released both by explicit calls to fclose and by the resource handle going out of scope.
I reported this as a bug in November 2014 and may update this question and answer pair if it is ever resolved. In case I get eaten by piranhas before then, you can check the bug report yourself to see if this behaviour has been fixed: https://bugs.php.net/bug.php?id=68509

PHP Singleton class for all requests

I have a simple problem. I use php as server part and have an html output. My site shows a status about an other server. So the flow is:
Browser user goes on www.example.com/status
Browser contacts www.example.com/status
PHP Server receives request and ask for stauts on www.statusserver.com/status
PHP Receives the data, transforms it in readable HTML output and send it back to the client
Browser user can see the status.
Now, I've created a singleton class in php which accesses the statusserver only 8 seconds. So it updates the status all 8 seconds. If a user requests for update inbetween, the server returns the locally (on www.example.com) stored status.
That's nice isn't it? But then I did an easy test and started 5 browser windows to see if it works. Here it comes, the php server created a singleton class for each request. So now 5 Clients requesting all 8 seconds the status on the statusserver. this means I have every 8 second 5 calls to the status server instead of one!
Isn't there a possibility to provide only one instance to all users within an apache server? That would be solve the problem in case 1000 users are connecting to www.example.com/status....
thx for any hints
=============================
EDIT:
I already use a caching on harddrive:
public function getFile($filename)
{
$diff = (time()-filemtime($filename));
//echo "diff:$diff<br/>";
if($diff>8){
//echo 'grösser 8<br/>';
self::updateFile($filename);
}
if (is_readable($filename)) {
try {
$returnValue = #ImageCreateFromPNG($filename);
if($returnValue == ''){
sleep(1);
return self::getFile($filename);
}else{
return $returnValue;
}
} catch (Exception $e){
sleep(1);
return self::getFile($filename);
}
} else {
sleep(1);
return self::getFile($filename);
}
}
this is the call in the singleton. I call for a file and save it on harddrive. but all the request call it at same time and start requesting the status server.
I think the only solution would be a standalone application which does an update every 8 seconds on the file... All request should just read the file and nomore able to update it.
This standalone could be a perl script or something similar...
Php requests are handled by different processes and each of them have a different state, there isn't any resident process like in other web development framework. You should handle that behavior directly in your class using for instance some caching.
The method which query the server status should have this logic
public function getStatus() {
if (!$status = $cache->load()) {
// cache miss
$status = // do your query here
$cache->save($status); // store the result in cache
}
return $status;
}
In this way only one request of X will fetch the real status. The X value depends on your cache configuration.
Some cache library you can use:
APC
Memcached
Zend_Cache which is just a wrapper for actual caching engines
Or you can store the result in plain text file and on every request check for the m_time of the file itself and rewrite it if more than xx seconds are passed.
Update
Your code is pretty strange, why all those sleep calls? Why a try/catch block when ImageCreateFromPNG does not throw?
You're asking a different question, since php is not an application server and cannot store state across processes your approach is correct. I suggest you to use APC (uses shared memory so it would be at least 10x faster than reading a file) to share status across different processes. With this approach your code could become
public function getFile($filename)
{
$latest_update = apc_fetch('latest_update');
if (false == $latest_update) {
// cache expired or first request
apc_store('latest_update', time(), 8); // 8 is the ttl in seconds
// fetch file here and save on local storage
self::updateFile($filename);
}
// here you can process the file
return $your_processed_file;
}
With this approach the code in the if part will be executed from two different processes only if a process is blocked just after the if line, which should not happen because is almost an atomic operation.
Furthermore if you want to ensure that you should use something like semaphores to handle that, but it would be an oversized solution for this kind of requirement.
Finally imho 8 seconds is a small interval, I'd use something bigger, at least 30 seconds, but this depends from your requirements.
As far as I know it is not possible in PHP. However, you surely can serialize and cache the object instance.
Check out http://php.net/manual/en/language.oop5.serialization.php

PHP mutual exclusion (mutex)

Read some texts about locking in PHP.
They all, mainly, direct to http://php.net/manual/en/function.flock.php .
This page talks about opening a file on the hard-disk!!
Is it really so? I mean, this makes locking really expensive - it means each time I want to lock I'll have to access the hard-disk )=
Can anymore comfort me with a delightful news?
Edit:
Due to some replies I've got here, I want to ask this;
My script will run only by one thread, or several? Because if it's by one then I obviously don't need a mutex. Is there a concise answer?
What exactly I'm trying to do
Asked by ircmaxell.
This is the story:
I have two ftp servers. I want to be able to show at my website how many online users are online.
So, I thought that these ftp servers will "POST" their stats to a certain PHP script page. Let's assume that the URL of this page is "http://mydomain.com/update.php".
On the website's main page ("http://mydomain.com/index.php") I will display the cumulative statistics (online users).
That's it.
My problem is that I'm not sure if, when one ftp server updates his stats while another does it too, the info will get mixed.
Like when multi-threading; Two threads increase some "int" variable at the same time. It will not happen as expected unless you sync between them.
So, will I have a problem? Yes, no, maybe?
Possible solution
Thinking hard about it all day long, I have an idea here and I want you to give your opinion.
As said these ftp servers will post their stats, once every 60sec.
I'm thinking about having this file "stats.php".
It will be included at the updating script that the ftp servers go to ("update.php") and at the "index.php" page where visitors see how many users are online.
Now, when an ftp server updates, the script at "update.php" will modify "stats.php" with the new cumulative statistics.
First it will read the stats included at "stats.php", then accumulate, and then rewrite that file.
If I'm not mistaken PHP will detect that the file ("stats.php") is changed and load the new one. Correct?
Well, most of PHP runs in a different process space (there are few threading implementations). The easy one is flock. It's guaranteed to work on all platforms.
However, if you compile in support, you can use a few other things such as the Semaphore extension. (Compile PHP with --enable-sysvsem). Then, you can do something like (note, sem_acquire() should block. But if it can't for some reason, it will return false):
$sem = sem_get(1234, 1);
if (sem_acquire($sem)) {
//successful lock, go ahead
sem_release($sem);
} else {
//Something went wrong...
}
The other options that you have, are MySQL user level locks GET_LOCK('name', 'timeout'), or creating your own using something like APC or XCache (Note, this wouldn't be a true lock, since race conditions could be created where someone else gets a lock between your check and acceptance of the lock).
Edit: To answer your edited question:
It all depends on your server configuration. PHP May be run multi-threaded (where each request is served by a different thread), or it may be run multi-process (where each request is served by a different process). It all depends on your server configuration...
It's VERY rare that PHP will serve all requests serially, with only one process (and one thread) serving all requests. If you're using CGI, then it's multi-process by default. If you're using FastCGI, it's likely multi-process and multi-thread. If you're using mod_php with Apache, then it depends on the worker type:
mpm_worker will be both multi-process and multi-thread, with the number of processes dictated by the ServerLimit variable.
prefork will be multi-process
perchild will be multi-process as well
Edit: To answer your second edited question:
It's quite easy. Store it in a file:
function readStatus() {
$f = fopen('/path/to/myfile', 'r');
if (!$f) return false;
if (flock($f, LOCK_SH)) {
$ret = fread($f, 8192);
flock($f, LOCK_UN);
fclose($f);
return $ret;
}
fclose($f);
return false;
}
function updateStatus($new) {
$f = fopen('/path/to/myfile', 'w');
if (!$f) return false;
if (flock($f, LOCK_EX)) {
ftruncate($f, 0);
fwrite($f, $new);
flock($f, LOCK_UN);
fclose($f);
return true;
}
fclose($f);
return false;
}
function incrementStatus() {
$f = fopen('/path/to/myfile', 'rw');
if (!$f) return false;
if (flock($f, LOCK_EX)) {
$current = fread($f, 8192);
$current++;
ftruncate($f, 0);
fwrite($f, $current);
flock($f, LOCK_UN);
fclose($f);
return true;
}
fclose($f);
return false;
}
The question is: Where will you store the stats that the FTP servers are pushing with POST to your update.php file? If it's a local file, than ircmaxell in the second post has answered you. You can do this with a mutex as well - the semaphore functions. Another solution is to use MySQL MyISAM table to store the stats and use something like update info_table set value = value + 1. It should lock the table, and serialize your requests, and you will have no problems.
I recently created my own simple implementation of a mutex-like mechanism using the flock function of PHP. Of course the code below can be improved, but it is working for most use cases.
function mutex_lock($id, $wait=10)
{
$resource = fopen(storage_path("app/".$id.".lck"),"w");
$lock = false;
for($i = 0; $i < $wait && !($lock = flock($resource,LOCK_EX|LOCK_NB)); $i++)
{
sleep(1);
}
if(!$lock)
{
trigger_error("Not able to create a lock in $wait seconds");
}
return $resource;
}
function mutex_unlock($id, $resource)
{
$result = flock($resource,LOCK_UN);
fclose($resource);
#unlink(storage_path("app/".$id.".lck"));
return $result;
}
Yes that's true, as PHP is run by Apache, and Apache can organize the threads of execution as it deems the best (see the various worker model). So if you want to access a resource one at a time, you either lock to a file (which is good if you are dealing with cron jobs for example), or you rely on database transaction mechanism, ACID features, and database resources locking, if you are dealing with data.
PHP doesn't support multithreading, every request (and therefore every PHP script) will be executed in only one thread (or even process, depending on the way you run PHP).

How do I restore this script after a hardware failure?

I know this is a bit generic, but I'm sure you'll understand my explanation. Here is the situation:
The following code is executed every 10 minutes. Variable "var_x" is always read/written to an external text file when its refereed to.
if ( var_x != 1 )
{
var_x = 1;
//
// here is where the main body of the script is.
// it can take hours to completely execute.
//
var_x = 0;
}
else
{
// exit script as it's already running.
}
The problem is: if I simulate a hardware failure (do a hard reset when the script is executing) then the main script logic will never execute again because "var_x" will always be "1". (I already have logic to work out the restore point).
Thanks.
You should lock and unlock files with flock:
$fp = fopen($your_file);
if (flock($fp, LOCK_EX)) { )
{
//
// here is where the main body of the script is.
// it can take hours to completely execute.
//
flock($fp, LOCK_UN);
}
else
{
// exit script as it's already running.
}
Edit:
As flock seems not to work correctly on Windows machines, you have to resort to other solutions. From the top of my head an idea for a possible solution:
Instead of writing 1 to var_x, write the process ID retrieved via getmypid. When a new instance of the script reads the file, it should then lookup for a running process with this ID, and if the process is a PHP script. Of course, this can still go wrong, as there is the possibility of another PHP script obtaining the same PID after a hardware failure, so the solution is far from optimal.
Don't you think this would be better solved using file locks? (When the reset occurs file locks are reset as well)
http://php.net/flock
It sounds like you're doing some kind of manual semaphore for process management.
Rather than writing to a file, perhaps you should use an environment variable instead. That way, in the event of failure, your script will not have a closed semaphore when you restore.

Php showing 97% of CPU usage

I have a game site developed using flash and php. The php code contains 4000 lines and it will run as a cron. Inside the code, there is one while loop which will run infinitely for checking any data is written in the socket and call different functions accordingly and will send the results back to the sockets. From flash, it will get the results and will be displayed.
The problem Im facing is, somewhere from the php code, it is leaking memory. Since it is very big, I can not find out from where it is happening. Moreover it can be run only as a cron. Is there any tool to find out the memory leakage ? I have heard about xdebug but I didnt use. Any other ?
check.php (as cron)
$sock = fsockopen(IP_ADDRESS, PORT, $sock_error_code, $sock_error_string, 10); if (!$sock){
$message = "Server was down, restarting...\n\n";
$last_line = system("php -q gameserver/server.php", $retval);} else {
$message = "Server is up...";
$message .= $sock_error_string." (".$sock_error_code.")\n\n";}
server.php (only some part)
class gameserver {
var $server_running = true;
function gameserver() {
global $cfg, $db;
$this->max_connections = $cfg["server"]["max-connections"];
$this->start_socket();
echo "Gameserver initialized\n";
while ($this->server_running) {
$read = $this->get_socket_list();
$temp = socket_select($read, $null, $null, 0, 15);
if (!empty($read)) {
$this->read_sockets($read);
}
$db->reconnection();
$this->update_DB_records();
$this->check_games_progress();
if ($this->soft_shutdown && $this->active_games == 0) {
$this->server_running = false;
echo "soft shutdown complete\n";
}
}
$this->stop_socket();
echo "Server shut down\n";
}} $server = new gameserver();
Two things, first, ensure that you sleep at least once inside the loop, to ensure that you don't use 97% cpu.
Second, a trick I've found is, if there is any database activity, to call mysql_free_result (or it's equivalent for other DBMS') to free up the memory used to store the result of the query.
Are you starting never-ending programs from CRON? Cron will start a new instance according to the schedule you specify and you'll end up with several running programs doing the same thing.. Could this be your problem?
I am assuming that you are not starting a new instance using cron every minute, and having each run an infinite loop:
XDebug is probably your best bet. Other than that, you could use memory_get_usage() and log memory usage in specific points of your loop.
Could it simply be that your script accumulates data and doesn't clean it up properly on the end of each loop?

Categories