Let's consider a sample php script which deletes a line by user input:
$DELETE_LINE = $_GET['line'];
$out = array();
$data = #file("foo.txt");
if($data)
{
foreach($data as $line)
if(trim($line) != $DELETE_LINE)
$out[] = $line;
}
$fp = fopen("foo.txt", "w+");
flock($fp, LOCK_EX);
foreach($out as $line)
fwrite($fp, $line);
flock($fp, LOCK_UN);
fclose($fp);
I want to know if some user is currently executing this script and file "foo.txt" is locked, in same time or before completion of its execution, if some other user calls this script, then what will happen?
Will second users process wait for unlocking of file by first users? or line deletion by second users input will fail?
If you try to acquire an exclusive lock while another process has the file locked, your attempt will wait until the file is unlocked. This is the whole point of locking.
See the Linux documentation of flock(), which describes how it works in general across operating systems. PHP uses fcntl() under the hood so NFS shares are generally supported.
There's no timeout. If you want to implement a timeout yourself, you can do something like this:
$count = 0;
$timeout_secs = 10; //number of seconds of timeout
$got_lock = true;
while (!flock($fp, LOCK_EX | LOCK_NB, $wouldblock)) {
if ($wouldblock && $count++ < $timeout_secs) {
sleep(1);
} else {
$got_lock = false;
break;
}
}
if ($got_lock) {
// Do stuff with file
}
Related
I developed a very simple counter in PHP. It works as expected but occasionally it resets to zero. No idea why. I suspect it could be related to concurrent visitors but I have no idea how to prevent that in case I am correct. Here is the code:
function updateCounter($logfile) {
$count = (int)file_get_contents($logfile);
$file = fopen($logfile, 'w');
if (flock($file, LOCK_EX)) {
$count++ ;
fwrite($file, $count);
flock($file, LOCK_UN);
}
fclose($file);
return number_format((float)$count, 0, ',', '.') ;
}
Thank you in advance.
file_get_contents on a locked file will probably get a "false" (== 0) and the logfile is probably unlocked again, when it comes to writing.
A classic race condition...
As file_get_contents() can return false accessing a previously locked file, the consequent fwrite() may write a zero or 1, resetting our counter to zero.
So we try to read the counter file after the locking has been succeeded for us.
function updateCounter($logfile) {
//$count = (int)file_get_contents($logfile);
if(file_exists($logfile)) {
$mode = 'r+';
} else {
$mode = 'w+';
}
//
$file = fopen($logfile, $mode);
//
if (flock($file, LOCK_EX)) {
//
// read counter file:
//
$count = (int) fgets($file);
$count++ ;
//
// point to the beginning of the file:
//
rewind($file);
fwrite($file, $count);
flock($file, LOCK_UN);
}
fclose($file);
return number_format((float)$count, 0, ',', '.') ;
}
//
$logfile = "counter.log";
echo updateCounter($logfile);
Please see usernotes on https://www.php.net/manual/en/function.flock.php .
I would append a character into the file and use strlen on the file contents to get the hits. Please note that your file will get big overtime but this can be easily solved with a cronjob that sums it up and cache it into another readonly file.
You can also use !is_writeable and check if its locked and if so you can miss the hit or wait with a while loop until its writable. Tricky but it works. It depends how valuable each hit will be and how much effort you would like to invest in this counter.
When I run this function on multiple scripts one script generated warning:
fread(): Length parameter must be greater than 0
function test($n){
echo "<h4>$n at ".time()."</h4>";
for ($i = 0; $i<50; $i++ ){
$fp = fopen("$n.txt", "r");
$s = fread($fp, filesize("$n.txt") );
fclose($fp);
$fp = fopen("$n.txt", "w");
$s = $_SERVER['HTTP_USER_AGENT'].' '.time();
if (flock($fp, LOCK_EX)) { // acquire an exclusive lock
fwrite($fp, $s);
// fflush($fp);// flush output before releasing the lock
flock($fp, LOCK_UN); // release the lock
} else {
echo "Couldn't get the lock!";
}
}
}
I try to write reading of the file for multiple users, but only one user can write the file. I know that when I use fwrite with flock - LOC_EX, next scripts must wait till the write is finished. But here it seems like filesize doesn't wait till the write operation is finished. My opinion is that it tries to reach the file when the file size is 0, and as a result this produces the problem: 0 bytes will be read from the file, when it is written by original script.
Is it possible to fix this for fread function?
Purpose of this script is to test fread with some limit and to check the data which I read later, if the data are really written when I did not used fflush.
function test($n){
echo "<h4>$n at ".time()."</h4>";
for ($i = 0; $i<50; $i++ ){
$start = microtime(true);
$fp = fopen("$n.txt", "r");
if(filesize($n.txt) > 0)
{
$s = fread($fp, filesize($n.txt) );
fclose($fp);
$fp = fopen("$n.txt", "w");
$s = $_SERVER['HTTP_USER_AGENT'].' '.time();
if (flock($fp, LOCK_EX)) { // acquire an exclusive lock
fwrite($fp, $s);
// fflush($fp);// flush output before releasing the lock
flock($fp, LOCK_UN); // release the lock
} else {
echo "Couldn't get the lock!";
}
}
else
{
echo "Filesize must be greater than 0";
}
}
}
please change $s variables name its use same things two time
$fp = fopen("$n.txt", "r");
$s = fread($fp, filesize("$n.txt") );
fclose($fp);
The error occurs in the middle line of the above three lines.
Firstly, these three lines could be rewritten into a single line as follows:
$s = file_get_contents("$n.txt");
However, these isn't necessary, as these three lines are entirely redundant in your code. They don't do anything useful.
What they do is open a file, store its contents to $s and then close it.
But you are then immediately setting $s to a different value, thus throwing away the previous value, and making it pointless to have read it from the file in the first place.
If you need to keep the original contents of the file, then use file_get_contents() and make sure you don't overwrite the contents of the variable.
If you don't need the original contents of the file, then just delete those three lines from your code.
Incidentally, this error highlights a couple of good coding practices that you should take on board: Firstly, never re-use a variable for two different things, and secondly always give your variables (and functions) good names. $s is not a good name; $previousFileContents would be a better name; it would have made the error much more obvious.
Here is my Code with filename
it does work without problems if lets say i just use
update.php?pokemon=pikachu
it updates pikachu value in my found.txt +0.0001
But now my problem, when i have multiple threads running and randomly
2 threads are
update.php?pokemon=pikachu
and
update.php?pokemon=zaptos
i see the found.txt file
is empty than!!
so nothing is written in it then anymore.
So i guess its a bug when the php file is opened and another request is posted to the server.
How can i solve this problem this does accour often
found.txt
pikachu:2.2122
arktos:0
zaptos:0
lavados:9.2814
blabla:0
update.php
<?php
$file = "found.txt";
$fh = fopen($file,'r+');
$gotPokemon = $_GET['pokemon'];
$users = '';
while(!feof($fh)) {
$user = explode(':',fgets($fh));
$pokename = trim($user[0]);
$infound = trim($user[1]);
// check for empty indexes
if (!empty($pokename)) {
if ($pokename == $gotPokemon) {
if ($gotPokemon == "Pikachu"){
$infound+=0.0001;
}
if ($gotPokemon == "Arktos"){
$infound+=0.0001;
}
if ($gotPokemon == "Zaptos"){
$infound+=0.0001;
}
if ($gotPokemon == "Lavados"){
$infound+=0.0001;
}
}
$users .= $pokename . ':' . $infound;
$users .= "\r\n";
}
}
file_put_contents('found.txt', $users);
fclose($fh);
?>
I would create an exclusive lock after open the file and then release the lock before closing the file:
For creating an exclusive lock over the file:
flock($fh, LOCK_EX);
To delete it:
flock($fh, LOCK_UN);
Anyway you will need to check if other threads hot already the lock, so the first idea coming up is to try a few attempts to get the lock and if it's not finally possible, to inform the user, throw an exception or whatever other action to avoid an infinite loop:
$fh = fopen("found.txt", "w+");
$attempts = 0;
do {
$attempts++;
if ($attempts > 5) {
// throw exception or return response with http status code = 500
}
if ($attempts != 1) {
sleep(1);
}
} while (!flock($fh, LOCK_EX));
// rest of your code
file_put_contents('found.txt', $users);
flock($fh, LOCK_UN); // release the lock
fclose($fh);
Update
Probably the issue still remains because the reading part, so let's create also a shared lock before start reading and also let's simplify the code:
$file = "found.txt";
$fh = fopen($file,'r+');
$gotPokemon = $_GET['pokemon'];
$users = '';
$wouldblock = true;
// we add a shared lock for reading
$locked = flock($fh, LOCK_SH, $wouldblock); // it will wait if locked ($wouldblock = true)
while(!feof($fh)) {
// your code inside while loop
}
// we add an exclusive lock for writing
flock($fh, LOCK_EX, $wouldblock);
file_put_contents('found.txt', $users);
flock($fh, LOCK_UN); // release the locks
fclose($fh);
Let's see if it works
I have a script i use that checks an IP address stored within my hosts.allow file against what IP is mapped to my dyndns hostname so i can log into my servers once i've synced my current IP to that hostname. For some reason though the script seems to cause really intermittent issues.
within my hosts.allow file i have a section like this:
#SOme.gotdns.com
sshd : 192.168.0.1
#EOme.gotdns.com
#SOme2.gotdns.com
sshd : 192.168.0.2
#EOme2.gotdns.com
I have a script running on a cron (every minute) that looks like this:
#!/usr/bin/php
<?php
$hosts = array('me.gotdns.com','me2.gotdns.com');
foreach($hosts as $host)
{
$ip = gethostbyname($host);
$replaceWith = "#SO".$host."\nsshd : ".$ip."\n#EO".$host;
$filename = '/etc/hosts.allow';
$handle = fopen($filename,'r');
$contents = fread($handle, filesize($filename));
fclose($handle);
if (preg_match('/#SO'.$host.'(.*?)#EO'.$host.'/si', $contents, $regs))
{
$result = $regs[0];
}
if($result != $replaceWith)
{
$newcontents = str_replace($result,$replaceWith,$contents);
$handle = fopen($filename,'w');
if (fwrite($handle, $newcontents) === FALSE) {
}
fclose($handle);
}
}
?>
The problem i have is that intermittently characters are being dropped (i assume during the replace) that causes future updates to fail as it inserts something like:
#SOme.gotdns.com
sshd : 192.168.0.1
#EOme.gotdn
note the missing "s.com"
This of course means i lose access to the server, any ideas why this would be happening?
Thanks.
that might be because of script execution time - can be too short- OR 1 min interval is too short. While cron is doing the job, another process of script starts and it may effect the first one.
This is almost certainly because the script hasn't finished executing within the one minute time period before it's started again via cron. You need to implement some sort of locking, or use a tool that only allows once instance of the script to be run. There are several tools available out there that can do this, for example lockrun.
I would say that in order to do this safely, you should acquire an exclusive lock on the file at the beginning of the script, read it all into memory once, modify it in memory, then write it back to the file at the end. This would also be considerably more efficient in terms of disk I/O.
You should also alter the cron job to run less frequently. It is likely that the reason you currently have this problem is because two processes are running at the same time - by locking the file, if this is the case, you risk having the processes stack up waiting to acquire a lock. Setting it for every 5 minutes should be good enough - your IP shouldn't change that often!
So do this (FIXED):
#!/usr/bin/php
<?php
// Settings
$hosts = array(
'me.gotdns.com',
'me2.gotdns.com'
);
$filename = '/etc/hosts.allow';
// No time limit (shouldn't be necessary with CLI, but just in case)
set_time_limit(0);
// Open the file in read/write mode and lock it
// flock() should block until it gets a lock
if ((!$handle = fopen($filename, 'r+')) || !flock($handle, LOCK_EX)) exit(1);
// Read the file
if (($contents = fread($handle, filesize($filename)) === FALSE) exit(1);
// Will be set to true if we actually make any changes to the file
$changed = FALSE;
// Loop hosts list
foreach ($hosts as $host) {
// Get current IP address of host
if (($ip = gethostbyname($host)) == $host) continue;
// Find the entry in the file
$replaceWith = "#SO{$host}\nsshd : {$ip}\n#EO{$host}";
if (preg_match("/#SO{$host}(.*?)#EO{$host}/si", $contents, $regs)) {
// Only do this if there was a match - otherise risk overwriting previous
// entries because you didn't reset the value of $result
if ($regs[0] != $replaceWith) {
$changed = TRUE;
$contents = str_replace($regs[0], $replaceWith, $contents);
}
}
}
// We'll only change the contents of the file if the data changed
if ($changed) {
ftruncate($handle, 0); // Zero the length of the file
rewind($handle); // start writing from the beginning
fwrite($handle, $contents); // write the new data
}
flock($handle, LOCK_UN); // Unlock
fclose($handle); // close
When a user upload a file(users can upload multiple files)
exec('nohup php /main/apache2/work/upload/run.php &');
I am using nohup as the it needs to be executed in the back end.
In my original design run.php scans the directory using scandir everytime it's executed. Get an exclusive lock LOCK_EX on the file using flock and use LOCK_NB to skip the file if it has a lock and go the next one. If a file has a lock //Do logic. The problem is that the server is missing fcntl() library and since flock uses that library to execute the locking mechanism, flock won't work at the moment. It's going to take a month or two to get that installed(I have no control over that).
So my work around for that is have a temporary file lock.txt that acts a lock. If the filename exists in lock.txt skip the file and go to the next one.
$dir = "/main/apache2/work/upload/files/";
$files = scandir($dir);
$fileName = "lock.txt";
for($i=0; $i<count($files); $i++)
{
if(substr(strrchr($files[$i],'.csv'),-4) == '.csv')
{
if($file_handle = fopen("$fileName", "rb"))
{
while(!feof($file_handle))
{
$line = fgets($file_handle);
$line = rtrim($line);
if($line == "")
{
break;
}
else
{
if($files[$i] == $line)
{
echo "Reading from lock: ".$line."</br>";
$i++; //Go to next file
}
}
}
fclose($file_handle);
}
if($i >= count($files))
{
die("$i End of file");
}
if($file_handle = fopen("$fileName", "a+"))
{
if(is_writable($fileName))
{
$write = fputs($file_handle, "$files[$i]"."\n");
//Do logic
//Delete the file name - Stuck here
fclose($file_handle);
}
}
}
else
{
//Do nothing
}
}
How can I delete the filename from lock.txt?
More importantly, is there a better way to lock a file in php without using flock?
Having a shared lock database simply moves the locking problem to that file; it doesn't solve it.
A much better solution is to use one lock file per real file. If you want to lock access to myFile.csv then you check file_exists('myFile.csv.lock') and touch('myFile.csv.lock') if it doesn't exist. And unlink('myFile.csv.lock') when done.
Now, there is a possible race-condition between file_exists() and touch(), which can be mitigated by storing the PID in the file and checking if getmypid() is indeed the process holding the lock.