I had a newcomer (the next door teenager) write some php code to track some usage on my web site. I'm not familiar with php so I'm asking a bit about concurrent file access.
My native app (on Windows), occasionally logs some data to my site by hitting the URL that contains my php script. The native app does not examine the returned data.
$fh = fopen($updateFile, 'a') or die("can't open file");
fwrite($fh, $ip);
fwrite($fh, ', ');
fwrite($fh, $date);
fwrite($fh, ', ');
fwrite($fh, implode(', ', $_GET));
fwrite($fh, "\r\n");
fclose($fh);
This is a low traffic site, and the data is not critical. But what happens if two users collide and two instances of the script each try to add a line to the file? Is there any implicit file locking in php?
Is the code above at least safe from locking up and never returning control to my user? Can the file get corrupted? If I have the script above delete the file every month, what happens if another instance of the script is in the middle of writing to the file?
You should put a lock on the file:
$fp = fopen($updateFile, 'w+');
if (flock($fp, LOCK_EX)) {
fwrite($fp, 'a');
flock($fp, LOCK_UN);
} else {
echo 'can\'t lock';
}
fclose($fp);
For the record, I worked in a library that does that:
https://github.com/EFTEC/DocumentStoreOne
It allows to CRUD documents by locking the file. I tried 100 concurrent users (100 calls to the PHP script at the same time) and it works.
However, it doesn't use flock but mkdir:
while (!#mkdir("file.lock")) {
// use the file
fopen("file"...)
#rmdir("file.lock")
}
Why?
mkdir is atomic, so the lock is atomic: In a single step, you lock or you don't.
It's faster than flock(). Apparently flock requires several calls to the file system.
flock() depends on the system.
I did a stress test and it worked.
Since this is an append to the file, the best way would be to aggregate the data and write it to the file in one fwrite(), providing the data to be written is not bigger then the file buffer. Ofcourse you don't always know the size of the buffer, so flock(); is always a good option.
Related
I have a cache file that is updated every hour or so. The file size ranges from 100KB to 1MB. The way the cache is updated is with the file_put_contents() method.
Only the server writes to the file. However, there is continuous access to the file. The file is accessed by users by a script that performs a one time read through readfile() to echo it to the user.
If the file is being read by the caching script, and the server runs the user reading script, or the other way around, would there be a problem? Or is this handled automatically by PHP>
Basically, you should lock the file while writing or reading. At least, it guarantees that there is no problem. It is the way of good programming!. The example is shown below.
<?php
$fp = fopen("/tmp/lock.txt", "w+");
if (flock($fp, LOCK_EX)) { // do an exclusive lock
fwrite($fp, "Write something here\n");
flock($fp, LOCK_UN); // release the lock
} else {
echo "Couldn't lock the file !";
}
fclose($fp);
?>
More information
In normal condition , everything is ok, an I can write and create new files with fopen() and fwrite() but under "heavy" DDOS attacks , when file pointer is located at 0 , i cant write anything to file.eg. using "w" mod ,result will be a blank file , but by using "a" or "c" mod , if file not exist or be empty, nothing will be written (and just create a blank file too) , but if file has some characters , it will writes after characters or will clear and rewrite new characters respectively.
and when DDOS stopped , everything would be Ok.
here is simple code that I'm using for test, what is the problem? Can I fix it?
I'm using php5 in ubuntu with apache and lighttpd...
<?php
$fp = fopen('data.txt', 'w');
fwrite($fp, '1');
fputs($fp, '23');
fclose($fp);
?>
The way I understood the question is that you have problems running this code when there are multiple requests accessing the .php file (and thus the file you are writing to) at the same time.
Now, while it is far from being foolproof, flock() is there to help with this. The basic concept is that you'd ask for a lock of the file before writing and only write to a file if you're able to get the lock to that file, like
$fp = fopen( $filename,"w"); // open it for WRITING ("w")
if (flock($fp, LOCK_EX | LOCK_NB)) {
// do your file writes here
// when you're done,
// flush your file writes to a file before unlocking
fflush($fp);
// unlock the file
flock($fp, LOCK_UN);
} else {
// flock() returned false, no lock obtained
print "Could not lock $filename!\n";
}
fclose($fp);
You can read some more details from the manual entry or this article.
I am building a system to create files that will range from a few Kb to say, around 50Mb, and this question is more out of curiosity than anything else. I couldn't find any answers online.
If I use
$handle=fopen($file,'w');
where is the $handle stored before I call
fclose($handle);
? Is it stored in the system's memory, or in a temp file somewhere?
Secondly, I am building the file using a loop that takes 1024 bytes of data at a time, and each time writes data as:
fwrite($handle, $content);
It then calls
fclose($handle);
when the loop is complete and all data is written. However, would it be more efficient or memory friendly to use a loop that did
$handle = fopen($file, 'a');
fwrite($handle, $content);
fclose($handle);
?
In PHP terminology, fopen() (as well as many other functions) returns a resource. So $handle is a resource that references the file handle that is associated with your $file.
Resources are in-memory objects, they are not persisted to the file system by PHP.
Your current methodology is the more efficient of the two options. Opening, writing to, and then closing the same file over and over again is less efficient than just opening it once, writing to it many times, and then closing it. Opening and closing the file requires setting up input and output buffers and allocating other internal resources, which are comparatively expensive operations.
Your file handle is just another memory reference and is stored in the stack memory just like other program variables and resources. Also in terms of file I/O, open and close once, and write as many times as you need - that is the most efficient way.
$handle = fopen($file, 'a'); //open once
while(condition){
fwrite($handle, $content); //write many
}
fclose($handle); //close once
According to PHP DOCS fopen() creates a stream which is a File handle. It is associated with file in filesystem.
Creating new File handle every time you need to write another 1024 bytes would be terribly slow.
I am trying to save tokens to a php file using this code, but after 2kb the file mysteriously empties and I lose all the data. Why does this happen? how do I prevent it?
$fh = fopen('token.txt', 'a+');
fwrite($fh, $access_token . "\n");
fclose($fh);
As per comments elsewhere, using files to store data from multiple concurrent processes is a recipe for failure. You can minimise the damage (at the risk of introducing deadlocks and race conditions) by ensuring you get a valid flock() on the file before attempting to read or write from it.
$fh = fopen('token.txt', 'a');
if (flock($fh, LOCK_EX)) {
fwrite($fh, $access_token . "\n");
fflush($fh);
flock($fh, LOCK_UN);
} else {
trigger_error("failed to lock file");
}
fclose($fh);
If you're just logging then use the syslog facility. If you're performing the full set of CRUD operations, then use a DBMS.
Some ideas:
Multiple invocations of your PHP page are stomping over each other. If two processes/threads open the same file for append simultaneously, I would not be surprised if the result is an empty file.
Change open mode from a+ to a. From your code it appears that you need only to write, not to read/write.
Check your filesystem available space (df -h) and your user's disk quota (quota -h). Are you out of space?
How can I make it so when a user clicks on a link on my web page, it writes to a .txt file named "Count.txt", which contains only a number and adds 1 to that number? Thank you.
If you forego any validity checking you could do it with something as simple as:
file_put_contents($theCounterFile, file_get_contents($theCounterFile)+1);
Addition:
There's talk about concurrency in this thread and it should be noted that it is a good idea to use a database and transactions to deal with concurrency, I'd highly recommend against writing a bunch of plumbing code to do this in a file.
If you've ever had, or think you might ever have two requests for the resource in the same second you should look into PDO with mysql, or PDO with SQLite instead of a file, use transactions (and InnoDB or better if you're going for mysql).
But really, even if you get two requests in the same microsecond (highly unlikely), chances of locking the file are slim as it will not be kept open and the two requests will probably not be handled parallel enough to lock anyway. Reality check: how many hits on the same resource do you get on average in the same minute?...
If you decide to do anything more advanced, like say two numbers, you may want to consider using SQLite. It's about as about as fast and as simple as opening and closing a file, but is much more flexible.
Open the file, lock the file (VERY important), read the number currently in there, add 1 to the number, write number back to file, release the lock and close the file.
ie. something like :
$fp = fopen("count.txt", "r+");
if (flock($fp, LOCK_EX)) { // do an exclusive lock
$num = fread($fp, 10);
$num++;
fseek($fp, 0);
fwrite($fp, $num);
flock($fp, LOCK_UN); // release the lock
} else {
// handle error
}
fclose($fp);
should work (not tested).
Generally this is quite easy:
$count = (int)file_get_contents('/path/to/Count.txt');
file_put_contents('/path/to/Count.txt', $count++, LOCK_EX);
But you'll run into concurrency problems using this code. One way to generate a lock safe from any race condition is:
$countFile = '/path/to/Count.txt';
$countTemp = tempnam(dirname($countFile), basename($countFile));
$countLock = $countFile . '.lock';
$f_lock = fopen($countLock, 'w');
if(flock($f_lock, LOCK_EX)) {
$currentCount = (int)file_get_contents($countFile);
$f_temp = fopen($countTemp, 'w');
if(flock($f_temp, LOCK_EX)) {
fwrite($f_temp, $currentCount++);
flock($f_temp, LOCK_UN);
fclose($f_temp);
if(!rename($countTemp, $countFile)) {
unlink($countTemp);
}
}
flock($f_lock, LOCK_UN);
fclose($f_lock);
}