A visitor log on one of my sites just lost all visitors. It's been working for a year and a half but all entries were lost overnight. The log page is here (if you visit one of the site pages, your information will be added to the log):
http://mykindred.com/dalton/hoax/viewlog.php
The log is kept in a text file ($filename) which should limit to $maxloglines = 300. $logline contains the new visitor to be added to the log. The code that generates the log:
$lines = file($filename, FILE_IGNORE_NEW_LINES | FILE_SKIP_EMPTY_LINES);
$logline .= implode("\n", array_slice($lines, 0, $maxloglines));
file_put_contents($filename, $logline);
Any suggestions as to why my log would lose its entries? Do I have a coding error?
When writing files using file_put_contents, you are essentially using a series of fopen(), fwrite() and fclose() where the file is opened with file mode w:
Open for writing only; place the file pointer at the beginning of the
file and truncate the file to zero length. If the file does not exist,
attempt to create it.
As you do not have an exclusive lock on the file, it is possible to read the file at the point at which the file has been truncated and the file pointer placed at the start. In this case, the contents of your file() command would be empty.
Instead, you should use the option LOCK_EX to secure an exclusive lock of the file while writing:
file_put_contents($filename, $logline, LOCK_EX);
You can read more about that at in the flock() documentation.
Related
I have a text file which multiple users will be simultaneously editing (limited to an individual line per edit, per user). I have already found a solution for the "line editing" part of the required functionality right here on StackOverflow.com, specifically, the 4th solution (for large files) offered by #Gnarf in the following question:
how to replace a particular line in a text file using php?
It basically rewrites the entire file contents to a new temporary file (with the user's edit included) and then renames the temporary file to the original file once finished. It's great!
To avoid one user's edit causing a conflict with another user's edit if they are both attempting an edit at the same time, I have introduced flock() functionality, as can be seen in my variation on the code here:
$reading = fopen($file, 'r');
$writing = fopen($temp, 'w');
$replaced = false;
if ((flock($reading, LOCK_EX)) and (flock($writing, LOCK_EX))) {
echo 'Lock acquired.<br>';
while (!feof($reading)) {
$line = fgets($reading);
$values = explode("|",$line);
if ($values[0] == $id) {
$line = $id."|comment edited!".PHP_EOL;
$replaced = true;
}
fputs($writing, $line);
}
flock($reading, LOCK_UN);
flock($writing, LOCK_UN);
fclose($reading);
fclose($writing);
} else {
echo 'Lock not acquired.<br>';
}
I've made sure the $temp file always has a unique filename. Full code here: https://pastebin.com/E31hR9Mz
I understand that flock() will force any other execution of the script to wait in a queue until the first execution has finished and the flock() has been released. So far so good.
However, the problem starts at the end of the script, when the time has come to rename() the temporary file to replace the original file.
if ($replaced) {
rename($temp, $file);
} else {
unlink($temp);
}
From what I have seen, rename() will fail if the original file still has a flock(), so I need to release the flock() before this point. However, I also need it to remain locked, or rename() will fail when another user running the same script immediately opens a new flock() as soon as the previous flock() is released. When this happens, it will return:
Warning: rename(temporary.txt,original.txt): Access is denied. (code: 5)
tl;dr: I seem to be in a bit of a Catch-22. It looks like rename() won't work on a locked file, but unlocking the file will allow another user to immediately lock it again before the rename() can take place.
Any ideas?
update: After some extensive research into how flock() works, (in layman's terms, there is no guarantee that another script will respect the "lock", and therefore it is not really a "lock" at all as one would assume from the literal meaning of the word) I have opted for this solution instead which works like a charm:
https://docstore.mik.ua/orelly/webprog/pcook/ch18_25.htm
"Good lock" on your locking adventures.
I wanted to wait for all processes reading a certain file in PHP by obtaining an exclusive lock on that file, and after that delete (unlink) the file. This concerns files like profile pictures which a user can delete or change. The name of the file will be something like the user ID.
My code:
//Obtain lock
$file = fopen("path/to/file", "r"); //(I'm not sure which mode to use here btw)
flock($file, LOCK_EX);
//Delete file
unlink("path/to/file");
Line 3 waits for all locks to be released, which is good, but the unlink function throws an error: Warning: unlink(path/to/file): Resource temporarily unavailable in path/to/script on line xx
To prevent this I could release the lock before calling unlink, but this means another process could lock on the file again, which would cause the same error.
My questions are:
Is it possible to delete a file in PHP without releasing the lock? That is, without the risk of other processes trying to use the file at the same time.
If not:
Is this possible in Windows at all? How about Unix?
Should I involve my database for this matter and lock on rows in the database instead, or is there a better way?
Another option I can see is repeating this piece of code, including a release of the lock before calling unlink, until unlink succeeds, but this seems a bit messy, right?
Hey I'm struggling with this too, 2 years later. Kind of seems dumb you can't acquire an exclusive lock on a file when trying to rename or unlink it, or at least the documentation isn't there for doing this.
One solution is open the file for writing, acquire an exclusive lock, clear contents of the file using ftruncate, close it, and then unlink it. When you're reading from the file, you can check the size to make sure the file has contents.
When deleting (untested code):
$fh = fopen('yourfile.txt', 'c'); // 'w' mode truncates file, you don't want to do that yet!
flock($fh, LOCK_EX); // blocking, but you can use LOCK_EX | LOCK_NB for nonblocking and a loop + sleep(1) for a timeout
ftruncate($fh, 0); // truncate file to 0 length
fclose($fh);
unlink('yourfile.txt');
When reading (untested code):
if (!file_exists('yourfile.txt') || filesize('yourfile.txt') <= 0) {
print 'nah.jpg, must be dELeTeD :O';
}
I noticed while testing two fopen() handles on one file, that the handles or channels mix, and the file contents empty when i call fread(). One handle is read, and one handle is write.
Example code:
$rh = fopen('existingfilewithcontent.txt', 'r');
$wh = fopen('existingfilewithcontent.txt', 'w');
echo fread($rh, 1000);
fclose($rh);
fclose($wh);
// file is now blank
This is tested on Linux & Windows.
I could not find anything in the PHP docs about it.
Please do not ask my why I would want two handles on one file as that is not the question.
Thankyou
Opening a file for write is destructive.
Manual says:
'w' - Open for writing only; place the file pointer at the beginning of the file and truncate the file to zero length. If the file does not exist, attempt to create it.
You probably want:
'r+' - Open for reading and writing; place the file pointer at the beginning of the file.
OR
'a+' - Open for reading and writing; place the file pointer at the end of the file. If the file does not exist, attempt to create it. In this mode, fseek() only affects the reading position, writes are always appended.
See manual: http://php.net/manual/en/function.fopen.php
I would like to cut off the beginning of a large file in PHP. Use of file_get_contents() is not possible due to memory restrictions.
What is the best way to delete the first $n characters from a file?
If it is possible to do it without creating a second file, I would prefer that solution.
Update After the file has been modified, it will be used by other scripts.
If you don't have enough memory to buffer the entire file, you'll need to create two files (at least temporarily) regardless of your solution.
Look into fseek(), which allows you to go to a particular byte position within a file.
// Open the file
$filename = 'somefile.dat';
$file = fopen($filename, 'r');
// Skip the first 1 KB
fseek($file, 1024);
// Your processing goes here...
// Close the file
fclose($file);
In your case, you could open the original file for reading and the temp file for writing concurrently. Seek the original file. Loop over the original file, reading a small chunk and writing it to temp. Then rename temp to have the same name as original.
In normal condition , everything is ok, an I can write and create new files with fopen() and fwrite() but under "heavy" DDOS attacks , when file pointer is located at 0 , i cant write anything to file.eg. using "w" mod ,result will be a blank file , but by using "a" or "c" mod , if file not exist or be empty, nothing will be written (and just create a blank file too) , but if file has some characters , it will writes after characters or will clear and rewrite new characters respectively.
and when DDOS stopped , everything would be Ok.
here is simple code that I'm using for test, what is the problem? Can I fix it?
I'm using php5 in ubuntu with apache and lighttpd...
<?php
$fp = fopen('data.txt', 'w');
fwrite($fp, '1');
fputs($fp, '23');
fclose($fp);
?>
The way I understood the question is that you have problems running this code when there are multiple requests accessing the .php file (and thus the file you are writing to) at the same time.
Now, while it is far from being foolproof, flock() is there to help with this. The basic concept is that you'd ask for a lock of the file before writing and only write to a file if you're able to get the lock to that file, like
$fp = fopen( $filename,"w"); // open it for WRITING ("w")
if (flock($fp, LOCK_EX | LOCK_NB)) {
// do your file writes here
// when you're done,
// flush your file writes to a file before unlocking
fflush($fp);
// unlock the file
flock($fp, LOCK_UN);
} else {
// flock() returned false, no lock obtained
print "Could not lock $filename!\n";
}
fclose($fp);
You can read some more details from the manual entry or this article.