I have one file: configuration.txt.
This file gets read by PHP, then wrote by the same PHP, while a C++ program reads the content of the same file at a regular interval.
PHP:
$closeFlag = false;
$arrayInputs = new SplFixedArray(3);
$arrayInputs[0] = "URL not entered";
$arrayInputs[1] = "3";
$arrayInputs[2] = "50";
$configFilePath = "/var/www/html/configuration.txt";
$currentSettingsFile = fopen($configFilePath, "r");
if(flock($currentSettingsFile, LOCK_SH)) {
$arrayInputs = explode(PHP_EOL, fread($currentSettingsFile, filesize($configFilePath)));
flock($currentSettingsFile, LOCK_UN);
$closeFlag = fclose($currentSettingsFile);
}
if(isset( $_POST['save_values'])) {
if(!empty($_POST['getURL'])) {
$arrayInputs[0] = $_POST['getURL'];
}
if(!empty($_POST['getURR'])) {
$arrayInputs[1] = $_POST['getURR'];
}
if(!empty($_POST['getBrightness'])) {
$arrayInputs[2] = $_POST['getBrightness'];
}
}
if(!$closeFlag) fclose($currentSettingsFile);
$currentSettingsFile = fopen($configFilePath, "w");
if(flock($currentSettingsFile, LOCK_SH)) {
foreach ($arrayInputs as $key => $value) {
if($value != '')
fwrite($currentSettingsFile,$value.PHP_EOL);
}
flock($currentSettingsFile, LOCK_UN);
fclose($currentSettingsFile);
}
?>
C++
char configFilePath[]="/var/www/html/configuration.txt";
std::fstream configFile;
configFile.open(configFilePath, std::fstream::in);
if(configFile.is_open()){
// do stuff
} else {
std::cout<<"Error ! Could not open Configuration file to read"<<std::endl;
}
The c++ returned no error so far. It can open the file. And php will return Warning: fread(): Length parameter must be greater than 0 because the file is empty.
I believe that PHP is deleting the file's content.
When locking a file in PHP, you lock a LOCK file, not the main file. Example:
$myfile = 'myfile.txt';
$lockfile = 'myfile.lock';
$lock = fopen($lockfile,'a');
if(flock($lock, LOCK_EX)) // The lock file is locked in exclusive mode - so I can write to it.
{
$fp = fopen($myfile,'w');
fputs($fp, "I am writing safely!");
fclose($fp);
flock($lock, LOCK_UN); // Always unlock it!
}
fclose($lock);
You work similarly in C++ because PHP is not locking the actual file. It is locking a lock file. The exact syntax depends heavily on your version of C/C++ and the operating system. So, I will use minimal syntax.
int lock=fopen(lockfile, "r+");
if(flock(fileno(lock), LOCK_EX))
{
//Locked. You can open a stream to ANOTHER file and play with it.
flock(fileno(lock), LOCK_UN));
}
fclose(lock);
Related
I need to transfer files of any type or size over HTTP/GET in ~1k chunks. The resulting file hash needs to match the source file. This needs to be done in native PHP without any special tools. I have a basic strategy but I'm getting odd results. This proof of concept just copies the file locally.
CODE
<?php
$input="/home/lm1/Music/Ellise - Feeling Something Bad.mp3";
$a=pathinfo($input);
$output=$a["basename"];
echo "\n> ".md5_file($input);
$fp=fopen($input,'rb');
if ($fp) {
while(!feof($fp)) {
$buffer=base64_encode(fread($fp,1024));
// echo "\n\n".md5($buffer);
write($output,$buffer);
}
fclose($fp);
echo "\n> ".md5_file($output);
echo "\n";
}
function write($file,$buffer) {
// echo "\n".md5($buffer);
$fp = fopen($file, 'ab');
fwrite($fp, base64_decode($buffer));
fclose($fp);
}
?>
OUTPUT
> d31e102b1cae9c73bbf5a12615a8ea36
> 9f03f6c88ed61c07cb534922d6d31864
Thanks in advance.
fread already advances the file pointer position, so there's no need to keep track of it. Same with frwite, so consecutive calls automatically append to the given file. Thus, you could simplify your approach to (code adapted from this answer on how to efficiently write a large input stream to a file):
$src = "a.test";
$dest = "b.test";
$fp_src = fopen($src, 'rb');
if ($fp_src) {
$fp_dest = fopen($dest, 'wb');
$buffer_size = 1024;
while(!feof($fp_src)) {
fwrite($fp_dest, fread($fp_src, $buffer_size));
}
fclose($fp_src);
fclose($fp_dest);
echo md5_file($src)."\n"; // 88e4af2f85080a280e7f00e50d96b7f7
echo md5_file($dest)."\n"; // 88e4af2f85080a280e7f00e50d96b7f7
}
If you want to keep both processes separated, you'd do:
$src = "a.test";
$dest = "b.test";
if (file_exists($dest)) {
unlink($dest); // So we don't append to an existing file
}
$fp = fopen($src,'rb');
if ($fp) {
while(!feof($fp)){
$buffer = base64_encode(fread($fp, 1024));
write($dest, $buffer);
}
fclose($fp);
}
function write($file, $buffer) {
$fp = fopen($file, 'ab');
fwrite($fp, base64_decode($buffer));
fclose($fp);
}
echo md5_file($src)."\n"; // 88e4af2f85080a280e7f00e50d96b7f7
echo md5_file($dest)."\n"; // 88e4af2f85080a280e7f00e50d96b7f7
As for how to stream files over HTTP, you might want to have a look at:
Streaming a large file using PHP
Here is my Code with filename
it does work without problems if lets say i just use
update.php?pokemon=pikachu
it updates pikachu value in my found.txt +0.0001
But now my problem, when i have multiple threads running and randomly
2 threads are
update.php?pokemon=pikachu
and
update.php?pokemon=zaptos
i see the found.txt file
is empty than!!
so nothing is written in it then anymore.
So i guess its a bug when the php file is opened and another request is posted to the server.
How can i solve this problem this does accour often
found.txt
pikachu:2.2122
arktos:0
zaptos:0
lavados:9.2814
blabla:0
update.php
<?php
$file = "found.txt";
$fh = fopen($file,'r+');
$gotPokemon = $_GET['pokemon'];
$users = '';
while(!feof($fh)) {
$user = explode(':',fgets($fh));
$pokename = trim($user[0]);
$infound = trim($user[1]);
// check for empty indexes
if (!empty($pokename)) {
if ($pokename == $gotPokemon) {
if ($gotPokemon == "Pikachu"){
$infound+=0.0001;
}
if ($gotPokemon == "Arktos"){
$infound+=0.0001;
}
if ($gotPokemon == "Zaptos"){
$infound+=0.0001;
}
if ($gotPokemon == "Lavados"){
$infound+=0.0001;
}
}
$users .= $pokename . ':' . $infound;
$users .= "\r\n";
}
}
file_put_contents('found.txt', $users);
fclose($fh);
?>
I would create an exclusive lock after open the file and then release the lock before closing the file:
For creating an exclusive lock over the file:
flock($fh, LOCK_EX);
To delete it:
flock($fh, LOCK_UN);
Anyway you will need to check if other threads hot already the lock, so the first idea coming up is to try a few attempts to get the lock and if it's not finally possible, to inform the user, throw an exception or whatever other action to avoid an infinite loop:
$fh = fopen("found.txt", "w+");
$attempts = 0;
do {
$attempts++;
if ($attempts > 5) {
// throw exception or return response with http status code = 500
}
if ($attempts != 1) {
sleep(1);
}
} while (!flock($fh, LOCK_EX));
// rest of your code
file_put_contents('found.txt', $users);
flock($fh, LOCK_UN); // release the lock
fclose($fh);
Update
Probably the issue still remains because the reading part, so let's create also a shared lock before start reading and also let's simplify the code:
$file = "found.txt";
$fh = fopen($file,'r+');
$gotPokemon = $_GET['pokemon'];
$users = '';
$wouldblock = true;
// we add a shared lock for reading
$locked = flock($fh, LOCK_SH, $wouldblock); // it will wait if locked ($wouldblock = true)
while(!feof($fh)) {
// your code inside while loop
}
// we add an exclusive lock for writing
flock($fh, LOCK_EX, $wouldblock);
file_put_contents('found.txt', $users);
flock($fh, LOCK_UN); // release the locks
fclose($fh);
Let's see if it works
Let's consider a sample php script which deletes a line by user input:
$DELETE_LINE = $_GET['line'];
$out = array();
$data = #file("foo.txt");
if($data)
{
foreach($data as $line)
if(trim($line) != $DELETE_LINE)
$out[] = $line;
}
$fp = fopen("foo.txt", "w+");
flock($fp, LOCK_EX);
foreach($out as $line)
fwrite($fp, $line);
flock($fp, LOCK_UN);
fclose($fp);
I want to know if some user is currently executing this script and file "foo.txt" is locked, in same time or before completion of its execution, if some other user calls this script, then what will happen?
Will second users process wait for unlocking of file by first users? or line deletion by second users input will fail?
If you try to acquire an exclusive lock while another process has the file locked, your attempt will wait until the file is unlocked. This is the whole point of locking.
See the Linux documentation of flock(), which describes how it works in general across operating systems. PHP uses fcntl() under the hood so NFS shares are generally supported.
There's no timeout. If you want to implement a timeout yourself, you can do something like this:
$count = 0;
$timeout_secs = 10; //number of seconds of timeout
$got_lock = true;
while (!flock($fp, LOCK_EX | LOCK_NB, $wouldblock)) {
if ($wouldblock && $count++ < $timeout_secs) {
sleep(1);
} else {
$got_lock = false;
break;
}
}
if ($got_lock) {
// Do stuff with file
}
I'm using a simple unzip function (as seen below) for my files so I don't have to unzip files manually before they are processed further.
function uncompress($srcName, $dstName) {
$string = implode("", gzfile($srcName));
$fp = fopen($dstName, "w");
fwrite($fp, $string, strlen($string));
fclose($fp);
}
The problem is that if the gzip file is large (e.g. 50mb) the unzipping takes a large amount of ram to process.
The question: can I parse a gzipped file in chunks and still get the correct result? Or is there a better other way to handle the issue of extracting large gzip files (even if it takes a few seconds more)?
gzfile() is a convenience method that calls gzopen, gzread, and gzclose.
So, yes, you can manually do the gzopen and gzread the file in chunks.
This will uncompress the file in 4kB chunks:
function uncompress($srcName, $dstName) {
$sfp = gzopen($srcName, "rb");
$fp = fopen($dstName, "w");
while (!gzeof($sfp)) {
$string = gzread($sfp, 4096);
fwrite($fp, $string, strlen($string));
}
gzclose($sfp);
fclose($fp);
}
try with
function uncompress($srcName, $dstName) {
$fp = fopen($dstName, "w");
fwrite($fp, implode("", gzfile($srcName)));
fclose($fp);
}
$length parameter is optional.
If you are on a Linux host, have the required privilegies to run commands, and the gzip command is installed, you could try calling it with something like shell_exec
SOmething a bit like this, I guess, would do :
shell_exec('gzip -d your_file.gz');
This way, the file wouldn't be unzip by PHP.
As a sidenote :
Take care where the command is run from (ot use a swith to tell "decompress to that directory")
You might want to take a look at escapeshellarg too ;-)
As maliayas mentioned, it may lead to a bug. I experienced an unexpected fall out of the while loop, but the gz file has been decompressed successfully. The whole code looks like this and works better for me:
function gzDecompressFile($srcName, $dstName) {
$error = false;
if( $file = gzopen($srcName, 'rb') ) { // open gz file
$out_file = fopen($dstName, 'wb'); // open destination file
while (($string = gzread($file, 4096)) != '') { // read 4kb at a time
if( !fwrite($out_file, $string) ) { // check if writing was successful
$error = true;
}
}
// close files
fclose($out_file);
gzclose($file);
} else {
$error = true;
}
if ($error)
return false;
else
return true;
}
I want to make movement such as the tail command with PHP,
but how may watch append to the file?
I don't believe that there's some magical way to do it. You just have to continuously poll the file size and output any new data. This is actually quite easy, and the only real thing to watch out for is that file sizes and other stat data is cached in php. The solution to this is to call clearstatcache() before outputting any data.
Here's a quick sample, that doesn't include any error handling:
function follow($file)
{
$size = 0;
while (true) {
clearstatcache();
$currentSize = filesize($file);
if ($size == $currentSize) {
usleep(100);
continue;
}
$fh = fopen($file, "r");
fseek($fh, $size);
while ($d = fgets($fh)) {
echo $d;
}
fclose($fh);
$size = $currentSize;
}
}
follow("file.txt");
$handle = popen("tail -f /var/log/your_file.log 2>&1", 'r');
while(!feof($handle)) {
$buffer = fgets($handle);
echo "$buffer\n";
flush();
}
pclose($handle);
Checkout php-tail on Google code. It's a 2 file implementation with PHP and Javascript and it has very little overhead in my testing.
It even supports filtering with a grep keyword (useful for ffmpeg which spits out frame rate etc every second).
$handler = fopen('somefile.txt', 'r');
// move you at the end of file
fseek($handler, filesize( ));
// move you at the begining of file
fseek($handler, 0);
And probably you will want to consider a use of stream_get_line
Instead of polling filesize you regular checking the file modification time: filemtime
Below is what I adapted from above. Call it periodically with an ajax call and append to your 'holder' (textarea)... Hope this helps... thank you to all of you who contribute to stackoverflow and other such forums!
/* Used by the programming module to output debug.txt */
session_start();
$_SESSION['tailSize'] = filesize("./debugLog.txt");
if($_SESSION['tailPrevSize'] == '' || $_SESSION['tailPrevSize'] > $_SESSION['tailSize'])
{
$_SESSION['tailPrevSize'] = $_SESSION['tailSize'];
}
$tailDiff = $_SESSION['tailSize'] - $_SESSION['tailPrevSize'];
$_SESSION['tailPrevSize'] = $_SESSION['tailSize'];
/* Include your own security checks (valid user, etc) if required here */
if(!$valid_user) {
echo "Invalid system mode for this page.";
}
$handle = popen("tail -c ".$tailDiff." ./debugLog.txt 2>&1", 'r');
while(!feof($handle)) {
$buffer = fgets($handle);
echo "$buffer";
flush();
}
pclose($handle);