I need to read a file that is changing all the time. the file only and will only ever have one line that changes all the time.
I found the following code that should do what I want here: PHP: How to read a file live that is constantly being written to
But the code does not work, the page just keeps loading, I tried to add a "flush" like one user suggested, but I still cant make it work.
Here's the code
$file='/home/user/youfile.txt';
$lastpos = 0;
while (true) {
usleep(300000); //0.3 s
clearstatcache(false, $file);
$len = filesize($file);
if ($len < $lastpos) {
//file deleted or reset
$lastpos = $len;
}
elseif ($len > $lastpos) {
$f = fopen($file, "rb");
if ($f === false)
die();
fseek($f, $lastpos);
while (!feof($f)) {
$buffer = fread($f, 4096);
echo $buffer;
flush();
}
$lastpos = ftell($f);
fclose($f);
}
}
Please could someone have a look and let me know how to fix it.
Thanks in advance.
If your file have only one string, and you need to read it on change, use this code:
$file = '/path/to/test.txt';
$last_modify_time = 0;
while (true) {
sleep(1); // 1 s
clearstatcache(true, $file);
$curr_modify_time = filemtime($file);
if ($last_modify_time < $curr_modify_time) {
echo file_get_contents($file);
}
$last_modify_time = $curr_modify_time;
}
Note:
filemtime() returns last file modification time in seconds, so if you need to check modification more than one time per second, probably you'll need to find other solutions.
Also, you may need to add set_time_limit(0); it depends on your requirements.
Update:
index.html
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1/jquery.min.js"
type="text/javascript">
</script>
</head>
<body>
<div id="file_content"></div>
<script>
var time = 0;
setInterval(function() {
$.ajax({
type: "POST",
data: {time : time},
url: "fileupdate.php",
success: function (data) {
var result = $.parseJSON(data)
if (result.content) {
$('#file_content').append('<br>' + result.content);
}
time = result.time;
}
});
}, 1000);
</script>
</body>
fileupdate.php
<?php
$file = 'test.txt';
$result = array();
clearstatcache(true, $file);
$data['time'] = filemtime($file);
$data['content'] = $_POST['time'] < $data['time']
? file_get_contents($file)
: false;
echo json_encode($data);
You might be dealing with 3 drawbacks:
First, the code you already have is holding a $lastpos. Meaning, it will always look for what is added at the end of the file. You are not clear in the OP, but I think this is not what you want. I think your one and only line is continuously changing, but not necessarily changing size. So you might want to remove the line $lastpos = ftell($f);.
Secondly, in regards to the same, you are check the file size to know if the file has changed. But as I explained, the file might have changed, while the file size stayed equal. Try changing the check of the file size to checking the file last-edit date.
Third, and probably most importantly: your web browser might be buffering your output until the php script is done running, before it releases the buffered output to the browser. Disable output buffering in both PHP and your web server. Things like gzip/compression by the web server can also be forcing output-buffering effects.
Related
I have no luck when the subject is reading text files. I have a small script to read a log file (real time updated) but I want to send some data to DB.
And the problem is, if I don't stat reading from the end of the files, I will get duplicated entries in database. Wich can't happen!
// Keep alive
for (;;)
{
$handle = fopen("data.log", "r");
if (!$handle) die("Open error - data.log");
while (!feof($handle))
{
$line = fgets($handle, 4096);
// If match with, I output the result
if (strpos($line, ':gshop_trade:') > 0)
{
if (!preg_match('/([\d-: ]+)\s*.*\sformatlog:gshop_trade:userid=(\d+):(.*)item_id=(\d+):expire=(\d+):item_count=(\d+):cash_need=(\d+):cash_left=(\d+).*$/', $line, $data))
{
echo "Parsing error on line: {$line}";
}
// show the data
}
}
sleep(5);
}
This script is working, but as I mentioned above, I need to send the data to BD. But also, I need to leave script running, with this current code, the script match the wanted string, and instead of wait for new entries on data.log he starting reading the whole file again.
I see this question here and I tested but doesn't work. I'll start the script when I start the service that generates "data.log" but to prevent duplicate entries in database, I need to read the last lines.
How can I do that?
Keep a track of the file offset from the previous reading using ftell() and keeping that result in a variable, and jump to that offset in the file when you re-open it for the next reading using fseek()
$lastPos = 0;
for (;;)
{
$handle = fopen("data.log", "r");
if (!$handle) die("Open error - data.log");
fseek($handle, $lastPos); // <--- jump to last read position
while (!feof($handle))
{
$line = fgets($handle, 4096);
$lastPos = ftell($handle); // <--- maintain last read position
// If match with, I output the result
if (strpos($line, ':gshop_trade:') > 0)
{
if (!preg_match('/([\d-: ]+)\s*.*\sformatlog:gshop_trade:userid=(\d+):(.*)item_id=(\d+):expire=(\d+):item_count=(\d+):cash_need=(\d+):cash_left=(\d+).*$/', $line, $data))
{
echo "Parsing error on line: {$line}";
}
// show the data
}
}
sleep(5);
}
Maybe you can use file_get_contents, explode and read the array backwards?
$arr = explode(PHP_EOL, file_get_contents("data.log")); // or file("data.log");
$arr = array_reverse($arr);
foreach($arr as $line){
// do stuff here in reverse order
}
From comments above I suggest this method to only use the new data in your code.
It will read your log and a text file with what has been read last time.
Remove what was read last time and use the new data in the code.
$logfile = file_get_contents("data.log");
$ReadData = file_get_contents("readdata.txt");
$newdata = str_replace($ReadData, "", $logfile); // this is what is new since last run.
file_put_contents("readdata.txt", $logfile); // save what has been read.
$arr = explode(PHP_EOL, $newdata);
foreach($arr as $line){
// do your stuff here with the new data.
}
?>
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="refresh" content="5"> <!-- This will run the page every five seconds.
</head>
</html>
I have a simple php service set up on a IIS web server. It is used by my client to retrieve files from the server. It looks like this:
<?php
if (isset($_GET['file']))
{
$filepath = "C:\\files\\" . $_GET['file'];
if (!strpos(pathinfo($filepath, PATHINFO_DIRNAME), "..") && file_exists($filepath) && !is_dir($filepath))
{
set_time_limit(0);
$fp = #fopen($filepath, "rb");
while(!feof($fp))
{
print(#fread($fp, 1024*8));
ob_flush();
flush();
}
}
else
{
echo "ERROR at www.testserver.com\r\n";
}
exit;
}
?>
I retrieve the files using WinHttp's WinHttpReadData in C++.
EDIT #2: Here is the C++ code. This is not exactly how it appears in my program. I had to pull pieces from multiple classes, but the gist should be apparent.
session = WinHttpOpen(appName.c_str(), WINHTTP_ACCESS_TYPE_NO_PROXY, WINHTTP_NO_PROXY_NAME, WINHTTP_NO_PROXY_BYPASS, 0);
if (session) connection = WinHttpConnect(session, hostName.c_str(), INTERNET_DEFAULT_HTTP_PORT, 0);
if (connection) request = WinHttpOpenRequest(connection, NULL, requestString.c_str(), NULL, WINHTTP_NO_REFERER, WINHTTP_DEFAULT_ACCEPT_TYPES, 0);
bool results = false;
if (request)
{
results = (WinHttpSendRequest(request, WINHTTP_NO_ADDITIONAL_HEADERS, 0, WINHTTP_NO_REQUEST_DATA, 0, 0, 0) != FALSE);
}
if (results)
{
results = (WinHttpReceiveResponse(request, NULL) != FALSE);
}
DWORD bytesCopied = 0;
DWORD size = 0;
if (results)
{
do {
results = (WinHttpQueryDataAvailable(request, &size) != FALSE);
if (results)
{
// More available data?
if (size > 0)
{
// Read the Data.
size = min(bufferSize, size);
ZeroMemory(buffer, size);
results = (WinHttpReadData(request, (LPVOID)buffer, size, &bytesCopied) != FALSE);
}
}
if (bytesCopied > 0 && !SharedShutDown.GetValue())
{
tempFile.write((PCHAR)RequestBuffer, bytesCopied);
if (tempFile.fail())
{
tempFile.close();
return false;
}
fileBytes += bytesCopied;
}
} while (bytesCopied > 0 && !SharedShutDown.GetValue());
}
Everything works fine when I test (thousands of files) over the local network using the server computer name from either a Windows 7 or Windows 10 machine. It also works fine when I access the service over the internet from a Windows 7 machine. However, when I run the client on a Windows 10 machine accessing over the internet, I get dropped characters. The interesting thing is that it is a specific set of characters that gets dropped every time from XML files. (Other, binary, files are affected as well, but I have not yet determined what changes in them.)
If the XML file contains an element starting with "<Style", that text disappears. So, this:
<Element1>blah blah</Element1>
<Style_Element>hoopa hoopa</Style_Element>
<Element2>bip bop bam</Element2>
becomes this:
<Element1>blah blah</Element1>
_Element>hoopa hoopa</Style_Element>
<Element2>bip bop bam</Element2>
Notice that the beginning of the style element is chopped off. This is the only element that is affected, and it seems to only affect the first one if there are more than one in the file.
What perplexes me is why this doesn't happen running the client from Windows 7.
EDIT: Some of the other files, binary and text, are missing from 1 to 3 characters each. It seems that a drop only happens once in a file. The rest of the contents of the file are identical to the source.
I can't make sense of the above read routine, it is also incomplete. Just keep it simple like the example below.
The fact that you are having problems with binary files suggest you are not opening the output tempFile in binary mode.
std::ofstream tempFile(filename, std::ios::binary);
while(WinHttpQueryDataAvailable(request, &size) && size)
{
std::string buf(size, 0);
WinHttpReadData(request, &buf[0], size, &bytesCopied);
tempFile.write(buf.data(), bytesCopied);
}
Your php file can be simplified as follows:
<?php
readfile('whatever.bin');
?>
I solved the problem, it seems. My php service did not include header information (didn't think I needed it), so I figured I would try adding a header specification for content type application/octet-stream just to see what would result. My updated service looked like this:
if (isset($_GET['file']))
{
$filepath = "C:\\Program Files (Unrestricted)\\Sony Online Entertainment\\Everquest Yarko Client\\" . $_GET['file'];
if (!strpos(pathinfo($filepath, PATHINFO_DIRNAME), "..") && file_exists($filepath) && !is_dir($filepath))
{
header("Content-Type:application/octet-stream");
set_time_limit(0);
$fp = #fopen($filepath, "rb");
while(!feof($fp))
{
print(#fread($fp, 1024*8));
ob_flush();
flush();
}
}
else
{
echo "ERROR at www.lewiefitz.com\r\n";
}
exit;
}
Now, the files download without any corruption. Why I need such a header in this situation is beyond me. What part of the system is messing with the response message before it ended up in my buffer? I don't know.
I am looking to numerically increment to a file's content if the file was last modified within 24 hours otherwise reset the file's content to 1. However I want to ensure this continues to work regardless of how many users visit the script at the same time (the script would always need to execute but ensure it does not overwrite/calculate incorrectly - I believe this is where flock comes to use).
Please see below code:
$host_limit = 50;
$file = 'timer.txt';
$fh = fopen($file,'r+');
if (flock($fh,LOCK_EX)) {
$content = fgets($fh);
//FILE HAS NOT BEEN MODIFIED IN LAST 24 HOURS
if (strtotime('-24 hours') > filemtime($file)) {
$content = 1;
} else {
$content = ($content + 1);
}
fwrite($fh, $content);
fflush($fh);
flock($fh,LOCK_UN);
}
fclose($fh);
if ($content < $host_limit) {
//do stuff
}
Would the above work as I would like (as have no way to simulate what I am anticipating to test)?
Instead of using fopen and fwrite, you could use
file_get_contents($file);
and
file_put_contents($file, $content, LOCK_EX);
Check the manual:
http://php.net/manual/en/function.file-get-contents.php
http://php.net/manual/en/function.file-put-contents.php
I'm using a PHP script to stream a live video (i.e. a file which never ends) from a remote source. The output is viewed in VLC, not a web browser. I need to keep a count of the number of bytes transferred. Here is my code:
<?php
ignore_user_abort(true);
$stream = $_GET['stream'];
if($stream == "vid1")
{
$count = readfile('http://127.0.0.1:8080/');
logThis($count);
}
function logThis($c)
{
$myFile = "bytecount.txt";
$handle = fopen($myFile,'a');
fwrite($handle,"Count: " . $c . "\n");
fclose($handle);
}
?>
However it appears that when the user presses the stop button, logThis() is never called, even though I've put in ignore_user_abort(true);
Any ideas on what I'm doing wrong?
Thanks
Update2: I've changed my code as I shoudn't be using ignore_user_abort(true) as that would continue to download the file forever even after the client has gone. I've changed my code to this:
<?php
$count = 0;
function bye()
{
//Create Dummy File with the filename of equal to count
}
register_shutdown_function('bye');
set_time_limit(0);
ignore_user_abort(false);
$stream = $_GET['stream'];
if($stream == "vid1")
{
$GLOBALS['count'] = readfile('http://127.0.0.1:8080/');
exit();
}
?>
My problem now is that when the script is aborted (i.e. user presses stop), readfile won't return a value (i.e. count remains at 0). Any ideas on how I can fix this?
Thanks
When a PHP script is running normally the NORMAL state, is active. If the remote client disconnects the ABORTED state flag is turned on. A remote client disconnect is usually caused by the user hitting his STOP button. If the PHP-imposed time limit (see set_time_limit()) is hit, the TIMEOUT state flag is turned on.
so setting the set_time_limit to 0 should help.
Ok folks I managed to fix this. The trick was to not use readfile() but read the video stream byte by byte. Ok it may not be 100% accurate, however a few bytes inaccuracy here or there is ok.
<?php
$count = 0;
function logCount()
{
//Write out dummy file with a filename equal to count
}
register_shutdown_function('logCount');
set_time_limit(0);
ignore_user_abort(false);
$stream = $_GET['stream'];
if($stream == "vid1")
{
$filename = 'http://127.0.0.1:8080/';
$f = fopen($filename, "rb");
while($chunk = fread($f, 1024)) {
echo $chunk;
flush();
if(!connection_aborted()) {
$GLOBALS['count'] += strlen($chunk);
}
else {
exit();
}
}
}
?>
I have an app that processes images and use jQuery to display progress to the user.
I done this with writing to a textfile each time and image is processed and than read this status with a setInterval.
Because no images are actually written in the processing (I do it in PHP's memory) I thought a log.txt would be a solution, but I am not sure about all the fopen and fread's. Is this prone to issues?
I tried also with PHP sessions, but can't seem to get it to work, I don't get why..
HTML:
<a class="download" href="#">request download</a>
<p class="message"></p>
JS:
$('a.download').click(function() {
var queryData = {images : ["001.jpg", "002.jpg", "003.jpg"]};
$("p.message").html("initializing...");
var progressCheck = function() {
$.get("dynamic-session-progress.php",
function(data) {
$("p.message").html(data);
}
);
};
$.post('dynamic-session-process.php', queryData,
function(intvalId) {
return function(data) {
$("p.message").html(data);
clearInterval(intvalId);
}
} (setInterval(progressCheck, 1000))
);
return false;
});
process.php:
// session_start();
$arr = $_POST['images'];
$arr_cnt = count($arr);
$filename = "log.txt";
for ($i = 1; $i <= $arr_cnt; $i++) {
$content = "processing $val ($i/$arr_cnt)";
$handle = fopen($filename, 'w');
fwrite($handle, $content);
fclose($handle);
// $_SESSION['counter'] = $content;
sleep(3); // to mimic image processing
}
echo "<a href='#'>download zip</a>";
progress.php:
// session_start();
$filename = "log.txt";
$handle = fopen($filename, "r");
$contents = fread($handle, filesize($filename));
fclose($handle);
echo $contents;
// echo $_SESSION['counter'];
What if two clients process images at the same time?
You can try adding session_write_close() between setting the new status in the session, so that the new session data is stored, otherwise it will only get stored once your script finishes.
Another solution would be to save the status in memcache or to use a database,
perhaps separate the statuses with a userid or creating an md5 hash on the image data