PHP extract data from binary file with structure - php

I have i file with structure in image. I want to extract data to array from that:
function get_data($file, $number)
{
if(!$fp = fopen ($file, 'rb')) return 0;
$fsize = filesize($file);
if(!$data = fread ($fp, $fsize)) return 0;
$data_format=
'#100/'.
'smember_id/'.
'cmember_name_length/'.
'a' . $member_name_length . 'member_name/'.
'C100other_data/';
$data = unpack ($data_format, $data);
fclose($file);
return $data;
}
How can I get the $member_name_length from the file? I want to create a function that if user input the $number, it returns a array of $number(th) data.
Thank you.

Since you have a variable-length data blocks, you can read them only sequentially, so in order to read n-th block, you need to read all n first blocks:
function readDataBlock($f) {
$data = unpack('nmember_id', fread ($f, 2)); // I assume member_id is n, not s
if ($data['member_id'] == 0xFFFF) {
throw new \Exception('End of file');
}
$data = array_merge($data, unpack('Cmember_name_length', fread ($f, 1))); //again, it must be C, not c, as I can't imagine negative length.
$data = array_merge($data, unpack('a*member_name', fread ($f, $data['member_name_length']))); // be sure you understand how a differs from A
return array_merge($data, unpack('C100other_data', fread ($f, 100))); // are you sure C100 is what you want here?
}
function get_data($file, $number)
{
if(!$fp = fopen ($file, 'rb')) return 0;
fread ($fp, 100); //skip header
for($n = 0; $n <= $number; $n++) {
$data = readDataBlock($fp); // read single member
}
fclose($fp);
return $data; //return the n-th member
}
If the file is small enough to fit into memory, it might be better to read it once and return n-th member from memory:
$data = [];
while(true) {
try {
$data[] = readDataBlock($fp);
} catch(\Exception $e) {
break;
}
}
function get_data(&$data, $number)
{
return $data[$number];
}

Related

Replace values of a csv file

I need to find and replace all the values of rows of a CSV using PHP;
I am trying this but its replacing even the headers to 0 and row values are not doubling as it suppose to.
public function checkForNumericValues()
{
// Read the columns and detect numeric values
if (($this->handle = fopen($this->csvFile, "r")) !== FALSE)
{
$fhandle = fopen($this->csvFile,"r");
$content = fread($fhandle,filesize($this->csvFile));
while (($this->data = fgetcsv($this->handle, 1000, ",")) !== FALSE)
{
$this->num = count($this->data);
// Skipping the header
if($this->row == 1)
{
$this->row++;
continue;
}
$this->row++;
// Check and replace the numeric values
for ($j=0; $j < $this->num; $j++)
{
if(is_numeric($this->data[$j]))
{
$content = str_replace($this->data[$j], $this->data[$j] * 2, $content);
}
else
{
$content = str_replace($this->data[$j], 0, $content);
}
}
break;
// print_r($content);
}
$fhandle = fopen($this->csvFile,"w");
fwrite($fhandle,$content);
fclose($this->handle);
}
echo "Numeric and String values been changed in rows of the CSV!";
}
CSV is like this:
You shouldn't update the entire $contents when you're processing each field in the CSV, just update that field. Your str_replace() will replace substrings elsewhere in the file; for instance, if the current field contains 5, you'll replace all the 5's in the file with 10, so 125 will become 1210.
You can do it correctly by replacing the element in the $this->data array. After you do that, you can then join them back into a string with implode(). Then you can keep all the updated lines in a string, which you write back to the file at the end.
You can skip the header line by calling fgets() before the while loop.
public function checkForNumericValues()
{
// Read the columns and detect numeric values
if (($this->handle = fopen($this->csvFile, "r")) !== FALSE)
{
$output = "";
$output .= fgets($this->csvFile); // Copy header line to output
while (($this->data = fgetcsv($this->handle, 1000, ",")) !== FALSE)
{
$this->num = count($this->data);
// Check and replace the numeric values
for ($j=0; $j < $this->num; $j++)
{
if(is_numeric($this->data[$j]))
{
$this->data[$j] *= 2;
}
else
{
$this->data[$j] = 0;
}
}
$output .= implode(',', $this->data) . "\n";
}
fclose($this->handle);
$fhandle = fopen($this->csvFile,"w");
fwrite($fhandle,$output);
fclose($fhandle);
}
echo "Numeric and String values been changed in rows of the CSV!";
}

Read multiple csv files in folder

i need a help ^^
What i need is script which will open and read all .csv files in folder 'csv/files' and then do that thing in "if". Well, when i had only one file it worked fine. I managed to construct some script which is not working but no "error line" popping up either ...
So can somebody look at my code and tell me what i am doing wrong ?
<?php
foreach (glob("*.csv") as $filename) {
echo $filename."<br />";
if (($handle = fopen($filename, "r")) !== FALSE) {
while (($data = fgetcsv($handle, 1000, ";")) !== FALSE) {
$url = $data[0];
$path = $data[1];
$ch = curl_init($url);
$fp = fopen($path, 'wb');
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_exec($ch);
curl_close($ch);
fclose($fp);
}
fclose($handle);
}
}
?>
This is a prime candidate for multi-threading, and here's some code to do it:
<?php
class WebWorker extends Worker {
public function run() {}
}
class WebTask extends Stackable {
public function __construct($input, $output) {
$this->input = $input;
$this->output = $output;
$this->copied = 0;
}
public function run() {
$data = file_get_contents($this->input);
if ($data) {
file_put_contents(
$this->output, $data);
$this->copied = strlen($data);
}
}
public $input;
public $output;
public $copied;
}
class WebPool {
public function __construct($max) {
$this->max = $max;
$this->workers = [];
}
public function submit(WebTask $task) {
$random = rand(0, $this->max);
if (isset($this->workers[$random])) {
return $this->workers[$random]
->stack($task);
} else {
$this->workers[$random] = new WebWorker();
$this->workers[$random]
->start();
return $this->workers[$random]
->stack($task);
}
}
public function shutdown() {
foreach ($this->workers as $worker)
$worker->shutdown();
}
protected $max;
protected $workers;
}
$pool = new WebPool(8);
$work = [];
$start = microtime(true);
foreach (glob("csv/*.csv") as $file) {
$file = fopen($file, "r");
if ($file) {
while (($line = fgetcsv($file, 0, ";"))) {
$wid = count($work);
$work[$wid] = new WebTask(
$line[0], $line[1]);
$pool->submit($work[$wid]);
}
}
}
$pool->shutdown();
$runtime = microtime(true) - $start;
$total = 0;
foreach ($work as $job) {
printf(
"[%s] %s -> %s %.3f kB\n",
$job->copied ? "OK" : "FAIL",
$job->input,
$job->output,
$job->copied/1024);
$total += $job->copied;
}
printf(
"[TOTAL] %.3f kB in %.3f seconds\n",
$total/1024, $runtime);
?>
This will create a maximum number of pooled threads, it will then read through a directory of semi-colon seperated csv files where each line is input;output, it will then submit the task to read the input and write the output asynchronously to the pool for execution, while the main thread continues to read csv files.
I have used the simplest input/output file_get_contents and file_put_contents so that you can see how it works without cURL.
The worker selected when a task is submitted to the pool is random, this may not be desirable, it's possible to detect if a worker is busy but this would complicate the example.
Further reading:
https://gist.github.com/krakjoe/6437782
http://php.net/pthreads

PHP; assigning fgets() output to an array

I am attempting to assign the string returned by the fgets() function to an array in PHP. I have tried test strings and they work fine. I have also made sure that fgets() is returning items, but still no joy. Thinking that it may be a timing issue, I had the function run onload and that didn't work. My code is below; any help on this would be much appreciated.
function createDataArray()
{
global $resultsArray;
$i = 0;
$file = fopen("downloads/E0.csv","r");
while(! feof($file))
{
$line = fgets($file, 4096);
$resultsArray[$i] = $line; //This isn't working. Something is wrong with $line. It is a string, but it doesn't get assigned to the array.
$i = $i + 1;
}
fclose($file);
}
PLEASE return the array; do not use globals.
This fix should work:
function createDataArray()
{
$resultsArray = array();
$file = fopen("downloads/E0.csv","r");
while(! feof($file))
{
$line = fgets($file, 4096);
$resultsArray[] = $line;
}
fclose($file);
return $resultsArray;
}

Read a file backwards line by line using fseek

How do I read a file backwards line by line using fseek?
code can be helpful. must be cross platform and pure php.
many thanks in advance
regards
Jera
If you are going to read the entire file in anyways, just use file() to read the file in as an array (each line is each element in the array) and then use array_reverse() to flip the array backwards and loop through that. Or just do a reverse for loop where you start at the end and decrement on each loop.
$file = file("test.txt");
$file = array_reverse($file);
foreach($file as $f){
echo $f."<br />";
}
The question is asking using fseek, so can only assume that performance is an issue and file() is not the solution. Here is a simple approach using fseek:
My file.txt
#file.txt
Line 1
Line 2
Line 3
Line 4
Line 5
And the code:
<?php
$fp = fopen('file.txt', 'r');
$pos = -2; // Skip final new line character (Set to -1 if not present)
$lines = array();
$currentLine = '';
while (-1 !== fseek($fp, $pos, SEEK_END)) {
$char = fgetc($fp);
if (PHP_EOL == $char) {
$lines[] = $currentLine;
$currentLine = '';
} else {
$currentLine = $char . $currentLine;
}
$pos--;
}
$lines[] = $currentLine; // Grab final line
var_dump($lines);
Output:
array(5) {
[0]=>
string(6) "Line 5"
[1]=>
string(6) "Line 4"
[2]=>
string(6) "Line 3"
[3]=>
string(6) "Line 2"
[4]=>
string(6) "Line 1"
}
You don't have to append to the $lines array like I am, you can print the output straight away if that is the purpose of your script. Also it is easy to introduce a counter if you want to limit the number of lines.
$linesToShow = 3;
$counter = 0;
while ($counter <= $linesToShow && -1 !== fseek($fp, $pos, SEEK_END)) {
// Rest of code from example. After $lines[] = $currentLine; add:
$counter++;
}
<?php
class ReverseFile implements Iterator
{
const BUFFER_SIZE = 4096;
const SEPARATOR = "\n";
public function __construct($filename)
{
$this->_fh = fopen($filename, 'r');
$this->_filesize = filesize($filename);
$this->_pos = -1;
$this->_buffer = null;
$this->_key = -1;
$this->_value = null;
}
public function _read($size)
{
$this->_pos -= $size;
fseek($this->_fh, $this->_pos);
return fread($this->_fh, $size);
}
public function _readline()
{
$buffer =& $this->_buffer;
while (true) {
if ($this->_pos == 0) {
return array_pop($buffer);
}
if (count($buffer) > 1) {
return array_pop($buffer);
}
$buffer = explode(self::SEPARATOR, $this->_read(self::BUFFER_SIZE) . $buffer[0]);
}
}
public function next()
{
++$this->_key;
$this->_value = $this->_readline();
}
public function rewind()
{
if ($this->_filesize > 0) {
$this->_pos = $this->_filesize;
$this->_value = null;
$this->_key = -1;
$this->_buffer = explode(self::SEPARATOR, $this->_read($this->_filesize % self::BUFFER_SIZE ?: self::BUFFER_SIZE));
$this->next();
}
}
public function key() { return $this->_key; }
public function current() { return $this->_value; }
public function valid() { return ! is_null($this->_value); }
}
$f = new ReverseFile(__FILE__);
foreach ($f as $line) echo $line, "\n";
To completely reverse a file:
$fl = fopen("\some_file.txt", "r");
for($x_pos = 0, $output = ''; fseek($fl, $x_pos, SEEK_END) !== -1; $x_pos--) {
$output .= fgetc($fl);
}
fclose($fl);
print_r($output);
Of course, you wanted line-by-line reversal...
$fl = fopen("\some_file.txt", "r");
for($x_pos = 0, $ln = 0, $output = array(); fseek($fl, $x_pos, SEEK_END) !== -1; $x_pos--) {
$char = fgetc($fl);
if ($char === "\n") {
// analyse completed line $output[$ln] if need be
$ln++;
continue;
}
$output[$ln] = $char . ((array_key_exists($ln, $output)) ? $output[$ln] : '');
}
fclose($fl);
print_r($output);
Really though, Jonathan Kuhn has the best answer IMHO above. The only cases you'd not use his answer that I know of is if file or like functions are disabled via php.ini, yet the admin forgot about fseek, or when opening a huge file just get the last few lines of contents would magically save memory this way.
Note: Error handling not included. And, PHP_EOL didn't cooperate, so I used "\n" to denote end of line instead. So, above may not work in all cases.
You cannot fseek line by line, because you do not know how long the lines are until you read them.
You should either read the whole file into a list of lines, or if the file is too big for that and you only need the last lines, read fixed-sized chunks from the end of the file and implement a bit more complicated logic which detects lines from such data.
Reading the entire file into an array and reversing is fine unless the file is enormous.
You could perform a buffered read of your file from back to front with something like this:
establish a buffer_size B - this should be longer than the longest anticipated line otherwise you'll need some logic for growing the buffer size when lines are too long
set offset = file length - buffer_size
while the offset>=0
read buffer_size bytes from offset
read a line - it will be incomplete as we'll have jumped into the middle of a line, so we want to ensure the next buffer we read ends with it. Set offset = offset - buffer_size + line length
discard that line, read all following lines into an array and reverse them
process this array to do whatever you wanted to do
This code read file backwards. This code ignore modifications on reading, example apache access.log new lines on procressing.
$f = fopen('FILE', 'r');
fseek($f, 0, SEEK_END);
$pos = ftell($f);
$pos--;
while ($pos > 0) {
$chr = fgetc($f);
$pos --;
fseek($f, $pos);
if ($chr == PHP_EOL) {
YOUR_OWN_FUNCTION($rivi);
$rivi = NULL;
continue;
}
$rivi = $chr.$rivi;
}
fclose($f);
I know this has been answered already but I found another, maybe faster, way.
// Read last 5000 chars of 'foo.log'
if(file_exists('foo.log') && $file = fopen('foo.log', 'r')) {
fseek($file, -5000, SEEK_END);
$text = stream_get_line($file, 5000);
var_dump($text);
fclose($file);
}
Here's a drop in replacement(ish) for fgets($fp) called fgetsr() that reads lines from a file in reverse order.
This code is verbatim so you should (famous last words) be able to copy it into a file on your server and run it. Though you may well need to change the filename in the fopn() call.
<?php
header('Content-Type: text/plain');
$fp = fopen('post.html', 'r');
while($line = fgetsr($fp)) {
echo $line;
}
// Read a line from the file but starting from the end
//
// #param $fp integer The file pointer
//
function fgetsr($fp)
{
// Make this variable persistent inside this function
static $seeked;
// The line buffer that will eventually be returned
$line = '';
// Initially seek to the end of the file
if (!$seeked) {
fseek($fp, -1, SEEK_END);
$seeked = true;
}
// Loop through all of the characters in the file
while(strlen($char = fgetc($fp)) {
// fgetc() advances that pointer so go back TWO places
// instead of one
fseek($fp, -2, SEEK_CUR);
//
// Check for a newline (LF). If a newline is found
// then break out of the function and return the
// line that's stored in the buffer.
//
// NB The first line in the file (ie the last to
// be read)has a special case
//
if (ftell($fp) <= 0) {
fseek($fp, 0, SEEK_SET);
$line = fgets($fp);
fseek($fp, 0, SEEK_SET);
return $line;
} else if ($char === "\n") {
$line = strrev($line);
return $line . "\n";
} else {
$line .= $char;
}
}
}
?>
Functions to read a file line-by-line in reverse:
function revfopen($filepath, $mode)
{
$fp = fopen($filepath, $mode);
fseek($fp, -1, SEEK_END);
if (fgetc($fp) !== PHP_EOL) {
fseek($fp, 1, SEEK_END);
}
return $fp;
}
function revfgets($fp)
{
$s = '';
while (true) {
if (fseek($fp, -2, SEEK_CUR) === -1) {
return false;
}
if (($c = fgetc($fp)) === PHP_EOL) {
break;
}
$s = $c . $s;
}
return $s;
}
Example use case: parse a long file until some date:
$fp = revfopen('/path/to/file', 'r');
$buffer = '';
while (($line = revfgets($fp)) !== false) {
if (strpos($line, '05-10-2021') === 0) {
break;
}
array_unshift($buffer, $line);
}
echo implode("\n", $buffer);

Is there a way to access a string as a filehandle in php?

I'm on a server where I'm limited to PHP 5.2.6 which means str_getcsv is not available to me. I'm using, instead fgetcsv which requires "A valid file pointer to a file successfully opened by fopen(), popen(), or fsockopen()." to operate on.
My question is this: is there a way to access a string as a file handle?
My other option is to write the string out to a text file and then access it via fopen() and then use fgetcsv, but I'm hoping there's a way to do this directly, like in perl.
If you take a look in the user notes on the manual page for str_getcsv, you'll find this note from daniel, which proposes this function (quoting) :
<?php
if (!function_exists('str_getcsv')) {
function str_getcsv($input, $delimiter = ",", $enclosure = '"', $escape = "\\") {
$fiveMBs = 5 * 1024 * 1024;
$fp = fopen("php://temp/maxmemory:$fiveMBs", 'r+');
fputs($fp, $input);
rewind($fp);
$data = fgetcsv($fp, 1000, $delimiter, $enclosure); // $escape only got added in 5.3.0
fclose($fp);
return $data;
}
}
?>
It seems to be doing exactly what you asked for : it uses a stream, which points to a temporary filehandle in memory, to use fgetcsv on it.
See PHP input/output streams for the documentation about, amongst others, the php://temp stream wrapper.
Of course, you should test that it works OK for you -- but, at least, this should give you an idea of how to achieve this ;-)
I'm horrified that no one has answered this solution:
<?php
$string = "I tried, honestly!";
$fp = fopen('data://text/plain,' . $string,'r');
echo stream_get_contents($fp);
#fputcsv($fp, .......);
?>
And memory hungry perfect solution:
<?php
class StringStream
{
private $Variable = NULL;
protected $fp = 0;
final public function __construct(&$String, $Mode = 'r')
{
$this->$Variable = &$String;
switch($Mode)
{
case 'r':
case 'r+':
$this->fp = fopen('php://memory','r+');
fwrite($this->fp, #strval($String));
rewind($this->fp);
break;
case 'a':
case 'a+':
$this->fp = fopen('php://memory','r+');
fwrite($this->fp, #strval($String));
break;
default:
$this->fp = fopen('php://memory',$Mode);
}
}
final public function flush()
{
# Update variable
$this->Variable = stream_get_contents($this->fp);
}
final public function __destruct()
{
# Update variable on destruction;
$this->Variable = stream_get_contents($this->fp);
}
public function __get($name)
{
switch($name)
{
case 'fp': return $fp;
default: trigger error('Undefined property: ('.$name.').');
}
return NULL;
}
}
$string = 'Some bad-ass string';
$stream = new StringStream($string);
echo stream_get_contents($stream->fp);
#fputcsv($stream->fp, .......);
?>
To answer your general question, yes you can treat a variable as a file stream.
http://www.php.net/manual/en/function.stream-context-create.php
The following is a copy and paste from a few different comments on the PHP manual (so I cannot vouch for how production ready it is):
<?php
class VariableStream {
private $position;
private $varname;
public function stream_open($path, $mode, $options, &$opened_path) {
$url = parse_url($path);
$this->varname = $url["host"];
$this->position = 0;
return true;
}
public function stream_read($count) {
$p=&$this->position;
$ret = substr($GLOBALS[$this->varname], $p, $count);
$p += strlen($ret);
return $ret;
}
public function stream_write($data){
$v=&$GLOBALS[$this->varname];
$l=strlen($data);
$p=&$this->position;
$v = substr($v, 0, $p) . $data . substr($v, $p += $l);
return $l;
}
public function stream_tell() {
return $this->position;
}
public function stream_eof() {
return $this->position >= strlen($GLOBALS[$this->varname]);
}
public function stream_seek($offset, $whence) {
$l=strlen(&$GLOBALS[$this->varname]);
$p=&$this->position;
switch ($whence) {
case SEEK_SET: $newPos = $offset; break;
case SEEK_CUR: $newPos = $p + $offset; break;
case SEEK_END: $newPos = $l + $offset; break;
default: return false;
}
$ret = ($newPos >=0 && $newPos <=$l);
if ($ret) $p=$newPos;
return $ret;
}
}
stream_wrapper_register("var", "VariableStream");
$csv = "foo,bar\ntest,1,2,3\n";
$row = 1;
if (($handle = fopen("var://csv", "r")) !== FALSE) {
while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) {
$num = count($data);
echo "<p> $num fields in line $row: <br /></p>\n";
$row++;
for ($c=0; $c < $num; $c++) {
echo $data[$c] . "<br />\n";
}
}
fclose($handle);
}
?>
Of course, for your particular example, there are simpler stream methods that can be used.
You can use stream handles such as php://memory to achieve what you're after. Just open, fwrite, rewind, and you should be able to use fgetcsv.
Unfortunately, that is not possible. You cannot treat a string as if it's a stream from a file. You would indeed have to first write the string to a file, and then open said file using fopen.
And now for the obvious part, have you considered upgrading?

Categories