I am using a fairly straight-forward script to open and parse several xml files that are gzipped. I also need to do the same basic operation with a ZIP file. It seems like it should be simple, but I haven't been able to find what looked like equivalent code anywhere.
Here is the simple version of what I am already doing:
$import_file = "source.gz";
$sfp = gzopen($import_file, "rb"); ///// OPEN GZIPPED data
while ($string = gzread($sfp, 4096)) { //Loop through the data
/// Parse Output And Do Stuff with $string
}
gzclose($sfp);
What would do the same thing for a zipped file?
If you have PHP 5 >= 5.2.0, PECL zip >= 1.5.0 then you may use the ZipArchive libraries:
$zip = new ZipArchive;
if ($zip->open('source.zip') === TRUE)
{
for($i = 0; $i < $zip->numFiles; $i++)
{
$fp = $zip->getStream($zip->getNameIndex($i));
if(!$fp) exit("failed\n");
while (!feof($fp)) {
$contents = fread($fp, 8192);
// do some stuff
}
fclose($fp);
}
}
else
{
echo 'Error reading zip-archive!';
}
There is a smart way to do it with ZipArchive. You can use a for loop with ZipArchive::statIndex() to get all the information you need. You can access the files by their index (ZipArchive::getFromIndex()) or name (ZipArchive::getFromName()).
For example:
function processZip(string $zipFile): bool
{
$zip = new ZipArchive();
if ($zip->open($zipFile) !== true) {
echo '<p>Can\'t open zip archive!</p>';
return false;
}
// As long as statIndex() does not return false keep iterating
for ($idx = 0; $zipFile = $zip->statIndex($idx); $idx++) {
$directory = \dirname($zipFile['name']);
if (!\is_dir($zipFile['name'])) {
// file contents
$contents = $zip->getFromIndex($idx);
}
}
$zip->close();
}
Related
We currently open a csv from our server and then do some stuff with the data like this:
$CSVfile = fopen('filename.csv', "r");
if($CSVfile !== FALSE) {
$count = 0;
while(! feof($CSVfile)) {
$data = fgetcsv($CSVfile, 5000, ",");
if ($count > 0 && !empty($data)) {
// Do something
}
}
}
We need to change the system as the file will now be hosted on an external server so we need to SFTP into it to retrieve the file. I've installed phpseclib and got the connection working but it just keeps echoing the file contents on the screen. I have it set up like this:
include 'vendor/autoload.php';
$sftp = new \phpseclib\Net\SFTP('SERVER');
if (!$sftp->login('USERNAME', 'PASSWORD')) {
exit('Login Failed');
} else {
$file = $sftp->fetch('FILE_LOCATION');
}
$CSVfile = fopen($file, "r");
if($CSVfile !== FALSE) {
$count = 0;
while(! feof($CSVfile)) {
$data = fgetcsv($CSVfile, 5000, ",");
if ($count > 0 && !empty($data)) {
// Do Something
}
}
}
How do I get the new system to read the file contents and do something with it rather than just showing all the file contents?
As #verjas commented already, there's no fetch method in phpseclib.
If you want to download a file to a string, use SFTP::get:
$contents = $sftp->get("/remote/path/file.csv");
You can then use str_getcsv to parse the contents. According to a contributed note at the function documentation, this should do:
$data = str_getcsv($contents, "\n"); // parse the rows
foreach ($data as $line)
{
$row_data = str_getcsv($line); // parse the items in rows
}
I have one file: configuration.txt.
This file gets read by PHP, then wrote by the same PHP, while a C++ program reads the content of the same file at a regular interval.
PHP:
$closeFlag = false;
$arrayInputs = new SplFixedArray(3);
$arrayInputs[0] = "URL not entered";
$arrayInputs[1] = "3";
$arrayInputs[2] = "50";
$configFilePath = "/var/www/html/configuration.txt";
$currentSettingsFile = fopen($configFilePath, "r");
if(flock($currentSettingsFile, LOCK_SH)) {
$arrayInputs = explode(PHP_EOL, fread($currentSettingsFile, filesize($configFilePath)));
flock($currentSettingsFile, LOCK_UN);
$closeFlag = fclose($currentSettingsFile);
}
if(isset( $_POST['save_values'])) {
if(!empty($_POST['getURL'])) {
$arrayInputs[0] = $_POST['getURL'];
}
if(!empty($_POST['getURR'])) {
$arrayInputs[1] = $_POST['getURR'];
}
if(!empty($_POST['getBrightness'])) {
$arrayInputs[2] = $_POST['getBrightness'];
}
}
if(!$closeFlag) fclose($currentSettingsFile);
$currentSettingsFile = fopen($configFilePath, "w");
if(flock($currentSettingsFile, LOCK_SH)) {
foreach ($arrayInputs as $key => $value) {
if($value != '')
fwrite($currentSettingsFile,$value.PHP_EOL);
}
flock($currentSettingsFile, LOCK_UN);
fclose($currentSettingsFile);
}
?>
C++
char configFilePath[]="/var/www/html/configuration.txt";
std::fstream configFile;
configFile.open(configFilePath, std::fstream::in);
if(configFile.is_open()){
// do stuff
} else {
std::cout<<"Error ! Could not open Configuration file to read"<<std::endl;
}
The c++ returned no error so far. It can open the file. And php will return Warning: fread(): Length parameter must be greater than 0 because the file is empty.
I believe that PHP is deleting the file's content.
When locking a file in PHP, you lock a LOCK file, not the main file. Example:
$myfile = 'myfile.txt';
$lockfile = 'myfile.lock';
$lock = fopen($lockfile,'a');
if(flock($lock, LOCK_EX)) // The lock file is locked in exclusive mode - so I can write to it.
{
$fp = fopen($myfile,'w');
fputs($fp, "I am writing safely!");
fclose($fp);
flock($lock, LOCK_UN); // Always unlock it!
}
fclose($lock);
You work similarly in C++ because PHP is not locking the actual file. It is locking a lock file. The exact syntax depends heavily on your version of C/C++ and the operating system. So, I will use minimal syntax.
int lock=fopen(lockfile, "r+");
if(flock(fileno(lock), LOCK_EX))
{
//Locked. You can open a stream to ANOTHER file and play with it.
flock(fileno(lock), LOCK_UN));
}
fclose(lock);
I'm running 2 simple function:
<?php
$zipUrl = "path_of_original.zip"
$zipFilename = "local_path_and_name_of.zip"
$unzipPath = "destination_of_unzipped_files"
upload_archive ($zipUrl, $zipFilename);
unzip_archive ($zipFilename, $unzipPath);
?>
the 1st, upload a .zip archive on server
function upload_archive ($zipUrl, $zipFilename){
define('BUFSIZ', 4095);
$rfile = fopen($zipUrl, 'r');
$lfile = fopen($zipFilename, 'w');
while(!feof($rfile))
fwrite($lfile, fread($rfile, BUFSIZ), BUFSIZ);
fclose($rfile);
fclose($lfile);}
the 2nd, unzip the archive
function unzip_archive ($zipFilename, $unzipPath){
$zip = new ZipArchive;
$res = $zip->open($zipFilename);
if ($res === TRUE) {
$zip->extractTo($unzipPath);
$zip->close();
echo 'success!';
} else {
echo 'error!';
}
}
when these 2 functions are executed separately everything's fine, but when executed in sequence I can't appreciate the output of the second function (unzip).
I think the problem is that the .zip file is still locked in write by the first function.
any Suggestions?
Angelo.
I am reading a file containing around 50k lines using the file() function in Php. However, its giving a out of memory error since the contents of the file are stored in the memory as an array. Is there any other way?
Also, the lengths of the lines stored are variable.
Here's the code. Also the file is 700kB not mB.
private static function readScoreFile($scoreFile)
{
$file = file($scoreFile);
$relations = array();
for($i = 1; $i < count($file); $i++)
{
$relation = explode("\t",trim($file[$i]));
$relation = array(
'pwId_1' => $relation[0],
'pwId_2' => $relation[1],
'score' => $relation[2],
);
if($relation['score'] > 0)
{
$relations[] = $relation;
}
}
unset($file);
return $relations;
}
Use fopen, fread and fclose to read a file sequentially:
$handle = fopen($filename, 'r');
if ($handle) {
while (!feof($handle)) {
echo fread($handle, 8192);
}
fclose($handle);
}
EDIT after update of question and comments to answer of fabjoa:
There is definitely something fishy if a 700kb file eats up 140MB of memory with that code you gave (you could unset $relation at the end of the each iteration though). Consider using a debugger to step through it to see what happens. You might also want to consider rewriting the code to use SplFileObject's CSV functions as well (or their procedural cousins)
SplFileObject::setCsvControl example
$file = new SplFileObject("data.csv");
$file->setFlags(SplFileObject::READ_CSV);
$file->setCsvControl('|');
foreach ($file as $row) {
list ($fruit, $quantity) = $row;
// Do something with values
}
For an OOP approach to iterate over the file, try SplFileObject:
SplFileObject::fgets example
$file = new SplFileObject("file.txt");
while (!$file->eof()) {
echo $file->fgets();
}
SplFileObject::next example
// Read through file line by line
$file = new SplFileObject("misc.txt");
while (!$file->eof()) {
echo $file->current();
$file->next();
}
or even
foreach(new SplFileObject("misc.txt") as $line) {
echo $line;
}
Pretty much related (if not duplicate):
How to save memory when reading a file in Php?
If you don't know the maximum line length and you are not comfortable to use a magic number for the max line length then you'll need to do an initial scan of the file and determine the max line length.
Other than that the following code should help you out:
// length is a large number or calculated from an initial file scan
while (!feof($handle)) {
$buffer = fgets($handle, $length);
echo $buffer;
}
Old question but since I haven't seen anyone mentioning it, PHP generators is a great way to reduce save memory consumption.
For example:
function read($fileName)
{
$fileHandler = fopen($fileName, 'rb');
while(($line = fgets($fileHandler)) !== false) {
yield rtrim($line, "\r\n");
}
fclose($fileHandler);
}
foreach(read(__DIR__ . '/filenameHere') as $line) {
echo $line;
}
allocate more memory during the operation, maybe something like ini_set('memory_limit', '16M');. Don't forget to go back to initial memory allocation once operation is done
This is what I have so far:
<?php
$file = "18201010338AM16390621000846.png";
$test = file_get_contents($file, FILE_BINARY);
echo str_replace("\n","<br>",$test);
?>
The output is sorta what I want, but I really only need lines 3-7 (inclusively). This is what the output looks like now: http://silentnoobs.com/pbss/collector/test.php. I am trying to get the data from "PunkBuster Screenshot (±) AAO Bridge Crossing" to "Resulting: w=394 X h=196 sample=2". I think it'd be fairly straight forward to read through the file, and store each line in an array, line[0] would need to be "PunkBuster Screenshot (±) AAO Bridge Crossing", and so on. All those lines are subject to change, so I can't just search for something finite.
I've tried for a few days now, and it doesn't help much that I'm poor at php.
The PNG file format defines that a PNG document is split up into multiple chunks of data. You must therefore navigate your way to the chunk you desire.
The data you want to extract seem to be defined in a tEXt chunk. I've written the following class to allow you to extract chunks from PNG files.
class PNG_Reader
{
private $_chunks;
private $_fp;
function __construct($file) {
if (!file_exists($file)) {
throw new Exception('File does not exist');
}
$this->_chunks = array ();
// Open the file
$this->_fp = fopen($file, 'r');
if (!$this->_fp)
throw new Exception('Unable to open file');
// Read the magic bytes and verify
$header = fread($this->_fp, 8);
if ($header != "\x89PNG\x0d\x0a\x1a\x0a")
throw new Exception('Is not a valid PNG image');
// Loop through the chunks. Byte 0-3 is length, Byte 4-7 is type
$chunkHeader = fread($this->_fp, 8);
while ($chunkHeader) {
// Extract length and type from binary data
$chunk = #unpack('Nsize/a4type', $chunkHeader);
// Store position into internal array
if ($this->_chunks[$chunk['type']] === null)
$this->_chunks[$chunk['type']] = array ();
$this->_chunks[$chunk['type']][] = array (
'offset' => ftell($this->_fp),
'size' => $chunk['size']
);
// Skip to next chunk (over body and CRC)
fseek($this->_fp, $chunk['size'] + 4, SEEK_CUR);
// Read next chunk header
$chunkHeader = fread($this->_fp, 8);
}
}
function __destruct() { fclose($this->_fp); }
// Returns all chunks of said type
public function get_chunks($type) {
if ($this->_chunks[$type] === null)
return null;
$chunks = array ();
foreach ($this->_chunks[$type] as $chunk) {
if ($chunk['size'] > 0) {
fseek($this->_fp, $chunk['offset'], SEEK_SET);
$chunks[] = fread($this->_fp, $chunk['size']);
} else {
$chunks[] = '';
}
}
return $chunks;
}
}
You may use it as such to extract your desired tEXt chunk as such:
$file = '18201010338AM16390621000846.png';
$png = new PNG_Reader($file);
$rawTextData = $png->get_chunks('tEXt');
$metadata = array();
foreach($rawTextData as $data) {
$sections = explode("\0", $data);
if($sections > 1) {
$key = array_shift($sections);
$metadata[$key] = implode("\0", $sections);
} else {
$metadata[] = $data;
}
}
<?php
$fp = fopen('18201010338AM16390621000846.png', 'rb');
$sig = fread($fp, 8);
if ($sig != "\x89PNG\x0d\x0a\x1a\x0a")
{
print "Not a PNG image";
fclose($fp);
die();
}
while (!feof($fp))
{
$data = unpack('Nlength/a4type', fread($fp, 8));
if ($data['type'] == 'IEND') break;
if ($data['type'] == 'tEXt')
{
list($key, $val) = explode("\0", fread($fp, $data['length']));
echo "<h1>$key</h1>";
echo nl2br($val);
fseek($fp, 4, SEEK_CUR);
}
else
{
fseek($fp, $data['length'] + 4, SEEK_CUR);
}
}
fclose($fp);
?>
It assumes a basically well formed PNG file.
I found this problem a few days ago, so I made a library to extract the metadata (Exif, XMP and GPS) of a PNG in PHP, 100% native, I hope it helps. :) PNGMetadata
How about:
http://www.php.net/manual/en/function.getimagesize.php