I am downloading a gzip csv and writing the un-zipped string to a file. Using:
$file = gzopen($this->getTmpZipFileName(), 'rb');
$outPutFile = fopen($uncompressedFileName, 'wb');
while(!gzeof($file)){
fwrite($outPutFile, gzgets($file, $bufferSie));
}
At some point during this process something is breaking with a space " ". It is treating the " " as a new line. Which of course will 'break' the csv.
I believe it is something to do with the uncompressing of the gzip file. If I dump out
var_dump(var_dump(gzread($file,100000)));
die();
I get the same issue.
Uncompressing the csv through terminal the csv file is fine.
I am at a loss of what else I can try to open the file correctly.
Any help will be much appreciated.
It turns out when I was creating the gzip file something was messed up with the compression. Using another file from another source works as expected. This has driven me mad!!
Related
I want to download different feeds form some publishers. But the poor thing is, that they are first of all zipped as .gz and as second not in the right format. You can download one of the feeds and check it out. They do not have any filespec... So, I'm forced to add the .csv by myself..
My question now is, how can I unzip those files from the different urls?
How I do rename them, I know. But how do I unzip them?
I already searched for it and found this one:
//This input should be from somewhere else, hard-coded in this example
$file_name = '2013-07-16.dump.gz';
// Raising this value may increase performance
$buffer_size = 4096; // read 4kb at a time
$out_file_name = str_replace('.gz', '', $file_name);
// Open our files (in binary mode)
$file = gzopen($file_name, 'rb');
$out_file = fopen($out_file_name, 'wb');
// Keep repeating until the end of the input file
while (!gzeof($file)) {
// Read buffer-size bytes
// Both fwrite and gzread and binary-safe
fwrite($out_file, gzread($file, $buffer_size));
}
// Files are done, close files
fclose($out_file);
gzclose($file);
But with those feeds it doesn't work...
Here a two example files: file one | file two
Do you have an idea? - Would be very grateful!
Greetings!
windows 10 + php7.1.4 it's work.
The following code has the same effect.
ob_start();
readgzfile($file_name);
file_put_contents($output_filename', ob_get_contents());
ob_clean();
Or you can try to use the gzip command to decompress, and then use the it.
Program execution Functions
I have a gzip file. Into this file I have many csv files. I want to read only one of these files that are contained into the gzip folder. The name of the gzip folder that I want to read is 'com.instore'. And the name of the csv file is 'second'.
I use the gzopen() to read the gzip folder. But now I don't know how to read the csv file contained inside.
So, how can I do this?
Do I have to close with the gzclose()?
The gzip is into a server directory. How can I reach it?
When you gzopen you dont uncompress it. you just creates a file pointer to it.
<?php
// get contents of a gz-file into a string
$filename = "/usr/local/something.txt.gz";
$zd = gzopen($filename, "r");
$contents = gzread($zd, 10000);
gzclose($zd);
?>
while $contents is your csv file as a string.
more info # http://www.php.net/manual/en/function.gzread.php
You can extract the gzip file using this:
How can I extract or uncompress gzip file using php?
and than using this:
How to import csv file in PHP?
populate the database.
I have a file plain.cache which is little over 10MB and I made a gzcompressed file gz.cache out of the original plain.cache file. Then, I made two separate files which load each of the mentioned cache files and was kind of surprised that the page load speed of both files was almost the same. So, my question is - am I being right by concluding that gzcompressed file does not in any way benefit the load speed of the page? Now, I would conclude that the gzuncompress that I use in the gz.php file "makes" the same exact string just as when I read it from the plain file. Given all these staments - a general question is how can I (if it all in all is done this way) increase the load speed by compressing the file with gzcompress.
The image of the files is below, and the code of files is as follows:
_makeCache.php, in which I make the gzcompressed version of the plain.cache file:
$str = file_get_contents("plain.cache");
$strCompressed = gzcompress($str, 9);
$file = "gz.cache";
$fp = fopen($file, "w");
fwrite($fp, $strCompressed);
fclose($fp);
plain.php:
echo file_get_contents("plain.cache");
gz.php:
echo gzuncompress(file_get_contents("plain.cache"));
Your http server is compressing the plain.cache automatically on the fly, using gzip as well, and the client decompresses it. So you should see almost no difference.
I'm trying to debug this issue by posting raw PNG image data to the server with the help of Postman. Here's a screenshot, which might help to understand the issue:
On the server I'm receiving the file as follows:
$png = $GLOBALS["HTTP_RAW_POST_DATA"];
Then I write the data to a new file:
$fh = fopen($myFile, 'w') or die("can't open file");
fwrite($fh, $png);
fclose($fh);
The file gets saved correctly, but it now has a different file size,
417KB instead of 279KB which is the size of the original file.
Now of course, I can't do any image operation as none of the functions (such as getimagesize which returns bool(false)) recognizes the file as a valid image.
I have debugged this process to a point where the issue must be somewhere in the file operations, but I don't understand why the file just doesn't result in the very same file type and size as the original, when the only thing I am doing is using the same raw data.
UPDATE:
I've now compared the encodings of the original file with the uploaded one,
and the former is in ISO-8859-1 and it displays correctly, the latter is in UTF-8 and has about 138kB more in file size.
Now I've achieved to convert the file on the server to ISO-8859-1.
fwrite($fh, iconv("UTF-8", "ISO-8859-1", $png));
The resulting file does now have the same output file size (279kB),
but it is still not recognized as a PNG image, some information seems to still get lost.
UPDATE (1):
I've been able to examine the issue further and found out, that the original file is exactly 4 bytes bigger than the generated file, thus the resulting PNG seems to be corrupted.
UPDATE (2):
I'm now able to save the file and open it as a valid PNG. The following code seems to be saving the image correctly:
$input = fopen("php://input","r+");
$destination = fopen($myFile, 'w+');
stream_copy_to_stream($input, $destination);
fclose($input);
fclose($destination);
However when trying to open the file with the imagecreatefrompng function I get a 500 error. I'm now trying to figure out if it's a memory issue in PHP.
Problem might be the way you test your POST by copying the "binary" data into a text field.
If you paste the same data into a text editor you won't get a valid image file either when saving this with the png extension.
Try to build a simple form with file field to test your upload
I use nginx for uploads and haven't had a problem, but I use the standard PHP way of uploading files as per: http://www.php.net/manual/en/features.file-upload.post-method.php
I would suggest trying that.
Try using: < ?php $postdata = file_get_contents("php://input"); ?>
To get the raw data. I use it some times to get data sent from a ajax post on cake.
I'm trying to let users upload files onto my website, but unfortunately some of them seem to turn corrupt when reading them. I've tried both images and html files, and all the images come through corrupt (the HTML files come through fine).
To upload the files I'm using a standard HTML form and the PHP $_FILES array. I'm then using the following code to read the contents of the file:
$filename = $_FILES['varname']['tmp_name'];
$handle = fopen($filename, "r");`
$contents = fread($handle, filesize($filename));
fclose($handle);
Unfortunately the value of $contents is now slightly different to the file I uploaded (here's a snippet from the top of the file):
Original file:
ˇÿˇ·ExifII*ˇÏDucky<ˇÓAdobed¿ˇ€Ñ
New file:
ˇÿˇ· Exif II* ˇÏ Ducky < ˇÓ Adobe d¿ ˇ€ Ñ
As you can see there's a difference in the spacing. Any ideas what would be causing this? Am I handling the file read incorrectly for binary files? It seems odd that it's fine for any text files I upload..
Thanks!
I usually output files like this:
header("Content-Disposition: attachment; filename=\"$fileName\"");
readfile("$HOME_DIR/uploads/$fileName");
exit();
Anyway, to try to debug your problem, you should first understand which phase is failing. Upload or download? To check, just go to your webserver and download the file via FTP, then open it in a binary editor. If it is already corrupt then you need to investigate your upload phase, otherwise it's the other way around.
how do you print $contents ? Are you sure this is a problem with reading the file ?
I guess that maybe this is a problem with PRINTING the file to the output... Try printing it binary way. Something like:
$data = unpack("C*", $contents);
foreach ($data as $v)
{
echo $v, ' ';
}
and compare that with binary dump of the original file...