I want to download different feeds form some publishers. But the poor thing is, that they are first of all zipped as .gz and as second not in the right format. You can download one of the feeds and check it out. They do not have any filespec... So, I'm forced to add the .csv by myself..
My question now is, how can I unzip those files from the different urls?
How I do rename them, I know. But how do I unzip them?
I already searched for it and found this one:
//This input should be from somewhere else, hard-coded in this example
$file_name = '2013-07-16.dump.gz';
// Raising this value may increase performance
$buffer_size = 4096; // read 4kb at a time
$out_file_name = str_replace('.gz', '', $file_name);
// Open our files (in binary mode)
$file = gzopen($file_name, 'rb');
$out_file = fopen($out_file_name, 'wb');
// Keep repeating until the end of the input file
while (!gzeof($file)) {
// Read buffer-size bytes
// Both fwrite and gzread and binary-safe
fwrite($out_file, gzread($file, $buffer_size));
}
// Files are done, close files
fclose($out_file);
gzclose($file);
But with those feeds it doesn't work...
Here a two example files: file one | file two
Do you have an idea? - Would be very grateful!
Greetings!
windows 10 + php7.1.4 it's work.
The following code has the same effect.
ob_start();
readgzfile($file_name);
file_put_contents($output_filename', ob_get_contents());
ob_clean();
Or you can try to use the gzip command to decompress, and then use the it.
Program execution Functions
Related
When using force_download to download a zip file my code works for a zip file that is 268Mb (31 MP3 files) but not for a zip file that is 287Mb (32 MP3 files), the difference being 1 extra MP3 file added to the zip. The download attempts to start and appears as though it keeps starting over and over a couple of times and shows as failed with Chrome indicating that the zip file is incomplete. Windows reports the zip file which is only 61Kb is invalid when trying to open it.
The zip file gets created and MP3 files added to it by another area of code.
I have increased the memory_limit up to 1024M but its no different.
Below is the code I want working:
$this->load->helper("download");
$lastbasket = "uniquefilename.zip";
$zipdlpath = base_url()."uploads/zipped/".$lastbasket;
$fileContent = file_get_contents($zipdlpath);
force_download($lastbasket, $fileContent);
I have also tried using the following code:
$this->load->helper("download");
$lastbasket = "uniquefilename.zip";
$zipdlpath = FCPATH."uploads/zipped/".$lastbasket;
force_download($zipdlpath, NULL);
Providing a direct link to the zip file works fine (so I know the issue isnt with the zip file itself) but the force_download function in the controller appears to have an issue with larger files or is there a setting I am missing somewhere that is forcing a limit somehow?
PHP 7.1.33
CodeIgniter 3.1.9
Try to increase memory limit by adding this code:
ini_set('memory_limit','1024M');
increase memory limit and use fopen, fread
try this
$this->load->helper("download");
$lastbasket = "uniquefilename.zip";
$zipdlpath = FCPATH."uploads/zipped/".$lastbasket;
force_download($zipdlpath, NULL);
if (is_file($zipdlpath))
{
$chunkSize = 1024 * 1024;
$handle = fopen($zipdlpath, 'rb');
while (!feof($handle))
{
$buffer = fread($handle, $chunkSize);
echo $buffer;
ob_flush();
flush();
}
fclose($handle);
exit;
}
I've tried with the following custom download helper, may it will work for you.
Ref Link - https://github.com/bcit-ci/CodeIgniter/wiki/Download-helper-for-large-files
I have a file plain.cache which is little over 10MB and I made a gzcompressed file gz.cache out of the original plain.cache file. Then, I made two separate files which load each of the mentioned cache files and was kind of surprised that the page load speed of both files was almost the same. So, my question is - am I being right by concluding that gzcompressed file does not in any way benefit the load speed of the page? Now, I would conclude that the gzuncompress that I use in the gz.php file "makes" the same exact string just as when I read it from the plain file. Given all these staments - a general question is how can I (if it all in all is done this way) increase the load speed by compressing the file with gzcompress.
The image of the files is below, and the code of files is as follows:
_makeCache.php, in which I make the gzcompressed version of the plain.cache file:
$str = file_get_contents("plain.cache");
$strCompressed = gzcompress($str, 9);
$file = "gz.cache";
$fp = fopen($file, "w");
fwrite($fp, $strCompressed);
fclose($fp);
plain.php:
echo file_get_contents("plain.cache");
gz.php:
echo gzuncompress(file_get_contents("plain.cache"));
Your http server is compressing the plain.cache automatically on the fly, using gzip as well, and the client decompresses it. So you should see almost no difference.
I'm trying to let users upload files onto my website, but unfortunately some of them seem to turn corrupt when reading them. I've tried both images and html files, and all the images come through corrupt (the HTML files come through fine).
To upload the files I'm using a standard HTML form and the PHP $_FILES array. I'm then using the following code to read the contents of the file:
$filename = $_FILES['varname']['tmp_name'];
$handle = fopen($filename, "r");`
$contents = fread($handle, filesize($filename));
fclose($handle);
Unfortunately the value of $contents is now slightly different to the file I uploaded (here's a snippet from the top of the file):
Original file:
ˇÿˇ·ExifII*ˇÏDucky<ˇÓAdobed¿ˇ€Ñ
New file:
ˇÿˇ· Exif II* ˇÏ Ducky < ˇÓ Adobe d¿ ˇ€ Ñ
As you can see there's a difference in the spacing. Any ideas what would be causing this? Am I handling the file read incorrectly for binary files? It seems odd that it's fine for any text files I upload..
Thanks!
I usually output files like this:
header("Content-Disposition: attachment; filename=\"$fileName\"");
readfile("$HOME_DIR/uploads/$fileName");
exit();
Anyway, to try to debug your problem, you should first understand which phase is failing. Upload or download? To check, just go to your webserver and download the file via FTP, then open it in a binary editor. If it is already corrupt then you need to investigate your upload phase, otherwise it's the other way around.
how do you print $contents ? Are you sure this is a problem with reading the file ?
I guess that maybe this is a problem with PRINTING the file to the output... Try printing it binary way. Something like:
$data = unpack("C*", $contents);
foreach ($data as $v)
{
echo $v, ' ';
}
and compare that with binary dump of the original file...
I'm finding it difficult to phrase this question correctly, let me try to explain our problem...
We have an intranet running on Ubunutu box with Apache2/PHP 5.2.4. We have a bit of PHP code that reads a file from a directory that is not publically accessible and output it to the screen (code below):
$file_path = '/home/path/to/filename.gif';
if(file_exists($file_path)){
$output = FALSE;
//File Information
$path_parts = pathinfo($file_path);
$file_size = filesize($file_path);
$file_ext = (isset($path_parts['extension'])) ? strtolower($path_parts['extension']) : null;
$file_name = $path_parts['basename'];
//Sets up the headers
if($file_size > 0){
header('Content-Length: ' .$file_size);
}
header('Content-Disposition: attachment; filename="'.$file_name.'"');
header('Content-Type: application/octet-stream');
//Reads the File
if($file_size > 0){
$handle = fopen($file_path, "r");
$output = fread($handle, $file_size);
fclose($handle);
}
//Outputs the File
echo $output;
}
Inside our network when, browsing to the page that uses this code, the file is downloaded perfectly and quickly...
However, when accessing this page via our Cisco ASA/Proxy/VPN (not sure what to call it) this code locks up the browser, but does eventually download the file...
After a bit of experimenting, after taking out the headers and just echoing the contents of the file to the browser, it prints no problem. However as soon as I add the lines with the headers back into the code it causes the hanging again, but only when accessed via this box..
Anybody come across this problem before or have any idea what we can try to move forward?
Thanks for any advice...
Have you tried eliminating the content-size header entirely? The proxy may be taking that as a firm promise and if the data you're sending ends up being a different size, the proxy may wait for those last few "missing" bytes to show up.
Just as an aside, you should use [readfile()][1] instead of the fopen()/fread()/echo construct you have now.
As it stands now, you're slurping the contents of the entire file into memory and then echoing out. For large files and multiple requests, you'll kill the server with memory starvation. readfile will automatically stream the file in smaller chunks so that memory usage is minimal.
Your proxy obviously have problems with the Content-Type: application/octet-stream. Try setting it to the real MIME-type of each file. You can use the Fileinfo module to find out which MIME-type a certain file is, like this:
//You may need to specify the location of your system's magic file
//See http://php.net/finfo_open for more info
$finfo = new finfo(FILEINFO_MIME);
$mimetype = $finfo->file($file_path);
I'm using CodeIgniter and I can't figure out how to unzip files!
PHP itself has a number of functions for dealing with gzip files.
If you want to create a new, uncompressed file, it would be something like this.
Note: This doesn't check if the target file exists first, doesn't delete the input file, or do any error checking. You really should fix those before using this in production code.
// This input should be from somewhere else, hard-coded in this example
$file_name = 'file.txt.gz';
// Raising this value may increase performance
$buffer_size = 4096; // read 4kb at a time
$out_file_name = str_replace('.gz', '', $file_name);
// Open our files (in binary mode)
$file = gzopen($file_name, 'rb');
$out_file = fopen($out_file_name, 'wb');
// Keep repeating until the end of the input file
while(!gzeof($file)) {
// Read buffer-size bytes
// Both fwrite and gzread and binary-safe
fwrite($out_file, gzread($file, $buffer_size));
}
// Files are done, close files
fclose($out_file);
gzclose($file);
Note: This deals with gzip only. It doesn't deal with tar.
gzopen is way too much work. This is more intuitive:
$zipped = file_get_contents("foo.gz");
$unzipped = gzdecode($zipped);
works on http pages when the server is spitting out gzipped data also.
If you have access to system():
system("gunzip file.sql.gz");
Use the functions implemented by the Zlib Compression extension.
This snippet shows how to use some of the functions made available from the extension:
// open file for reading
$zp = gzopen($filename, "r");
// read 3 char
echo gzread($zp, 3);
// output until end of the file and close it.
gzpassthru($zp);
gzclose($zp);
Download the Unzip library
and include or autoload the unzip library
$this->load->library('unzip');