I am right at the start of trying to write some PHP code to run on a Linux box on an EC2 server that will read files from my S3 bucket, zip them then write the zip file back to the bucket.
I have instantly run in to problems with even creating a simple zip archive from some images on the local disk of the EC2 instance, I am using a script to test out the idea from the PHP manual online and also have tried out a script from David Walsh - http://davidwalsh.name/create-zip-php which looks like it will be great. Neither result in an actual zip file and both give me different status results -
the first snippet from the php manual (to which i add the variable $thisdir)-
<?php
$zip = new ZipArchive();
$filename = "test112.zip";
$thisdir = "/uploads/";
if ($zip->open($filename, ZIPARCHIVE::CREATE)!==TRUE) {
exit("cannot open <$filename>\n");
}
$zip->addFromString("testfilephp.txt" . time(), "#1 This is a test string added as testfilephp.txt.\n");
$zip->addFromString("testfilephp2.txt" . time(), "#2 This is a test string added as testfilephp2.txt.\n");
$zip->addFile($thisdir . "/too.php","/testfromfile.php");
echo "numfiles: " . $zip->numFiles . "\n";
echo "status:" . $zip->status . "\n";
$zip->close();
?>
output =
numfiles: 2 status:11
I dont see any zip file in my 'uploads' folder
The second bit of code i try ( i wont post the code here) - I pass real files and it returns
The zip archive contains 2 files with a status of 0
What are the status messages. I have checked to see if I have the correct libraries installed by looking at the output of phpinfo(); and under the ZIP heading I see -
Zip enabledExtension Version $Id: php_zip.c,v 1.1.2.43 2008/01/18 00:51:38 pajoye Exp $
Zip version 1.8.11
Libzip version 0.8.0-compatible
I have checked the permissions of the files PHP files with the code I am executing and they are set to 777 as is the folder I am trying to add the ziparchive to. I know that this should not remain at 777.
any ideas why I cannot see a zip file? what do the status values mean? Is there a good tutorial out there for using PHP to zip files on an Amazon S3 bucket? or a good utility to provide this functionality?
cheers
You can find it here:
http://www.php.net/manual/en/zip.constants.php
Error 11 is "Can't open file"
In my case i got error code 11 while opening a zip file for reading. The path and permissions are OK. I just changed the DIRECTORY SEPARATOR from back slash (\) to forward slash (/) and it worked. The strange thing is that forward slash (/) paths are in Linux where as I am working in windows server 2003.
Related
I want to read a the .txt file content from a .zip file that contains many other .txt files. But hope do not need to download the .zip file to the local machine..
How can I use functions like zip_open/zip_read to do it?
$path = 'Data/2021';
ftp_login($conn,$user,$pass);
ftp_chdir($conn,$path);
ftp_pasv($conn,TRUE);
foreach (ftp_nlist($conn,".") as $Lot_Zip) {
$Lot_Zip = Trim($Lot_Zip);
$Lot_Zip_Name = substr($Lot_Zip,0,10);
if (preg_match("/$Lot_Zip_Name/i",$ParsedLot_String)){
echo "processing ", $Lot_Zip, " ...\n";
// $Lot_Zip = $path.'/'.$Lot_Zip also can't be zip_open
$zip = zip_open($Lot_Zip);
echo "ZIP: ", $zip ,"\n"; //return value is 11
if ($zip)
{
while ($zip_entry = zip_read($zip))
{
echo "Name: " . zip_entry_name($zip_entry) . "\n";
}
}
}
}
Opening a connection to an FTP server won't make functions magically use files on the FTP server. The $Lot_Zip in zip_open($Lot_Zip) refers to the local file on the machine that runs your PHP script (webserver?), not files on the FTP server. With some PHP functions you can use ftp:// URL wrapper to refer to the files on the FTP server. But I believe that neither the deprecated zip_open, nor the up-to-date FtpArchive support the URL wrappers. They can work with local files only.
So the easiest solution is to download the archive from the FTP server to the local machine/webserver.
In general, it's rather complicated to work with ZIP files on an FTP server. Working with ZIP files requires a random access to the file contents, what is not easy to achieve with the FTP protocol.
See this related (Python) question that discusses the problem into more details:
Get files names inside a zip file on FTP server without downloading whole archive
I am trying to convert an Excel file to PDF (Base64).
This is the code that converts the Excel to PDF:
$spreadsheet = $this->objPHPExcel = \PhpOffice\PhpSpreadsheet\IOFactory::load("MyExcelFile.xlsx");
$class = \PhpOffice\PhpSpreadsheet\Writer\Pdf\Mpdf::class;
\PhpOffice\PhpSpreadsheet\IOFactory::registerWriter('Pdf', $class);
$this->objPHPExcel = \PhpOffice\PhpSpreadsheet\IOFactory::createWriter($spreadsheet, 'Pdf');
$this->objPHPExcel->writeAllSheets();
//Save the file. (THIS IS WHERE THE ERROR OCCUR)
$this->objPHPExcel->save(storage_path() . '/app/temp_files/' . $newFileName);
Everything works locally, but whenever I try to run the same code on my Laravel Forge server, I get below error:
unlink(/tmp/imagick-3.4.0.tgz): Operation not permitted
If I trace the error, it is in this specific line:
$this->objPHPExcel->save(storage_path() . '/app/temp_files/' . $newFileName);
As said, this code runs fine locally. The temp file $newFileName is created inside my /temp_files folder.
What am I doing wrong?
Ok so the solution to this was rather tricky. I found out that it had nothing to do with Phpspreadsheet but rather Mpdf.
The problem was that the file "imagick-3.4.0.tgz" file permission was set to read only. Mening that unlink could not work on this specific file. This goes all the way back to when I first installed the imagick library.
The solution was to go to the /tmp folder and delete the imagick-3.4.0.tgz file manually. This folder should actually be deleted when doing the imagick installation.
I'm trying to move file received from js using POST to another folder on linux sever using php
with shell_exec and also printing the move command in one text file for my reference so that i can verify the command is right or not
this is my code
$command = "mv ".$file_tmp_name." ".$curr_dir."/wiresharktraces/".$file_name;
$log_string = "command = $command \r\n";
fwrite($file, $log_string);
shell_exec("mv ".$file_tmp_name." ".$curr_dir."/wiresharktraces/".$file_name);
this code is running fine when file name are short but for bigger file name the code is not running properly it is not able to move the file
command = mv /tmp/phpZRDvIp /var/www/html/web/auto/exfo/excelautomation/wiresharktraces/5.3.3.1 Tracking Area Update procedure with Serving GW change(SGW_MME).pcap (this is my log file's output)
for this file the file is not moved in desired folder but when i change the file name to "ab" the file is moved
and i have also checked few website for maximum allowed length for mp command which is 131072 bytes much more than my command's length which is 153
thank you for your help in advance
found reason not solution
the reason for this strange behavior is "spaces" and "()" or any another character except "_" in file name but i want solution because i can not restrict user for file name and i want to save it with same name
the solution is very simple replace the other character(space, (, [, etc) in file name with underscore
$file_name = preg_replace("/[^0-9,a-z,A-Z,.]/", "_", $file_name);
this will replace all other character except number, alphabet and . so now Linux command will execute safely with no error
I've searched all over for a solution to this and I can't find one.
I'm doing something like the following:
$data = file_get_contents('zip://../files/' . $this->_file->getHash() . '.zip#' . $stat['name']);
The specific error is:
failed to open stream: operation failed
It works on a Linux machine I use, but I'm trying to get the project running on a Windows box. I know that the path to the zip file is correct and it is readable because in the same code that the above snippet is located in, I read it using the ZipArchive class, like this:
$this->_za = new ZipArchive();
$res = $this->_za->open('../files/' . $file->getHash() . '.zip');
And that works perfectly fine.
Why can't file_get_contents read from the zip? Is there something I have to do to the files directory? I'm not very good at folders and permissions on Windows.
Edit: to be absolute sure it wasn't an issue with my code, I set up a very simple test script, with a zip file in the exact same directory:
<?php
$za = new ZipArchive();
$res = $za->open('myZip.zip');
for($i=0; $i<$za->numFiles; $i++)
{
$stat = $za->statIndex($i);
$data = file_get_contents('zip://myZip.zip#' . $stat['name']);
echo $stat['name'];
echo $data;
}
This outputs the name of each file in the zip (from the echo $stat['name'] part) but then produces the exact same error as before, i.e.,
failed to open stream: operation failed
The fix was reasonably simple. Instead of:
$data = file_get_contents('zip://myZip.zip#' . $stat['name']);
If I do:
$data = file_get_contents('zip://' . realpath('myZip.zip') . '#' . $stat['name']);
It works.
On Linux the zip wrapper needs to be compiled with PHP in order to use zip:// which appears to be the case for you.
On Windows you need to enable php_zip.dll inside of php.ini in order to use it, and may need to install a PECL extension (not sure) http://pecl.php.net/package/zip.
Not sure where you are storing the files, but this bug on PHP might be relevant to your issue https://bugs.php.net/bug.php?id=54128
I have a PHP site with a lot of media files and users need to be able to download multiple files at a time as a .zip. I'm trying to use ZipStream to serve the zips on the fly with "store" compression so I don't actually have to create a zip on the server, since some of the files are huge and it's prohibitively slow to compress them all.
This works great and the resulting files can be opened by every zip program I've tried with no errors except for OS X's default unzipping program, Archive Utility. You double click the .zip file and Archive Utility decides it doesn't look a real zip and instead compresses into a .cpgz file.
Using unzip or ditto in the OS X terminal or StuffIt Expander unzips the file with no problem but I need the default program (Archive Utility) to work for the sake of our users.
What sort of things (flags, etc.) in otherwise acceptable zip files can trip Archive Utility into thinking a file isn't a valid zip?
I've read this question, which seems to describe a similar issue but I don't have any of the general purpose bitfield bits set so it's not the third bit issue and I'm pretty sure I have valid crc-32's because when I don't, WinRAR throws a fit.
I'm happy to post some code or a link to a "bad" zip file if it would help but I'm pretty much just using ZipStream, forcing it into "large file mode" and using "store" as the compression method.
Edit - I've tried the "deflate" compression algorithm as well and get the same results so I don't think it's the "store". It's also worth pointing out that I'm pulling down the files one a time from a storage server and sending them out as they arrive so a solution that requires all the files to be downloaded before sending anything isn't going to be viable (extreme example is 5GB+ of 20MB files. User can't wait for all 5GB to transfer to zipping server before their download starts or they'll think it's broken)
Here's a 140 byte, "store" compressed, test zip file that exhibits this behavior: http://teknocowboys.com/test.zip
The problem was in the "version needed to extract" field, which I found by doing a hex diff on a file created by ZipStream vs a file created by Info-zip and going through the differences, trying to resolve them.
ZipStream by default sets it to 0x0603. Info-zip sets it to 0x000A. Zip files with the former value don't seem to open in Archive Utility. Perhaps it doesn't support the features at that version?
Forcing the "version needed to extract" to 0x000A made the generated files open as well in Archive Utility as they do everywhere else.
Edit: Another cause of this issue is if the zip file was downloaded using Safari (user agent version >= 537) and you under-reported the file size when you sent out your Content-Length header.
The solution we employ is to detect Safari >= 537 server side and if that's what you're using, we determine the difference between the Content-Length size and the actual size (how you do this depends on your specific application) and after calling $zipStream->finish(), we echo chr(0) to reach the correct length. The resulting file is technically malformed and any comment you put in the zip won't be displayed, but all zip programs will be able to open it and extract the files.
IE requires the same hack if you're misreporting your Content-Length but instead of downloading a file that doesn't work, it just won't finish downloading and throws a "download interrupted".
use ob_clean(); and flush();
Example :
$file = __UPLOAD_PATH . $projectname . '/' . $fileName;
$zipname = "watherver.zip"
$zip = new ZipArchive();
$zip_full_path_name = __UPLOAD_PATH . $projectname . '/' . $zipname;
$zip->open($zip_full_path_name, ZIPARCHIVE::CREATE);
$zip->addFile($file); // Adding one file for testing
$zip->close();
if(file_exists($zip_full_path_name)){
header('Content-type: application/zip');
header('Content-Disposition: attachment; filename="'.$zipname.'"');
ob_clean();
flush();
readfile($zip_full_path_name);
unlink($zip_full_path_name);
}
I've had this exact issue but with a different cause.
In my case the php generated zip would open from the command line, but not via finder in OSX.
I had made the mistake of allowing some HTML content into the output buffer prior to creating the zip file and sending that back as the response.
<some html></....>
<?php
// Output a zip file...
The command line unzip program was evidently tolerant of this but the Mac unarchive function was not.
No idea. If the external ZipString class doesn't work, try another option. The PHP ZipArchive extension won't help you, since it doesn't support streaming but only ever writes to files.
But you could try the standard Info-zip utility. It can be invoked from within PHP like this:
#header("Content-Type: archive/zip");
passthru("zip -0 -q -r - *.*");
That would lead to an uncompressed zip file directly send back to the client.
If that doesn't help, then the MacOS zip frontend probably doesn't like uncompressed stuff. Remove the -0 flag then.
The InfoZip commandline tool I'm using, both on Windows and Linux, uses version 20 for the zip's "version needed to extract" field. This is needed on PHP as well, as the default compression is the Deflate algorithm. Thus the "version needed to extract" field should really be 0x0014. If you alter the "(6 << 8) +3" code in the referenced ZipStream class to just "20", you should get a valid Zip file across platforms.
The author is basically telling you that the zip file was created in OS/2 using the HPFS file system, and the Zip version needed predates InfoZip 1.0. Not many implementations know what to do about that one any longer ;)
For those using ZipStream in Symfony, here's your solution: https://stackoverflow.com/a/44706446/136151
use Symfony\Component\HttpFoundation\StreamedResponse;
use Aws\S3\S3Client;
use ZipStream;
//...
/**
* #Route("/zipstream", name="zipstream")
*/
public function zipStreamAction()
{
//test file on s3
$s3keys = array(
"ziptestfolder/file1.txt"
);
$s3Client = $this->get('app.amazon.s3'); //s3client service
$s3Client->registerStreamWrapper(); //required
$response = new StreamedResponse(function() use($s3keys, $s3Client)
{
// Define suitable options for ZipStream Archive.
$opt = array(
'comment' => 'test zip file.',
'content_type' => 'application/octet-stream'
);
//initialise zipstream with output zip filename and options.
$zip = new ZipStream\ZipStream('test.zip', $opt);
//loop keys useful for multiple files
foreach ($s3keys as $key) {
// Get the file name in S3 key so we can save it to the zip
//file using the same name.
$fileName = basename($key);
//concatenate s3path.
$bucket = 'bucketname';
$s3path = "s3://" . $bucket . "/" . $key;
//addFileFromStream
if ($streamRead = fopen($s3path, 'r')) {
$zip->addFileFromStream($fileName, $streamRead);
} else {
die('Could not open stream for reading');
}
}
$zip->finish();
});
return $response;
}
If your controller action response is not a StreamedResponse, you are likely going to get a corrupted zip containing html as I found out.
It's an old question but I leave what it worked for me just in case it helps someone else.
When setting the options you need set Zero header to true and enable zip 64 to false (this will limit the archive to archive to 4 Gb though):
$options->setZeroHeader(true);
$opt->setEnableZip64(false)
Everything else as described by Forer.
Solution found on https://github.com/maennchen/ZipStream-PHP/issues/71