Here's a very stripped down version of the code I'm using.
$url = "http://server.com/getDaFile";
//Get the file from the server
$zip_file_contents = file_get_contents($url);
//Write file to disk
file_put_contents("file.zip", $zip_file_contents);
//open zip file
$zip = zip_open("file.zip");
if(is_resource($zip))
{
while($zip_entry = zip_read($zip))
{
if(zip_entry_open($zip, $zip_entry, 'r'))
{
//Read the whole file
$buf = zip_entry_read($zip_entry, zip_entry_filesize($zip_entry));
/*
Do stuff with $buf!!!
*/
zip_entry_close($zip_entry);
}
}
zip_close($zip);
}
else
{
echo "Not a resource. Oh noes!\n";
}
So : get the file, save it to disk, unzip it to extract files it contains, do stuff with files. The problem here is that, for some reason I cannot figure out, zip_read returns FALSE, as if it couldn't read files inside the ZIP archive. $zip does contain a resource, I've checked with var_dump.
What makes this even stranger is that I downloaded the ZIP file on my PC using the URL on top, manually uploaded it to the PHP server, and commented out the calls to file_get_contents and file_put_contents so PHP uses the local version. When I do this, zip_read correctly finds the right amount of files inside the ZIP and processing proceeds as it should.
I also tried doing this : $zip = zip_open($url) but $zip fails the is_resource($zip) check.
Something is obviously wrong with my code since the URL works and returns a valid ZIP archive. What is it?
So I finally found out the problem. Following #diolemo's suggestion, I opened my server's ZIP archive in a hex editor. Here's what I found at the top, followed by the usual ZIP binary data : http://pastebin.com/vQEXJtTN
It turns out there was a PHP error mixed in with the actual content of the ZIP file. Unsure of how to fix this (but knowing it certainly had to do with HTTP headers), I tried this guy's code and, what do you know, my code works perfectly now!
Lessons learned? Never trust your data, even if it seems all right (both 7-Zip and Winrar managed to open the file without problem).
Related
I am writing a scanner that will look for possibly hacked/malware files. One requirement is to check if a zip (or any compressed) file is password-protected using some PHP function.
I don't want to add any extra software requirements, so should work on multiple servers, using PHP 5.3+ . (Yes I know that 5.3 is old, but the process may need to run on older PHP installations.) If this detection is available in newer PHP versions, then I could have code that would run only on newer PHP version.
I can use the file_get_contents() function to read the file's contents into a string. How do I check that string for an indication that the zip file is password-protected? Note that I don't want to uncompress the file, just check it for password-protection.
Thanks.
This code appears to work, but might be improved.
The process seems to involve two steps:
use zip_open to open the file, returning a resource. No resource, zip couldn't be opened, so it might be passworded
use zip_read to read the files inside the zip. If fails, then might be passworded
In either of those two cases, return true, indicating probable password on the zip file.
// try to open a zip file; if it fails, probably password-protected
function check_zip_password($zip_file = '') {
/*
open/read a zip file
return true if passworded
*/
if (!$zip_file) { // file not specified
return false;
}
$zip = zip_open($zip_file); // open the file
if (is_resource($zip)) { // file opened OK
$zipfile = zip_read($zip); // try read of zip file contents
if (!$zipfile) { // couldn't read inside, so passworded
return true;
}
else
{ // file opened and read, so not passworded
return false;
}
} else { // couldn't open the file, might be passworded
return true;
}
return false; // file exists, but not password protected
}
Note that the code only determines that the files inside the zip can't be accessed, so they are probably password-protected. The code doesn't try to do any processing of files inside the zip.
I have written the PHP script to generate the zip file. it's working fine when I use rar software to extract it but not getting extract with rar software. I can't ask to users to install rar software to extract downloaded zip file.
I don't know where i am commiting mistakes.
Here i attached error screen shot which i get when try to open zip file.
// Here is code snippet
$obj->create_zip($files_to_zip, $dir . '/download.zip');
// Code for create_zip function
//create the archive
$zip = new ZipArchive();
if ($zip->open($destination, $overwrite ? ZIPARCHIVE::OVERWRITE : ZIPARCHIVE::CREATE) !== true) {
return false;
}
//add the files
foreach ($valid_files as $file) {
$filearr = explode('/', $file);
$zip->addFile($file, end($filearr));
}
$zip->close();
If $valid_files is a glob'd array, use basename() instead of end(), your zip might not actually have added any files causing for it to be an invalid zip (however that would be visible in the size of the zip file).
Also try winrar/winzip/7zip and see what they return, microsoft's internal zip engine might not be up to date enough to open the zips.
I have also encountered this problem, using 7z solved the problem but we need to send the zip to somebody else so 7z is a nono.
I found that, in my case it is that the file path is too long:
When I use this:
$zip->addFile($files_path.'/people.txt');
And it generated a zip folder nested very deep e.g. ["/tmp/something/something1/something2/people.txt"]
So I need to use this instead
$zip->addFile($files_path.'/people.txt', 'people.txt');
Which generate a a zip folder with only 1 layer ["people.txt"], and Windows Zip read perfectly~
Hope this helps somebody that also have this problem!
I am writing a script that will go through all my .js files and minify them into one .php file to be included on the site. I just run this script after I have edited some js and want to upload it to the live site.
The issue: I can not load the content of jquery-2.1.4.min.js using file_get_contents. I have tried changing the name of the file to jquery.js and that did not help. I do not have any complex javascript in the other files (just random strings) but they open fine.
With the code:
if (!file_get_contents($filename)) {
die ("dammit");
}
I get the response of "dammit". All other files are fine though, so I know the file name and path are correct. One of the weird things is that there are no errors coming up (I have used error_reporting (-1); to make sure they will).
Is anyone else able to get the file contents of jquery? Any ideas what would cause this and if it will be a problem with other javascript or css?
As requested, here is the full code:
$buffer = $jsStartBuffer;
//get a list of files in the folder (only .js files)
$fileArray = array();
if (is_dir($jsMakeFile["SourcePath"])){
if ($dh = opendir($jsMakeFile["SourcePath"])){
while (($file = readdir($dh)) !== false){
$file_parts = pathinfo($jsMakeFile["SourcePath"].$file);
if ($file_parts['extension'] == "js") {
$fileArray[] = $file;
}
}
}
}
print_r($fileArray);
foreach ($fileArray as $nextRawFile) {
$buffer .= file_get_contents($jsMakeFile["SourcePath"].$nextRawFile);
if (!file_get_contents($jsMakeFile["SourcePath"].$nextRawFile)) {
die ("dammit");
}
echo $jsMakeFile["SourcePath"].$nextRawFile;
}
$buffer .= $jsEndBuffer;
echo $buffer;
$buffer = \JShrink\Minifier::minify($buffer);
file_put_contents($jsMakeFile["finalFile"]["path"].$jsMakeFile["finalFile"]["name"], $buffer);
When I put other .js files in there it is fine (I even tried lightbox.min.js and it worked fine!) I have tried a few different versions of jquery.min and they all seem to fail.
OK, solution found. It is to do with the actual file created by jquery.
The way I solved it was:
- Go to the query site, and instead of downloading the required file, open it in a new tab/window
- Copy all the content in this window
- Create a new file where required and name as required
- Paste the content into this file and save it
This new file will now be able to be read by file_get_contents. I would imagine this solution would help if you are trying to work with jquery (and other) files in php in any way and having issues.
I have a processing server with my database and a serving database to serve up files with a low bandwidth cost. On the processing server, php is not able to create files so everything must be done with streams and/or stay in memory before being sent over to another server for download. A few days ago I found out about the stream abstraction with 'php://memory' and that I can do something like
$fp=fopen('php://memory','w+');
fwrite($fp,"Hello world");
fseek($fp,0,SEEK_SET);
//make a ftp connection here with $conn_id
$upload = ftp_fput($conn_id,"targetpath/helloworld.txt",$fp,FTP_BINARY);
to make the file in memory and then allow me to ftp it over to my other server. This is exactly what I want, except I also want to zip the data before sending it -- preferably using only native parts of php like ziparchive and not additional custom classes for special stream manipulation. I know that I am very close with the following...
$zip = new ZipArchive();
if($zip->open('php://memory', ZIPARCHIVE::CREATE)) {
$zip->addFromString('testtext.txt','Hello World!');
$fp = $zip->getStream('test'); if(!$fp) print "no filepointer";
//make a ftp connection here with $conn_id
$upload = ftp_fput($conn_id,"targetpath/helloworld.zip",$fp,FTP_BINARY);
} else print "couldn't open a zip like that";
The point at which this fails is the call to getStream (which always returns false although I think I am using correctly). It appears that the zip is fine making the file in 'php://memory' but for some reason getStream still fails although perhaps I don't sufficiently understand how ZipArchive makes zips...
How can I go from the string to the zipped filepointer so that I can ftp the zip over to my other server? Remember I can't make any files or else I would just make the zip file then ftp it over.
EDIT: based on skinnynerd's suggestions below I tried the following
$zip = new ZipArchive();
if($zip->open('php://memory', ZIPARCHIVE::CREATE)) {
$zip->addFromString('testtext.txt','Hello World!');
$zip->close();
$fp = fopen('php://memory','r+');
fseek($fp,0,SEEK_SET);
//connect to ftp
$upload = ftp_fput($conn_id,"upload/transfer/helloworld.zip",$fp,FTP_BINARY);
}
This does make a zip and send it over but the zip is 0 bytes large so I don't think that 'php://memory' works the way I thought... it actually fails at the close step -- the $zip->close() returns false which makes me wonder if I can open zips into 'php://memory' at all. Does anyone know what I can try along these line to get the zip?
$zip->getStream('test') is getting a stream to extract the file 'test' from the archive. Since there's no file 'test' in the archive, this fails. This is not the function you want to use.
As you said, what you want to do is send the finished archive to the ftp server. In this case, you would want to close the zip archive, and then reopen php://memory as a normal file (using fopen) to send it.
I don't know, but you may also be able to use $zip as a resource directly, without having to close and reopen the file.
And I think you can try create a stream pipe directly from ftp server
<?php
$zip = new ZipArchive();
if($zip->open('ftp://user:password#ftp.host.com/upload/transfer/helloworld.zip', ZipArchive::CREATE))
{
$zip->addFromString('testtext.txt','Hello World!');
$zip->close();
}
else
print "couldn't open zip file on remote ftp host.";
Does it have to be a Zip archive? Since you're trying to save bandwith it could be a gzip too.
<?php
$ftp_credentials = "ftp://USER:PASSWORD#HOST/helloworld.gz";
$gz = gzencode("Hello World!", 9);
$options = array('ftp' => array('overwrite' => true));
$stream_context = stream_context_create($options);
file_put_contents($ftp_credentials, $gz, 0, $stream_context);
?>
So, I'm writing a chunked file transfer script that is intended to copy files--small and large--to a remote server. It almost works fantastically (and did with a 26 byte file I tested, haha) but when I start to do larger files, I notice it isn't quite working. For example, I uploaded a 96,489,231 byte file, but the final file was 95,504,152 bytes. I tested it with a 928,670,754 byte file, and the copied file only had 927,902,792 bytes.
Has anyone else ever experienced this? I'm guessing feof() may be doing something wonky, but I have no idea how to replace it, or test that. I commented the code, for your convenience. :)
<?php
// FTP credentials
$server = CENSORED;
$username = CENSORED;
$password = CENSORED;
// Destination file (where the copied file should go)
$destination = "ftp://$username:$password#$server/ftp/final.mp4";
// The file on my server that we're copying (in chunks) to $destination.
$read = 'grr.mp4';
// If the file we're trying to copy exists...
if (file_exists($read))
{
// Set a chunk size
$chunk_size = 4194304;
// For reading through the file we want to copy to the FTP server.
$read_handle = fopen($read, 'rb');
// For appending to the destination file.
$destination_handle = fopen($destination, 'ab');
echo '<span style="font-size:20px;">';
echo 'Uploading.....';
// Loop through $read until we reach the end of the file.
while (!feof($read_handle))
{
// So Rackspace doesn't think nothing's happening.
echo PHP_EOL;
flush();
// Read a chunk of the file we're copying.
$chunk = fread($read_handle, $chunk_size);
// Write the chunk to the destination file.
fwrite($destination_handle, $chunk);
sleep(1);
}
echo 'Done!';
echo '</span>';
}
fclose($read_handle);
fclose($destination_handle);
?>
EDIT
I (may have) confirmed that the script is dying at the end somehow, and not corrupting the files. I created a simple file with each line corresponding to the line number, up to 10000, then ran my script. It stopped at line 6253. However, the script is still returning "Done!" at the end, so I can't imagine it's a timeout issue. Strange!
EDIT 2
I have confirmed that the problem exists somewhere in fwrite(). By echoing $chunk inside the loop, the complete file is returned without fail. However, the written file still does not match.
EDIT 3
It appears to work if I add sleep(1) immediately after the fwrite(). However, that makes the script take a million years to run. Is it possible that PHP's append has some inherent flaw?
EDIT 4
Alright, further isolated the problem to being an FTP problem, somehow. When I run this file copy locally, it works fine. However, when I use the file transfer protocol (line 9) the bytes are missing. This is occurring despite the binary flags the two cases of fopen(). What could possibly be causing this?
EDIT 5
I found a fix. The modified code is above--I'll post an answer on my own as soon as I'm able.
I found a fix, though I'm not sure exactly why it works. Simply sleeping after writing each chunk fixes the problem. I upped the chunk size quite a bit to speed things up. Though this is an arguably bad solution, it should work for my uses. Thanks anyway, guys!