I use the PHP zip:// stream wrapper to parse large XML files line by line. For example:
$stream_uri = 'zip://' . __DIR__ . '/archive.zip#foo.xml';
$reader = new XMLReader();
$reader->open( $stream_uri, null );
$reader->read();
while ( true ) {
echo( $reader->readInnerXml() . PHP_EOL );
if ( ! $reader->next() ) {
break;
}
}
Quite often an XML file will include dodgy UTF control characters XMLReader doesn't like. So I'd like to implement a custom stream wrapper I can pass the output of the zip:// stream to, which will run a preg_replace on each line to remove those characters.
My dream is to be able to do this:
stream_wrapper_register( 'xmlchars', 'XML_Chars' );
$stream_uri = 'xmlchars://zip://' . __DIR__ . '/archive.zip#foo.xml';
and have XMLReader happily read the tidied-up nodes. I've figured out a way to reconstruct the zip stream URI based on the path passed to my wrapper:
class XML_Chars {
protected $stream_uri = '';
protected $handle;
function stream_open( $path, $mode, $options, &$opened_path ) {
$parsed_url = parse_url( $path );
$this->stream_uri = 'zip:' . $parsed_url['path'] . '#' . $parsed_url['fragment'];
return true;
}
}
But I'm puzzled about the best way to open the zip:// stream so I can modify its output and pass the result through to the XMLReader. Can anyone give me any pointers about how to implement that?
In case useful to anybody else, I've found a different way to solve the problem: a stream filter. You define it like this:
class UTF_Character_Filter extends php_user_filter {
public function filter( $in, $out, &$consumed, $closing ) {
while ( $bucket = stream_bucket_make_writeable( $in ) ) {
$consumed += $bucket->datalen;
// Remove characters in the hex range 0 - 8, B and C, E to 1F
// i.e. all control characters except newline, tab and return
$bucket->data = preg_replace( '|[\x0-\x8\xB-\xC\xE-\x1F]|ms', '', $bucket->data );
stream_bucket_append( $out, $bucket );
}
return PSFS_PASS_ON;
}
}
stream_filter_register( 'utf_character_filter', 'UTF_Character_Filter' );
And use it like this:
php://filter/read=utf_character_filter/resource=zip://archive.zip#import.xml
I'd still be interested to know if anyone's figured out how to make a stream wrapper that can accept the input of another stream wrapper though, as it could be a handy tool.
I am trying to zip a bunch of PDFs together and download in the browser, at the moment the PDF files are zipped and downloaded to the folder the PDFs are stored in, not via the users browser and into their download folder, I have a similar (and much simpler) function which downloads a single PDF so feel like I'm missing something fairly obvious here..
$id is a comma seperated list of filenames, this is then split into an array for looping through and adding to zip file. this bit works, thinking it may be a header issue or with the response.
Any help much appreciated.
public function downloadMultiple($id) {
$id_array = explode(',', $id);
$public_dir = storage_path();
$zipFileName = time().'.zip';
$zip = new ZipArchive;
if ($zip->open($public_dir . '/' . $zipFileName, ZipArchive::CREATE) === TRUE) {
foreach($id_array as $file) {
$file_path = storage_path($file).".pdf";
if (file_exists($file_path)) {
$zip->addFile($file_path,$file.".pdf");
}
}
if ($zip->close()) {
$filetopath = $public_dir.'/'.$zipFileName;
$headers = [
'Cache-control: maxage=1',
'Pragma: no-cache',
'Expires: 0',
'Content-Type : application/octet-stream',
'Content-Transfer-Encoding: binary',
'Content-Type: application/force-download',
'Content-Disposition: attachment; filename='.time().'.zip',
"Content-length: " . filesize($filetopath)
];
if (file_exists($filetopath)) {
$response = response()->download($filetopath, $zipFileName, $headers);
//$response->deleteFileAfterSend(true);
} else {
return ['status'=>'zip file does not exist'];
}
} else {
return ['status'=>'zip file could not close'];
}
} else {
return ['status'=>'Could not create new zip'];
}
}
Update:
Definitely gets to the return and does create the file, it just doesn't seem to download for the user, the below is what is brought back in the inspector so clearly something not working as expected
may be worth while mentioning the code which is sent to the controller
let xhr = new XMLHttpRequest(), self = this;
xhr.open('GET', window.location.origin+'/download-multiple/' + this.selected);
xhr.onload = function () {
};
xhr.send();
Assuming you're getting to that part of the controller method, I believe the problem is that you're not returning your response:
if (file_exists($filetopath)) {
// $response = response()->download($filetopath, $zipFileName, $headers);
// $response->deleteFileAfterSend(true);
return response()->download($filetopath, $zipFileName, $headers)->deleteFileAfterSend(true);
} else {
return ['status'=>'zip file does not exist'];
}
EDIT: The problem is you're trying to load the file via AJAX, which you can't do the way that you're trying to do it (see here for examples on how to do it). Change your javascript to:
let xhr = new XMLHttpRequest(), self = this;
window.location = window.location.origin+'/download-multiple/' + this.selected
I get my file via:
require_once 'google/appengine/api/cloud_storage/CloudStorageTools.php';
use google\appengine\api\cloud_storage\CloudStorageTools;
$public_link = CloudStorageTools::getPublicUrl("gs://bucket/file.pdf", false);
If I go to $public_link in the browser, it shows the PDF inside the browser. I am trying to figure out how I can force the download of this file.
Google App Engine only has a 60 second timeout so I'm afraid the serve function wont work via GAE. Does anyone have any suggestions?
--
EDIT
Andrei Volga's previous answer in this post suggests I use a Signed URL with a response-content-distribution header.
So far, I am able to create a signed URL that successfully shows the file but I am not able to generate a signed url that has any sort of header at all aka create a signed URL that will force the download instead of just showing it.
This is what I have so far, most of which is courtesy of mloureiro.
function googleBuildConfigurationString($method, $expiration, $file, array $options = [])
{
$allowedMethods = ['GET', 'HEAD', 'PUT', 'DELETE'];
// initialize
$method = strtoupper($method);
$contentType = $options['Content_Type'];
$contentMd5 = $options['Content_MD5'] ? base64_encode($options['Content_MD5']) : '';
$headers = $options['Canonicalized_Extension_Headers'] ? $options['Canonicalized_Extension_Headers'] . PHP_EOL : '';
$file = $file ? $file : $options['Canonicalized_Resource'];
// validate
if(array_search($method, $allowedMethods) === false)
{
throw new RuntimeException("Method '{$method}' is not allowed");
}
if(!$expiration)
{
throw new RuntimeException("An expiration date should be provided.");
}
return <<<TXT
{$method}
{$contentMd5}
{$contentType}
{$expiration}
{$headers}{$file}
TXT;
}
function googleSignString($p12FilePath, $string)
{
$certs = [];
if (!openssl_pkcs12_read(file_get_contents($p12FilePath), $certs, 'notasecret'))
{
echo "Unable to parse the p12 file. OpenSSL error: " . openssl_error_string(); exit();
}
$RSAPrivateKey = openssl_pkey_get_private($certs["pkey"]);
$signed = '';
if(!openssl_sign( $string, $signed, $RSAPrivateKey, 'sha256' ))
{
error_log( 'openssl_sign failed!' );
$signed = 'failed';
}
else $signed = base64_encode($signed);
return $signed;
}
function googleBuildSignedUrl($serviceEmail, $file, $expiration, $signature)
{
return "http://storage.googleapis.com{$file}" . "?GoogleAccessId={$serviceEmail}" . "&Expires={$expiration}" . "&Signature=" . urlencode($signature);
}
$serviceEmail = '<EMAIL>';
$p12FilePath = '../../path/to/cert.p12';
$expiration = (new DateTime())->modify('+3hours')->getTimestamp();
$bucket = 'bucket';
$fileToGet = 'picture.jpg';
$file = "/{$bucket}/{$fileToGet}";
$string = googleBuildConfigurationString('GET', $expiration, $file, array("Canonicalized_Extension_Headers" => ''));
$signedString = googleSignString($p12FilePath, $string);
$signedUrl = googleBuildSignedUrl($serviceEmail, $file, $expiration, $signedString);
echo $signedUrl;
For small files you can use serve option instead of public URL with save-as option set to true. See documentation.
For large files you can use a Signed URL with response-content-disposition parameter.
You can add and additional query string only.
https://cloud.google.com/storage/docs/xml-api/reference-headers#responsecontentdisposition
response-content-disposition
A query string parameter that allows content-disposition to be overridden for authenticated GET requests.
Valid Values URL-encoded header to return instead of the content-disposition of the underlying object.
Example
?response-content-disposition=attachment%3B%20filename%3D%22foo%22
I'm writing a RESTful API. I'm having trouble with uploading images using the different verbs.
Consider:
I have an object which can be created/modified/deleted/viewed via a post/put/delete/get request to a URL. The request is multi part form when there is a file to upload, or application/xml when there's just text to process.
To handle the image uploads which are associated with the object I am doing something like:
if(isset($_FILES['userfile'])) {
$data = $this->image_model->upload_image();
if($data['error']){
$this->response(array('error' => $error['error']));
}
$xml_data = (array)simplexml_load_string( urldecode($_POST['xml']) );
$object = (array)$xml_data['object'];
} else {
$object = $this->body('object');
}
The major problem here is when trying to handle a put request, obviously $_POST doesn't contain the put data (as far as I can tell!).
For reference this is how I'm building the requests:
curl -F userfile=#./image.png -F xml="<xml><object>stuff to edit</object></xml>"
http://example.com/object -X PUT
Does anyone have any ideas how I can access the xml variable in my PUT request?
First of all, $_FILES is not populated when handling PUT requests. It is only populated by PHP when handling POST requests.
You need to parse it manually. That goes for "regular" fields as well:
// Fetch content and determine boundary
$raw_data = file_get_contents('php://input');
$boundary = substr($raw_data, 0, strpos($raw_data, "\r\n"));
// Fetch each part
$parts = array_slice(explode($boundary, $raw_data), 1);
$data = array();
foreach ($parts as $part) {
// If this is the last part, break
if ($part == "--\r\n") break;
// Separate content from headers
$part = ltrim($part, "\r\n");
list($raw_headers, $body) = explode("\r\n\r\n", $part, 2);
// Parse the headers list
$raw_headers = explode("\r\n", $raw_headers);
$headers = array();
foreach ($raw_headers as $header) {
list($name, $value) = explode(':', $header);
$headers[strtolower($name)] = ltrim($value, ' ');
}
// Parse the Content-Disposition to get the field name, etc.
if (isset($headers['content-disposition'])) {
$filename = null;
preg_match(
'/^(.+); *name="([^"]+)"(; *filename="([^"]+)")?/',
$headers['content-disposition'],
$matches
);
list(, $type, $name) = $matches;
isset($matches[4]) and $filename = $matches[4];
// handle your fields here
switch ($name) {
// this is a file upload
case 'userfile':
file_put_contents($filename, $body);
break;
// default for all other files is to populate $data
default:
$data[$name] = substr($body, 0, strlen($body) - 2);
break;
}
}
}
At each iteration, the $data array will be populated with your parameters, and the $headers array will be populated with the headers for each part (e.g.: Content-Type, etc.), and $filename will contain the original filename, if supplied in the request and is applicable to the field.
Take note the above will only work for multipart content types. Make sure to check the request Content-Type header before using the above to parse the body.
Please don't delete this again, it's helpful to a majority of people coming here! All previous answers were partial answers that don't cover the solution as a majority of people asking this question would want.
This takes what has been said above and additionally handles multiple file uploads and places them in $_FILES as someone would expect. To get this to work, you have to add 'Script PUT /put.php' to your Virtual Host for the project per Documentation. I also suspect I'll have to setup a cron to cleanup any '.tmp' files.
private function _parsePut( )
{
global $_PUT;
/* PUT data comes in on the stdin stream */
$putdata = fopen("php://input", "r");
/* Open a file for writing */
// $fp = fopen("myputfile.ext", "w");
$raw_data = '';
/* Read the data 1 KB at a time
and write to the file */
while ($chunk = fread($putdata, 1024))
$raw_data .= $chunk;
/* Close the streams */
fclose($putdata);
// Fetch content and determine boundary
$boundary = substr($raw_data, 0, strpos($raw_data, "\r\n"));
if(empty($boundary)){
parse_str($raw_data,$data);
$GLOBALS[ '_PUT' ] = $data;
return;
}
// Fetch each part
$parts = array_slice(explode($boundary, $raw_data), 1);
$data = array();
foreach ($parts as $part) {
// If this is the last part, break
if ($part == "--\r\n") break;
// Separate content from headers
$part = ltrim($part, "\r\n");
list($raw_headers, $body) = explode("\r\n\r\n", $part, 2);
// Parse the headers list
$raw_headers = explode("\r\n", $raw_headers);
$headers = array();
foreach ($raw_headers as $header) {
list($name, $value) = explode(':', $header);
$headers[strtolower($name)] = ltrim($value, ' ');
}
// Parse the Content-Disposition to get the field name, etc.
if (isset($headers['content-disposition'])) {
$filename = null;
$tmp_name = null;
preg_match(
'/^(.+); *name="([^"]+)"(; *filename="([^"]+)")?/',
$headers['content-disposition'],
$matches
);
list(, $type, $name) = $matches;
//Parse File
if( isset($matches[4]) )
{
//if labeled the same as previous, skip
if( isset( $_FILES[ $matches[ 2 ] ] ) )
{
continue;
}
//get filename
$filename = $matches[4];
//get tmp name
$filename_parts = pathinfo( $filename );
$tmp_name = tempnam( ini_get('upload_tmp_dir'), $filename_parts['filename']);
//populate $_FILES with information, size may be off in multibyte situation
$_FILES[ $matches[ 2 ] ] = array(
'error'=>0,
'name'=>$filename,
'tmp_name'=>$tmp_name,
'size'=>strlen( $body ),
'type'=>$value
);
//place in temporary directory
file_put_contents($tmp_name, $body);
}
//Parse Field
else
{
$data[$name] = substr($body, 0, strlen($body) - 2);
}
}
}
$GLOBALS[ '_PUT' ] = $data;
return;
}
For whom using Apiato (Laravel) framework:
create new Middleware like file below, then declair this file in your laravel kernel file within the protected $middlewareGroups variable (inside web or api, whatever you want) like this:
protected $middlewareGroups = [
'web' => [],
'api' => [HandlePutFormData::class],
];
<?php
namespace App\Ship\Middlewares\Http;
use Closure;
use Symfony\Component\HttpFoundation\ParameterBag;
/**
* #author Quang Pham
*/
class HandlePutFormData
{
/**
* Handle an incoming request.
*
* #param \Illuminate\Http\Request $request
* #param \Closure $next
*
* #return mixed
*/
public function handle($request, Closure $next)
{
if ($request->method() == 'POST' or $request->method() == 'GET') {
return $next($request);
}
if (preg_match('/multipart\/form-data/', $request->headers->get('Content-Type')) or
preg_match('/multipart\/form-data/', $request->headers->get('content-type'))) {
$parameters = $this->decode();
$request->merge($parameters['inputs']);
$request->files->add($parameters['files']);
}
return $next($request);
}
public function decode()
{
$files = [];
$data = [];
// Fetch content and determine boundary
$rawData = file_get_contents('php://input');
$boundary = substr($rawData, 0, strpos($rawData, "\r\n"));
// Fetch and process each part
$parts = $rawData ? array_slice(explode($boundary, $rawData), 1) : [];
foreach ($parts as $part) {
// If this is the last part, break
if ($part == "--\r\n") {
break;
}
// Separate content from headers
$part = ltrim($part, "\r\n");
list($rawHeaders, $content) = explode("\r\n\r\n", $part, 2);
$content = substr($content, 0, strlen($content) - 2);
// Parse the headers list
$rawHeaders = explode("\r\n", $rawHeaders);
$headers = array();
foreach ($rawHeaders as $header) {
list($name, $value) = explode(':', $header);
$headers[strtolower($name)] = ltrim($value, ' ');
}
// Parse the Content-Disposition to get the field name, etc.
if (isset($headers['content-disposition'])) {
$filename = null;
preg_match(
'/^form-data; *name="([^"]+)"(; *filename="([^"]+)")?/',
$headers['content-disposition'],
$matches
);
$fieldName = $matches[1];
$fileName = (isset($matches[3]) ? $matches[3] : null);
// If we have a file, save it. Otherwise, save the data.
if ($fileName !== null) {
$localFileName = tempnam(sys_get_temp_dir(), 'sfy');
file_put_contents($localFileName, $content);
$files = $this->transformData($files, $fieldName, [
'name' => $fileName,
'type' => $headers['content-type'],
'tmp_name' => $localFileName,
'error' => 0,
'size' => filesize($localFileName)
]);
// register a shutdown function to cleanup the temporary file
register_shutdown_function(function () use ($localFileName) {
unlink($localFileName);
});
} else {
$data = $this->transformData($data, $fieldName, $content);
}
}
}
$fields = new ParameterBag($data);
return ["inputs" => $fields->all(), "files" => $files];
}
private function transformData($data, $name, $value)
{
$isArray = strpos($name, '[]');
if ($isArray && (($isArray + 2) == strlen($name))) {
$name = str_replace('[]', '', $name);
$data[$name][]= $value;
} else {
$data[$name] = $value;
}
return $data;
}
}
Pls note: Those codes above not all mine, some from above comment, some modified by me.
Quoting netcoder reply : "Take note the above will only work for multipart content types"
To work with any content type I have added the following lines to Mr. netcoder's solution :
// Fetch content and determine boundary
$raw_data = file_get_contents('php://input');
$boundary = substr($raw_data, 0, strpos($raw_data, "\r\n"));
/*...... My edit --------- */
if(empty($boundary)){
parse_str($raw_data,$data);
return $data;
}
/* ........... My edit ends ......... */
// Fetch each part
$parts = array_slice(explode($boundary, $raw_data), 1);
$data = array();
............
...............
I've been trying to figure out how to work with this issue without having to break RESTful convention and boy howdie, what a rabbit hole, let me tell you.
I'm adding this anywhere I can find in the hope that it will help somebody out in the future.
I've just lost a day of development firstly figuring out that this was an issue, then figuring out where the issue lay.
As mentioned, this isn't a symfony (or laravel, or any other framework) issue, it's a limitation of PHP.
After trawling through a good few RFCs for php core, the core development team seem somewhat resistant to implementing anything to do with modernising the handling of HTTP requests. The issue was first reported in 2011, it doesn't look any closer to having a native solution.
That said, I managed to find this PECL extension called Always Populate Form Data. I'm not really very familiar with pecl, and couldn't seem to get it working using pear. but I'm using CentOS and Remi PHP which has a yum package.
I ran yum install php-pecl-apfd and it literally fixed the issue straight away (well I had to restart my docker containers but that was a given).
I believe there are other packages in various flavours of linux and I'm sure anybody with more knowledge of pear/pecl/general php extensions could get it running on windows or mac with no issue.
I know this article is old.
But unfortunately, PHP still does not pay attention to form-data other than the Post method.
Thanks to friends (#netcoder, #greendot, #pham-quang) who suggested solutions above.
Using those solutions I wrote a library for this purpose:
composer require alireaza/php-form-data
You can also use composer require alireaza/laravel-form-data in Laravel.
$httpsock = #socket_create_listen("9090");
if (!$httpsock) {
print "Socket creation failed!\n";
exit;
}
while (1) {
$client = socket_accept($httpsock);
$input = trim(socket_read ($client, 4096));
$input = explode(" ", $input);
$input = $input[1];
$fileinfo = pathinfo($input);
switch ($fileinfo['extension']) {
default:
$mime = "text/html";
}
if ($input == "/") {
$input = "index.html";
}
$input = ".$input";
if (file_exists($input) && is_readable($input)) {
echo "Serving $input\n";
$contents = file_get_contents($input);
$output = "HTTP/1.0 200 OK\r\nServer: APatchyServer\r\nConnection: close\r\nContent-Type: $mime\r\n\r\n$contents";
} else {
//$contents = "The file you requested doesn't exist. Sorry!";
//$output = "HTTP/1.0 404 OBJECT NOT FOUND\r\nServer: BabyHTTP\r\nConnection: close\r\nContent-Type: text/html\r\n\r\n$contents";
function openfile()
{
$filename = "a.pl";
$file = fopen($filename, 'r');
$filesize = filesize($filename);
$buffer = fread($file, $filesize);
$array = array("Output"=>$buffer,"filesize"=>$filesize,"filename"=>$filename);
return $array;
}
$send = openfile();
$file = $send['filename'];
$filesize = $send['filesize'];
$output = 'HTTP/1.0 200 OK\r\n';
$output .= "Content-type: application/octet-stream\r\n";
$output .= 'Content-Disposition: attachment; filename="'.$file.'"\r\n';
$output .= "Content-Length:$filesize\r\n";
$output .= "Accept-Ranges: bytes\r\n";
$output .= "Cache-Control: private\n\n";
$output .= $send['Output'];
$output .= "Content-Transfer-Encoding: binary";
$output .= "Connection: Keep-Alive\r\n";
}
socket_write($client, $output);
socket_close ($client);
}
socket_close ($httpsock);
Hello, I am snikolov i am creating a miniwebserver with php and i would like to know how i can send the client a file to download with his browser such as firefox or internet explore i am sending a file to the user to download via sockets, but the cleint is not getting the filename and the information to download can you please help me here,if i declare the file again i get this error in my server
Fatal error: Cannot redeclare openfile() (previously declared in C:\User
s\fsfdsf\sfdsfsdf\httpd.php:31) in C:\Users\hfghfgh\hfghg\httpd.php on li
ne 29, if its possible, i would like to know if the webserver can show much banwdidth the user request via sockets, perl has the same option as php but its more hardcore than php i dont understand much about perl, i even saw that a miniwebserver can show much the client user pulls from the server would it be possible that you can assist me with this coding, i much aprreciate it thank you guys.
You are not sending the filename to the client, so how should it know which filename to use?
There is a drawback, you can provide the desired filename in the http header, but some browsers ignore that and always suggest the filename based on the last element in URL.
For example http://localhost/download.php?help.me would result in the sugested filename help.me in the file download dialogue.
see: http://en.wikipedia.org/wiki/List_of_HTTP_headers
Everytime you run your while (1) loop you declare openfile function. You can declare function only once. Try to move openfile declaration outside loop.
$httpsock = #socket_create_listen("9090");
if (!$httpsock) {
print "Socket creation failed!\n";
exit;
}
while (1) {
$client = socket_accept($httpsock);
$input = trim(socket_read ($client, 4096));
$input = explode(" ", $input);
$input = $input[1];
$fileinfo = pathinfo($input);
switch ($fileinfo['extension']) {
default:
$mime = "text/html";
}
if ($input == "/") {
$input = "index.html";
}
$input = ".$input";
if (file_exists($input) && is_readable($input)) {
echo "Serving $input\n";
$contents = file_get_contents($input);
$output = "HTTP/1.0 200 OK\r\nServer: APatchyServer\r\nConnection: close\r\nContent-Type: $mime\r\n\r\n$contents";
} else {
//$contents = "The file you requested doesn't exist. Sorry!";
//$output = "HTTP/1.0 404 OBJECT NOT FOUND\r\nServer: BabyHTTP\r\nConnection: close\r\nContent-Type: text/html\r\n\r\n$contents";
$filename = "dada";
$file = fopen($filename, 'r');
$filesize = filesize($filename);
$buffer = fread($file, $filesize);
$send = array("Output"=>$buffer,"filesize"=>$filesize,"filename"=>$filename);
$file = $send['filename'];
$output = 'HTTP/1.0 200 OK\r\n';
$output .= "Content-type: application/octet-stream\r\n";
$output .= "Content-Length: $filesize\r\n";
$output .= 'Content-Disposition: attachment; filename="'.$file.'"\r\n';
$output .= "Accept-Ranges: bytes\r\n";
$output .= "Cache-Control: private\n\n";
$output .= $send['Output'];
$output .= "Pragma: private\n\n";
// $output .= "Content-Transfer-Encoding: binary";
//$output .= "Connection: Keep-Alive\r\n";
}
socket_write($client, $output);
socket_close ($client);
}
socket_close ($httpsock);