Why does my php upload not work? - php

TL;DR: why does reuploding the uploaded data not work?
I' trying to upload the data a user uploaded to my file to another server. This means, I want to post my post data.
I want to upload some data to upload.php, which then should get posted to test.php (which simply outputs the raw post data). And due to memory saving, I wanted it to work without generating the post whole message as string. Thus I also don't want to use curl.
test.php
<?php
echo 'foo', file_get_contents('php://input'), 'bar';
upload.php
<?php
//new line variable
$NL = "\r\n";
//open the posted data as a resource
$client_upload = fopen('php://input', 'r');
//open the connection to the other server
$server_upload = fsockopen('ssl://example.com', 443);
//write the http headers to the socket
fwrite($server_upload, 'POST /test.php HTTP/1.1' . $NL);
fwrite($server_upload, 'Host: example.com' . $NL);
fwrite($server_upload, 'Connection: close' . $NL);
//header/body divider
fwrite($server_upload, $NL);
//loop until client upload reached the end
while (!feof($client_upload)) {
fwrite($server_upload, fread($client_upload, 1024));
}
//close the client upload resource - not needed anymore
fclose($client_upload);
//intitalize response variable
$response = '';
//loop until server upload reached the end
while (!feof($server_upload)) {
$response .= fgets($server_upload, 1024);
}
//close the server upload resource - not needed anymore
fclose($server_upload);
//output the response
echo $response;
When I post { "test": true } (from Fiddler) to test.php file, it outputs foo{ "test": true }bar.
Now when I try to do the same for upload.php, I just get foobar (and the http headers from test.php) but not the uploaded content.

Finally I managed to fix this error. Apparently the other server (and mine) depend on the http header Content-Length.
As you can see in my answer, I was not sending (neither calculating) this header. So when I finally now calculated and sent the Content-Length header everything worked. This are the missing lines:
$content_length = fstat($client_upload)['size'];
fwrite($server_upload, 'Content-Length: ' . $content_length . $NL);
The reupload (of the uploaded data to my server) didn't work, because the other server just read the body as long as it was specified in the Content-Length header. Because I was not sending this header, it didn't work.

Related

PHP cURL Upload file without processing

I am trying to upload a file to a Rest API (Tableau Server Rest API) endpoint using PHP cURL.
The file upload procedure exists of three steps:
Initiate file upload (request upload token)
Append file upload (upload file data)
Publish resource (save the file)
I was having issues with the server giving me a 500 status code on the second step. After contacting the support we figured out that the problem is most likely that the curl request seems to using the -data flag instead of the --data-binary flag which means some kind of encoding happens on the request body which should not be happening. This causes the server to respond with a 500 status code instead of an actual error message...
I would like to know how i can make a cURL request with the --data-binary flag in PHP.
Relevant sections of my current code:
// more settings
$curl_opts[CURLOPT_CUSTOMREQUEST] = $method;
$curl_opts[CURLOPT_RETURNTRANSFER] = true;
$curl_opts[CURLOPT_HTTPHEADER] = $headers;
$curl_opts[CURLOPT_POSTFIELDS] = $body;
//more settings
curl_setopt_array( $this->ch, $curl_opts );
$responseBody = curl_exec( $this->ch );
$method is "PUT", $headers contains an array with Content-Type: multipart/mixed; boundary=boundary-string and the $body is constructed like this:
$boundary = md5(date('r', time()));
$body = "--$boundary\n";
$body .= "Content-Disposition: name='request_payload'\nContent-Type: text/xml\n\n";
$body .= "\n--$boundary\n";
$body .= "Content-Disposition: name='tableau_file'; filename='$fileName'\nContent-Type: application/octet-stream\n\n";
$body .= file_get_contents( $path );
$body .= "\n--$boundary--";
Where the $boundary is the same as the boundary-string in the content-type header.
i am aware this is a kinda/very messy way to construct my body, i am planning to use Mustache as soon as i can actually upload my files :S
(i would like to mention that this is my first post here, please be gentle...)
CURLOPT_POSTFIELDS can accept an array of key=value field pairs. DON'T build your own mime body.
This is all you should have, really:
$data = array(
'tableau_file' => '#/path/to/file';
^---tell curl this field is a file
etc..
);
curl_setopt($this->ch, CURLOPT_POSTFIELDS, $data);
I ended up trying to implement a "--data-binary" using the exec function in PHP. Before i could move to this i had to cleanup some of my request body building code. This cleanup involved moving from string building ($body .= "this" . $that . "\n";) to a template engine (Mustache). Somehow this change fixed the problem.

Sending binary data to php-cgi via shell_exec

I have a script that sends a post request to /usr/bin/php-cgi. The script is working fine when dealing with plain text, but fails when the data is binary:
$data = file_get_contents('example.jpg');
$size = filesize('example.jpg') + 5;
$post_data = 'file='.$data;
$response = shell_exec('echo "'.$post_data.'" |
REDIRECT_STATUS=CGI
REQUEST_METHOD=POST
SCRIPT_FILENAME=/example/script.php
SCRIPT_NAME=/script.php
PATH_INFO=/
SERVER_NAME=localhost
SERVER_PROTOCOL=HTTP/1.1
REQUEST_URI=/example/index.html
HTTP_HOST=example.com
CONTENT_TYPE=application/x-www-form-urlencoded
CONTENT_LENGTH='.$size.' php-cgi');
I get the following error:
sh: -c: line 1: unexpected EOF while looking for matching `"'
sh: -c: line 5: syntax error: unexpected end of file
I guess this is because the data I'm trying to send is binary and must be encoded/escaped somehow.
Like I said the above code works if the data is plain text:
$post_data = "data=sample data to php-cgi";
$size = strlen($post_data);
I also tried to encode the data using base64_encode() but then I face another problem; the data must be decoded from within the receiving script. I was thinking that perhaps I could encode the data in base64 and then add some content or mime type header to force the php-cgi binary to make the conversation?
One other problem is that I like to send the data as an attachment and therefore I think we must set CONTENT_TYPE to multipart/form-data; boundary=<random_boundary> and CONTENT_DISPOSITION to form-data, but I'm not sure how to set these headers from the commandline.
You are trying to upload binary files through shell_exe to post the contents. shell_exe doesn't accept the binary encoding. If you change the image data to base64 then you problem would be solved. But you will get into another problem i.e. how to identify the submitted text/string i.e. text or image. Presently, I find no solution to identify the submitted value is image or text.
Since, you want to post the image and data, I would suggest you to use CURL and providing the way to submit the image and data through CURL which is used by me also:
$local_directory=dirname(__FILE__).'/local_files/';
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_VERBOSE, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible;)");
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_URL, 'http://localhost/curl_image/uploader.php' );
//most importent curl assues #filed as file field
$post_array = array(
"my_file"=>"#".$local_directory.'filename.jpg',
"upload"=>"Upload"
);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_array);
$response = curl_exec($ch);
echo $response;
You can also store all post data in a temporary file and then cat that file into php-cgi:
char file[20] = "/tmp/php"; //temp post data location
char name[10];
webserver_alpha_random(name, 10); //create random name
strcat(file, name);
int f = open(file, O_TRUNC | O_WRONLY | O_CREAT);
write(f, conn->content, conn->content_len); //post data from mongoose
close(f);
/* cat temp post data into php-cgi */
/* this function also takes care of all environment variables */
/* but the idea is understandable */
output = webserver_shell("cat %s | php-cgi %s", conn, request, file, request);
unlink(file);
Finally I got this working, the solution was to send a base64 encoded request which also contained a constant named field like ORIGINAL_QUERY_URI to a sort of gateway file that in turn would decode the request and bounce it to it's original destination.
Basically, the server does this:
encode any received file data in base64
add a form-data field named ORIGINAL_REQEUST_URI with the original url as value
assemble a valid http request body encoded as multipart/form-data based on above
send this data using shell_exec to a gateway file that will decode the content
Here is the command I used to send everything to php-cgi:
shell_exec('echo "' . $body . '" |
HOST=localhost
REDIRECT_STATUS=200
REQUEST_METHOD=POST
SCRIPT_FILENAME=/<path>/gate.php
SCRIPT_NAME=/gate.php
PATH_INFO=/
SERVER_NAME=localhost
SERVER_PROTOCOL=HTTP/1.1
REQUEST_URI=/example.php
HTTP_HOST=localhost
CONTENT_TYPE="multipart/form-data; boundary=' . $boundary . '"
CONTENT_LENGTH=' . $size . ' php-cgi');
Then inside the gate.php file I decoded the data and included the file pointed to by theORIGINAL_REQUEST_URI field.
// File: gate.php
if (isset($_FILES)) {
foreach ($_FILES as $key => $value) {
// decode the `base64` encoded file data
$content = file_get_contents($_FILES[$key]['tmp_name']);
$content = base64_decode($content);
file_put_contents($_FILES[$key]['tmp_name'], $content);
}
}
// bounce to original destination
include_once($_POST['ORIGINAL_REQUEST_URI']);

How to speed up file_get_contents?

Here's my code:
$language = $_GET['soundtype'];
$word = $_GET['sound'];
$word = urlencode($word);
if ($language == 'english') {
$url = "<the first url>";
} else if ($language == 'chinese') {
$url = "<the second url>";
}
$opts = array(
'http'=>array(
'method'=>"GET",
'header'=>"User-Agent: <my user agent>"
)
);
$context = stream_context_create($opts);
$page = file_get_contents($url, false, $context);
header('Content-Type: audio/mpeg');
echo $page;
But I've found that this runs terribly slow.
Are there any possible methods of optimization?
Note: $url is a remote url.
It's slow because file_get_contents() reads the entire file into $page, PHP waits for the file to be received before outputting the content. So what you're doing is: downloading the entire file on the server side, then outputting it as a single huge string.
file_get_contents() does not support streaming or grabbing offsets of the remote file. An option is to create a raw socket with fsockopen(), do the HTTP request, and read the response in a loop, as you read each chunk, output it to the browser. This will be faster because the file will be streamed.
Example from the Manual:
$fp = fsockopen("www.example.com", 80, $errno, $errstr, 30);
if (!$fp) {
echo "$errstr ($errno)<br />\n";
} else {
header('Content-Type: audio/mpeg');
$out = "GET / HTTP/1.1\r\n";
$out .= "Host: www.example.com\r\n";
$out .= "Connection: Close\r\n\r\n";
fwrite($fp, $out);
while (!feof($fp)) {
echo fgets($fp, 128);
}
fclose($fp);
}
The above is looping while there is still content available, on each iteration it reads 128 bytes and then outputs it to the browser. The same principle will work for what you're doing. You'll need to make sure that you don't output the response HTTP headers which will be the first few lines, because since you are doing a raw request, you will get the raw response with headers included. If you output the response headers you will end up with a corrupt file.
Instead of downloading the whole file before outputting it, consider streaming it out like this:
$in = fopen($url, 'rb', false, $context);
$out = fopen('php://output', 'wb');
header('Content-Type: video/mpeg');
stream_copy_to_stream($in, $out);
If you're daring, you could even try (but that's definitely experimental):
header('Content-Type: video/mpeg');
copy($url, 'php://output');
Another option is using internal redirects and making your web server proxy the request for you. That would free up PHP to do something else. See also my post regarding X-Sendfile and friends.
As explained by #MrCode, first downloading the file to your server, then passing it on to the client will of course incur a doubled download time. If you want to pass the file on to the client directly, use readfile.
Alternatively, think about if you can't simply redirect the client to the file URL using a header("Location: $url") so the client can get the file directly from the source.

How to redirect a live data stream adding to it another header and returning it on demand? (PHP)

I have a url like http://localhost:8020/stream.flv
On request to my php sctipt I want to return (be something like a proxy) all data I can get from that URL (so I mean my php code should get data from that url and give it to user) and my header and my beginning of file.
So I have my header and some data I want to write in the beginning of response like
# content headers
header("Content-Type: video/x-flv");
# FLV file format header
if($seekPos != 0)
{
print('FLV');
print(pack('C', 1));
print(pack('C', 1));
print(pack('N', 9));
print(pack('N', 9));
}
How to do such thing?
Here's the code I used to do something very similar, I also forwarded all headers from the original flv to the end user but you can just remove that section if you want to set your own.
$video = fopen(URL, "rb");
// Forward headers, $http_response_header is populated by fopen call
foreach ($http_response_header AS $header) {
header($header);
}
// Output contents of flv
while (!feof($video)) {
print (fgets($video));
}
fclose($video);
See the manual entries for fopen and fread.
I've accomplished this using libcurl,
$url = 'http://localhost:8020/stream.flv';
$ch = curl_init();
curl_exec($ch);
curl_close($ch);

Handle CURL headers before downloading body

Using PHP and CURL (unless there is a better alternative then CURL in this case), is it possible to have a php function handle the header response before downloading the file?
For example:
I have a script that downloads and processes urls supplied by the user. I would like to add a check so that if the file is not valid for my process (not a text file, too large, etc),the CURL request would be cancelled before the server wastes time downloading the file.
Update: Solution
PEAR class HTTP_Request2: http://pear.php.net/package/HTTP_Request2/
Gives you the ability to set observers to the connection and throw exceptions to cancel anytime. Works perfectly for my needs!
Using cURL, do a HTTP HEAD request to check the headers, then if it is valid (the status is 200) do the full HTTP GET request.
The basic option you must set is CURLOPT_NOBODY, which changes the requested to the type HEAD
curl_setopt($ch, CURLOPT_NOBODY, true);
Then after executing the query, you need to check the return status which can be done using curl_getinfo()
$status = curl_getinfo($ch, CURLINFO_HTTP_CODE);
I know this is an old topic but just in case people comes here in the future.
With CURL, you can use CURLOPT_WRITEFUNCTION, who let's you place a callback that will be called as soon as the body response starts coming and needs to be written. In that moment you can read the headers and cancel the process and the body will not be downloaded. All in one request.
For a deeper look and code examples see PHP/Curl: inspecting response headers before downloading body
This is an example how you can solve it:
// Include the Auth string in the headers
// Together with the API version being used
$headers = array(
"Authorization: GoogleLogin auth=" . $auth,
"GData-Version: 3.0",
);
// Make the request
curl_setopt($curl, CURLOPT_URL, "http://docs.google.com/feeds/default/private/full");
curl_setopt($curl, CURLOPT_HTTPHEADER, $headers);
$response = curl_exec($curl);
curl_close($curl);
// Parse the response
$response = simplexml_load_string($response);
// Output data
foreach($response->entry as $file)
{
//now you can do what ever if file type is a txt
//if($file->title =="txt")
// do something
else
// do soething
echo "File: " . $file->title . "<br />";
echo "Type: " . $file->content["type"] . "<br />";
echo "Author: " . $file->author->name . "<br /><br />";
}

Categories