I'm making a PHP image proxy script. I need it to not only echo the contents of the image it requests, but also identically reproduce the header of the image request.
I've seen one, and the other, but not both together... and these cURL option things confuse me. How would I do this?
Sorry, I'm not sure what is you want.
This is the example to get from a image url, echo header and save image to a file.
But, if you want a proxy, you should use web server (Nginx, Apache, etc), PHP is no need
<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://img3.cache.netease.com/www/logo/logo_png.png");
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_REFERER, "http://www.163.com/");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, 1);
$return = curl_exec($ch);
curl_close($ch);
list($header, $image) = explode("\r\n\r\n", $return, 2);
echo $header;
file_put_contents("/tmp/logo.png", $image);
You can get all the headers (not as raw text) with getallheaders()
http://www.php.net/manual/en/function.getallheaders.php
Then string them back together:
$headers = "";
foreach (getallheaders() as $name => $value) {
$headers = "$name: $value\r\n";
}
$headers .= "\r\n"; // Double newline to signal end of headers (HTTP spec)
Then I think the best way is to use a socket connection, rather than CURL like so:
$response = '';
$fp = fsockopen('example.org', 80);
fputs($fp, $headers);
while (!feof($fp)) {
$response .= fgets($fp, 128);
}
fclose($fp);
Note that you may need to modify the host/request headers (because this is an identical copy, as you asked), and you may need to implement redirect following.
Related
Situation
I am trying to make a call to the Shopify REST API where I have more than 50-250 results but I am not able to get the Link Header from the cURL Response which contains the Pagination Links.
Sample of Link Headers from the API Documentation for Cursor-Pagination (https://shopify.dev/tutorials/make-paginated-requests-to-rest-admin-api)
#...
Link: "<https://{shop}.myshopify.com/admin/api/{version}/products.json?page_info={page_info}&limit={limit}>; rel={next}, <https://{shop}.myshopify.com/admin/api/{version}/products.json?page_info={page_info}&limit={limit}>; rel={previous}"
#...
The link rel parameter does show up, but the Link is empty as below.
My Shopify Call function
function shopify_call($token, $shop, $api_endpoint, $query = array(), $method = 'GET', $request_headers = array()) {
// Build URL
$url = "https://" . $shop . ".myshopify.com" . $api_endpoint;
if (!is_null($query) && in_array($method, array('GET', 'DELETE'))) $url = $url . "?" . http_build_query($query);
$headers = [];
// Configure cURL
$curl = curl_init($url);
curl_setopt($curl, CURLOPT_HEADER, TRUE);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, TRUE);
// this function is called by curl for each header received
curl_setopt($curl, CURLOPT_HEADERFUNCTION,
function($ch, $header) use (&$headers)
{
$len = strlen($header);
$header = explode(':', $header, 2);
if (count($header) < 2) // ignore invalid headers
return $len;
$headers[trim($header[0])] = trim($header[1]);
return $len;
}
);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($curl, CURLOPT_MAXREDIRS, 3);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, FALSE);
// curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, 3);
// curl_setopt($curl, CURLOPT_SSLVERSION, 3);
curl_setopt($curl, CURLOPT_USERAGENT, 'Sphyx App v.1');
curl_setopt($curl, CURLOPT_CONNECTTIMEOUT, 30);
curl_setopt($curl, CURLOPT_TIMEOUT, 30);
curl_setopt($curl, CURLOPT_CUSTOMREQUEST, $method);
curl_setopt($curl,CURLOPT_ENCODING,'');
// Setup headers
$request_headers[] = "";
if (!is_null($token)) $request_headers[] = "X-Shopify-Access-Token: " . $token;
$request_headers[] = 'Accept: */*'; // Copied from POSTMAN
$request_headers[] = 'Accept-Encoding: gzip, deflate, br'; // Copied from POSTMAN
curl_setopt($curl, CURLOPT_HTTPHEADER, $request_headers);
if ($method !== 'GET' && in_array($method, array('POST', 'PUT'))) {
if (is_array($query)) $query = http_build_query($query);
curl_setopt ($curl, CURLOPT_POSTFIELDS, $query);
}
// Send request to Shopify and capture any errors
$result = curl_exec($curl);
$response = preg_split("/\r\n\r\n|\n\n|\r\r/", $result, 2);
$error_number = curl_errno($curl);
$error_message = curl_error($curl);
// Close cURL to be nice
curl_close($curl);
// Return an error is cURL has a problem
if ($error_number) {
return $error_message;
} else {
// Return headers and Shopify's response
return array('headers' => $headers, 'response' => json_decode($response[1],true));
}
}
But when I use a POSTMAN Collection, I get a proper formatted response without the Link getting truncated/processed.
I have tried a lot of things here available via the StackOverflow Forums as well as Shopify Community, but I'm unable to parse the Response Header the same way as shown by API Examples or POSTMAN
My issue does seem to be with the PHP Code, but I'm not a pro with cURL. Thus, I'm not able to make it further :(
Also, I'm not able to understand why POSTMAN's Headers are in Proper Case whereas mine are in Lower Case
Thanks in Advance!
Found my answer :
https://community.shopify.com/c/Shopify-APIs-SDKs/Help-with-cursor-based-paging/m-p/579640#M38946
I was using a browser to view my log files. So the data is there but it's hidden because of your use of '<'s around the data. I had to use the browser inspector to see the data. Not sure who decided this syntax was a good idea. Preference would be two headers that one can see and more easily parse since using link syntax is not relative to using an API.
My suggestion would be 2 headers:
X-Shopify-Page-Next: page_info_value (empty if no more pages)
X-Shopify-Page-Perv: page_info_value (empty on first page or if there is no previous page).
Easy to parse and use.
But having this buried as an invalid xml tag, having them both in the same header and using 'rel=' syntax makes no sense at all from an API perspective.
How I can get downloading file with its original name?
URL doesn't have name.
URL's format: http://someurl/?RANDOMSYMBOLS
URL returns named file, but all of known me PHP's methods drop to server RENAMED file, with MY name, NOT original.
I using file_put_contents("Tmpfile.zip", fopen("http://someurl/?RANDOMSYMBOLS", 'r')), but it drops file as "Tmpfile.zip", not with original name (name of file which common user with common browser gets when navigate to url).
I want to know name of that file and drop with that. No problem when URL has format "example.com/?filename", but no, it doesn't contain name.
you have to read http response headers , http headers contains fileName , size... etc
and its necessary to use curl because fopen dont show you headers
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://www.example.com/file.zip");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
$response = curl_exec($ch);
// Then, after your curl_exec call:
$header_size = curl_getinfo($ch, CURLINFO_HEADER_SIZE);
$header = substr($response, 0, $header_size);
$body = substr($response, $header_size);
print $header to see headers list with
var_dump($header);
and $body contains file data
I have a url something like this
$url ="www.domain.com/image.php?id=123&idlocation=987&number=01";
Previously i was getting the extension using following code
$img_details= pathinfo($url);
But this won't work any more since the url has other variables also . So in this case how to get the Image name and extension .
I know i should first download the file using
$contenido = file_get_contents($url);
But don't know how to get the files name/extention from this
Thanks in advance
$ch = curl_init();
$url = 'http://www.domain.com/image.php?id=123&idlocation=987&number=01';
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_NOBODY, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$results = split("\n", trim(curl_exec($ch)));
foreach($results as $line) {
if (strtok($line, ':') == 'Content-Type') {
$parts = explode(":", $line);
echo trim($parts[1]);
}
}
Return: image/png
Already answered in: Get mime type of external file using cURL and php
The above answers focus both on the mime-type and they will work in most cases they both need additional resource usage - either a second network call to get the mime type or more disk reads/writes to save the file to disk and use the exif-imagetype function. And both will not return the file name which was a part of the question. Here is a download function using curl that will return a downloaded file as an array with name,mime type and content. Additionally it will try to get the name from the URL if possible.
Sample usage
$file=downloadfile($url);
echo "Name: ".$file["name"]."<br>";
echo "Type: ".$file["mimetype"]."<br>";
Code
function downloadfile($url){
global $headers;
$headers=array();
$file=array("content"=>"","mimetype"=>"","name"=>"");
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_HEADERFUNCTION, 'readHeaders');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$file["content"]=curl_exec($ch);
if(sizeof($headers)){
foreach($headers as $header){
if(!strncasecmp($header,'content-type:',13)){
$file["mimetype"]=trim(substr($header,13));
}
elseif(!strncasecmp($header,'content-disposition:',20)){
$file["name"]=trim(substr(strstr($header,'filename='),9));
}
}
}
if(!$file["name"]){
$query=strpos("?",$url);
$file["name"]=basename(substr($url,0,($query?$query:strlen($url))));
}
unset($headers);
return $file;
}
function readHeaders($ch, $header) {
global $headers;
array_push($headers, $header);
return strlen($header);
}
I am using cUrl in PHP to request from some external service.
Interesting enough, the server is responding with raw "multipart/form-data" instead of binary file data.
My website is using a shared hosting, therefore PECL HTTP is not an option.
Is there a way to parse this data with PHP?
Sample code:
$response = curl_exec($cUrl);
/* $response is raw "multipart/form-data" string
--MIMEBoundaryurn_uuid_DDF2A2C71485B8C94C135176149950475371
Content-Type: application/xop+xml; charset=utf-8; type="text/xml"
Content-Transfer-Encoding: binary
(xml data goes here)
--MIMEBoundaryurn_uuid_DDF2A2C71485B8C94C135176149950475371
Content-Type: application/zip
Content-Transfer-Encoding: binary
(binary file data goes here)
*/
EDIT: I tried piping the response to a localhost HTTP request, but the respond data is likely to exceed the allowed memory size in PHP process. Expending mem limit is not very practical, this action also dramatically reduces the server performance dramatically.
If there is no alternatives to the original question, you may suggest a way to handle very large POST requests, along with XML parsing, in terms of streams in PHP.
I know this would be hard, please comment. I am open for discussions.
if you need the zip file from the response I guess you could just write a tmp file to save the curl response to, and stream that as a workaround:
Never tried that with multipart curls, but I guess it should work.
$fh = fopen('/tmp/foo', 'w');
$cUrl = curl_init('http://example.com/foo');
curl_setopt($cUrl, CURLOPT_FILE, $fh); // redirect output to filehandle
curl_exec($cUrl);
curl_close($cUrl);
fclose($fh); // close filehandle or the file will be corrupted
if you do NOT need anything but the xml part of the response you might want to disable headers
curl_setopt($cUrl, CURLOPT_HEADER, FALSE);
and add option to only accept xml as a response
curl_setopt($cUrl, CURLOPT_HTTPHEADER, array('Accept: application/xml'));
//That's a workaround since there is no available curl option to do so but http allows that
[EDIT]
A Shot in the dark...
can you test with these curlopt settings to see if modifiying these help anything
$headers = array (
'Content-Type: multipart/form-data; boundary=' . $boundary,
'Content-Length: ' . strlen($requestBody),
'X-EBAY-API-COMPATIBILITY-LEVEL: ' . $compatLevel, // API version
'X-EBAY-API-DEV-NAME: ' . $devID,
'X-EBAY-API-APP-NAME: ' . $appID,
'X-EBAY-API-CERT-NAME: ' . $certID,
'X-EBAY-API-CALL-NAME: ' . $verb,
'X-EBAY-API-SITEID: ' . $siteID,
);
$cUrl = curl_init();
curl_setopt($cUrl, CURLOPT_URL, $serverUrl);
curl_setopt($cUrl, CURLOPT_TIMEOUT, 30 );
curl_setopt($cUrl, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($cUrl, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($cUrl, CURLOPT_HTTPHEADER, $headers);
curl_setopt($cUrl, CURLOPT_POST, 1);
curl_setopt($cUrl, CURLOPT_POSTFIELDS, $requestBody);
curl_setopt($cUrl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($cUrl, CURLOPT_FAILONERROR, 0 );
curl_setopt($cUrl, CURLOPT_FOLLOWLOCATION, 1 );
curl_setopt($cUrl, CURLOPT_HEADER, 0 );
curl_setopt($cUrl, CURLOPT_USERAGENT, 'ebatns;xmlstyle;1.0' );
curl_setopt($cUrl, CURLOPT_HTTP_VERSION, 1 ); // HTTP version must be 1.0
$response = curl_exec($cUrl);
if ( !$response ) {
print "curl error " . curl_errno($cUrl ) . PHP_EOL;
}
curl_close($cUrl);
[EDIT II]
This is just a try, as mentioned I cannot get my curled pages to respond with a multipart form data. So be gentle with me here ;)
$content_type = ""; //use last know content-type as a trigger
$tmp_cnt_file = "tmp/tmpfile";
$xml_response = ""; // this will hold the "usable" curl response
$hidx = 0; //header index.. counting the number of different headers received
function read_header($cUrl, $string)// this will be called once for every line of each header received
{
global $content_type, $hidx;
$length = strlen($string);
if (preg_match('/Content-Type:(.*)/', $string, $match))
{
$content_type = $match[1];
$hidx++;
}
/*
should set $content_type to 'application/xop+xml; charset=utf-8; type="text/xml"' for the first
and to 'application/zip' for the second response body
echo "Header: $string<br />\n";
*/
return $length;
}
function read_body($cUrl, $string)
{
global $content_header, $xml_response, $tmp_cnt_file, $hidx;
$length = strlen($string);
if(stripos ( $content_type , "xml") !== false)
$xml_response .= $string;
elseif(stripos ($content_type, "zip") !== false)
{
$handle = fopen($tmp_cnt_file."-".$hidx.".zip", "a");
fwrite($handle, $string);
fclose($handle);
}
/*
elseif {...} else{...}
depending on your needs
echo "Received $length bytes<br />\n";
*/
return $length;
}
and of course set the proper curlopts
// Set callback function for header
curl_setopt($cUrl, CURLOPT_HEADERFUNCTION, 'read_header');
// Set callback function for body
curl_setopt($cUrl, CURLOPT_WRITEFUNCTION, 'read_body');
don't forget to NOT save the curl response to a variable because of the memory issues,
hopefully all you need will be in the $xml_response above anyways.
//$response = curl_exec($cUrl);
curl_exec($cUrl);
And for parsing your code you can refer to $xml_response and the temp files you created starting with tmp/tmpfile-2 in this scenario. Again, I have not been able to test the code above in any way. So this might not work (but it should imho ;))
[EDIT III]
Say we want curl to write all incoming data directly to another (outgoing) stream, in this case a socket connection
I'm not sure if it is as easy as this:
$fs = fsockopen($host, $port, $errno, $errstr);
$cUrl = curl_init('http://example.com/foo');
curl_setopt($cUrl, CURLOPT_FILE, $fs); // redirect output to sockethandle
curl_exec($cUrl);
curl_close($cUrl);
fclose($fs); // close handle
else we will have to use our known write and header functions with just a little trick
//first open the socket (before initiating curl)
$fs = fsockopen($host, $port, $errno, $errstr);
// now for the new callback function
function socket_pipe($cUrl, $string)
{
global $fs;
$length = strlen($string);
fputs($fs, $string); // add NOTHING to the received line just send it to $fs; that was easy wasn't it?
return $length;
}
// and of course for the CURLOPT part
// Set callback function for header
curl_setopt($cUrl, CURLOPT_HEADERFUNCTION, 'socket_pipe');
// Set the same callback function for body
curl_setopt($cUrl, CURLOPT_WRITEFUNCTION, 'socket_pipe');
// do not forget to
fclose($fs); //when we're done
The thing is, not editing the result and simply piping it to $fs will make it necessary that apache is listening on a certain port which you then assign your script to.
Or you will need to add ONE header line directly after fsockopen
fputs($fp, "POST $path HTTP/1.0\n"); //where path is your script of course
I'm sorry i can't help much because you did not put much code but i remember i was having a similar issue when i was playing with curl_setopt options.
Did you use CURLOPT_BINARYTRANSFER?
From php documentation -> CURLOPT_BINARYTRANSFER-> TRUE to return the raw output when CURLOPT_RETURNTRANSFER is used.
just set CURLOPT_RETURNTRANSFER CURLOPT_POST
$c = curl_init($url);
curl_setopt($c, CURLOPT_RETURNTRANSFER, true);
curl_setopt($c, CURLOPT_CONNECTTIMEOUT, 1);
curl_setopt($c, CURLOPT_TIMEOUT, 1);
curl_setopt($c, CURLOPT_POST, 1);
curl_setopt($c, CURLOPT_POSTFIELDS,
array());
$rst_str = curl_exec($c);
curl_close($c);
You can re-assemble you Binary data doing something like this, I hope it helps.
$file_array = explode("\n\r", $file, 2);
$header_array = explode("\n", $file_array[0]);
foreach($header_array as $header_value) {
$header_pieces = explode(':', $header_value);
if(count($header_pieces) == 2) {
$headers[$header_pieces[0]] = trim($header_pieces[1]);
}
}
header('Content-type: ' . $headers['Content-Type']);
header('Content-Disposition: ' . $headers['Content-Disposition']);
echo substr($file_array[1], 1);
If you don't need binary data, have you tried below?
curl_setopt($c, CURLOPT_NOBODY, true);
first time asker, but many times you helped me back in the day. Great job! I ask this because I'm struggling here with and issue I'm unable to solve, and as my PHP (and cURL) knowledge is so scarce, I'm lost.
The Background
I'm developing a Javascript app, that needs to connect to several different servers and make XMLRPC calls to them. The app is working perfectly running it locally (disabling cross-domain security), but to make it run online I knew I had to use a cross-domain proxy, so after several days of searching and investigating, I didn't found one that could make the work, so I managed to make one myself (not without blood and sweat). Know what? It (almost) works!!!
This is my proxy.php:
<?
function readHeader($ch, $header) {
//extracting data to send it to the client
$headers = explode("\n", $header);
foreach ($headers as $item) {
// $string= str_replace($delimiter, $mainDelim, $string);
if (strpos($item, 'Set-Cookie:') !== false) {
$cookie = trim(substr($item,strlen('Set-Cookie:')));
header('X-Set-Cookie:' . $cookie);
} else {
header($item);
}
}
return strlen($header);
}
$allowed_domains = array('domain1.com', 'domain2.com');
header('Content-Type: text/html; charset=iso-8859-1');
$REFERRER = $_SERVER['HTTP_REFERER'];
if ($REFERRER == '') {
// What do you do here?
exit(header('Location: index.html'));
}
$domain = substr($REFERRER, strpos($REFERRER, '://') + 3);
$domain = substr($domain, 0, strpos($domain, '/'));
if (!in_array($domain, $allowed_domains)) {
exit(header('Location: index.html'));
}
$XMLRPC_SERVICE = $_SERVER['HTTP_X_PROXY_URL'];
$xml = $HTTP_RAW_POST_DATA;
$header[] = "Content-type: text/xml; charset=utf-8";
$header[] = "Connection: close";
$header[] = "Accept: text/xml";
if ($_SERVER['HTTP_X_SET_COOKIE'])
$cookie = $_SERVER['HTTP_X_SET_COOKIE'];
if ($_SERVER['HTTP_X_PROXY_URL'] === "other-domain.com")
$header[] = "x-custom-header: value";
$ch = curl_init($XMLRPC_SERVICE);
//URL to post to
curl_setopt($ch, CURLOPT_URL, $XMLRPC_SERVICE);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'POST');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
if ($cookie)
curl_setopt($ch, CURLOPT_COOKIE, $cookie);
curl_setopt($ch, CURLOPT_POSTFIELDS, $xml);
curl_setopt($ch, CURLOPT_HEADERFUNCTION, 'readHeader');
$response = curl_exec($ch);
if (curl_errno($ch)) {
echo curl_error($ch);
} else {
curl_close($ch);
echo $response;
}
?>
The Issue
As I've said, I got it working partially. In fact, it works for most of the usual XMLRPC needs.
It gets the remote server address from the HTTP_X_PROXY_URL header of the request, and using cURL makes the call and returns the values to the javascript client without issues.
The problem comes when I need to get/send a session cookie (probably when getting it, because the cookie value is pretty different when I make calls directly from the app locally). In any case, I can't get the cookie stuff to work. As you see, I'm surrounding the Set-Cookie browser protection on AJAX calls with my own X-Set-Cookie header, that the proxy gets to use or translates accordingly, but the issue with cookies is here, and I can't use cookies, that are critical for app functionality.