Related
Situation
I am trying to make a call to the Shopify REST API where I have more than 50-250 results but I am not able to get the Link Header from the cURL Response which contains the Pagination Links.
Sample of Link Headers from the API Documentation for Cursor-Pagination (https://shopify.dev/tutorials/make-paginated-requests-to-rest-admin-api)
#...
Link: "<https://{shop}.myshopify.com/admin/api/{version}/products.json?page_info={page_info}&limit={limit}>; rel={next}, <https://{shop}.myshopify.com/admin/api/{version}/products.json?page_info={page_info}&limit={limit}>; rel={previous}"
#...
The link rel parameter does show up, but the Link is empty as below.
My Shopify Call function
function shopify_call($token, $shop, $api_endpoint, $query = array(), $method = 'GET', $request_headers = array()) {
// Build URL
$url = "https://" . $shop . ".myshopify.com" . $api_endpoint;
if (!is_null($query) && in_array($method, array('GET', 'DELETE'))) $url = $url . "?" . http_build_query($query);
$headers = [];
// Configure cURL
$curl = curl_init($url);
curl_setopt($curl, CURLOPT_HEADER, TRUE);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, TRUE);
// this function is called by curl for each header received
curl_setopt($curl, CURLOPT_HEADERFUNCTION,
function($ch, $header) use (&$headers)
{
$len = strlen($header);
$header = explode(':', $header, 2);
if (count($header) < 2) // ignore invalid headers
return $len;
$headers[trim($header[0])] = trim($header[1]);
return $len;
}
);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($curl, CURLOPT_MAXREDIRS, 3);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, FALSE);
// curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, 3);
// curl_setopt($curl, CURLOPT_SSLVERSION, 3);
curl_setopt($curl, CURLOPT_USERAGENT, 'Sphyx App v.1');
curl_setopt($curl, CURLOPT_CONNECTTIMEOUT, 30);
curl_setopt($curl, CURLOPT_TIMEOUT, 30);
curl_setopt($curl, CURLOPT_CUSTOMREQUEST, $method);
curl_setopt($curl,CURLOPT_ENCODING,'');
// Setup headers
$request_headers[] = "";
if (!is_null($token)) $request_headers[] = "X-Shopify-Access-Token: " . $token;
$request_headers[] = 'Accept: */*'; // Copied from POSTMAN
$request_headers[] = 'Accept-Encoding: gzip, deflate, br'; // Copied from POSTMAN
curl_setopt($curl, CURLOPT_HTTPHEADER, $request_headers);
if ($method !== 'GET' && in_array($method, array('POST', 'PUT'))) {
if (is_array($query)) $query = http_build_query($query);
curl_setopt ($curl, CURLOPT_POSTFIELDS, $query);
}
// Send request to Shopify and capture any errors
$result = curl_exec($curl);
$response = preg_split("/\r\n\r\n|\n\n|\r\r/", $result, 2);
$error_number = curl_errno($curl);
$error_message = curl_error($curl);
// Close cURL to be nice
curl_close($curl);
// Return an error is cURL has a problem
if ($error_number) {
return $error_message;
} else {
// Return headers and Shopify's response
return array('headers' => $headers, 'response' => json_decode($response[1],true));
}
}
But when I use a POSTMAN Collection, I get a proper formatted response without the Link getting truncated/processed.
I have tried a lot of things here available via the StackOverflow Forums as well as Shopify Community, but I'm unable to parse the Response Header the same way as shown by API Examples or POSTMAN
My issue does seem to be with the PHP Code, but I'm not a pro with cURL. Thus, I'm not able to make it further :(
Also, I'm not able to understand why POSTMAN's Headers are in Proper Case whereas mine are in Lower Case
Thanks in Advance!
Found my answer :
https://community.shopify.com/c/Shopify-APIs-SDKs/Help-with-cursor-based-paging/m-p/579640#M38946
I was using a browser to view my log files. So the data is there but it's hidden because of your use of '<'s around the data. I had to use the browser inspector to see the data. Not sure who decided this syntax was a good idea. Preference would be two headers that one can see and more easily parse since using link syntax is not relative to using an API.
My suggestion would be 2 headers:
X-Shopify-Page-Next: page_info_value (empty if no more pages)
X-Shopify-Page-Perv: page_info_value (empty on first page or if there is no previous page).
Easy to parse and use.
But having this buried as an invalid xml tag, having them both in the same header and using 'rel=' syntax makes no sense at all from an API perspective.
I am developing a PHP script that:
Gets 10 rows from DB (works well)
Sends addresses from these rows to Map API
Obtains data
Save results in DB (works well)
I don't know how to obtain the final response of curl_exec. curl_exec
in $response = call('GET', $query);
- even when hasn't completed returns something in response.
These my loop that is waiting for the response of the call function. But it doesn't work json_decode($stringJSON) is invoked earlier than it is obtained the final response
$requestHandled = false;
$response = "";
$response = call('GET', $query);
while(!$requestHandled) {
if(strlen($response) > 5){
$response = mb_convert_encoding($response, "UTF-8");
$stringJSON = get_magic_quotes_gpc() ? stripslashes($response) : $response;
echo $stringJSON;
$jsonObject = "+";
echo $jsonObject;
$jsonObject = json_decode($stringJSON);
echo $jsonObject;
$requestHandled = true;
}
}
This is my curl call function
function call($method, $url, $data = false) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, $method);
curl_setopt($ch, CURLOPT_URL, $url);
if ($data) {
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
$headers = array();
$headers[] = 'Content-Type: application/json';
$headers[] = 'Content-Length: ' . strlen($data);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
}
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
return curl_exec($ch);
Please help. Has spent a half of the day solving it
So the result of the var_dump( $response ) is an empty string?
In that case, it simply means the response from the server is empty, because I just tested your function call() and it seems to work just fine. In other words, make sure the URL (with the method GET) you are trying to call actually returns data to begin with.
I wouldn't be surprised if you simply have a typographical error somewhere in the URL.
Also, for debugging purposes, temporarily replace
return curl_exec($ch);
with
$result = curl_exec($ch);
var_dump(curl_getinfo($ch));
return $result;
and investigate the info in the var_dump() result, to make sure the response code is 200 (or any other that indicates success) and not in the 4xx or 5xx ranges (respectively indicating either a client or server error), for instance.
See curl_getinfo() for more information about what useful information will be returned about your last curl transfer.
To address OPs comment(s):
Please try this complete script (nothing else: no while loop, no stripslashes(), no json_decode(), etc):
<?php
function call($method, $url, $data = false) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, $method);
curl_setopt($ch, CURLOPT_URL, $url);
if ($data) {
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
$headers = array();
$headers[] = 'Content-Type: application/json';
$headers[] = 'Content-Length: ' . strlen($data);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
}
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$result = curl_exec($ch);
var_dump(curl_getinfo($ch)); // check this result and make sure the response code is 200
return $result;
}
$query = 'fill in the correct URL! (with http(s)://)';
$response = call('GET', $query);
var_dump( $response ); // check this result and see if it's an empty string
die;
If the var_dump( $response ); returns an empty string, it means your script works fine but the URL you are calling simply returned an empty response.
If $response is not empty and actually contains JSON data, replace
var_dump( $response );
with
$stringJSON = mb_convert_encoding( $response, "UTF-8" );
echo $stringJSON; // make sure this contains valid JSON data
// stripslashes() should not be needed
var_dump( json_decode( $stringJSON ) ); // if JSON data was valid, you should get a valid PHP data structure
Here is my code
$url = "partial_response.php";
$sac_curl = curl_init();
curl_setopt($sac_curl, CURLOPT_HTTPGET, true);
curl_setopt($sac_curl, CURLOPT_URL, $url);
curl_setopt($sac_curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($sac_curl, CURLOPT_HEADER, false);
curl_setopt($sac_curl, CURLOPT_TIMEOUT, 11);
$resp = curl_exec($sac_curl);
curl_close($sac_curl);
echo $resp;
Partial_response.php
header( 'Content-type: text/html; charset=utf-8' );
echo 'Job waiting ...<br />';
for( $i = 0 ; $i < 10 ; $i++ )
{
echo $i . '<br/>';
flush();
ob_flush();
sleep(1);
}
echo 'End ...<br/>';
From the about code am trying to get a partial response from partial_response.php. what I want is, I need curl to return me "Job waiting.." alone instead of waiting for the partial_response.php to complete the loop and return the entire data. so when I reduce CURLOPT_TIMEOUT below 11 i dont get any response at all. Kindly clarify my doubt.
Thanks in advance.
I later realized that cURL could not do what I want, I used stream_context_get_options
to achieve what I wanted. Here it is, http://www.php.net/manual/en/function.stream-context-get-options.php.
No, I'm afraid not. At least not that I know of any, this is simply because PHP is a synchronous language, meaning you cannot "skip" tasks. (I.e. curl_exec() will always - no matter what - be executed until the request is completed)
I'm not sure abut the timeout, but you can get partial response using cURL by using the CURLOPT_WRITEFUNCTION flag:
curl_setopt($ch, CURLOPT_WRITEFUNCTION, $callback);
Where $ch is the Curl handler, and $callback is the callback function name. This command will stream response data from remote site. The callback function can look something like:
$result = '';
$callback = function ($ch, $str) {
global $result;
//$str has the chunks of data streamed back.
$result .= $str;
// here you can mess with the stream data either with $result or $str.
// i.e. look for the "Job waiting" string and terminate the response.
return strlen($str);//don't touch this
};
If not interrupted at the end $result will contain all the response from remote site.
So combining everything will look something like:
$result = '';
$callback = function ($ch, $str) {
global $result;
//$str has the chunks of data streamed back.
$result .= $str;
// here you can mess with the stream data either with $result or $str.
// i.e. look for the "Job waiting" string and terminate the response.
return strlen($str);//don't touch this
};
$url = "partial_response.php";
$sac_curl = curl_init();
curl_setopt($sac_curl, CURLOPT_HTTPGET, true);
curl_setopt($sac_curl, CURLOPT_URL, $url);
curl_setopt($sac_curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($sac_curl, CURLOPT_HEADER, false);
curl_setopt($sac_curl, CURLOPT_TIMEOUT, 11);
curl_setopt($ch, CURLOPT_WRITEFUNCTION, $callback);
curl_exec($sac_curl); // the response is now in $result.
curl_close($sac_curl);
echo $result;
I am using cUrl in PHP to request from some external service.
Interesting enough, the server is responding with raw "multipart/form-data" instead of binary file data.
My website is using a shared hosting, therefore PECL HTTP is not an option.
Is there a way to parse this data with PHP?
Sample code:
$response = curl_exec($cUrl);
/* $response is raw "multipart/form-data" string
--MIMEBoundaryurn_uuid_DDF2A2C71485B8C94C135176149950475371
Content-Type: application/xop+xml; charset=utf-8; type="text/xml"
Content-Transfer-Encoding: binary
(xml data goes here)
--MIMEBoundaryurn_uuid_DDF2A2C71485B8C94C135176149950475371
Content-Type: application/zip
Content-Transfer-Encoding: binary
(binary file data goes here)
*/
EDIT: I tried piping the response to a localhost HTTP request, but the respond data is likely to exceed the allowed memory size in PHP process. Expending mem limit is not very practical, this action also dramatically reduces the server performance dramatically.
If there is no alternatives to the original question, you may suggest a way to handle very large POST requests, along with XML parsing, in terms of streams in PHP.
I know this would be hard, please comment. I am open for discussions.
if you need the zip file from the response I guess you could just write a tmp file to save the curl response to, and stream that as a workaround:
Never tried that with multipart curls, but I guess it should work.
$fh = fopen('/tmp/foo', 'w');
$cUrl = curl_init('http://example.com/foo');
curl_setopt($cUrl, CURLOPT_FILE, $fh); // redirect output to filehandle
curl_exec($cUrl);
curl_close($cUrl);
fclose($fh); // close filehandle or the file will be corrupted
if you do NOT need anything but the xml part of the response you might want to disable headers
curl_setopt($cUrl, CURLOPT_HEADER, FALSE);
and add option to only accept xml as a response
curl_setopt($cUrl, CURLOPT_HTTPHEADER, array('Accept: application/xml'));
//That's a workaround since there is no available curl option to do so but http allows that
[EDIT]
A Shot in the dark...
can you test with these curlopt settings to see if modifiying these help anything
$headers = array (
'Content-Type: multipart/form-data; boundary=' . $boundary,
'Content-Length: ' . strlen($requestBody),
'X-EBAY-API-COMPATIBILITY-LEVEL: ' . $compatLevel, // API version
'X-EBAY-API-DEV-NAME: ' . $devID,
'X-EBAY-API-APP-NAME: ' . $appID,
'X-EBAY-API-CERT-NAME: ' . $certID,
'X-EBAY-API-CALL-NAME: ' . $verb,
'X-EBAY-API-SITEID: ' . $siteID,
);
$cUrl = curl_init();
curl_setopt($cUrl, CURLOPT_URL, $serverUrl);
curl_setopt($cUrl, CURLOPT_TIMEOUT, 30 );
curl_setopt($cUrl, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($cUrl, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($cUrl, CURLOPT_HTTPHEADER, $headers);
curl_setopt($cUrl, CURLOPT_POST, 1);
curl_setopt($cUrl, CURLOPT_POSTFIELDS, $requestBody);
curl_setopt($cUrl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($cUrl, CURLOPT_FAILONERROR, 0 );
curl_setopt($cUrl, CURLOPT_FOLLOWLOCATION, 1 );
curl_setopt($cUrl, CURLOPT_HEADER, 0 );
curl_setopt($cUrl, CURLOPT_USERAGENT, 'ebatns;xmlstyle;1.0' );
curl_setopt($cUrl, CURLOPT_HTTP_VERSION, 1 ); // HTTP version must be 1.0
$response = curl_exec($cUrl);
if ( !$response ) {
print "curl error " . curl_errno($cUrl ) . PHP_EOL;
}
curl_close($cUrl);
[EDIT II]
This is just a try, as mentioned I cannot get my curled pages to respond with a multipart form data. So be gentle with me here ;)
$content_type = ""; //use last know content-type as a trigger
$tmp_cnt_file = "tmp/tmpfile";
$xml_response = ""; // this will hold the "usable" curl response
$hidx = 0; //header index.. counting the number of different headers received
function read_header($cUrl, $string)// this will be called once for every line of each header received
{
global $content_type, $hidx;
$length = strlen($string);
if (preg_match('/Content-Type:(.*)/', $string, $match))
{
$content_type = $match[1];
$hidx++;
}
/*
should set $content_type to 'application/xop+xml; charset=utf-8; type="text/xml"' for the first
and to 'application/zip' for the second response body
echo "Header: $string<br />\n";
*/
return $length;
}
function read_body($cUrl, $string)
{
global $content_header, $xml_response, $tmp_cnt_file, $hidx;
$length = strlen($string);
if(stripos ( $content_type , "xml") !== false)
$xml_response .= $string;
elseif(stripos ($content_type, "zip") !== false)
{
$handle = fopen($tmp_cnt_file."-".$hidx.".zip", "a");
fwrite($handle, $string);
fclose($handle);
}
/*
elseif {...} else{...}
depending on your needs
echo "Received $length bytes<br />\n";
*/
return $length;
}
and of course set the proper curlopts
// Set callback function for header
curl_setopt($cUrl, CURLOPT_HEADERFUNCTION, 'read_header');
// Set callback function for body
curl_setopt($cUrl, CURLOPT_WRITEFUNCTION, 'read_body');
don't forget to NOT save the curl response to a variable because of the memory issues,
hopefully all you need will be in the $xml_response above anyways.
//$response = curl_exec($cUrl);
curl_exec($cUrl);
And for parsing your code you can refer to $xml_response and the temp files you created starting with tmp/tmpfile-2 in this scenario. Again, I have not been able to test the code above in any way. So this might not work (but it should imho ;))
[EDIT III]
Say we want curl to write all incoming data directly to another (outgoing) stream, in this case a socket connection
I'm not sure if it is as easy as this:
$fs = fsockopen($host, $port, $errno, $errstr);
$cUrl = curl_init('http://example.com/foo');
curl_setopt($cUrl, CURLOPT_FILE, $fs); // redirect output to sockethandle
curl_exec($cUrl);
curl_close($cUrl);
fclose($fs); // close handle
else we will have to use our known write and header functions with just a little trick
//first open the socket (before initiating curl)
$fs = fsockopen($host, $port, $errno, $errstr);
// now for the new callback function
function socket_pipe($cUrl, $string)
{
global $fs;
$length = strlen($string);
fputs($fs, $string); // add NOTHING to the received line just send it to $fs; that was easy wasn't it?
return $length;
}
// and of course for the CURLOPT part
// Set callback function for header
curl_setopt($cUrl, CURLOPT_HEADERFUNCTION, 'socket_pipe');
// Set the same callback function for body
curl_setopt($cUrl, CURLOPT_WRITEFUNCTION, 'socket_pipe');
// do not forget to
fclose($fs); //when we're done
The thing is, not editing the result and simply piping it to $fs will make it necessary that apache is listening on a certain port which you then assign your script to.
Or you will need to add ONE header line directly after fsockopen
fputs($fp, "POST $path HTTP/1.0\n"); //where path is your script of course
I'm sorry i can't help much because you did not put much code but i remember i was having a similar issue when i was playing with curl_setopt options.
Did you use CURLOPT_BINARYTRANSFER?
From php documentation -> CURLOPT_BINARYTRANSFER-> TRUE to return the raw output when CURLOPT_RETURNTRANSFER is used.
just set CURLOPT_RETURNTRANSFER CURLOPT_POST
$c = curl_init($url);
curl_setopt($c, CURLOPT_RETURNTRANSFER, true);
curl_setopt($c, CURLOPT_CONNECTTIMEOUT, 1);
curl_setopt($c, CURLOPT_TIMEOUT, 1);
curl_setopt($c, CURLOPT_POST, 1);
curl_setopt($c, CURLOPT_POSTFIELDS,
array());
$rst_str = curl_exec($c);
curl_close($c);
You can re-assemble you Binary data doing something like this, I hope it helps.
$file_array = explode("\n\r", $file, 2);
$header_array = explode("\n", $file_array[0]);
foreach($header_array as $header_value) {
$header_pieces = explode(':', $header_value);
if(count($header_pieces) == 2) {
$headers[$header_pieces[0]] = trim($header_pieces[1]);
}
}
header('Content-type: ' . $headers['Content-Type']);
header('Content-Disposition: ' . $headers['Content-Disposition']);
echo substr($file_array[1], 1);
If you don't need binary data, have you tried below?
curl_setopt($c, CURLOPT_NOBODY, true);
Is there any way to get both headers and body for a cURL request using PHP? I found that this option:
curl_setopt($ch, CURLOPT_HEADER, true);
is going to return the body plus headers, but then I need to parse it to get the body. Is there any way to get both in a more usable (and secure) way?
Note that for "single request" I mean avoiding issuing a HEAD request prior of GET/POST.
One solution to this was posted in the PHP documentation comments: http://www.php.net/manual/en/function.curl-exec.php#80442
Code example:
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
// ...
$response = curl_exec($ch);
// Then, after your curl_exec call:
$header_size = curl_getinfo($ch, CURLINFO_HEADER_SIZE);
$header = substr($response, 0, $header_size);
$body = substr($response, $header_size);
Warning: As noted in the comments below, this may not be reliable when used with proxy servers or when handling certain types of redirects. #Geoffrey's answer may handle these more reliably.
Many of the other solutions offered this thread are not doing this correctly.
Splitting on \r\n\r\n is not reliable when CURLOPT_FOLLOWLOCATION is on or when the server responds with a 100 code RFC-7231, MDN.
Not all servers are standards compliant and transmit just a \n for new lines (and a recipient may discard the \r in the line terminator) Q&A.
Detecting the size of the headers via CURLINFO_HEADER_SIZE is also not always reliable, especially when proxies are used Curl-1204 or in some of the same redirection scenarios.
The most correct method is using CURLOPT_HEADERFUNCTION.
Here is a very clean method of performing this using PHP closures. It also converts all headers to lowercase for consistent handling across servers and HTTP versions.
This version will retain duplicated headers
This complies with RFC822 and RFC2616, please do not make use of the mb_ (and similar) string functions, it is a not only incorrect but even a security issue RFC-7230!
$ch = curl_init();
$headers = [];
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
// this function is called by curl for each header received
curl_setopt($ch, CURLOPT_HEADERFUNCTION,
function($curl, $header) use (&$headers)
{
$len = strlen($header);
$header = explode(':', $header, 2);
if (count($header) < 2) // ignore invalid headers
return $len;
$headers[strtolower(trim($header[0]))][] = trim($header[1]);
return $len;
}
);
$data = curl_exec($ch);
print_r($headers);
Curl has a built in option for this, called CURLOPT_HEADERFUNCTION. The value of this option must be the name of a callback function. Curl will pass the header (and the header only!) to this callback function, line-by-line (so the function will be called for each header line, starting from the top of the header section). Your callback function then can do anything with it (and must return the number of bytes of the given line). Here is a tested working code:
function HandleHeaderLine( $curl, $header_line ) {
echo "<br>YEAH: ".$header_line; // or do whatever
return strlen($header_line);
}
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://www.google.com");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADERFUNCTION, "HandleHeaderLine");
$body = curl_exec($ch);
The above works with everything, different protocols and proxies too, and you dont need to worry about the header size, or set lots of different curl options.
P.S.: To handle the header lines with an object method, do this:
curl_setopt($ch, CURLOPT_HEADERFUNCTION, array($object, 'methodName'))
is this what are you looking to?
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Expect:'));
$response = curl_exec($ch);
list($header, $body) = explode("\r\n\r\n", $response, 2);
If you specifically want the Content-Type, there's a special cURL option to retrieve it:
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$response = curl_exec($ch);
$content_type = curl_getinfo($ch, CURLINFO_CONTENT_TYPE);
Just set options :
CURLOPT_HEADER, 0
CURLOPT_RETURNTRANSFER, 1
and use curl_getinfo with CURLINFO_HTTP_CODE (or no opt param and you will have an associative array with all the informations you want)
More at : http://php.net/manual/fr/function.curl-getinfo.php
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
$parts = explode("\r\n\r\nHTTP/", $response);
$parts = (count($parts) > 1 ? 'HTTP/' : '').array_pop($parts);
list($headers, $body) = explode("\r\n\r\n", $parts, 2);
Works with HTTP/1.1 100 Continue before other headers.
If you need work with buggy servers which sends only LF instead of CRLF as line breaks you can use preg_split as follows:
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
$parts = preg_split("#\r?\n\r?\nHTTP/#u", $response);
$parts = (count($parts) > 1 ? 'HTTP/' : '').array_pop($parts);
list($headers, $body) = preg_split("#\r?\n\r?\n#u", $parts, 2);
My way is
$response = curl_exec($ch);
$x = explode("\r\n\r\n", $v, 3);
$header=http_parse_headers($x[0]);
if ($header=['Response Code']==100){ //use the other "header"
$header=http_parse_headers($x[1]);
$body=$x[2];
}else{
$body=$x[1];
}
If needed apply a for loop and remove the explode limit.
Here is my contribution to the debate ... This returns a single array with the data separated and the headers listed. This works on the basis that CURL will return a headers chunk [ blank line ] data
curl_setopt($ch, CURLOPT_HEADER, 1); // we need this to get headers back
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_VERBOSE, true);
// $output contains the output string
$output = curl_exec($ch);
$lines = explode("\n",$output);
$out = array();
$headers = true;
foreach ($lines as $l){
$l = trim($l);
if ($headers && !empty($l)){
if (strpos($l,'HTTP') !== false){
$p = explode(' ',$l);
$out['Headers']['Status'] = trim($p[1]);
} else {
$p = explode(':',$l);
$out['Headers'][$p[0]] = trim($p[1]);
}
} elseif (!empty($l)) {
$out['Data'] = $l;
}
if (empty($l)){
$headers = false;
}
}
The problem with many answers here is that "\r\n\r\n" can legitimately appear in the body of the html, so you can't be sure that you're splitting headers correctly.
It seems that the only way to store headers separately with one call to curl_exec is to use a callback as is suggested above in https://stackoverflow.com/a/25118032/3326494
And then to (reliably) get just the body of the request, you would need to pass the value of the Content-Length header to substr() as a negative start value.
Just in case you can't / don't use CURLOPT_HEADERFUNCTION or other solutions;
$nextCheck = function($body) {
return ($body && strpos($body, 'HTTP/') === 0);
};
[$headers, $body] = explode("\r\n\r\n", $result, 2);
if ($nextCheck($body)) {
do {
[$headers, $body] = explode("\r\n\r\n", $body, 2);
} while ($nextCheck($body));
}
A better way is to use the verbose CURL response which can be piped to a temporary stream. Then you can search the response for the header name. This could probably use a few tweaks but it works for me:
class genericCURL {
/**
* NB this is designed for getting data, or for posting JSON data
*/
public function request($url, $method = 'GET', $data = array()) {
$ch = curl_init();
if($method == 'POST') {
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POSTFIELDS, $string = json_encode($data));
}
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_VERBOSE, true);
//open a temporary stream to output the curl log, which would normally got to STDERR
$err = fopen("php://temp", "w+");
curl_setopt($ch, CURLOPT_STDERR, $err);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$server_output = curl_exec ($ch);
//rewind the temp stream and put it into a string
rewind($err);
$this->curl_log = stream_get_contents($err);
curl_close($ch);
fclose($err);
return $server_output;
}
/**
* use the curl log to get a header value
*/
public function getReturnHeaderValue($header) {
$log = explode("\n", str_replace("\r\n", "\n", $this->curl_log));
foreach($log as $line) {
//is the requested header there
if(stripos($line, '< ' . $header . ':') !== false) {
$value = trim(substr($line, strlen($header) + 3));
return $value;
}
}
//still here implies not found so return false
return false;
}
}
Improvement of Geoffreys answer:
I couldn't get the right length for header with $headerSize = curl_getinfo($this->curlHandler, CURLINFO_HEADER_SIZE);- i had to calculate header size on my own.
In addition some improvements for better readability.
$headerSize = 0;
curl_setopt_array($this->curlHandler, [
CURLOPT_URL => $yourUrl,
CURLOPT_POST => 0,
CURLOPT_RETURNTRANSFER => 1,
// this function is called by curl for each header received
CURLOPT_HEADERFUNCTION =>
function ($curl, $header) use (&$headers, &$headerSize) {
$lenghtCurrentLine = strlen($header);
$headerSize += $lenghtCurrentLine;
$header = explode(':', $header, 2);
if (count($header) > 1) { // store only vadid headers
$headers[strtolower(trim($header[0]))][] = trim($header[1]);
}
return $lenghtCurrentLine;
},
]);
$fullResult = curl_exec($this->curlHandler);
$result = substr($fullResult, $headerSize);
Return response headers with a reference parameter:
<?php
$data=array('device_token'=>'5641c5b10751c49c07ceb4',
'content'=>'测试测试test'
);
$rtn=curl_to_host('POST', 'http://test.com/send_by_device_token', array(), $data, $resp_headers);
echo $rtn;
var_export($resp_headers);
function curl_to_host($method, $url, $headers, $data, &$resp_headers)
{$ch=curl_init($url);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $GLOBALS['POST_TO_HOST.LINE_TIMEOUT']?$GLOBALS['POST_TO_HOST.LINE_TIMEOUT']:5);
curl_setopt($ch, CURLOPT_TIMEOUT, $GLOBALS['POST_TO_HOST.TOTAL_TIMEOUT']?$GLOBALS['POST_TO_HOST.TOTAL_TIMEOUT']:20);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
curl_setopt($ch, CURLOPT_HEADER, 1);
if ($method=='POST')
{curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($data));
}
foreach ($headers as $k=>$v)
{$headers[$k]=str_replace(' ', '-', ucwords(strtolower(str_replace('_', ' ', $k)))).': '.$v;
}
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
$rtn=curl_exec($ch);
curl_close($ch);
$rtn=explode("\r\n\r\nHTTP/", $rtn, 2); //to deal with "HTTP/1.1 100 Continue\r\n\r\nHTTP/1.1 200 OK...\r\n\r\n..." header
$rtn=(count($rtn)>1 ? 'HTTP/' : '').array_pop($rtn);
list($str_resp_headers, $rtn)=explode("\r\n\r\n", $rtn, 2);
$str_resp_headers=explode("\r\n", $str_resp_headers);
array_shift($str_resp_headers); //get rid of "HTTP/1.1 200 OK"
$resp_headers=array();
foreach ($str_resp_headers as $k=>$v)
{$v=explode(': ', $v, 2);
$resp_headers[$v[0]]=$v[1];
}
return $rtn;
}
?>
Try this if you are using GET:
$curl = curl_init($url);
curl_setopt_array($curl, array(
CURLOPT_URL => $url,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_ENCODING => "",
CURLOPT_MAXREDIRS => 10,
CURLOPT_TIMEOUT => 30,
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
CURLOPT_CUSTOMREQUEST => "GET",
CURLOPT_HTTPHEADER => array(
"Cache-Control: no-cache"
),
));
$response = curl_exec($curl);
curl_close($curl);
If you don't really need to use curl;
$body = file_get_contents('http://example.com');
var_export($http_response_header);
var_export($body);
Which outputs
array (
0 => 'HTTP/1.0 200 OK',
1 => 'Accept-Ranges: bytes',
2 => 'Cache-Control: max-age=604800',
3 => 'Content-Type: text/html',
4 => 'Date: Tue, 24 Feb 2015 20:37:13 GMT',
5 => 'Etag: "359670651"',
6 => 'Expires: Tue, 03 Mar 2015 20:37:13 GMT',
7 => 'Last-Modified: Fri, 09 Aug 2013 23:54:35 GMT',
8 => 'Server: ECS (cpm/F9D5)',
9 => 'X-Cache: HIT',
10 => 'x-ec-custom-error: 1',
11 => 'Content-Length: 1270',
12 => 'Connection: close',
)'<!doctype html>
<html>
<head>
<title>Example Domain</title>...
See http://php.net/manual/en/reserved.variables.httpresponseheader.php