How to remove HTTP headers from CURL response? - php

I have a php script that returns just plain text without any html. Now I want to make a cURL request to that script and I get the following response:
HTTP/1.1 200 OK
Date: Mon, 28 Feb 2011 14:21:51 GMT
Server: Apache/2.2.14 (Ubuntu)
X-Powered-By: PHP/5.2.12-nmm2
Vary: Accept-Encoding
Content-Length: 6
Content-Type: text/html
6.8320
The actuall response is just 6.8320 as text without any html. I want to retrieve it from the response above by just removing the header information.
I already minified the script a bit:
$url = $_GET['url'];
if ( !$url ) {
// Passed url not specified.
$contents = 'ERROR: url not specified';
$status = array( 'http_code' => 'ERROR' );
} else if ( !preg_match( $valid_url_regex, $url ) ) {
// Passed url doesn't match $valid_url_regex.
$contents = 'ERROR: invalid url';
$status = array( 'http_code' => 'ERROR' );
} else {
$ch = curl_init( $url );
if ( strtolower($_SERVER['REQUEST_METHOD']) == 'post' ) {
curl_setopt( $ch, CURLOPT_POST, true );
curl_setopt( $ch, CURLOPT_POSTFIELDS, $_POST );
}
curl_setopt( $ch, CURLOPT_FOLLOWLOCATION, true );
curl_setopt( $ch, CURLOPT_HEADER, true );
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, true );
curl_setopt( $ch, CURLOPT_USERAGENT, $_GET['user_agent'] ? $_GET['user_agent'] : $_SERVER['HTTP_USER_AGENT'] );
list( $header, $contents ) = preg_split( '/([\r\n][\r\n])\\1/', curl_exec( $ch ), 2 );
$status = curl_getinfo( $ch );
curl_close( $ch );
}
// Split header text into an array.
$header_text = preg_split( '/[\r\n]+/', $header );
if ( true ) {
if ( !$enable_native ) {
$contents = 'ERROR: invalid mode';
$status = array( 'http_code' => 'ERROR' );
}
// Propagate headers to response.
foreach ( $header_text as $header ) {
if ( preg_match( '/^(?:Content-Type|Content-Language|Set-Cookie):/i', $header ) ) {
header( $header );
}
}
print $contents;
}
Any idea what I need to change to remove the header information from the response?

Just set CURLOPT_HEADER to false.

Make sure you put set the header flag:
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, true );
curl_setopt($ch, CURLOPT_TIMEOUT, Constants::HTTP_TIMEOUT);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, Constants::HTTP_TIMEOUT);
$response = curl_exec($ch);
Do this after your curl call:
$header_size = curl_getinfo($ch, CURLINFO_HEADER_SIZE);
$headerstring = substr($response, 0, $header_size);
$body = substr($response, $header_size);
EDIT:
If you'd like to have header in assoc array, add something like this:
$headerArr = explode(PHP_EOL, $headerstring);
foreach ($headerArr as $headerRow) {
preg_match('/([a-zA-Z\-]+):\s(.+)$/',$headerRow, $matches);
if (!isset($matches[0])) {
continue;
}
$header[$matches[1]] = $matches[2];
}
Result print_r($header):
(
[content-type] => application/json
[content-length] => 2848
[date] => Tue, 06 Oct 2020 10:29:33 GMT
[last-modified] => Tue, 06 Oct 2020 10:17:17 GMT
)
Don't forget to close connection curl_close($ch);

Update the value of CURLOPT_HEADER to 0 for false
curl_setopt($ch, CURLOPT_HEADER, 0);

Just for a later use if anyone else needs. I was into same situation, but just need to remove header text, not content. The response i was getting in the header was (including white space):
HTTP/1.1 200 OK
Cache-Control: private, no-cache, no-store, must-revalidate
Content-Language: en
Content-Type: text/html
Date: Tue, 25 Feb 2014 20:59:29 GMT
Expires: Sat, 01 Jan 2000 00:00:00 GMT
Pragma: no-cache
Server: nginx
Vary: Cookie, Accept-Language, Accept-Encoding
transfer-encoding: chunked
Connection: keep-alive
I wanted to remove starting from HTTP till keep-alive with white space:
$contents = preg_replace('/HTTP(.*)alive/s',"",$contents);
that did for me.

If you are using nuSoap, you can access data without headers with $nsoap->responseData or $nsoap->response, if you want the full headers.
Just in case someone needs that.

If for some reason you have to curl_setopt($ch, CURLOPT_HEADER, 1); to get cookies for example, the following worked for me. Not sure if it's 100% reliable but worth a try
$foo = preg_replace('/HTTP(.*)html/s',"",$curlresult);

$content = null;
$ch = curl_init();
$rs = curl_exec($ch);
if (CURLE_OK == curl_errno($ch)) {
$content = substr($rs, curl_getinfo($ch, CURLINFO_HEADER_SIZE));
}
curl_close($ch);
echo $content;

If someone already saved the curl response to a file (like me) and therefore don't know how big the header was to use substr, try:
$file = '/path/to/file/with/headers';
file_put_contents($file, preg_replace('~.*\r\n\r\n~s', '', file_get_contents($file)));

Just do not set the curl_header in the curl request or set it to z or false
like this
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_HEADER, false);

Just don't set CURLOPT_HEADER!

Related

PHP Curl - eBay CSV FileExchange

I've tried to send a CSV File to eBay FileExchange Service.
I'm writing an application to update a lot of products on eBay at the same time.
When I upload the test.csv by using the eBay CSV-Manager the update will be success, but with the script nothing will happens after post the data.
I've treid the following steps:
Create a separate token for FileExchange.
https://signin.ebay.de/ws/eBayISAPI.dll?SignIn&runame=F-FILEEXL51P1EHH6L899Q9B969GE134DK-FileUpload
Then I use the following script:
$token = 'AgAAAA**AQAAAA**aAAAAA************';
$ebay_url = 'https://bulksell.ebay.com/ws/eBayISAPI.dll?FileExchangeUpload';
$sendheaders = array(
'User-Agent: My Client App v1.0'
);
$fields = array(
'token' => $token,
'file' => '#test.csv'
);
$ch = curl_init($ebay_url);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_HEADER, 1); // set to 0 to eliminate header info from response
curl_setopt($ch, CURLOPT_NOBODY, 0); // set to 1 to eliminate body info from response
curl_setopt($ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_0); // use HTTP/1.0 instead of 1.1
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); // Returns response data instead of TRUE(1)
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE); // uncomment this line if you get no gateway response. ###
curl_setopt($ch, CURLOPT_HTTPHEADER, $sendheaders);
curl_setopt($ch, CURLOPT_POSTFIELDS, $fields); // use HTTP POST to send form data
$resp = curl_exec($ch); //execute post and get results
if(!curl_exec($ch)) {
die('Error: ' . curl_error($ch) . ' - Code: ' . curl_errno($ch));
}
curl_close ($ch);
I've used this csv File-format (test.csv)
Action;ItemID;DispatchTimeMax
Revise;28*********916;30
The results after post:
print_r($resp);
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Set-Cookie: dp1=bu1p/QEBfX0BAX19AQA**617d3da3^bl/DE617d3da3^; Domain=.ebay.com; Expires=Sat, 30-Oct-2021 12:42:11 GMT; Path=/
Set-Cookie: s=CgAD4ACBdvCgjMjFkNjZkMDcxNmUwYTBmMTc1MTA0ZmEwZmZmYjEyZWFY39RE; Domain=.ebay.com; Path=/
Set-Cookie: nonsession=CgADKACBhfT2jMjFkNjZkMDcxNmUwYTBmMTc1MTA0ZmEwZmZmYjEyZWIAywABXbrdqzHTwXKU; Domain=.ebay.com; Expires=Sat, 30-Oct-2021 12:42:11 GMT; Path=/
Cache-Control: private
Pragma: no-cache
Content-Type: text/html;charset=UTF-8
Content-Length: 731
Date: Thu, 31 Oct 2019 12:42:11 GMT
Connection: keep-alive
File upload successful. Your ref # is .
Close
Thanks for helping me.
I have found a solution:
Obviously at php 7.1 the # has no effect, and the file post ist empty to ebay.
I use the curl_file_create function and it's work.
if (!function_exists('curl_file_create'))
{
function curl_file_create($filename, $mimetype = '')
{
return "#$filename;filename="
. ($mimetype ? ";type=$mimetype" : '');
}
}
$fields = array(
"token" => $token,
"file" => curl_file_create ($_GET['filename'], 'text/csv')
);
Hope that help's anybody.

Need to get error code alone from header and also the json response in MS Dynamics web api in php

I'm using Prefer:return=representation to get specific fields while creating accounts
API URL
https://xxxxxx-xxxx-xxxxx-xxxx/api/data/v9.0/accounts?$select=accountid,name,telephone1.
Request
$params['firstname'] = $first_name;
$params['lastname'] = $last_name;
$authHeader = 'Authorization: Bearer ' . $access_token;
$headers = array (
$authHeader,
'Content-Type:application/json',
'Prefer:return=representation', //NEWLY ADDED TO GET SPECIFIC
FIELDS
'Accept:application/json;'
);
$data_string = json_encode ( $params );
$custom_vars [CURLOPT_VERBOSE] = true;
$custom_vars [CURLOPT_HEADER] = true;
customLogs('case_create.log',print_r($data_string,true),3);
$result = gUnifyCURLCall ( $url, 'POST', $data_string, $headers, true, false, $custom_vars );
function gUnifyCURLCall($url, $method_type = 'GET', $data_string = null, $header = null, $ssl_verify = true,
$tls_version = false, $custom_vars = array()) {
$ch = curl_init ();
curl_setopt ( $ch, CURLOPT_URL, $url );
curl_setopt ( $ch, CURLOPT_RETURNTRANSFER, TRUE );
curl_setopt ( $ch, CURLOPT_CUSTOMREQUEST, $method_type );
if ($data_string != null) {
curl_setopt ( $ch, CURLOPT_POSTFIELDS, $data_string );
}
if ($header != null) {
curl_setopt ( $ch, CURLOPT_HTTPHEADER, $header );
}
if(!empty($custom_vars)) {
foreach ( $custom_vars as $key=>$val) {
curl_setopt($ch, trim($key), 1);
}
}
$result = curl_exec ( $ch );
curl_close ( $ch );
return $result;
}
I want to get server error code and also the response, so i'm passing CURLOPT_VERBOSE:TRUE,CURLOPT_HEADER:TRUE in the header. It returns following response from MS Dynamics.
HTTP/1.1 201 Created
Cache-Control: no-cache
Allow: OPTIONS,GET,HEAD,POST
Content-Type: application/json; odata.metadata=minimal
Expires: -1
Server:
x-ms-service-request-id: 85aed055-abff-42b0-a9bd-a4ad617638b1
REQ_ID: 85aed055-abff-42b0-a9bd-a4ad617638b1
AuthActivityId: xxxxxxxxxxxxxxx
Preference-Applied: return=representation
x-ms-ratelimit-burst-remaining-xrm-requests: 5995
x-ms-ratelimit-time-remaining-xrm-requests: 1,199.79
OData-Version: 4.0
Public: OPTIONS,GET,HEAD,POST
Set-Cookie: ApplicationGatewayAffinity=69ed338020af0cda5b08cef8314523419f00ed630d46bde35fe61b31f28285cf;Path=/;Domain=xxxxxxxx.xxxxxxx.dynamics.com
Date: Wed, 10 Jul 2019 10:43:48 GMT
Content-Length: 244
{"#odata.context":"https://xxxxxxxxxx.xxxxx.dynamics.com/api/data/v9.0/$metadata#accounts(accountid,name,fax)/$entity","#odata.etag":"W/\"11660448\"","accountid":"xxxxxxxxxx-xxxx-xxxx-xxxxxxx","name":"xxxxxx","fax":"9500293527"}
)
previously we were only working with the response code in the header to get the entity_id,now MS Dynamics web api is updated ,so now we want the response code from header and also the response using PHP

PHP - curl post view request body (parameters)

I get an error response for missing parameter when posting cURL POST method,
I'm adding an array of parameters to CURLOPT_POSTFIELDS the following way:
$service = "AutoInsuranceFormPostService";
$method = "autoInsurancePublisherFormPost";
$userAgent = "Mozilla%2F5.0+%28Linux%3B+Android+4.4.4%3B+Z752C+Build%2FKTU84P%29+AppleWebKit%2F537.36+%28KHTML%2C+like+Gecko%29+Chrome%2F36.0.1985.135+Mobile+Safari%2F537.36";
$payload = $encodedPayLoad;
$parameters = array (
'service' => $service,
'method' => $method,
'UserAgent' => $userAgent,
'payload' => $payload
);
With:
curl_setopt($ch,CURLOPT_POSTFIELDS,$parameters);
Since the response is saying missing parameter "service", I figured I need to debug the request body.
I managed to get the headers with:
curl_getinfo($ch)
I also attempted to use:
curl_setopt($ch, CURLOPT_VERBOSE, true);
But unfortunately in both cases I only got the headers and not the body (the parameters values).
Full curl execution function:
function openurl($url, $postvars) {
$ch=curl_init();
curl_setopt($ch, CURLOPT_VERBOSE, true);
$verbose = fopen('php://temp', 'w+');
curl_setopt($ch, CURLOPT_STDERR, $verbose);
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_POST, TRUE);
curl_setopt($ch,CURLOPT_POSTFIELDS,$postvars);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: application/x-www-form-urlencoded'));
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch,CURLOPT_TIMEOUT, '3');
$result = curl_exec($ch);
rewind($verbose);
$verboseLog = stream_get_contents($verbose);
echo "Verbose information:\n<pre>", htmlspecialchars($verboseLog), "</pre>\n";
return $result;
}
Verbos information:
Content-Length: 6659
Expect: 100-continue
Content-Type: application/x-www-form-urlencoded; boundary=------------------------45b2d9f6776306b0
< HTTP/1.1 100 Continue
< HTTP/1.1 200 OK
< Date: Thu, 12 Jul 2018 16:32:52 GMT
< Server: Apache
< Cache-Control: public
< ORIGIN: S_CACHE
< Vary: User-Agent,Accept-Encoding
< Set-Cookie: _qs_origin=s-cache; path=/;
< Set-Cookie: _qs_deviceType=; path=/;
< Content-Length: 141
< Content-Type: application/json;charset=ISO-8859-1
<
This output is useless for me since I cannot see how the parameters were sent and those cannot fix their format.
The response I get is:
{"Status":"Fail","StatusCode":"400","ResponseMessage":"\"service\" parameter empty! || \"method\" parameter empty! ","SkipMatchingFlag":"No"}
I have been searching for a solution all day long, I've seen a ton of answers on "How to see the RESPONSE body", and "How to see the request HEADERS".
But none for "How to see the request body", so any help would be much appreciated,
Best regards.

Subscribe using Superfeedr PubSubHubbub generating error hub.topic not found

I want to integrate Superfeedr API using PubSubHubbub in PHP. I am following this and my code is:
<?php
require_once('Superfeedr.class.php')
$superfeedr = new Superfeedr('http://push-pub.appspot.com/feed',
'http://mycallback.tld/push?feed=http%3A%2F%2Fpush-pub.appspot.com%2Ffeed',
'http://wallabee.superfeedr.com');
$superfeedr->verbose = true;
$superfeedr->subscribe();
?>
And my subscribe() function is
public function subscribe()
{
$this->request('subscribe');
}
private function request($mode)
{
$data = array();
$data['topic'] = $this->topic;
$data['callback'] = $this->callback;
$post_data = array (
"hub.mode" => 'subscribe',
"hub.verify" => "sync",
"hub.callback" => urlencode($this->callback),
"hub.topic" => urlencode($this->topic),
"hub.verify_token" => "26550615cbbed86df28847cec06d3769",
);
//echo "<pre>"; print_r($post_data); exit;
// url-ify the data for the POST
foreach ($post_data as $key=>$value) {
$post_data_string .= $key.'='. $value.'&';
}
rtrim($fields_string,'&');
// curl request
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $this->hub);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_data_string);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Accept: application/json'));
curl_setopt($ch, CURLOPT_USERPWD, 'USERNAME:PASSWORD');
$output = curl_exec($ch);
if ($this->verbose) {
print('<pre>');
print_r($output);
print('</pre>');
}
}
But after execution I am getting this error
HTTP/1.1 422 Unprocessable Entity
X-Powered-By: The force, Luke
Vary: X-HTTP-Method-Override, Accept-Encoding
Content-Type: text/plain; charset=utf-8
X-Superfeedr-Host: supernoder16.superfeedr.com
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
Access-Control-Allow-Methods: GET, POST, PUT, DELETE
Access-Control-Allow-Headers: Authorization
Content-Length: 97
ETag: W/"61-db6269b5"
Date: Wed, 24 Aug 2016 14:01:47 GMT
Connection: close
Please provide a valid hub.topic (feed) URL that is accepted on this hub. The hub does not match.
Same data (topic and callback etc..) requesting from https://superfeedr.com/users/testdata/push_console
is working fine. But I don't know why I am getting this error on my local. If anyone has any experienced with same problom then please help me. Thanks.
You are using a strange hub URL. You should use HTTPS://push.superfeedr.com in the last param of your class constructor.

php - cURL response header different than browser response header

I want to send a request using cURL and retrieve the response header.
Using a browser the response header is as follow:
HTTP/1.0 302 Moved Temporarily
Content-Type: text/html; charset=utf-8
Cache-Control: no-cache
Location: "Correct URL"
Expires: Fri, 01 Jan 1990 00:00:00 GMT
Date: Tue, 30 Oct 2012 08:32:24 GMT
Server: Google Frontend
Content-Length: 0
But when I send the request using cURL the response header is as follow:
HTTP/1.1 302 Found
Content-Type: text/html; charset=utf-8
Cache-Control: no-cache
Location: "Wrong URL"
Expires: Fri, 01 Jan 1990 00:00:00 GMT
Date: Tue, 30 Oct 2012 09:12:14 GMT
Server: Google Frontend
Content-Length: 0
I want to know what is causing the response to return different URLs. This is a small php sample out of many samples and things I tried with no avail.
<?php
$url = "url";
$ch = curl_init( $url );
curl_setopt( $ch, CURLOPT_FOLLOWLOCATION, true );
curl_setopt( $ch, CURLOPT_HEADER, true );
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, true );
curl_setopt( $ch, CURLOPT_COOKIEJAR, "cookie.txt" );
curl_setopt( $ch, CURLOPT_COOKIEFILE, "cookie.txt" );
curl_setopt( $ch, CURLOPT_USERAGENT, "Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.4) Gecko/20091030 Gentoo Firefox/3.5.4" );
list( $header, $contents ) = preg_split( '/([\r\n][\r\n])\\1/', curl_exec( $ch ), 2 );
curl_close( $ch );
$header_text = preg_split( '/[\r\n]+/', $header );
foreach ( $header_text as $headers ) {
echo $headers . "</br>";
}
?>
There is some difference between the requests sent through the browser and through curl (almost certainly in the HTTP headers) that causes the difference in the responses.
You should capture the request from the browser (perhaps using an HTTP proxy like Fiddler for convenience) and compare its headers to those from your curl request. One (or more) of the differences you will find is the reason for what you are seeing.

Categories