PHP's curl doesn't seem to include the Host header when doing a request:
<?php
$handle = curl_init('http://example.com/');
curl_setopt_array($handle, array(
CURLOPT_USERAGENT => 'test ua',
CURLOPT_REFERER => 'http://example.org/',
CURLOPT_RETURNTRANSFER => true,
CURLOPT_PROXY => '127.0.0.1:8080',
CURLOPT_PROXYTYPE => CURLPROXY_HTTP,
//Doesn't work if you specify it manually or leave it out:
CURLOPT_HTTPHEADER => array(
'Host: example.com'
)
));
print_r(curl_getinfo($handle));
echo curl_exec($handle);
?>
Host header is defined by the argument of curl_init()
The Host header is already automatically by PHP when you specify the request URL.
The following example oughta be sufficient confirmation:
<?php
$handle = curl_init('http://pgl.yoyo.org/http/browser-headers.php');
curl_setopt($handle, CURLOPT_RETURNTRANSFER, true);
echo curl_exec($handle);
?>
The output HTML lists the request headers, among which we see Host: pgl.yoyo.org, as expected.
This was a totally unrelated bug, which happened by having a linebreak inside the user-agent, eg.:
curl_setopt_array($handle, array(
CURLOPT_USERAGENT => "user-agent\n"
));
Related
I have been able to use the Advanced Rest Client Extension for chrome to send POST queries to an specific HTTPS server and I get Status Code: 200 - OK with the same body fields as the ones I used in this code, but when I run the following code I get this response: 403 - Access Denied.
<?php
$postData = array(
'type' => 'credentials',
'id' => 'exampleid',
'secret_key' => 'gsdDe32dKa'
);
// Setup cURL
$ch = curl_init('https://www.mywebsite.com/oauth/token');
curl_setopt_array($ch, array(
CURLOPT_POST => TRUE,
CURLOPT_RETURNTRANSFER => TRUE,
CURLOPT_HTTPHEADER => array('Content-Type: application/x-www-form-urlencoded'
),
CURLOPT_POSTFIELDS => json_encode($postData)
));
// Send the request
$response = curl_exec($ch);
var_dump($response);
// Check for errors
if($response === FALSE){
die(curl_error($ch));
}
// Decode the response
$responseData = json_decode($response, TRUE);
// Print the date from the response
echo $responseData['published'];
?>
I've noticed as well that when I use Advanced Rest Client Extension for chrome and if I set the Content-Type to application/json I have to enter a login and a password that I don't know what are those because even if I enter the id and secret key that I have in the code it returns 401 Unauthorized. So I'm guessing this code that I wrote is not forcing it to the content-type: application/x-www-form-urlencoded, but I'm not sure. Thank you for any help on this issue!
Can you try like that and see if it helps:
curl_setopt_array($ch, array(
CURLOPT_POST => TRUE,
CURLOPT_RETURNTRANSFER => TRUE,
CURLOPT_COOKIEFILE => 'cookie.txt',
CURLOPT_COOKIEJAR => 'cookie.txt',
CURLOPT_USERPWD => 'username:password', //Your credentials goes here
CURLOPT_HTTPHEADER => array('Content-Type: application/x-www-form-urlencoded'),
CURLOPT_POSTFIELDS => http_build_query($postData),
));
I guess the site expect simple authentication on top of the secret_key that you already provided.
Also it is possible to send a Cookie, so just in case it is good idea to store it and use it again in the next Curl calls.
When I do
$h = get_headers('http://www.weebly.com');
It works just fine... The headers for that page are promptly returned.
But if I try to retrieve the headers via an explicit HEAD request using curl...
$url = 'http://www.weebly.com';
$request_headers = array(
'Connection: close',
);
$user_agent = 'curl/7.19.7 (x86_64-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3 libidn/1.15';
$ch = curl_init($url);
curl_setopt_array($ch, array(
CURLOPT_CUSTOMREQUEST => 'HEAD',
CURLOPT_HEADER => TRUE,
CURLOPT_HTTPHEADER => $request_headers,
CURLOPT_RETURNTRANSFER => TRUE,
CURLOPT_CONNECTTIMEOUT => 10,
CURLOPT_FOLLOWLOCATION => TRUE,
CURLOPT_MAXREDIRS => 10,
CURLOPT_USERAGENT => $user_agent,
));
$result = curl_exec($ch);
The request does not finish.
What's wrong with my CURL setup? This works for other websites for http://www.google.com but for some like weebly it ends up hanging.
Because libcurl will act as if a GET is used but you change the method only in the actual request to HEAD, and that screw things up since the HEAD response probably has a Content-Length: header but no response body.
The "proper" way to have libcurl do a HEAD and expect a HEAD response is to use CURLOPT_NOBODY instead.
CURLOPT_CUSTOMREQUEST should only be used to change method keyword, not to change behavior.
I'm trying to get the content of a stream over HTTPS, but I have to go over an HTTP proxy.
I'd like not to use cURL but rather use fopen with a context argument.
The thing is, I can't make it work over HTTPS (HTTP is working fine though).
This DOES NOT work :
$stream = stream_context_create(Array("http" => Array("method" => "GET",
"timeout" => 20,
"proxy" => "tcp://my-proxy:3128",
'request_fulluri' => True
)));
echo file_get_contents('https://my-stream', false, $context);
This DOES work (cURL) :
$url = 'https://my-stream';
$proxy = 'my-proxy:3128';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_PROXY, $proxy);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
$curl_scraped_page = curl_exec($ch);
curl_close($ch);
echo $curl_scraped_page;
Does somebody know what is wrong with the first piece of code ? if it works with cURL there has to be a way to make it work with a context.
I tried to change the context options to a bunch of different values woth no luck.
Any help would be greatly appreciated !
Thanks.
You did not specifiy the exact error message, try adding ignore_errors => true. But if you are getting a 400 Bad Request from Apache, the problem you are probably hitting, is a Server Name Indication & host header mismatch. There is also a PHP bug related to this: https://bugs.php.net/bug.php?id=63519
Try the following fix until this bug is resolved:
$stream = stream_context_create(array(
'http' => array(
'timeout' => 20,
'proxy' => 'tcp://my-proxy:3128',
'request_fulluri' => true
),
'ssl' => array(
'SNI_enabled' => false // Disable SNI for https over http proxies
)
));
echo file_get_contents('https://my-stream', false, $context);
I'm new to php and I'm trying to create a simple example for calling our company api. I got NetBeans IDE 7.1.2 to work last night (yay) but I cannot seem to get the following code to show me anything. I can run it in the debugger. I can step through it. I even get to the end without errors, but the curl_exec returns just 0. I have added the CURLOPT_PROXYPORT so that I can get fiddler to see the traffic, but fiddler sees nothing. I am also trying to run this as a php command line (if that has any bearing).
I know I'm doing something stupid... but that's the problem with stupid.
<?php
$url = 'https://target.boomerang.com/api/JobCreate';
$authToken = 'phptest';
$data = array(
"emailHTML" => "Howdy",
"jobKind" => "email",
"senderEmail" => "bob#boomerang.com",
"subject" => "My howdy email"
);
$data_string = json_encode($data);
$headers = array(
'Content-type: application/json',
'auth_token: ' . $authToken,
'Accept: application/json',
'Expect:'
);
$ch = curl_init();
$args = array(
CURLOPT_URL => $url,
CURLOPT_FOLLOWLOCATION => TRUE,
CURLOPT_RETURNTRANSFER => TRUE,
CURLOPT_POST => TRUE,
CURLOPT_POSTFIELDS => $data_string,
CURLOPT_HTTPHEADER => $headers,
CURLOPT_PROXYPORT => "localhost:8888"
);
curl_setopt_array($ch, $args);
$res = curl_exec($ch);
$res_data = json_decode($res, true);
print($res_data);
?>
Thanks for any help.
So the problem was caused by SSL certificate mismatch? I suppose it can be solved by adding this...
CURLOPT_SSL_VERIFYHOST => 0,
CURLOPT_SSL_VERIFYPEER => 0,
...into $args array.
I have a class function to interface with the RESTful API for Last.FM - its purpose is to grab the most recent tracks for my user. Here it is:
private static $base_url = 'http://ws.audioscrobbler.com/2.0/';
public static function getTopTracks($options = array())
{
$options = array_merge(array(
'user' => 'bachya',
'period' => NULL,
'api_key' => 'xxxxx...', // obfuscated, obviously
), $options);
$options['method'] = 'user.getTopTracks';
// Initialize cURL request and set parameters
$ch = curl_init();
curl_setopt_array($ch, array(
CURLOPT_URL => self::$base_url,
CURLOPT_POST => TRUE,
CURLOPT_POSTFIELDS => $options,
CURLOPT_RETURNTRANSFER => TRUE,
CURLOPT_TIMEOUT => 30,
CURLOPT_USERAGENT => 'Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)'
));
$results = curl_exec($ch);
return $results;
}
This returns "Empty reply from server". I know that some have suggested that this error comes from some fault in network infrastructure; I do not believe this to be true in my case. If I run a cURL request through the command line, I get my data; the Last.FM service is up and accessible.
Before I go to those folks and see if anything has changed, I wanted to check with you fine folks and see if there's some issue in my code that would be causing this.
Thanks!
ANSWER: #Jan Kuboschek helped me stumble onto what is (maybe) going on here. By giving CURLOPT_POSTFIELDS an associative array, a particular content-type is specified that may not work with certain RESTful services. A smarter solution is to manually create a URL-encoded version of that data and pass that as the CURLOPT_POSTFIELDS.
For more info, check out: http://www.brandonchecketts.com/archives/array-versus-string-in-curlopt_postfields
A common issue are spaces in the URL - beginning, in the middle, or trailing. Did you check that out?
Edit - per comments below, spacing is not the issue.
I ran your code and had the same problem - no output whatsoever. I tried the URL and with a GET request, the server talks to me. I would do the following:
Use the following as $base_url: $base_url = 'http://ws.audioscrobbler.com/2.0/?user=bachya&period=&api_key=xxx&method=user.getTopTracks';
Remove the post fields from your request.
Edit
I moved your code out of the class since I didn't have the rest and modified it. The following code runs perfect for me. If these changes don't work for you, I suggest that your error is in a different function.
<?php
function getTopTracks()
{
$base_url = 'http://ws.audioscrobbler.com/2.0/?user=bachya&period=&api_key=8066d2ebfbf1e1a8d1c32c84cf65c91c&method=user.getTopTracks';
$options = array_merge(array(
'user' => 'bachya',
'period' => NULL,
'api_key' => 'xxxxx...', // obfuscated, obviously
));
$options['method'] = 'user.getTopTracks';
// Initialize cURL request and set parameters
$ch = curl_init($base_url);
curl_setopt_array($ch, array(
CURLOPT_URL => $base_url,
CURLOPT_RETURNTRANSFER => TRUE,
CURLOPT_TIMEOUT => 30,
CURLOPT_USERAGENT => 'Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)'
));
$results = curl_exec($ch);
return $results;
}
echo getTopTracks();
?>
The server received your request, but sent an empty response. Check the result of curl_getinfo($ch, CURLINFO_HTTP_CODE) to find out if the server responded with an HTTP error code.
Update: Ok so the server responds with the 100 Continue HTTP status code. In that case, this should solve your problem:
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Expect:'));
I found this here: PHP and cURL: Disabling 100-continue header. Hope it works!
I came acorss the same issue. My Http_code returned 200 but my response was empty. There could be many reasons for this as i experienced.
--Your hedaers might be incorrect
CURLOPT_HTTPHEADER => array('Content-Type:application/json', 'Expect:')
--You might need to send data as post fields in culr and not attached to the URl like url?p1=a1&p2=a2
$data = array (p1=>a1, p2=>a2)
CURLOPT_POSTFIELDS => $data
So your options array would be similar to the below
array(
CURLOPT_URL => $url,
CURLOPT_FAILONERROR => TRUE, // FALSE if in debug mode
CURLOPT_RETURNTRANSFER => TRUE,
CURLOPT_TIMEOUT => 4,
CURLOPT_HTTPHEADER => array('Content-Type:application/json', 'Expect:'),
CURLOPT_POST => TRUE,
CURLOPT_POSTFIELDS => $data,
);
According to Last.FM API documentation you should use GET method instead of POST to pass parameters. When I've changed POST to GET I've received the answer about incorrect key.
And here's the code for get Album Info from Laft.FM even if return error:
The Function:
function getAlbum($xml,$artist,$album)
{
$base_url = $xml;
$options = array_merge(array(
'user' => 'YOUR_USERNAME',
'artist'=>$artist,
'album'=>$album,
'period' => NULL,
'api_key' => 'xYxOxUxRxxAxPxIxxKxExYxx',
));
$options['method'] = 'album.getinfo';
// Initialize cURL request and set parameters
$ch = curl_init($base_url);
curl_setopt_array($ch, array(
CURLOPT_URL => 'http://ws.audioscrobbler.com/2.0/',
CURLOPT_POST => TRUE,
CURLOPT_POSTFIELDS => $options,
CURLOPT_RETURNTRANSFER => TRUE,
CURLOPT_TIMEOUT => 30,
CURLOPT_HTTPHEADER => array( 'Expect:' ) ,
CURLOPT_USERAGENT => 'Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)'
));
$results = curl_exec($ch);
unset ($options);
return $results;
}
Usage:
// Get the XML
$xml_error = getAlbum($xml,$artist,$album);
// Show XML error
if (preg_match("/error/i", $xml_error)) {
echo " <strong>ERRO:</strong> ".trim(strip_tags($xml_error));
}