Weird behaviour of cURL request in PHP script - php

I'm experiencing weird behaviour of a cURL request that I do in my PHP code.
I'm running the code locally on a standard WAMP Apache server.
Here's the code:
$auth_info = "...";
$some_url = "...";
$channel = curl_init();
curl_setopt($channel, CURLOPT_URL, $some_url);
curl_setopt($channel, CURLOPT_HTTPHEADER,
array("Authorization: BASIC " . base64_encode($auth_info))
);
curl_setopt($channel, CURLOPT_RETURNTRANSFER, true);
curl_setopt($channel, CURLOPT_HEADER, true);
$response = curl_exec($channel);
var_dump(curl_getinfo($channel));
$header_size = curl_getinfo($channel, CURLINFO_HEADER_SIZE);
$header = substr($response, 0, $header_size);
$body = substr($response, $header_size);
var_dump($header);
var_dump($body);
curl_close($channel);
If I'm executing this little PHP code snippet via my CLI (Powershell, since I'm running on Windows), everything is fine, all var_dumps work and I can see the $header and $body and everything and the data I expect is actually present.
Now for the weird behaviour. If I'm opening the script file with the above snippet in any browser, it just gives me:
array (size=26)
'url' => string 'the_url_here' (length=39)
'content_type' => null
'http_code' => int 0
'header_size' => int 0
'request_size' => int 0
'filetime' => int -1
'ssl_verify_result' => int 0
'redirect_count' => int 0
'total_time' => float 1.234
'namelookup_time' => float 0
'connect_time' => float 0.109
'pretransfer_time' => float 0
'size_upload' => float 0
'size_download' => float 0
'speed_download' => float 0
'speed_upload' => float 0
'download_content_length' => float -1
'upload_content_length' => float -1
'starttransfer_time' => float 0
'redirect_time' => float 0
'redirect_url' => string '' (length=0)
'primary_ip' => string 'here_is_the_ip' (length=12)
'certinfo' =>
array (size=0)
empty
'primary_port' => int 443
'local_ip' => string 'here_is_my_ip' (length=13)
'local_port' => int -> my_local_port_here
boolean false
boolean false
I'm completely puzzled since I cannot see a difference between the script beeing started by the CLI and beeing started by the browser. Has anyone got an idea on this?
EDIT Note: If I'm using Guzzle for the request, everything works fine, both in CLI and browser. I'm still interested in an answer why cURL is causing problems here.

Have you tried logging verbose output for the Curl request?
Normally I find this the best way to figure out what's going on under the hood...https://stackoverflow.com/a/14436877/682754
Also not as popular but this approach looks simple to implement and is a lot cleaner...https://stackoverflow.com/a/26611776/682754

Related

getting data with curl failed

I'm trying to get the data returned from this link with CURL. However, I'm getting false as the output.
Here's my code and output:
Code:
$url = "https://airport.api.aero/airport/distance/LOS/ABV?user_key=MY_USER_KEY_GOES_HERE&units=mile";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 0);
$data = curl_exec($ch);
var_dump( curl_getinfo($ch) );
curl_close($ch);
var_dump($data);
Dump of curl_getinfo call
array (size=26)
'url' => string 'https://airport.api.aero/airport/distance/LOS/ABV?user_key=MY_USER_KEY&units=mile' (length=102)
'content_type' => null
'http_code' => int 0
'header_size' => int 0
'request_size' => int 0
'filetime' => int -1
'ssl_verify_result' => int 0
'redirect_count' => int 0
'total_time' => float 0.702
'namelookup_time' => float 0.249
'connect_time' => float 0.468
'pretransfer_time' => float 0
'size_upload' => float 0
'size_download' => float 0
'speed_download' => float 0
'speed_upload' => float 0
'download_content_length' => float -1
'upload_content_length' => float -1
'starttransfer_time' => float 0
'redirect_time' => float 0
'redirect_url' => string '' (length=0)
'primary_ip' => string '107.22.197.175' (length=14)
'certinfo' =>
array (size=0)
empty
'primary_port' => int 443
'local_ip' => string '192.168.1.17' (length=12)
'local_port' => int 53767
Dump of $data
boolean false
Is there something I'm doing wrong?
I was able to successfully send a get request (you should hide your key from your question lol)
Try this:
$url = "https://airport.api.aero/airport/distance/LOS/ABV";
$qry_str = "?units=mile&user_key=MY_USER_KEY_GOES_HERE";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url . $qry_str);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_TIMEOUT, '3');
curl_setopt($ch, CURLOPT_HEADER, 0);
$content = trim(curl_exec($ch));
curl_close($ch);
print $content;
Found the solution here. Since the url was using https, I needed to include this option curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);. That did it.
Are you able to make a curl request via command line?
i.e curl 'https://airport.api.aero/airport/distance/LOS/ABV?user_key=MY_USER_KEY_GOES_HERE&units=mile'

curl not working on certain networks

I seem to be facing a weird problem. I have a PHP script in laptop (running under wamp) which uses 'curl' and works perfectly fine at work. But when at home and connected to my ISP, the same
script fails to return any data. Even a sample code like below fails:
$ch = curl_init("http://www.yahoo.com/");
curl_setopt($ch, CURLOPT_URL, "http://www.example.com/");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLINFO_HEADER_OUT, true);
curl_exec($ch);
var_dump(curl_getinfo($ch));
curl_getinfo dows not provide any clues to further debug. The HTTP CODE is zero. Do I need to add any headers>
I am able to access example.com through the browser, so no network issues I guess. curl_err and curl_errno is null.
<pre>
array (size=22)
'url' => string 'http://www.example.com/' (length=23)
'content_type' => null
'http_code' => int 0
'header_size' => int 0
'request_size' => int 0
'filetime' => int -1
'ssl_verify_result' => int 0
'redirect_count' => int 0
'total_time' => float 0
'namelookup_time' => float 0
'connect_time' => float 0
'pretransfer_time' => float 0
'size_upload' => float 0
'size_download' => float 0
'speed_download' => float 0
'speed_upload' => float 0
'download_content_length' => float -1
'upload_content_length' => float -1
'starttransfer_time' => float 0
'redirect_time' => float 0
'certinfo' =>
array (size=0)
empty
'redirect_url' => string '' (length=0)
</pre>
null
null
After thinking this through I think this is what you should do:
Ensure you can connect to the site from PHP:
<?php
$urlNoHttp = 'www.example.com'; // try 'example.com', too
$hostIP = gethostbyname($urlNoHttp);
var_dump($hostIP);
If it does not get the IP you have connection issues, so proceed to point 2.
Otherwise - it's not the network issues, it's curl setting itself - try point 3 and following.
Try pinging/wget'ting your destination from command line (ping www.example.com). If you get responses - your host is reachable for your OS and not the PHP, so
If you got the IP - then try reading about curl_setopt() int PHP docs. I'd start with setting proper HTTP headers (like Content-type, User-Agent, non-empty body to the request etc.). Also, try adding curl_setopt($c, CURLOPT_FRESH_CONNECT, 1); which forces not using a cached version of the url.
If it's still not working - try mixing in some proxy options, like curl_setopt($c, CURLOPT_HTTPPROXYTUNNEL, TRUE); to force "transparcence" of proxies, if there are any.
Generally you have to figure out what changes when you switch network environments. You have to know if there are any proxies involved, some strange NAT systems or maybe your router's firewall has some strange settings, that assume this kind of request as insecure and block it.
Hope this helps.

Using PHP's CURL to store API's JSON to variable

I'm trying to store data exposed with the following URL in a PHP variable ($curl_response) for further manipulation, but my current code is not executing properly. I've copied/pasted the data exposed in the following URL into the body of an HTML file and have tried running the below script on that HTML file, and CURL works properly. I'm guessing the issue then has to do with getting a response correctly from this particular site. Perhaps there is a CURL option I'm overlooking.
http://tdm.prod.obanyc.com/api/pullouts/agency/MTABC/list
Thoughts?
<?php
$ch = curl_init("http://tdm.prod.obanyc.com/api/pullouts/agency/MTABC/list");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$curl_response = curl_exec($ch);
if ($curl_response === false) {
$info = curl_getinfo($ch);
curl_close($ch);
die('error occured during curl exec. Additioanl info: ' . var_export($info));
}
else echo $curl_response ;
curl_close($ch);
?>
The debugging output is:
array ( 'url' => '', 'content_type' => NULL, 'http_code' => 0, 'header_size' => 0, 'request_size' => 0, 'filetime' => -1, 'ssl_verify_result' => 0, 'redirect_count' => 0, 'total_time' => 63.145232, 'namelookup_time' => 0.006015, 'connect_time' => 0, 'pretransfer_time' => 0, 'size_upload' => 0, 'size_download' => 0, 'speed_download' => 0, 'speed_upload' => 0, 'download_content_length' => -1, 'upload_content_length' => -1, 'starttransfer_time' => 0, 'redirect_time' => 0, 'certinfo' => array ( ), 'primary_ip' => '10.137.36.11', 'primary_port' => 80, 'local_ip' => '', 'local_port' => 0, 'redirect_url' => '', )error occured during curl exec. Additioanl info:
Is your else statement in the question the same way it is in the code? If yes, then I would start by fixing the else statement
Change
else $curl_response ;
to
else
{
$curl_response; // You might want to echo $curl_response or store it or do something with it. In it's current state it just does nothing.
}
I was informed that the IP address of the site is blocked for servers not in-network. That resolves the no response error and null value of the variable upon CURL execution.
Thanks all

cURL with REST API from PHP

I'm trying to create connection between two system, one of them is main. The connection will be provided throught cURL. I have this function:
private function build_post($url, $params, $authentificate = true) {
//url-ify the data for the POST
$params_string= http_build_query($params);
$curl = curl_init($url);
//set the url, number of POST vars, POST data
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl,CURLOPT_POST, count($params));
curl_setopt($curl,CURLOPT_POSTFIELDS, $params_string);
if ($authentificate) {
curl_setopt($curl, CURLOPT_HTTPHEADER, array(
'x-api-key: key1',
'x-api-skey: key2'
));
}
//execute post
$curl_response = curl_exec($curl);
if ($curl_response === false) {
$info = curl_getinfo($curl);
curl_close($curl);
echo('error occured during curl exec. Additioanl info: '.var_export($info));
}
else
{
echo 'Responze OK';
curl_close($curl);
}
}
From the system where I am trying to connect, I only have 2insctructions:
Each request to REST API must be authenticated by providing valid access & secret key of sender in request header.
for example:
x-api-key: D6d03y3h0iEW7Iz3xW9127a0xrh1ib
x-api-skey: sc1vcOTTFLuqjFa5u08UKtKaWl48XSqlm8jMQvrnXnuPvRjqTPgIDI6P1YcR
and second instruction is how should requested post look like.
I have no idea what is going wrong, all the time i get this:
array ( 'url' => 'https://rest.api.profitee.com/public/subscribe', 'content_type' => NULL, 'http_code' => 0, 'header_size' => 0, 'request_size' => 0, 'filetime' => -1, 'ssl_verify_result' => 1, 'redirect_count' => 0, 'total_time' => 0.037060000000000002662314813051125383935868740081787109375, 'namelookup_time' => 0.0003939999999999999817472395857720357525977306067943572998046875, 'connect_time' => 0.037070999999999999785504911642419756390154361724853515625, 'pretransfer_time' => 0, 'size_upload' => 0, 'size_download' => 0, 'speed_download' => 0, 'speed_upload' => 0, 'download_content_length' => -1, 'upload_content_length' => -1, 'starttransfer_time' => 0, 'redirect_time' => 0, 'certinfo' => array ( ), 'primary_ip' => '54.229.78.229', 'redirect_url' => '', )error occured during curl exec. Additioanl info: NULL
Can someone help me make it work? Thanks
Add the following lines and see if it works.
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, 2);

Testing if a web site is up by checking CURLINFO_HTTP_CODE returns 200 even if web site is false

I'm testing this code snippet on a Windows 7 machine running EasyPHP and PHP5, when I run the code it still returns code 200, I'm using a non-existent url so it shouldn't, should it? Is this because a redirection takes place for example to http://www.123-reg.co.uk? Thats what I suspect but I am struggling to find how I get around that.
$url = "http://www.bnkbkvbirbcx.co.uk";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, TRUE);
curl_setopt($ch, CURLOPT_NOBODY, TRUE);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, FALSE);
$hr = curl_exec($ch);
$httpreturncode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);
echo $httpreturncode;
inserting var_dump($hr, curl_getinfo($ch)) produces this output;
string 'HTTP/1.1 200 OK
Date: Fri, 28 Sep 2012 20:18:16 GMT
Server: Apache
Cache-control: no-cache, no-store
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Pragma: no-cache
Connection: close
Content-Type: text/html; charset=UTF-8
' (length=224)
array
'url' => string 'http://www.bnkbkvbirbcx.co.uk' (length=29)
'content_type' => string 'text/html; charset=UTF-8' (length=24)
'http_code' => int 200
'header_size' => int 224
'request_size' => int 62
'filetime' => int -1
'ssl_verify_result' => int 0
'redirect_count' => int 0
'total_time' => float 0.063
'namelookup_time' => float 0.016
'connect_time' => float 0.031
'pretransfer_time' => float 0.031
'size_upload' => float 0
'size_download' => float 0
'speed_download' => float 0
'speed_upload' => float 0
'download_content_length' => float -1
'upload_content_length' => float 0
'starttransfer_time' => float 0.063
'redirect_time' => float 0
'certinfo' =>
array
empty
'redirect_url' => string '' (length=0)
I would like to suggest to set another DNS server for your computer/server. It seems that your provider redirects non-existing domains to their homepage.
I don't know your provider virginmedia but with the german telekom there is an option to deactivate this option on their homepage.
An option would be to use an open/free DNS server like them from Google which are aviable via 8.8.8.8 and 8.8.4.4
I hope that helps!
Well I finally got to the bottom of this, I feel stupid it took me so long now! My isp DOES intervene with a "handy feature" as follows: "If an address you enter doesn't locate a site, this handy feature will convert the incorrect address into a web search, so instead of an error message you will get a list of our closest matches". I also discovered I can choose to "switch off" this handy feature which I did and now I get the results I'd expect. Thanks for all your input, as you can tell from some of my shakey posts I'm new to stackoverflow but have found it good to share my problem.
The script below might be helpful to whomever ends their search here.
The fakePhrase is used to detect ISP "Search Assist" adware HTTP responses.
#!/bin/bash
fakePhrase="verizon"
siteList=(
'http://google.com'
'https://google.com'
'http://wikipedia.org'
'https://wikipedia.org'
'http://cantgettherefromhere'
'http://searchassist.verizon.com'
)
exitStatus=0
function isUp {
http=`curl -sL -w "%{http_code}" "$1" -o temp_isUp`
fakeResponse=`cat temp_isUp | grep $fakePhrase`
if [ -n "$fakeResponse" ]; then
http=$fakePhrase
fi
case $http in
[2]*)
;;
[3]*)
echo 'Redirect'
;;
[4]*)
exitStatus=4
echo "$1 is DENIED with ${http}"
;;
[5]*)
exitStatus=5
echo "$1 is ERROR with ${http}"
;;
*)
exitStatus=6
echo "$1 is NO RESPONSE with ${http}"
;;
esac
}
for var in "${siteList[#]}"
do
isUp $var
done
if [ "$exitStatus" -eq "0" ]; then
echo 'All up'
fi
rm temp_isUp
exit $exitStatus

Categories