I am just wondering if there is a way to check if an encrypted image exist over https.
I am using this code below to check whether an image exist. It will return 200 if it does, and 404 if it doesn't. But this only work on image URL that does not contain https.
function exists_url($uri){
$ch = curl_init($uri);
curl_setopt($ch, CURLOPT_NOBODY, true);
curl_exec($ch);
return $code = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);
}
If I try to verify this url ( https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcTup5KSMveqkgrDKZR6p-0ANhPkJ7srbJOlKR78DUqqh85I_3MUrw ) it gives me a result of 0 instead of 200 or 404.
I have thought to use getimagesize() function to accomplish this task. But even this function only works for image that is not send over HTTPS.
Since you're using SSL/TLS you must point cURL to a valid CA Certificate bundle. The bundle must be in .pem format. You can obtain such a bundle from http://curl.haxx.se/ca/.
Next you need to tell cURL to use your CA bundle. You can do this with the CURLOPT_CAINFO option.
curl_setopt($ch, CURLOPT_CAINFO, '/path_to_file/cacert.pem');
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, true);
You could simply deactivate the verify of peer with
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, FALSE);
So you have not to worry about nothing.
Watch out: #user555 answer is better for security purposes
A response code of 0 from Curl typically means there was an error connecting to the server or something along those lines (See PHP cURL HTTP CODE return 0 and countless other questions).
I did however run this code myself using PHP 5.5.x and it returned a 200 as expected. getimagesize() not working also leads me to think it's most likely a problem with curl or php on your system, what versions of them do you have?
Related
My Deepcrawl crawl only give null value.
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"https://api.deepcrawl.com/accounts/00000/projects/195334/crawls/1306396/reports/thin_pages_basic");
curl_setopt($ch, CURLOPT_POST, 0);
curl_setopt($ch,CURLOPT_HTTPHEADER,array('X-Auth-Token:Private'));
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$server_output2 = curl_exec ($ch);
curl_close ($ch);
$results = json_decode($server_output2);
To create a session you first need to manually generate an API Key. This is a key/value pair that can be generated in the API Access page. When you click Generate New API Key, a popup will be displayed with the API Key Value and in the table of Active Keys you can find the API Key ID. This is then sent via Basic Authentication in a POST call to the sessions route.
curl -X POST -u '123:abcdef' https://api.deepcrawl.com/sessions
{
"token":"abcdef123",
"_user_href":"/users/example-user",
...
}
The token returned from the call to sessions is then passed to all API calls as the X-Auth-Token header:
curl -X GET -H 'X-Auth-Token:' https://api.deepcrawl.com/accounts/1/projects
Can someone explain me further about the authentication of deepcrawl? How can I able to curl it using X-Auth-Token only. Sorry for my bad English. Thank you
CURL may be tricky sometimes.
When I face problems with CURL I first try to do the same on command line, in this case it worked perfectly so we can reject there is any kind of problem on API or on CURL so we must assume the problem is on the php call.
When you get a NULL it's likely there is an error set on curl handler, you just need to look for it and bring it to screen:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"https://api.deepcrawl.com/accounts/00000/projects/195334/crawls/1306396/reports/thin_pages_basic");
curl_setopt($ch, CURLOPT_POST, 0);
curl_setopt($ch,CURLOPT_HTTPHEADER,array('X-Auth-Token:Private'));
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$server_output2 = curl_exec ($ch);
if (curl_errno($ch)) {
print curl_error($ch);
die();
// or whatever you might want to do with this error
}
curl_close ($ch);
$results = json_decode($server_output2);
In this case the error was:
SSL certificate problem: unable to get local issuer certificate
Here you can see how to fix this problem, just need to add a valid CA cert to your php.ini (in the example cacert is used) like that:
1) Download the latest cacert.pem from https://curl.haxx.se/ca/cacert.pem
2) Add the following line to php.ini (if this is shared hosting and you don't have access to php.ini then you could add this to .user.ini in public_html)
curl.cainfo="/path/to/downloaded/cacert.pem"
I have a cURL request in my code which works fine when running locally:
$url = "http://ipinfo.io/{$_SERVER['REMOTE_ADDR']}";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
$response = curl_exec($ch);
$locale = json_decode($response);
and returns a JSON as expected. Our production system is on Google App Engine, however, where I get the website version for a browser rather than the JSON.
I can get this cURL request to work if I change
google_app_engine.enable_curl_lite = "1"
in the php.ini in the root directory of my project to
extension = "curl.so"
but Google's documentation insists the former is to be used on production. Additionally, using the latter breaks things like Monolog's SlackHandler.
Is there a way to get the JSON from this cURL request while still using Google's "cURL Lite"?
From the ipinfo.io documentation:
"We do a little bit of magic on the server to determine if we should send the JSON response or the webpage. We usually get it right, but if you're seeing the webpage instead of the JSON (or want to check the JSON output in a browser) you can force the JSON response by adding /json to the end of the URL"
Adding /json to the end of the URL worked for me in this case, but I wanted a more general solution. Since Google's cURL Lite uses their URL Fetch in the background, ipinfo.io's "magic" is somehow getting confused. I found that by specifying the Accept header then the /json addition wasn't required. In PHP:
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Accept: application/json'));
Thanks to the folks on the #php channel of NashDev Slack for helping me on this one!
For some reason my curl call is very slow. Here is the code I used.
$postData = "test"
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postData);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, false);
$result = curl_exec($ch);
Executing this code takes on average 250ms to finish.
However when I just open the url in a browser, firebug says it only takes about 80ms.
Is there something I am doing wrong? Or is this the overhead associated with PHP Curl.
It's the call to
curl_exec
That is taking up all the time.
UPDATE:
So I figured out right after I posted this that if I set the curl option
curl_setopt($ch, CURLOPT_POSTFIELDS, $postData);
It significantly slows down
curl_exec
The post data could be anything and it will slow it down.
Even if I set
curl_setopt($ch, CURLOPT_POST, false);
It's slow.
I'll try to work around it by just adding the parameters to the URI as a query string.
SECOND UPDATE:
Confirmed that if I just call the URI using GET and passing parameters
as a query string it is much faster than using POST and putting the parameters in the body.
CURL has some problems with DNS look-ups. Try using IP address instead of domain name.
Curl has the ability to tell exactly how long each piece took and where the slowness is (name lookup, connect, transfer time). Use curl_getinfo (http://www.php.net/manual/en/function.curl-getinfo.php) after you run curl_exec.
If curl is slow, it is generally not the PHP code, it's almost always network related.
try this
curl_setopt($ch, CURLOPT_IPRESOLVE, CURL_IPRESOLVE_V4 );
Adding "curl_setopt($ch, CURLOPT_POSTREDIR, CURL_REDIR_POST_ALL);" solved here. Any problem with this solution?
I just resolved this exact problem by removing the following two options:
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postData);
Somehow on the site I was fetching, the POST request to over ten full seconds. If it's GET, it's less than a second.
So... in my wrapper function that does the Curl requests, it now only sets those two options when there is something in $postData
I just experienced a massive speed-up through compression. By adding the Accept-Encoding header to "gzip, deflate", or just to all formats which Curl supports, my ~200MB download took 6s instead of 20s:
curl_setopt($ch, CURLOPT_ENCODING, '');
Notes:
If an empty string, "", is set, a header containing all supported encoding types is sent.
you do not even have to care about decompression after the download, as this is done by Curl internally.
CURLOPT_ENCODING requires Curl 7.10+
The curl functions in php directly use the curl command line tool under *nix systems.
Therefore it really only depends on the network speed since in general curl itself is much faster than a webbrowser since it (by default) does not load any additional data like included pictures, stylesheets etc. of a website.
It might be possible that you are not aware, that the network performance of the server on which you were testing your php script is way worse than on your local computer where you were testing with the browser. Therefore both measurements are not really comparable.
generally thats acceptable when you are loading contents or posting to slower end of world. curl call are directly proportional to your network speed and throughput of your webserver
EDIT/UPDATE:
1) I tried the URL with just http (not https), and it worked in my browser. But, it did not work with PHP and cURL!
2) I read the curl error message, and it said Couldn't resolve host 'ajax.googleapis.com'. But, again, it could resolve the host from my web browser on the same machine!
3) Google explicitly stated that I needed the CURLOPT_REFERER to be set, so I'm keeping it.
Any other ideas? Thanks!
ORIGINAL POST:
When I enter this URL into my web browser, I get the JSON response I want. But, when I run the following cURL code in PHP5 (via Apache 2), the request fails. Can anyone point to some possible problems?
$url = "https://ajax.googleapis.com/ajax/services/language/translate?v=1.0&q=hola&langpair=es%7Cen&key=I-REMOVED-MY-API-KEY-FOR-STACKOVERFLOW-POST";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch, CURLOPT_REFERER, "http://my.ip.addr.ess/");
$response = curl_exec($ch);
var_dump($response);
The output is bool(false);
I have no idea what's wrong... do you? Thanks!
When the response if false, there was an error. Check for errors doing something like this:
if (($response = curl_exec($ch)) === FALSE) {
echo curl_error($ch);
exit();
}
In production code you definitely want to do something else on an error condition (instead of outputting the error message and exiting), but this will help you for debugging.
Probably because you're accessing a HTTPS resource.
Quick fix:
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
Use http:// instead of https://. Code works fine without the key in the query string. CURLOPT_REFERER is also not necessary.
I was thinking of doing a head request with cURL, was wondering if this is the way to go?
The best solution which follows the KISS principle
$head = array_change_key_case(get_headers("http://example.com/file.ext", 1));
$filesize = $head['content-length'];
I'm guessing using curl to send a HEAD request is a nice possibility ; something like this would probably do :
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://sstatic.net/so/img/logo.png');
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_NOBODY, true);
curl_exec($ch);
$size = curl_getinfo($ch, CURLINFO_CONTENT_LENGTH_DOWNLOAD);
var_dump($size);
And will get you :
float 3438
This way, you are using a HEAD request, and not downloading the whole file -- still, you depend on the remote server send a correct Content-length header.
Another option you might think about would be to use filesize... But this will fail : the documentation states (quoting) :
As of PHP 5.0.0, this function can
also be used with some URL wrappers.
Refer to List of Supported
Protocols/Wrappers for a listing
of which wrappers support stat()
family of functionality.
And, unfortunately, with HTTP and HTTPS wrappers, stat() is not supported...
If you try, you'll get an error, like this :
Warning: filesize() [function.filesize]: stat failed
for http://sstatic.net/so/img/logo.png
Too bad :-(
Yes. Since the file is remote, you're completely dependent on the value of the Content-Length header (unless you want to download the whole file). You'll want to curl_setopt($ch, CURLOPT_NOBODY, true) and curl_setopt($ch, CURLOPT_HEADER, true).
Using a HEAD request and checking for Content-Length is the standard way to do it, but you can't rely on it in general, since the server might not support it. The Content-Length header is optional, and further the server might not even implement the HEAD method. If you know which server you're probing, then you can test if it works, but as a general solution it isn't bullet proof.
If you don't need a bulletproof solution you can just do:
strlen(file_get_contents($url));