Retrieve Twitter's X-RateLimit-Limit in PHP - php

With every Twitter request I make, the returned HTTP headers should include X-RateLimit-Limit.
However, I seem unable to retrieve these using PHP. Can someone tell me what bone-headed mistake I've made?
I've set my curl up in the normal way and am able to successfully GET and POST requests.
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Expect:'));
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_HEADER, FALSE);
curl_setopt($ch, CURLINFO_HEADER_OUT, TRUE);
curl_setopt($ch, CURLOPT_VERBOSE, TRUE);
$response = curl_exec($ch);
$response_info=curl_getinfo($ch);
$erno = curl_errno($ch);
$er = curl_error($ch);
curl_close($ch);
I'm able to get some response information, like http_code
$response_info['http_code']
But this line just returns null
//Doesn't bloody work. No idea why!
$rate_limit = $response_info['X-RateLimit-Limit'];
I'm running PHP Version 5.3.10.
EDIT
This is the result of print_r($response_info);
Array
(
[url] => https://api.twitter.com/1/statuses/home_timeline.json...
[content_type] => application/json;charset=utf-8
[http_code] => 200
[header_size] => 695
[request_size] => 410
[filetime] => -1
[ssl_verify_result] => 0
[redirect_count] => 0
[total_time] => 1.239977
[namelookup_time] => 0.007361
[connect_time] => 0.155783
[pretransfer_time] => 0.465397
[size_upload] => 0
[size_download] => 99425
[speed_download] => 80182
[speed_upload] => 0
[download_content_length] => 99425
[upload_content_length] => 0
[starttransfer_time] => 0.794829
[redirect_time] => 0
[certinfo] => Array()
[redirect_url] =>
[request_header] => GET /1/statuses/home_timeline.json... HTTP/1.1
Host: api.twitter.com
Accept: */*
)

curl_getinfo does not return the response headers, only other meta info about the request. To retrieve headers, set CURLOPT_HEADER to true. That will include the headers in the output. To separate them from the response body do:
list($headers, $body) = explode("\n\n", $response, 2);
To parse the headers, explode again:
$headers = explode("\n", $headers);
foreach ($headers as $header) {
list($key, $value) = explode(':', $header, 2);
$headers[trim($key)] = trim($value);
}
echo $headers['X-RateLimit-Limit'];

Related

PHP - Using cURL to get a cookie from a redirecting URL but getting no response

I have a cURL command using which I get authenticated to a website and it gives a cookie in response, then subsequently using this cookie I can make REST API calls to this service
Here is the working cURL command:
curl -v -l -H "Content-Type: application/x-www-form-urlencoded" -H "Referrer:https://mywebsiteurl.com?ticket=unique_ticket_id" -d "_charset_=UTF-8&errorMessage=User+name+and+password+do+not+match&resource=%2F&username=username%40domain.com&password=XXXXXX&nextpage=welcomeCM.jsp&viewInfo=&ticket=unique_ticket_id" -X POST https://mywebsiteurl.com/index.jsp
In the above command ticket parameter contains a unique ticket id that is passed along with the website url
Now, I'm trying to accomplish the same using cURL PHP, here is the PHP code:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://mywebsiteurl.com/index.jsp");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, "_charset_=UTF-8&errorMessage=User+name+and+password+do+not+match&resource=%2F&username=username%40domain.com&password=XXXXXX&nextpage=welcomeCM.jsp&viewInfo=&ticket=unique_ticket_id");
curl_setopt($ch, CURLOPT_POST, 1);
$headers = array(
'Content-Type: application/x-www-form-urlencoded',
'Referrer: https://mywebsiteurl.com?ticket=unique_ticket_id'
);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
$result = curl_exec($ch);
if (curl_errno($ch)) {
echo 'Error:' . curl_error($ch);
}
echo $result;
curl_close ($ch);
Here echo $result shows nothing
I then tried var_dump($result); and it gives this output: string(0) "" which means nothing is getting returned in $response
After this I tried curl_getinfo and added the following code:
echo "<pre>";
$info = curl_getinfo($ch);
print_r($info);
echo "</pre>";
And this gave me the following array:
Array
(
[url] => https://mywebsiteurl.com/index.jsp
[content_type] => text/html;charset=UTF-8
[http_code] => 302
[header_size] => 1306
[request_size] => 619
[filetime] => -1
[ssl_verify_result] => 0
[redirect_count] => 0
[total_time] => 1.540687
[namelookup_time] => 0.028262
[connect_time] => 0.171774
[pretransfer_time] => 0.462507
[size_upload] => 280
[size_download] => 0
[speed_download] => 0
[speed_upload] => 181
[download_content_length] => 0
[upload_content_length] => 280
[starttransfer_time] => 1.54066
[redirect_time] => 0
[redirect_url] => https://mywebsiteurl.com/welcomeCM.jsp?username=username#domain.com&locale=en_US
[primary_ip] => YY.YYY.YYY.YY
[certinfo] => Array
(
)
[primary_port] => 443
[local_ip] => XX.XX.XXX.XX
[local_port] => 47692
)
Now my goal is to get the cookie in $result, but it is empty & nothing is getting returned into it
Is it that something is missing in the cURL command equivalent PHP code?
Can someone please help me out & point me in the direction to retrieve the cookie
Thanks a lot!
UPDATE
I added curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); to the request to have it follow the 302 redirect after successful login
But this time after the execution of cURL, echo $result is redirecting my current local webpage to /Dashboard?viewInfo= which obviously does not exist
Also this time var_dump($result); is resulting in: string(1462) " "
And curl_getinfo($ch) this time is giving the following array:
Array
(
[url] => https://mywebsiteurl.com/welcomeCM.jsp?username=username#domain.com&locale=en_US
[content_type] => text/html;charset=UTF-8
[http_code] => 200
[header_size] => 1821
[request_size] => 956
[filetime] => -1
[ssl_verify_result] => 0
[redirect_count] => 1
[total_time] => 1.746911
[namelookup_time] => 1.5E-5
[connect_time] => 1.5E-5
[pretransfer_time] => 6.2E-5
[size_upload] => 0
[size_download] => 1462
[speed_download] => 836
[speed_upload] => 0
[download_content_length] => 1462
[upload_content_length] => 0
[starttransfer_time] => 0.146587
[redirect_time] => 1.6003
[redirect_url] =>
[primary_ip] => YY.YYY.YYY.YY
[certinfo] => Array
(
)
[primary_port] => 443
[local_ip] => YY.YY.YYY.YY
[local_port] => 47705
)
Still can't get the required cookie here,
Please help!
If you want to use curls given functionality you can try using CURLOPT_COOKIEJAR / CURLOPT_COOKIEFILE, to store and get your cookie. This if file based.
In this case, curl will then write any Cookie related Information in a file.
You can define where this file will be located.
By using CURLOPT_COOKIEJAR , you will tell curl that it shall capture the cookie data.
// create cookie file
$cookie = __DIR__ . "/where/you/want/to/store/your/cookie/mycookie.txt";
fopen($cookie, "w");
// you may want to use some random identicator in your cookie file
// to make sure it does not get overwritten by some action
// prepare login
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://mywebsiteurl.com/index.jsp");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
// tell curl to follow redirects
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURlOPT_POSTFIELDS, "username=usernam&password=XXXXXX");
curl_setopt($ch, CURLOPT_POST, 1);
// set a jar, to store your cookie
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie);
$headers = array(
'Content-Type: application/x-www-form-urlencoded',
'Referrer: https://mywebsiteurl.com?ticket=unique_ticket_id'
);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
// do login
$loginResult = curl_exec($ch);
// check if your login has worked according to the given result by any method you want
if (preg_match("/Welcome/", $loginResult)) {
// simple regex for demonstration purpose
}
// now you could load your cookie out of the file, or just use the file for future requests
If your login has worked, you will have all information stored in your cookie file.
Now you can use that cookie again with the curlopt CURLOPT_COOKIEFILE for future requests.
// do other request, with your cookie
curl_setopt($ch, CURLOPT_URL, "https://mywebsiteurl.com/api/rest.php");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, "your new post data");
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
// set a jar to update your cookie if needed
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie);
// give curl the location of your cookie file, so it can send it
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie);

cURL takes too long to load

I am calling a REST endpoint in PHP using cURL to fetch some JSON data:
<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$result = curl_exec($ch);
echo $result;
curl_close($ch);
It takes 2.5 seconds to fetch the data using the above code on my localhost. The same code takes around 7.5 seconds when run on the live server. When the URL is opened directly on a browser it takes only 1.5 seconds.
My question is: Why does it take so long for cURL to fetch data on the live server and how can I solve this problem?
Below is the output of curl_getinfo($ch) on the server:
Array
(
[content_type] => application/json
[http_code] => 200
[header_size] => 420
[request_size] => 113
[filetime] => -1
[ssl_verify_result] => 0
[redirect_count] => 0
[total_time] => 7.305496
[namelookup_time] => 0.150378
[connect_time] => 0.473187
[pretransfer_time] => 0.473237
[size_upload] => 0
[size_download] => 1291504
[speed_download] => 176785
[speed_upload] => 0
[download_content_length] => -1
[upload_content_length] => 0
[starttransfer_time] => 1.787901
[redirect_time] => 0
[redirect_url] =>
[certinfo] => Array
(
)
[primary_port] => 80
[local_port] => 53962
)
I found the solution to my problem. As I had mentioned in the question, the service was loading the fastest in browsers. So, I checked the 'Request Headers' of the request in the 'Network' tab of Google Chrome Inspector. I copied those headers and used them in my cURL request in PHP. After scraping those headers I found that all I needed to do was to add an Accept-Encoding header. I passed a value of gzip like so:
curl_setopt($ch, CURLOPT_ENCODING, 'gzip');
but setting it to an empty string also works.
curl_setopt($ch, CURLOPT_ENCODING, '');
According to the php.net manual for CURLOPT_ENCODING:
The contents of the "Accept-Encoding: " header. This enables decoding
of the response. Supported encodings are "identity", "deflate", and
"gzip". If an empty string, "", is set, a header containing all
supported encoding types is sent.
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "set ur url");
curl_setopt($ch, CURLOPT_ENCODING , "gzip");
curl_setopt($ch, CURLOPT_ENCODING, '');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$response = curl_exec($ch);
curl_close($ch);
Please check this example

Trouble POSTing using cURL, PHP, and Basic Authentication

I made a similar post last week, which can be found here. I was having problems with authentication, which I believe I've solved. The error message is different, at least.
I'm trying to recreate a successful POST I made to our vendor via POSTman by executing cURL commands in PHP.
Here is the example cURL command from their documentation:
curl -i -k -u '<api_key>': -XPOST --data-urlencode assessment_data#/path/to/test/file.json "https://<your_subdomain>.vendor.org/api/v1/import"; echo ""
This is my PHP code, which does not work:
<?php session_start();
include('../connect.php');
include('../functions.php');
function curlPost($url, $headers, $username, $password, $post) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_CAINFO, 'certificate.pem');
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLINFO_HEADER_OUT, true);
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
curl_setopt($ch, CURLOPT_USERPWD, $username . ":" . $password);
curl_setopt($ch, CURLOPT_TIMEOUT, 30);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
echo '<pre>';
echo curl_exec($ch) . '<br><br>';
echo print_r(curl_getinfo($ch)) . '<br><br>';
echo curl_error($ch) . '<br><br>';
echo '</pre>';
curl_close($ch);
}
$data = 'JSON_object={"JSON_key":"JSON_value"}';
curlPost('https://company.vendor.org/api/v1/import',
['Content-Type: multipart/form-data'],
'api-key',
'',
$data
);
and instead produces this output:
{"success":false,"errors":["500 Internal Server Error"],"warnings":[],"info":[],"meta":[],"results":[]}
Array
(
[url] => https://company.vendor.org/api/v1/import
[content_type] => application/json; charset=utf-8
[http_code] => 500
[header_size] => 547
[request_size] => 248
[filetime] => -1
[ssl_verify_result] => 0
[redirect_count] => 0
[total_time] => 1.586493
[namelookup_time] => 0.135777
[connect_time] => 0.177182
[pretransfer_time] => 0.286958
[size_upload] => 1999
[size_download] => 103
[speed_download] => 64
[speed_upload] => 1260
[download_content_length] => 103
[upload_content_length] => 1999
[starttransfer_time] => 0.345878
[redirect_time] => 0
[redirect_url] =>
[primary_ip] => xx.xxx.xx.xxx
[certinfo] => Array
(
'This array is actually empty, I didn't delete anything.'
)
[primary_port] => 443
[local_ip] => xxx.xxx.xx.xxx
[local_port] => 60532
[request_header] => POST /api/v1/import HTTP/1.1
Authorization: Basic xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Host: company.vendor.org
Accept: */*
Content-Type: multipart/form-data
Content-Length: 1999
Expect: 100-continue
)
1
For good measure, here is a screenshot of a successful POST using POSTman (It's censored a little differently, but I'm passing in the exact same information):
Their documentation contains an example JSON object, which they save as a file and send as form-data. I've managed to recreate the object with live data, output it to a properly formatted string, and send that as the value instead. This works in POSTman, but not in my PHP, so I suspect the problem lies there somehow. CURLOPT_POSTFIELDS requires a string (tried an array - it got mad), and I've tried formatting it every way I can think of. Any help would be greatly appreciated.

PHP cURL Problems: HTTP Code 0

I am trying to bring up a website/API through cURL and PHP. Whatever URL I try I am getting an HTTP Code of 0. I have tried several different URLs. No matter what I try I am getting the following curl_getinfo back (see below). I have verified that cURL is enabled in PHP.ini file.
Code:
// create a new cURL resource
$ch = curl_init();
// set URL and other appropriate options
curl_setopt($ch, CURLOPT_URL, "http://www.yahoo.com");
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_POST, TRUE);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, FALSE);
$report=curl_getinfo($ch);
print_r($report);
// grab URL and pass it to the browser
curl_exec($ch);
if(curl_errno($ch)) {
echo 'Curl error: ' . curl_error($ch);
}
print curl_error($ch);
// close cURL resource, and free up system resources
curl_close($ch);
cURL_getinfo:
Array
(
[url] => http://www.yahoo.com
[content_type] =>
[http_code] => 0
[header_size] => 0
[request_size] => 0
[filetime] => 0
[ssl_verify_result] => 0
[redirect_count] => 0
[total_time] => 0
[namelookup_time] => 0
[connect_time] => 0
[pretransfer_time] => 0
[size_upload] => 0
[size_download] => 0
[speed_download] => 0
[speed_upload] => 0
[download_content_length] => -1
[upload_content_length] => -1
[starttransfer_time] => 0
[redirect_time] => 0
[certinfo] => Array
(
)
[primary_ip] =>
[primary_port] => 0
[local_ip] =>
[local_port] => 0
[redirect_url] =>
)
You need to call curl_exec($ch); before curl_getinfo($ch); cause this is the actual connection to the server:
also there is no need in the flag CURLOPT_POST since it's a get call:
// create a new cURL resource
$ch = curl_init();
//for post calls:
//$post = 'a=b&d=c';
//$headers[] = 'Content-type: application/x-www-form-urlencoded;charset=utf-8';
//$headers[] = 'Content-Length: ' . strlen($post);
//for get calls:
$headers = array();
$headers[] = 'Content-type: charset=utf-8';
$headers[] = 'Connection: Keep-Alive';
// set URL and other appropriate options
curl_setopt($ch, CURLOPT_URL, "http://www.yahoo.com");
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
//curl_setopt($ch, CURLOPT_POST, TRUE);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, FALSE);
curl_exec($ch);
$report=curl_getinfo($ch);
print_r($report);
// grab URL and pass it to the browser
if(curl_errno($ch)) {
echo 'Curl error: ' . curl_error($ch);
}
print curl_error($ch);
// close cURL resource, and free up system resources
curl_close($ch);

Curl not returning all headers

I am using Curl in PHP to call an API.
According to their documentation, they are returning "Authentication-Callback" within the returned page's header.
It works perfectly when I paste the URL into the browser, but Curl seems to leave it out.
Here is my code
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://api.themoviedb.org/3/authentication/token/new?api_key=[MY_API_KEY]&language=en');
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FAILONERROR, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$results = curl_exec($ch);
$headers = curl_getinfo($ch);
Here is the returned header
Array
(
[url] => http://api.themoviedb.org/3/authentication/token/new?api_key=[MY_API_KEY]&language=en&
[content_type] => application/json;charset=utf-8
[http_code] => 200
[header_size] => 470
[request_size] => 137
[filetime] => -1
[ssl_verify_result] => 0
[redirect_count] => 0
[total_time] => 0.109
[namelookup_time] => 0
[connect_time] => 0.047
[pretransfer_time] => 0.047
[size_upload] => 0
[size_download] => 116
[speed_download] => 1064
[speed_upload] => 0
[download_content_length] => 116
[upload_content_length] => 0
[starttransfer_time] => 0.109
[redirect_time] => 0
[certinfo] => Array
(
)
)
As far as I can tell, everything is right. Curl returns the data that I need perfectly, just not the correct headers.
Any help is appreciated!
What you're doing right now is getting stored information about the header via curl_getinfo() which only gets the information in the OPT list on that page.
What you should do instead is to return the header and then manually separate it:
curl_setopt($ch, CURLOPT_HEADER, 1);
// The rest of your options
$output = curl_exec($ch);
// Since the end of the header is always delimited by two newlines
$output = explode("\n\n", $output, 2);
$header = $output[0];
$content = $output[1];
This is more work but will get you the real headers.
This is my code to do what phsource has suggested the headers are put into the $headers array
# Extract headers from response
preg_match_all('%HTTP/\\d\\.\\d.*?(\\r\\n|\\n){2,}%si', $curl_result, $header_matches);
$headers = preg_split('/\\r\\n/', str_replace("\r\n\r\n",'',array_pop($header_matches[0])));
# Convert headers into an associative array
if(is_array($headers))
{
foreach ($headers as $header)
{
preg_match('#(.*?)\:\s(.*)#', $header, $header_matches);
if(isset($header_matches[1]))
{
$headers[$header_matches[1]] = $header_matches[2];
$headers['lowercase'][strtolower($header_matches[1])] = $header_matches[2];
}
}
}
# Remove the headers from the response body
$curl_result = preg_replace('%HTTP/\\d\\.\\d.*?(\\r\\n|\\n){2,}%si','',$curl_result);
you may want to replace \r\n with PHP_EOL as you see fit

Categories