PHP cURL error: "Empty reply from server" - php

I have a class function to interface with the RESTful API for Last.FM - its purpose is to grab the most recent tracks for my user. Here it is:
private static $base_url = 'http://ws.audioscrobbler.com/2.0/';
public static function getTopTracks($options = array())
{
$options = array_merge(array(
'user' => 'bachya',
'period' => NULL,
'api_key' => 'xxxxx...', // obfuscated, obviously
), $options);
$options['method'] = 'user.getTopTracks';
// Initialize cURL request and set parameters
$ch = curl_init();
curl_setopt_array($ch, array(
CURLOPT_URL => self::$base_url,
CURLOPT_POST => TRUE,
CURLOPT_POSTFIELDS => $options,
CURLOPT_RETURNTRANSFER => TRUE,
CURLOPT_TIMEOUT => 30,
CURLOPT_USERAGENT => 'Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)'
));
$results = curl_exec($ch);
return $results;
}
This returns "Empty reply from server". I know that some have suggested that this error comes from some fault in network infrastructure; I do not believe this to be true in my case. If I run a cURL request through the command line, I get my data; the Last.FM service is up and accessible.
Before I go to those folks and see if anything has changed, I wanted to check with you fine folks and see if there's some issue in my code that would be causing this.
Thanks!
ANSWER: #Jan Kuboschek helped me stumble onto what is (maybe) going on here. By giving CURLOPT_POSTFIELDS an associative array, a particular content-type is specified that may not work with certain RESTful services. A smarter solution is to manually create a URL-encoded version of that data and pass that as the CURLOPT_POSTFIELDS.
For more info, check out: http://www.brandonchecketts.com/archives/array-versus-string-in-curlopt_postfields

A common issue are spaces in the URL - beginning, in the middle, or trailing. Did you check that out?
Edit - per comments below, spacing is not the issue.
I ran your code and had the same problem - no output whatsoever. I tried the URL and with a GET request, the server talks to me. I would do the following:
Use the following as $base_url: $base_url = 'http://ws.audioscrobbler.com/2.0/?user=bachya&period=&api_key=xxx&method=user.getTopTracks';
Remove the post fields from your request.
Edit
I moved your code out of the class since I didn't have the rest and modified it. The following code runs perfect for me. If these changes don't work for you, I suggest that your error is in a different function.
<?php
function getTopTracks()
{
$base_url = 'http://ws.audioscrobbler.com/2.0/?user=bachya&period=&api_key=8066d2ebfbf1e1a8d1c32c84cf65c91c&method=user.getTopTracks';
$options = array_merge(array(
'user' => 'bachya',
'period' => NULL,
'api_key' => 'xxxxx...', // obfuscated, obviously
));
$options['method'] = 'user.getTopTracks';
// Initialize cURL request and set parameters
$ch = curl_init($base_url);
curl_setopt_array($ch, array(
CURLOPT_URL => $base_url,
CURLOPT_RETURNTRANSFER => TRUE,
CURLOPT_TIMEOUT => 30,
CURLOPT_USERAGENT => 'Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)'
));
$results = curl_exec($ch);
return $results;
}
echo getTopTracks();
?>

The server received your request, but sent an empty response. Check the result of curl_getinfo($ch, CURLINFO_HTTP_CODE) to find out if the server responded with an HTTP error code.
Update: Ok so the server responds with the 100 Continue HTTP status code. In that case, this should solve your problem:
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Expect:'));
I found this here: PHP and cURL: Disabling 100-continue header. Hope it works!

I came acorss the same issue. My Http_code returned 200 but my response was empty. There could be many reasons for this as i experienced.
--Your hedaers might be incorrect
CURLOPT_HTTPHEADER => array('Content-Type:application/json', 'Expect:')
--You might need to send data as post fields in culr and not attached to the URl like url?p1=a1&p2=a2
$data = array (p1=>a1, p2=>a2)
CURLOPT_POSTFIELDS => $data
So your options array would be similar to the below
array(
CURLOPT_URL => $url,
CURLOPT_FAILONERROR => TRUE, // FALSE if in debug mode
CURLOPT_RETURNTRANSFER => TRUE,
CURLOPT_TIMEOUT => 4,
CURLOPT_HTTPHEADER => array('Content-Type:application/json', 'Expect:'),
CURLOPT_POST => TRUE,
CURLOPT_POSTFIELDS => $data,
);

According to Last.FM API documentation you should use GET method instead of POST to pass parameters. When I've changed POST to GET I've received the answer about incorrect key.

And here's the code for get Album Info from Laft.FM even if return error:
The Function:
function getAlbum($xml,$artist,$album)
{
$base_url = $xml;
$options = array_merge(array(
'user' => 'YOUR_USERNAME',
'artist'=>$artist,
'album'=>$album,
'period' => NULL,
'api_key' => 'xYxOxUxRxxAxPxIxxKxExYxx',
));
$options['method'] = 'album.getinfo';
// Initialize cURL request and set parameters
$ch = curl_init($base_url);
curl_setopt_array($ch, array(
CURLOPT_URL => 'http://ws.audioscrobbler.com/2.0/',
CURLOPT_POST => TRUE,
CURLOPT_POSTFIELDS => $options,
CURLOPT_RETURNTRANSFER => TRUE,
CURLOPT_TIMEOUT => 30,
CURLOPT_HTTPHEADER => array( 'Expect:' ) ,
CURLOPT_USERAGENT => 'Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)'
));
$results = curl_exec($ch);
unset ($options);
return $results;
}
Usage:
// Get the XML
$xml_error = getAlbum($xml,$artist,$album);
// Show XML error
if (preg_match("/error/i", $xml_error)) {
echo " <strong>ERRO:</strong> ".trim(strip_tags($xml_error));
}

Related

php script login to td ameritrade

I need to log into the TD Ameritrade website with my account. Note, I am not talking about the official API, I'm talking about their actual UI. I need to scrape from the site itself, unfortunately. I'm using curl in PHP, but any language you can get to work is probably fine. I've started with postman, which I'm usually able to get to work for this kind of thing, but it isn't returning anything when I copy the transactions from Chrome Dev Tools and importing into Postman. For example, it either loops back to the login site, or if I play with the settings it will flat out not return anything. The current issue with my php is that I get Maximum (10) redirects followed, it's an infinite loop where the login forwards back to itself. Unfortunately, for obvious reasons, I cannot post my username and password, so you would need an account to test. They are free, but not the easiest to set up. In any case, if you have any ideas that would be greatly appreciated.
<?php
header('Content-type: text/javascript');
$curl = curl_init();
curl_setopt_array($curl, array(
CURLOPT_URL => 'https://auth.tdameritrade.com/oauth?client_id=MOBI%40AMER.OAUTHAP&response_type=code&redirect_uri=https%3A%2F%2Finvest.ameritrade.com%2Fgrid%2Fp%2Fsite&code_challenge=xNghkUhiiE_DSuVXLcSF1ufBVSPTQRa1ITybIyw2USc&code_challenge_method=S256',
CURLOPT_RETURNTRANSFER => true,
CURLOPT_ENCODING => '',
CURLOPT_MAXREDIRS => 10,
CURLOPT_TIMEOUT => 0,
CURLOPT_FOLLOWLOCATION => true,
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
CURLOPT_CUSTOMREQUEST => 'POST',
CURLOPT_SSL_VERIFYPEER => false,
CURLOPT_SSL_VERIFYHOST => false,
CURLOPT_HEADER => 1, // needed for getting cookies
CURLOPT_USERAGENT => "Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.7.3) Gecko/20041001 Firefox/0.10.1",
CURLOPT_POSTFIELDS => 'su_username=' . $username . '&su_password=' . $password,
CURLOPT_VERBOSE => true,
CURLOPT_POST => true,
CURLOPT_AUTOREFERER => true,
// CURLOPT_HTTPHEADER => array(
// 'Content-Type: application/x-www-form-urlencoded'
// ),
));
//curl_setopt($curl, CURLOPT_USERPWD, $username . ":" . $password);
// $cookie = dirname(__FILE__) . "/cookie.txt";
// curl_setopt($curl, CURLOPT_COOKIEFILE, $cookie);
// curl_setopt($curl, CURLOPT_COOKIEJAR, $cookie);
$res = (object)array(
'response' => curl_exec($curl),
'info' => (object)curl_getinfo($curl),
'errors' => curl_error($curl)
);
print_r($res);

Coinmarketcap PHP API no data returned

Just a simple test to get some data via API doensn't work. I use the PHP example from their own website, but no result is printed. Source: https://pro.coinmarketcap.com/api/v1#section/Quick-Start-Guide
Is there anything I do wrong? I use the sandbox-environment with the demo API key.
Curl is installed at my server. Thanks a lot!
<?php
$url = 'https://sandbox-api.coinmarketcap.com/v1/cryptocurrency/listings/latest';
$parameters = [
'start' => '1',
'limit' => '5000',
'convert' => 'USD'
];
$headers = [
'Accepts: application/json',
'X-CMC_PRO_API_KEY: b54bcf4d-1bca-4e8e-9a24-22ff2c3d462c'
];
$qs = http_build_query($parameters); // query string encode the parameters
$request = "{$url}?{$qs}"; // create the request URL
$curl = curl_init(); // Get cURL resource
// Set cURL options
curl_setopt_array($curl, array(
CURLOPT_URL => $request, // set the request URL
CURLOPT_HTTPHEADER => $headers, // set the headers
CURLOPT_RETURNTRANSFER => 1 // ask for raw response instead of bool
));
$response = curl_exec($curl); // Send the request, save the response
print_r(json_decode($response)); // print json decoded response
curl_close($curl); // Close request
?>
When I var_dump($response) I'm getting:
error code: 1020
This is caused by the API detecting a bot/script.
Consider adding a user agent:
PHP cURL how to add the User Agent value OR overcome the Servers blocking cURL requests?
curl_setopt_array($curl, array(
CURLOPT_URL => $request,
CURLOPT_HTTPHEADER => $headers,
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_USERAGENT => 'Mozilla/5.0 (Windows NT 6.2; WOW64; rv:17.0) Gecko/20100101 Firefox/17.0'
));
Now, the script outputs:
stdClass Object
(
[status] => stdClass Object
(
[timestamp] => 2021-08-30T15:06:20.200Z
[error_code] => 0
[error_message] =>
[elapsed] => 0
[credit_count] => 1
[notice] =>
)
[data] => Array
... and a lot more ...

PHP cURL login fails

There is a website http://www.tremorgames.com and I want to do a cURL login. I have a code which I use on my server http://shalva97.x10.mx/tremorlogin/ but it just does not work, but after a lot of testing I decided to install Apache on my PC and tested the exactly same code and it worked.
Here is the code :
<?php
$url="http://www.tremorgames.com/index.php";
$postdata = "loginuser=shalva&loginpassword=shalva&Submit=&saveme=1";
$ch = curl_init();
$f = fopen('request.txt', 'w');
curl_setopt_array($ch, array(
CURLOPT_URL => $url,
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_FOLLOWLOCATION => 1,
CURLOPT_VERBOSE => 1,
CURLOPT_STDERR => $f,
CURLOPT_USERAGENT => 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0',
CURLOPT_POST => true,
CURLOPT_POSTFIELDS => $postdata,
CURLOPT_HEADER => 1,
CURLOPT_HTTPHEADER => array("Accept-Language: en-US;q=0.6,en;q=0.4", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "Connection: keep-alive"),
CURLOPT_COOKIEFILE => "cookies.txt", //Save cookies
CURLOPT_COOKIEJAR => "cookies.txt", //Cookies located
CURLOPT_REFERER => 'http://www.tremorgames.com/index.php',
CURLOPT_ENCODING => 'gzip,deflate'
));
echo curl_exec($ch);
fclose($f);
curl_close($ch);
?>
Executing this same code on both my PC and server, I get different headers, on PC it receives multiple "Set-Cookie" header, but on server it is only one.
Why it does not work on the server? Why it works on my PC?
well after a lot of thinking i found out that it is not only headers is sent but the ip and my location. The site does not allow logins from other countries thats why it didnot loged in, the code is correct and it was all about server location. i hope this will help others who also try to do same thing.

CURL + POST + multipart/form-data

I am trying to scrape a website using PHP, CURL and POST method in order to submit a form before web scraping the page. The problem I am experiencing is that there is connected with POST method: no data is submitted to the server, so the scraped webpage doesn't contain what I am looking for.
I quit sure the problem is connected with the form type: enctype="multipart/form-data".
How can I manage this POST request, considering that the form is multipart/form-data?
Do I have to encode the post_string in a special way?
Here's the code I'm using:
function curl($url) {
//POST string
$post_string="XXXX";
$options = Array(
CURLOPT_RETURNTRANSFER => TRUE,
CURLOPT_FOLLOWLOCATION => TRUE,
CURLOPT_AUTOREFERER => TRUE,
CURLOPT_CONNECTTIMEOUT => 120,
CURLOPT_TIMEOUT => 120,
CURLOPT_MAXREDIRS => 10,
CURLOPT_USERAGENT => "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1a2pre) Gecko/2008073000 Shredder/3.0a2pre ThunderBrowse/3.2.1.8",
CURLOPT_URL => $url,
CURLOPT_CAINFO => dirname(__FILE__)."/cacert.pem",
CURLOPT_POSTFIELDS => $post_string,
);
$ch = curl_init();
curl_setopt_array($ch, $options);
$data = curl_exec($ch);
curl_error($ch);
curl_close($ch);
return $data;
}
$scraped_page = curl("XXXURLXXX");
echo $scraped_page;
Thank you!
Set the CURLOPT_POST to true:
CURLOPT_POST = true
Then fill your post fields like this 'setup':
$postfields = array();
$postfields['field1'] = 'value1';
$postfields['field2'] = 'value2';
CURLOPT_POSTFIELDS => $postfields
If value is an array, the Content-Type header will be set to multipart/form-data.
The PHP manual
Yes, $post_string needs to be an array.
Also set CURLOPT_POST to true.

Yet another blank page with cURL

I have read many similar title questions, but none of them worked for me...
The problem is that when I'm sending an cURL query to a website all I get is a blank page.
Here is my code:
<?php
$action = "http://www.website.com/index.php?section=login&do=process";
$fields = array(
'username' => $user,
'rememberMe' => '1'
);
$login = curl_post($action, $fields);
var_dump($login);
function curl_post($url, array $post = NULL, array $options = array())
{
$defaults = array(
CURLOPT_POST => 1,
CURLOPT_HEADER => 0,
CURLOPT_HTTPHEADER => array('Accept-Language: pl,en-us;q=0.7,en;q=0.3', 'Accept-Charset: ISO-8859-2,utf-8;q=0.7,*;q=0.7'),
CURLOPT_USERAGENT => "Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3",
CURLOPT_URL => $url,
CURLOPT_FRESH_CONNECT => 1,
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_FORBID_REUSE => 1,
CURLOPT_TIMEOUT => 4,
CURLOPT_NOBODY => false,
CURLOPT_POSTFIELDS => http_build_query($post)
);
$ch = curl_init();
curl_setopt_array($ch, ($options + $defaults));
if( !$result = curl_exec($ch))
{
return(curl_error($ch));
}
curl_close($ch);
return $result;
}
?>
Of course I have my cURL enabled in PHP so its not about that.
If you have any ideas, please share!
Update
I have also been trying adding the following lines at the top of my PHP file:
ini_set("display_errors", 1);
error_reporting(E_ALL);
But the problem still appears - result is a blank page. When I use file_get_contents("http://website.com/"); I can see the page content, so it doesnt work with cURL only.
Running this locally and pointing it at Google, I see two things immediately:
PHP Notice: Undefined variable: user
Google returns a 'Error 405 (Method Not Allowed)!!1' error page but probably because I'm trying to post to it
What happens when you define $user and try again?

Categories