I'm trying to make a php api request to simply make a simple paste to pastebin and i found an example at http://pastebin.com/api and it's pretty strait forward so i didnt think there would be any troubles. But the example doesn't seem to be working. I keep getting the response
Bad API request, invalid api_option
But you can clearly see it sets up api_option=paste in the string it creates...
and in the documentation it says
Creating A New Paste, [Required Parameters]
Include all the following POST parameters when you request the URL:
1. api_dev_key - which is your unique API Developers Key.
2. api_option - set as 'paste', this will indicate you want to create a new paste.
3. api_paste_code - this is the text that will be written inside your paste.
Leaving any of these parameters out will result in an error.
So....i figured it looks right, besides its the example they provided.
Anyone have any ideas what is going on here?
<?php
$api_dev_key = '1234'; // your api_developer_key
$api_paste_code = 'some random text to test'; // your paste text
$api_paste_private = '0'; // 0=public 1=private
$api_paste_name = 'savelogtest'; // name or title of your paste
$api_paste_expire_date = '10M';
$api_paste_format = 'php';
$api_user_key = ''; // if invalid key or no key is used, the paste will be create as a guest
$api_paste_name = urlencode($api_paste_name);
$api_paste_code = urlencode($api_paste_code);
$url = 'http://pastebin.com/api/api_post.php';
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, 'api_option=paste&api_user_key='.$api_user_key.'&api_paste_private='.$api_paste_private.'&api_paste_name='.$api_paste_name.'&api_paste_expire_date='.$api_paste_expire_date.'&api_paste_format='.$api_paste_format.'&api_dev_key='.$api_dev_key.'&api_paste_code='.$api_paste_code.'');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_NOBODY, 0);
$response = curl_exec($ch);
echo $response;
?>
The API example works fine. I just ran your code (of course changed the $api_dev_key, and it worked first time. Output: http://pastebin.com/eyn9tWNS
Try and add this at the top of your script:
error_reporting(E_ALL);
ini_set("display_errors", "on");
It should give you some better error report of what is going on.
Related
I am writing a script that does foreign exchange using an API to do it in realtime.
I presume that the problem is in my implementation of curl, as I get no output from this :
$currencyBase = "USD";
$currencyForeign = "EUR";
$url = 'https://www.alphavantage.co/query?function=CURRENCY_EXCHANGE_RATE&from_currency=' . $currencyBase . '&to_currency=' . $currencyForeign . '&apikey=KCUGBA9AP3Z1E2P8';
echo $currencyBase;
// create curl resource
$c = curl_init($url); //Initialize a cURL session
curl_setopt($c, CURLOPT_HEADER, 0); //Set an option for a cURL transfer
curl_setopt($c, CURLOPT_RETURNTRANSFER, 1); //Set an option for a cURL transfer
$this->fxRate = doubleval(curl_exec($c));
curl_close($c); //Close a cURL session
I have hardcoded
$currencyBase = "USD";
$currencyForeign = "EUR";
for debugging and I have also tried print_r($c) to see if I'm passing anything but I still get no output.
I know that my API call works because I have tried the link with entered USD and EUR and I get a response when I enter it in the browser as follows:
https://www.alphavantage.co/query?function=CURRENCY_EXCHANGE_RATE&from_currency=USD&to_currency=EUR&apikey=KCUGBA9AP3Z1E2P8
But I always get an empty response when I print the output from the script.
Your problem is this part of the code: $this->fxRate = doubleval(curl_exec($c));
curl_exec returns a string (which happens to be json), which you are casting to doubleval and that results in 0. Try it like this instead:
$this->fxRate = json_decode(curl_exec($c))->{"Realtime Currency Exchange Rate"}->{"5. Exchange Rate"};
I've programmed a very basic web-scraping tool in PHP using cURL and DOM. I'm running it locally on a Windows 10 box using XAMPP (Apache & MySQL). It scrapes approximately 5 values on 400 pages (~2,000 values in total) on one specific website. The job typically completes in < 120 seconds, but intermittently (about once every 5 runs) it'll stop around the 60 second mark with the following error:
Recv failure: Connection was reset
Probably irrelevant, but all of my scraped data is being thrown into a MySQL table, and a separate .php file is styling the data and presenting it. This part is working fine. The error is being thrown by cURL. Here's my (very trimmed) code:
$html = file_get_html('http://IPAddressOfSiteIAmScraping/subpage/listofitems.html');
//Some code that creates my SQL table.
//Finds all subpages on the site - this part works like a charm.
foreach($html->find('a[href^=/subpage/]') as $uniqueItems){
//3 array variables defined here, which I didn't include in this example.
$path = $uniqueItems->href;
$url = 'http://IPAddressOfSiteIAmScraping' . $path;
//Here's the cURL part - I suspect this is the problem. I am an amateur!
$curl = curl_init($url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_URL, trim($url));
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, 0); //An attempt to fix it - didn't work.
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, 0); //An attempt to fix it - didn't work.
curl_setopt($curl, CURLOPT_CONNECTTIMEOUT, 0);
curl_setopt($curl, CURLOPT_TIMEOUT, 1200); //Amount of time I let cURL execute for.
$page = curl_exec($curl);
//This is the part that throws up the connection reset error.
if(curl_errno($curl)) {
echo 'Scraping error: ' . curl_error($curl);
exit; }
curl_close($curl);
//Here we use DOM to begin collecting specific cURLed values we want in our SQL table.
$dom = new DOMDocument;
$dom->encoding = 'utf-8'; //Alows the DOM to display html entities for special characters like รถ.
#$dom->loadHTML(utf8_decode($page)); //Loads the HTML of the cURLed page.
$xpath = new DOMXpath($dom); //Allows us to use Xpath values.
//Xpaths that I've set - this is for the SQL part. Probably irrelevant.
$header = $xpath->query('(//div[#id="wrapper"]//p)[#class="header"][1]');
$price = $xpath->query('//tr[#class="price_tr"]/td[2]');
$currency = $xpath->query('//tr[#class="price_tr"]/td[3]');
$league = $xpath->query('//td[#class="left-column"]/p[1]');
//Here we collect specifically the item name from the DOM.
foreach($header as $e) {
$temp = new DOMDocument();
$temp->appendChild($temp->importNode($e,TRUE));
$val = $temp->saveHTML();
$val = strip_tags($val); //Removes the <p> tag from the data that goes into SQL.
$val = mb_convert_encoding($val, 'html-entities', 'utf-8'); //Allows the HTML entity for special characters to be handled.
$val = html_entity_decode($val); //Converts HTML entities for special characters to the actual character value.
$final = mysqli_real_escape_string($conn, trim($val)); //Defense against SQL injection attacks by canceling out single apostrophes in item names.
$item['title'] = $final; //Here's the item name, ready for the SQL table.
}
//Here's a bunch of code where I write to my SQL table. Again, this part works great!
}
I am not opposed to switching to regex if I need to ditch DOM, but I did three days worth of lurking before I chose DOM over regex. I have spent a lot of time researching this problem, but everything I'm seeing says "Recv failure: Connection was reset by peer", which is not what I am getting. I'm really frustrated that I have to ask for help - I've been doing so great so far - just learning as I go. This is the first thing I've ever written in PHP.
TL;DR: I wrote a cURL web-scraper that works brilliantly only 80% of the time. 20% of the time, for an unknown reason, it errors out with "Recv failure: Connection was reset".
Hopefully someone can help me!! :) Thanks for reading even if you can't!
P.S. if you'd like to see my FULL code, it's at: http://pastebin.com/vf4s0d5L.
After researching this at length (I'd already been researching it for days before posting my question), I've caved in and accepted that this error is probably tied to the site I'm trying to scrape and therefore out of my control.
I did manage to work around it though, so I'll drop my workaround here...
$curl = curl_init($url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_URL, trim($url));
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($curl, CURLOPT_CONNECTTIMEOUT, 0);
curl_setopt($curl, CURLOPT_TIMEOUT, 1200); //Amount of time I let cURL execute for.
$page = curl_exec($curl);
if(curl_errno($curl)) {
echo 'Scraping error: ' . curl_error($curl) . '</br>';
echo 'Dropping table...</br>';
$sql = "DROP TABLE table_item_info";
if (!mysqli_query($conn, $sql)) {
echo "Could not drop table: " . mysqli_error($conn);
}
mysqli_close($conn);
echo "TABLE has been dropped. Restarting.</br>";
goto start;
exit; }
curl_close($curl);
Basically, what I've done is implemented error-checking. If the error comes up under curl_errno($curl), I assume it's the connection reset error. That being the case, I drop my SQL table and then jump back to the start of my script using "goto start". Then, at the top of my file I have "start:"
This fixed my problem! Now I don't need to worry about whether the connection was reset or not. My code is smart enough to determine that on its own and reset the script if that was the case.
Hope this helps!
<?php error_reporting(0);
$currency_code = $_GET['currency_code'];
$currency_opt = strtoupper($currency_code)."INR";
$jsn_response = file_get_contents('http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20yahoo.finance.xchange%20where%20pair%20in%20%28%22' .$currency_opt. '%22%29&format=json&env=store://datatables.org/alltableswithkeys&callback=');
$currencyrate_arr = json_decode($jsn_response, true);
$currency_rate = $currencyrate_arr['query']['results']['rate']['Rate'];
//var_dump($currency_rate);
if($currency_rate > 0){
echo $currency_text = $currency_rate;
}
else{
echo $currency_text = "SORRY! ERROR..";
}
?>
It was working fine but now I am getting error while using this piece of code for currency conversion.
The script is working for me, and produces correct results.
If you are receiving an error, it's either because of a server configuration, or an API limit.
You can check https://developer.yahoo.com/yql/faq/
Rate limits in YQL are based on your authentication. If you use IP-based authentication, then you are limited to 2,000 calls/hour/IP to the public YQL Web service URL (/v1/public/*)
If you need to exceed the 2000 calls per hour limit, read the above link for more information.
PS: If you want to debug, set:
error_reporting(E_ALL);
ini_set('display_errors', 1);
Edit: Since allow_url_fopen is disabled, you can't use file_get_contents on outside URLs, but you can still use CURL.
So just replace the file_get_contents with:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20yahoo.finance.xchange%20where%20pair%20in%20%28%22' .$currency_opt. '%22%29&format=json&env=store://datatables.org/alltableswithkeys&callback=');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$jsn_response = curl_exec($ch);
curl_close($ch);
I'll try to be short: if you need more info, I'll tell you.
I'm using this code to get infos from Google Maps:
<?php
function getData($url) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0); //Change this to a 1 to return headers
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
$url = 'http://maps.google.com/maps/geo?output=xml&q=' . urlencode($startPlace);
$url = 'http://maps.google.com/maps/api/geocode/json?sensor=false&gl=IT&address=' . urlencode($startPlace);
$xml = simplexml_load_string($this->getData($this->url)) or die("Error loading xml data");
$points = $xml->Response->Placemark->Point->coordinates;
$provincia = $xml->Response->Placemark->AddressDetails->Country->AdministrativeArea->SubAdministrativeArea->SubAdministrativeAreaName;
$regione =$xml->Response->Placemark->AddressDetails->Country->AdministrativeArea->AdministrativeAreaName;
echo $regione."<br>";
preg_match_all("/-*[0-9.]*(?=,)/", $points[0], $matches);
$longitude = $matches[0][0];
$latitude = $matches[0][2];
The code is used to retrieve infos about italian locations and till three days ago, all worked fine, but this morning I saw something strange: $regione returned by code ($xml->Response->Placemark->AddressDetails->Country->AdministrativeArea->AdministrativeAreaName;) had an english name.
Let's say the location found be a little town in Lombardia (where 'Lombardia' is the name of the Administrative Area), the Administartive Area name returned by Google Maps was no more 'Lombardia' but 'Lombardy'.
Since this data is used to search in a local database other places in the Administrative area and since the name used in the database is obviously italian name, application doesnìt work anymore.
I'll be grateful for any advice
The problem is solved using a different url, specifying language parameter:
'http://maps.google.com/maps/api/geocode/xml?sensor=false&language=IT&address=' . urlencode($startPlace);
This url type return correct results but defferently formed so it is necessary change the code to access the infos and put them into variables, but this solved my problem
I'm trying to run a CURL script in wordpress but I'm having a problem.
When i test it, i get a 500 internal error as WP changes the URL.
So the script is at www.site.com/curl_script.php - When i test that (navigate to www.site.com/curl_script.php) I end up going to www.site.com/curl_script.php/wp-admin/install.php which returns a 500 internal error.
Now after playing around with the script, I've noticed the problem. It seems to be a function that I'm running (the curl function) thats causing wordpress to take me to that url.
Ive had similar issues to this but have managed to fix it by simply changing the names of the functions, but this doesn't seem to work anymore.
The function:
function verify_user($ref, $username, $uu_name){
$ch = curl_init($server_root);
curl_setopt($ch,CURLOPT_URL,"http://site.com/con1.php");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch,CURLOPT_POSTFIELDS,$fields_string);
curl_setopt($ch, CURLOPT_POST, 1);
$result = curl_exec($ch);
$data = json_decode($result);
global $ref_;
$ref_ = $data->ref_id;
//fetch some more info
$chh = curl_init($server_root);
curl_setopt($chh,CURLOPT_URL,"http://site.com/con2.php");
curl_setopt($chh, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($chh, CURLOPT_POST, 1);
$resultt_2 = curl_exec($chh);
$data_custt = json_decode($resultt_2);
$cust_st = $data__->user_status;
if ($cust_st == "FAILED"){
echo "this is bad";
}
elseif ($cust_st == "PASSED") {
echo "this is good";
}
}
}
Now when i call this function:
verify_user_info($ref, $username, $uu_name);
Wordpress plays up...
But when i leave the function out (don't call it), everything works fine.
It seems that WP is assuming the user is attempting to run the installation, when that's not the case.
Any ideas on how to fix this, dynamically as others will use this script too?
If sounds like you are getting redirected somehow, even though should shouldn't be if CURLOPT_FOLLOWLOCATION is not set. Try using the curl_getinfo function to debug the URL that is being accessed.