cURL Geocode OVER_QUERY_LIMIT - php

I have a problem with google's geocoding api:
I have an address which should be converted to latitude and longitude by google geocoding api using cURL. You will see my code below. It worked fine, however suddenly it stopped working and I got an "OVER_QUERY_LIMIT" answer. I looked it up and it normally happens if api is requested more than 2500 times a day. This is impossible, because my website is just upon finishing and has about 20-40 geocode requests per day.
So what is really the problem that "OVER_QUERY_LIMIT" occurs? Is something wrong with my code that somehow google blocks it?!
ini_set('allow_url_fopen', true);
$CookiePath = 'mycookiepath/geocookie.txt';
$userAgent = "mysite.com";
$ListURLRoot = "http://maps.googleapis.com/maps/api/geocode/json";
$ListURLSuffix = '&sensor=false';
$Curl_Obj = curl_init(); // Setup cURL
curl_setopt($Curl_Obj, CURLOPT_COOKIEJAR, $CookiePath);
curl_setopt($Curl_Obj, CURLOPT_USERAGENT, $userAgent);
curl_setopt($Curl_Obj, CURLOPT_HEADER, 0);
curl_setopt($Curl_Obj, CURLOPT_AUTOREFERER, TRUE);
curl_setopt($Curl_Obj, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($Curl_Obj, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($Curl_Obj, CURLOPT_TIMEOUT, 30);
curl_setopt($Curl_Obj, CURLOPT_POST, 0);
$stAddr = str_replace(' ','+', $ast);
$City = str_replace(' ','+', $ac);
$address = "$stAddr,+$City,+{$aco},+{$ap}";
$address = str_replace(' ', '+', $address);
$ListURL = "{$ListURLRoot}?address=$address$ListURLSuffix";
curl_setopt ($Curl_Obj, CURLOPT_URL, $ListURL);
$output = curl_exec ($Curl_Obj);
GetLocation($output);
function GetLocation($output) {
global $db_connect;
$Loc = json_decode($output, true);
if(isset($Loc)) {
$i = 1;
while ($Loc['status']=="OVER_QUERY_LIMIT") {
if ($i > 14) {
echo "Geocode failed!";
exit();
}
sleep(1);
$i++;
}
if(isset($Loc['status']) && stristr($Loc['status'], 'OK')) {
if(isset($Loc['results'][0]['geometry']['location'])) {
$Lat = $Loc['results'][0]['geometry']['location']['lat'];
$Lng = $Loc['results'][0]['geometry']['location']['lng'];
}
}
else {
error_log($Loc['status'], 0);
echo "Unknown error occured!";
exit();
}
}
}
Thanks in advance!

Mostly it's a Google problem. Try to avoid server side geocodeing. Instead use a client side javascript solver.

Related

Google Maps Geocode to include name of premises

Using the code below which works well:
function geocode($address){
$return = array();
$address = urlencode($address);
$key = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX";
$url = "https://maps.google.com/maps/api/geocode/json?key=$key&address={$address}";
if (!function_exists("\curl_init")){
\load_curl();
}
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_TIMEOUT,6000);
curl_setopt($ch, CURLOPT_FRESH_CONNECT, TRUE);
$resp_json = \curl_exec($ch);
curl_close($ch);
$resp = json_decode($resp_json, true);
if($resp['status']!=='OK') return false;
foreach($resp['results'] as $res){
$loc = array(
"zipcode"=>null,
"formatted"=>null
);
foreach($res['address_components'] as $comp){
if(in_array("postal_code", $comp['types']))
$loc['zipcode'] = $comp['short_name'];
}
$loc['formatted'] = $res['formatted_address'];
$loc['lng'] = $res['geometry']['location']['lng'];
$loc['lat'] = $res['geometry']['location']['lat'];
$return[] = $loc;
}
return $resp;
}
However when I search for an address such as "ZOOM DIGITAL PRINT UNIT 36 BINLEY IND EST HOTCHKISS WAY COVENTRY CV3 2RL" the formatted address is brought back as "Starley Court, Hotchkiss Way, Coventry CV3 2RL, UK"
Is there a way to get the business name included in this request or can it be found by another API?
The Geocoding API is used to convert addresses into lat/lng coordinates and vice-versa or to find addresses for a given place ID.
To get business information you should use the Places API. E.g. try searching for "Zoom Digital Print Ltd" in Google's Autocomplete example and you'll get this business name.
Hope this helps!

CURL 404 (Yahoo)

I was testing an script that comprove if the domain its working or not and I tested 50 sites (that should work) but just one site (Yahoo) returned "Died" (Not Working) and it returns 404, I'm not sure if the problem is from my code or from the website, I think the problem is the "Redirection" but I don't know what to do.
How can I follow it?
$host = "www.yahoo.com";
$ip = gethostbyname($host);
$domain = $ip;
//Starting process to check the domain
$check = curl_init($domain);
curl_setopt($check, CURLOPT_TIMEOUT, 10);
curl_setopt($check, CURLOPT_CONNECTTIMEOUT, 10);
curl_setopt($check, CURLOPT_RETURNTRANSFER, true);
$data = curl_exec($check);
$httpcode = curl_getinfo($check, CURLINFO_HTTP_CODE);
curl_close($check);
if ($httpcode >= 200 && $httpcode <= 350) {
$data_array[$key]['status'] = "Alive";
} else {
$data_array[$key]['status'] = "Died";
}
}
I changed the URL like $host = "www.yahoo.com/"; and added curl_setopt($check, CURLOPT_FOLLOWLOCATION, 1); it is work for me now.

Save facebook profile image using cURL

I'm trying to save a users profile image on facebook using CURL. When I use the code below, I save a jpeg image but it has zero bytes in it. But if I exchange the url value to https://fbcdn-profile-a.akamaihd.net/hprofile-ak-snc4/211398_812269356_2295463_n.jpg, which is where http://graph.facebook.com/' . $user_id . '/picture?type=large redirects the browser, the image is saved without a problem. What am I doing wrong here?
<?php
$url = 'http://graph.facebook.com/' . $user_id . '/picture?type=large';
$file_handler = fopen('pic_facebook.jpg', 'w');
$curl = curl_init($url);
curl_setopt($curl, CURLOPT_FILE, $file_handler);
curl_setopt($curl, CURLOPT_HEADER, false);
curl_exec($curl);
curl_close($curl);
fclose($file_handler);
?>
There is a redirect, so you have to add this option for curl
// safemode if off:
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
but if you have safemode if on, then:
// safemode if on:
<?php
function curl_redir_exec($ch)
{
static $curl_loops = 0;
static $curl_max_loops = 20;
if ($curl_loops++ >= $curl_max_loops)
{
$curl_loops = 0;
return FALSE;
}
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$data = curl_exec($ch);
#list($header, $data) = #explode("\n\n", $data, 2);
$http_code = curl_getinfo($ch, CURLINFO_HTTP_CODE);
if ($http_code == 301 || $http_code == 302)
{
$matches = array();
preg_match('/Location:(.*?)\n/', $header, $matches);
$url = #parse_url(trim(array_pop($matches)));
if (!$url)
{
//couldn't process the url to redirect to
$curl_loops = 0;
return $data;
}
$last_url = parse_url(curl_getinfo($ch, CURLINFO_EFFECTIVE_URL));
if (!$url['scheme'])
$url['scheme'] = $last_url['scheme'];
if (!$url['host'])
$url['host'] = $last_url['host'];
if (!$url['path'])
$url['path'] = $last_url['path'];
$new_url = $url['scheme'] . '://' . $url['host'] . $url['path'] . (#$url['query']?'?'.$url['query']:'');
return $new_url;
} else {
$curl_loops=0;
return $data;
}
}
function get_right_url($url) {
$curl = curl_init($url);
curl_setopt($curl, CURLOPT_HEADER, false);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
return curl_redir_exec($curl);
}
$url = 'http://graph.facebook.com/' . $user_id . '/picture?type=large';
$file_handler = fopen('pic_facebook.jpg', 'w');
$curl = curl_init(get_right_url($url));
curl_setopt($curl, CURLOPT_FILE, $file_handler);
curl_setopt($curl, CURLOPT_HEADER, false);
curl_exec($curl);
curl_close($curl);
fclose($file_handler);
If you can't process the redirect, try this instead:
Make the request to https://graph.facebook.com/<USER ID>?fields=picture and parse the response, which will be in JSON format and look like this - e.g. for Zuck you get this response:
{
"picture": "http://profile.ak.fbcdn.net/hprofile-ak-snc4/157340_4_3955636_q.jpg"
}
Then make your curl request directly to retrieve the image from that cloud storage URL
set
CURLOPT_FOLLOWLOCATION to true
so that it follows the 301/302 redirect the reads the image file from final location.
i.e.
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
I managed to do it this way, works perfectly fine:
$data = file_get_contents('https://graph.facebook.com/[App-Scoped-ID]/picture?width=378&height=378&access_token=[Access-Token]');
$file = fopen('fbphoto.jpg', 'w+');
fputs($file, $data);
fclose($file);
You just need an App Access Token (APPID . '|' . APPSECRET), and you can specify width and height.
You can also add "redirect=false" to the URL, to get a JSON object with the URL (For example: https://fbcdn-profile-a.akamaihd.net/hprofile-ak-xpa1...)
CURLOPT_FOLLOWLOCATION has been removed in PHP5.4, so it´s not really an option anymore.

How to get Google +1 count for current page in PHP?

I want to get count of Google +1s for current web page ? I want to do this process in PHP, then write number of shares or +1s to database. That's why, I need it. So, How can I do this process (getting count of +1s) in PHP ?
Thanks in advance.
This one works for me and is faster than the CURL one:
function getPlus1($url) {
$html = file_get_contents( "https://plusone.google.com/_/+1/fastbutton?url=".urlencode($url));
$doc = new DOMDocument(); $doc->loadHTML($html);
$counter=$doc->getElementById('aggregateCount');
return $counter->nodeValue;
}
also here for Tweets, Pins and Facebooks
function getTweets($url){
$json = file_get_contents( "http://urls.api.twitter.com/1/urls/count.json?url=".$url );
$ajsn = json_decode($json, true);
$cont = $ajsn['count'];
return $cont;
}
function getPins($url){
$json = file_get_contents( "http://api.pinterest.com/v1/urls/count.json?callback=receiveCount&url=".$url );
$json = substr( $json, 13, -1);
$ajsn = json_decode($json, true);
$cont = $ajsn['count'];
return $cont;
}
function getFacebooks($url) {
$xml = file_get_contents("http://api.facebook.com/restserver.php?method=links.getStats&urls=".urlencode($url));
$xml = simplexml_load_string($xml);
$shares = $xml->link_stat->share_count;
$likes = $xml->link_stat->like_count;
$comments = $xml->link_stat->comment_count;
return $likes + $shares + $comments;
}
Note: Facebook numbers are the sum of likes+shares and some people said plus comments (I didn't search this yet), anyway use the one you need.
This will works if your php settings allow open external url, check your "allow_url_open" php setting.
Hope helps.
function get_plusones($url) {
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, "https://clients6.google.com/rpc");
curl_setopt($curl, CURLOPT_POST, 1);
curl_setopt($curl, CURLOPT_POSTFIELDS, '[{"method":"pos.plusones.get","id":"p","params":{"nolog":true,"id":"' . $url . '","source":"widget","userId":"#viewer","groupId":"#self"},"jsonrpc":"2.0","key":"p","apiVersion":"v1"}]');
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_HTTPHEADER, array('Content-type: application/json'));
$curl_results = curl_exec ($curl);
curl_close ($curl);
$json = json_decode($curl_results, true);
return intval( $json[0]['result']['metadata']['globalCounts']['count'] );
}
echo get_plusones("http://www.stackoverflow.com")
from internoetics.com
The cURL and API way listed in the other posts here no longer works.
There is still at least 1 method, but it's ugly and Google clearly doesn't support it. You just rip the variable out of the JavaScript source code for the official button with a regular expression:
function shinra_gplus_get_count( $url ) {
$contents = file_get_contents(
'https://plusone.google.com/_/+1/fastbutton?url='
. urlencode( $url )
);
preg_match( '/window\.__SSR = {c: ([\d]+)/', $contents, $matches );
if( isset( $matches[0] ) )
return (int) str_replace( 'window.__SSR = {c: ', '', $matches[0] );
return 0;
}
The next PHP script works great so far for retrieving Google+ count on shares and +1's.
$url = 'http://nike.com';
$gplus_type = true ? 'shares' : '+1s';
/**
* Get Google+ shares or +1's.
* See out post at stackoverflow.com/a/23088544/328272
*/
function get_gplus_count($url, $type = 'shares') {
$curl = curl_init();
// According to stackoverflow.com/a/7321638/328272 we should use certificates
// to connect through SSL, but they also offer the following easier solution.
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
if ($type == 'shares') {
// Use the default developer key AIzaSyCKSbrvQasunBoV16zDH9R33D88CeLr9gQ, see
// tomanthony.co.uk/blog/google_plus_one_button_seo_count_api.
curl_setopt($curl, CURLOPT_URL, 'https://clients6.google.com/rpc?key=AIzaSyCKSbrvQasunBoV16zDH9R33D88CeLr9gQ');
curl_setopt($curl, CURLOPT_POST, 1);
curl_setopt($curl, CURLOPT_POSTFIELDS, '[{"method":"pos.plusones.get","id":"p","params":{"nolog":true,"id":"' . $url . '","source":"widget","userId":"#viewer","groupId":"#self"},"jsonrpc":"2.0","key":"p","apiVersion":"v1"}]');
curl_setopt($curl, CURLOPT_HTTPHEADER, array('Content-type: application/json'));
}
elseif ($type == '+1s') {
curl_setopt($curl, CURLOPT_URL, 'https://plusone.google.com/_/+1/fastbutton?url='.urlencode($url));
}
else {
throw new Exception('No $type defined, possible values are "shares" and "+1s".');
}
$curl_result = curl_exec($curl);
curl_close($curl);
if ($type == 'shares') {
$json = json_decode($curl_result, true);
return intval($json[0]['result']['metadata']['globalCounts']['count']);
}
elseif ($type == '+1s') {
libxml_use_internal_errors(true);
$doc = new DOMDocument();
$doc->loadHTML($curl_result);
$counter=$doc->getElementById('aggregateCount');
return $counter->nodeValue;
}
}
// Get Google+ count.
$gplus_count = get_gplus_count($url, $gplus_type);
Google does not currently have a public API for getting the +1 count for URLs. You can file a feature request here. You can also use the reverse engineered method mentioned by #DerVo. Keep in mind though that method could change and break at anytime.
I've assembled this code to read count directly from the iframe used by social button.
I haven't tested it on bulk scale, so maybe you've to slow down requests and/or change user agent :) .
This is my working code:
function get_plusone($url)
{
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, "https://plusone.google.com/_/+1/fastbutton?
bsv&size=tall&hl=it&url=".urlencode($url));
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
$html = curl_exec ($curl);
curl_close ($curl);
$doc = new DOMDocument();
$doc->loadHTML($html);
$counter=$doc->getElementById('aggregateCount');
return $counter->nodeValue;
}
Usage is the following:
echo get_plusones('http://stackoverflow.com/');
Result is: 3166
I had to merge a few ideas from different options and urls to get it to work for me:
function getPlusOnes($url) {
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, "https://plusone.google.com/_/+1/fastbutton?url=".urlencode($url));
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
$html = curl_exec ($curl);
curl_close ($curl);
$doc = new DOMDocument();
$doc->loadHTML($html);
$counter=$doc->getElementById('aggregateCount');
return $counter->nodeValue;
}
All I had to do was update the url but I wanted to post a complete option for those interested.
echo getPlusOnes('http://stackoverflow.com/')
Thanks to Cardy for using this approach, then I just had to just get a url that worked for me...
I've released a PHP library retrieving count for major social networks. It currently supports Google, Facebook, Twitter and Pinterest.
Techniques used are similar to the one described here and the library provides a mechanism to cache retrieved data. This library also have some other nice features: installable through Composer, fully tested, HHVM support.
http://dunglas.fr/2014/01/introducing-the-socialshare-php-library/

Reverse Geocoding Responses randomly in php

I have tried google reverse geocode.Following function called in for loop multiple times...this Works randomly...Sometimes response address perfectly..sometimes no response got...What is the problem here...
function reversegeo($ilatt,$ilonn)
{
$url1='http://maps.googleapis.com/maps/api/geocode/json?latlng='.$ilatt.','.$ilonn.'&sensor=false';
$ch1 = curl_init();
curl_setopt($ch1, CURLOPT_URL, $url1);
curl_setopt($ch1, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch1, CURLOPT_REFERER, 'http://www.mywebsiteurl.com/Trackfiles/report.php');
$body1 = curl_exec($ch1);
curl_close($ch1);
$json1 = json_decode($body1);
$add=$json1->results[0]->formatted_address;
return $add;
}
You are probably hitting the server too often, or too fast. Add some delays in there with sleep().
Also when you say "no response got" you need to be more specific. Google will give an error code if you are hitting it too often, it won't just be nothing.
Instead of:
$body1 = curl_exec($ch1);
Do:
if(($body1 = curl_exec($ch1)) === false) {
echo 'Curl error: ' . curl_error($ch);
}

Categories