I'm using the Google AdWords PHP API to access statistics from our account. However, I'm getting some really strange read outs from the statistics through the api. I'm trying to access the stats for individuals Ads or Adgroups. The statistics returned, however, are way off what they are in the client center. The code I'm using:
$user->SetClientCustomerId($clientId);
$adService = $user->GetService("AdGroupAdService", ADWORDS_VERSION);
$selector = new Selector();
$selector->fields = array("Id", "Name", "Clicks", "Impressions", "Cost");
$selector->predicates[] = new Predicate("AdGroupId", "IN", array($adGroupId));
$selector->dateRange = $dateRange;
$selector->paging = new Paging(0, AdWordsConstants::RECOMMENDED_PAGE_SIZE);
do {
// Make the get request.
$page = $adService->get($selector);
if (isset($page->entries)) {
foreach ($page->entries as $ad) {
$newLineObject->adName = $ad->name;
$newLineObject->clicks = $ad->ad->AdStats->clicks;
$newLineObject->impressions = $ad->adStats->impressions;
$newLineObject->cost = $ad->ad->AdStats->cost->microAmount/ AdWordsConstants::MICROS_PER_DOLLAR;
}
}
else {
print "No matching ads were found.\n";
}
$selector->paging->startIndex += AdWordsConstants::RECOMMENDED_PAGE_SIZE;
} while ($page->totalNumEntries > $selector->paging->startIndex);
When I print the results I get numbers that are considerably larger than those displayed in the client center. For example, for one partiuclar Ad the API reported 2.000.000 impressions, while the client center showed 56.000.
What am I doing wrong?
Your code seems correct to me. However, you problem may be that your date range in your code is different to the one you see in your client center. Make sure that you keep the same date range when you cross check.
Having tried using the method detailed above extensively, I have altered my code completely. I now use AdHoc Reporting (described here https://developers.google.com/adwords/api/docs/guides/reporting). This method was suggested to me by an AdWords developer. While this does not literally solve my question (i.e. why does the above code return incorrect statistics), it does provide an easy and clean way to obtain the data correctly.
Related
I am trying to check for toll-free numbers, and it is working as expected. However, my problem is that some countries don't have the TollFree numbers.
Using the same code on these countries throws a 404 error and stops the code there.
The only way I could think of is making a massive if statement and adding each country manually which offers toll-free option, but I don't like this solution at all as it will be hardcoded. Is there a way to overcome this issue, so it works for the countries that has the .json and ignore the ones that doesn't (instead of crashing the code)?
$twilio = new Client(env('TWILIO_ID'), env('TWILIO_TOKEN'));
$iso = 'CY';
$params = ["excludeLocalAddressRequired" => "true"];
$tollFreeNumbers = $twilio->availablePhoneNumbers($iso)->tollFree->read($params);
This is the response:
"[HTTP 404] Unable to fetch page: The requested resource /2010-04-01/Accounts/ACxxxxx/AvailablePhoneNumbers/CY/TollFree.json was not found"
Using this code will crash with CY but will work with UK, US, CA and many more. Should I add an if statement with hardcoded countries? (I really dislike this solution, but this is what I can think of). What I mean is:
if ($iso == 'GB' || $iso == 'US' || $iso == 'CA') { // and many more
$tollFreeNumbers = $twilio->availablePhoneNumbers($iso)->tollFree->read($params);
}
Twilio developer evangelist here.
Rather than guarding up front with a conditional (which could become out of date as we add toll free numbers in other countries in the future), why not catch the error and return a message to the user to say that toll free numbers are not available in the country they are searching in.
Something like:
try {
$tollFreeNumbers = $twilio->availablePhoneNumbers($iso)->tollFree->read($params);
} catch (Exception $e) {
$tollFreeNumbers = [];
$message = "Toll free numbers are not available in this country.";
}
Let me know if that helps at all.
Why not just wrap it in a try catch?
try {
$tollFreeNumbers = $twilio->availablePhoneNumbers($iso)->tollFree->read($params);
} catch(\Exception $e) {
$tollFreeNumbers = [];
}
I'm trying to audit a vast amount of company data from companycheck.co.uk my current script appears to be looping the first 10 results from only the first page. I had the script gather more than 10 results at one point, but this caused a fatal error after around 600 results (not a timeout error, but a connection error of some sort), I need the script to be more reliable as I'm fetching over 40,000 results.
My code so far:
<?php
set_time_limit(0);
ini_set('max_execution_time', 0);
require 'vendor/autoload.php';
require "Guzzle/guzzle.phar";
// Add this to allow your app to use Guzzle and the Cookie Plugin.
use Guzzle\Http\Client as GuzzleClient;
use Guzzle\Plugin\Cookie\Cookie;
use Guzzle\Plugin\Cookie\CookiePlugin;
use Guzzle\Plugin\Cookie\CookieJar\ArrayCookieJar;
use Guzzle\Plugin\Cookie\CookieJar\CookieJarInterface;
$Pagesurl = 'http://companycheck.co.uk/search/UpdateSearchCompany?searchTerm=cars&type=name';
$pagesData = json_decode(file_get_contents($Pagesurl), true);
$resultsFound = $pagesData["hits"]["found"];
$pages = ceil($resultsFound / 10);
//echo $pages;
echo "<br>";
for ($p = 0; $p < $pages; $p++) {
$url = 'http://companycheck.co.uk/search/UpdateSearchCompany?searchTerm=cars&type=name&companyPage=' . $p . '';
$data = json_decode(file_get_contents($url), true);
for ($i = 0; $i < 11; $i++) {
$id = $data["hits"]["hit"][$i]["id"];
$TradingAddress = $data["hits"]["hit"][$i]["data"]["address"][0];
$companyName = $data["hits"]["hit"][$i]["data"]["companyname"][0];
$companyNumber = $data["hits"]["hit"][$i]["data"]["companynumber"][0];
$finalURL = "http://companycheck.co.uk/company/" . $id . "";
$httpClient = new GuzzleClient($finalURL);
$httpClient->setSslVerification(FALSE);
$cookieJar = new ArrayCookieJar();
// Create a new cookie plugin
$cookiePlugin = new CookiePlugin($cookieJar);
// Add the cookie plugin to the client
$httpClient->addSubscriber($cookiePlugin);
$httpClient->setUserAgent("Opera/9.23 (Windows NT 5.1; U; en-US)");
$request = $httpClient->get($finalURL);
$response = $request->send();
$body = $response->getBody(true);
$matches = array();
preg_match_all('/<table.*?>(.*?)<\/table>/si', $body, $table);
preg_match('/<meta name=\"keywords\" content=\"(.*?)\"\/>/si', $body, $metaName);
preg_match('/<p itemprop="streetAddress".*?>(.*?)<\/p>/si', $body, $regOffice);
echo "<table><tbody>";
echo "<tr><th>Company Name</th><td>";
echo $companyName;
echo "</td></tr>";
echo "<tr><th>Company Number</th><td>";
echo $companyNumber;
echo "</td></tr>";
echo "<tr><th>Registar Address</th><td>";
echo str_replace("<br>", " ", $regOffice[0]);
echo "</td></tr>
<tr><th>Trading Address</th><td>";
echo $TradingAddress;
echo "</td></tr>
<tr>
<th>Director Name</th>
<td>";
$name = explode(',', $metaName[1]);
echo $name[2];
echo "</td>
</tr></tbody></table>";
echo $table[0][1];
echo "<br><br><br>";
}
}
To get each page, I use http://companycheck.co.uk/search/UpdateSearchCompany?searchTerm=cars&type=name&companyPage=1 which returns json for each page from http://companycheck.co.uk/search/results?SearchCompaniesForm[name]=cars&yt1= and some data, but not all.
With this I can get the ID of each company to navigate to each link and scrape some data from the frontend of the site.
For example the first result is:
"hits":{"found":42842,"start":0,"hit":[{"id":"08958547","data":{"address":["THE ALEXANDER SUITE SILK POINT, QUEENS AVENUE, MACCLESFIELD, SK10 2BB"],"assets":[],"assetsnegative":[],"cashatbank":[],"cashatbanknegative":[],"companyname":["CAR2CARS LIMITED"],"companynumber":["08958547"],"dissolved":["0"],"liabilities":[],"liabilitiesnegative":[],"networth":[],"networthnegative":[],"postcode":["SK10 2BB"],"siccode":[]}}
So the first link is: http://companycheck.co.uk/company/08958547
Then from this I can pull table data such as:
Registered Office
THE ALEXANDER SUITE SILK POINT
QUEENS AVENUE
MACCLESFIELD
SK10 2BB
And information from the meta tags such as:
<meta name="keywords" content="CAR2CARS LIMITED, 08958547,INCWISE COMPANY SECRETARIES LIMITED,MR ROBERT CARTER"/>
An example of one of the results returned:
Company Name CAR2CARS LIMITED
Company Number 08958547
Registar Address
THE ALEXANDER SUITE SILK POINT QUEENS AVENUE MACCLESFIELD SK10 2BB
Trading Address THE ALEXANDER SUITE SILK POINT, QUEENS AVENUE, MACCLESFIELD, SK10 2BB
Director Name INCWISE COMPANY SECRETARIES LIMITED
Telephone No telephone number available.
Email Address No email address available.
Contact Person No contact person available.
Business Activity No Business Activity on record.
Each json page contains 10 company IDs to put into the URL to find the company, from each of these companies I need to scrape data from the full URL, then after these 10 move onto the next page and get the next 10 and loop this up until the last page.
It is almost certainly blocking you deliberately due to an excessive number of requests. Try putting a pause in between requests - that might help you fly under their radar.
The website you are intending to scrape appears to be a private company that is reformatting and republishing data from Companies House, the official record of company information in the UK. This company offers an API which allows 10K requests per month, and this is either free or costs GBP200/month, depending on what data you need. Since you want 40K results immediately, it is no wonder they operate IP blocks.
The rights and wrongs of scraping are complicated, but there is an important point to understand: by copying someone else's data, you are attempting to avoid the costs of collating the data yourself. By taking them from someone else's server, you are also adding to their operational costs without reimbursing them, an economic phenomenon known as an externality.
There are some cases where I am sympathetic to passing on costs in this way, such as where the scrape target is engaged in potential market abuse (e.g. monopolistic practices) and scraping has an alleviating effect. I have heard that some airline companies operate anti-scraping devices because they don't want price scrapers to bring prices down. Since bringing prices down is in the interest of the consumer one could argue that the externality can be justified (on moral, if not legal grounds).
In your case, I would suggest obtaining this data directly from Companies House, where it might be available for a much lower cost. In any case, if you republish valuable data obtained from a scrape, having dodged technical attempts to block you, you may find yourself in legal trouble anyway. If in doubt (and if there is no moral or public interest defence such as I outlined earlier) get in touch with the site operator and ask if what you want to do is OK.
I searched for this but most of the questions related to this are for API's with other services.
I'm building an API that allows game developers to send and retrieve user info from my database.
I was finally able to put together the API, but now I need to call the API.
1st when the game initiates, it sends us the game developers key their developer id and game id.
//Game loads, get developer key, send token and current high score
// == [ FIRST FILTER - FILTER GET REQUEST ] == //
$_GET = array_map('_INPUT', $_GET); // filter all input
// ====================================== //
// ============[ ACTION MENU ]=========== //
// ====================================== //
if(!empty($_GET['action']) && !empty($_GET['user']) && !empty($_GET['key']) && !empty($_GET['email']) && !empty($_GET['password'])): // if key data exists
switch($_GET['action']):
//athenticate game developer return and high score
case 'authenticate':
$db = new PDO('mysql:host=localhost;dbname=xxxx', 'xxxx', 'xxxx');
$db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_WARNING);
$st = $db->prepare("SELECT * FROM `game_developers_games` WHERE `id` = :gameid AND `developer_id`=:user AND `key`= :key AND `developer_active` = '1'"); // need to filter for next auction
$st->bindParam(':user', $_GET['user']); // filter
$st->bindParam(':key', $_GET['key']); // filter
$st->execute();
$r = $st->fetch(PDO::FETCH_ASSOC);
if($st->rowCount() == 0):
$return = array('DBA_id'=>'0000');
echo json_encode($return);
else:
$token = initToken($_GET['key'],$_GET['user']);
if($token == $r['API_Token']):
$return = array(
'DBA_id'=>$token,
'DBA_servertime'=>time(),
'DBA_highscore'=>$r['score'],
);
echo json_encode($return);
endif;
endif;
break;
Here's the script the game developer will have to add to their game to get the data when the game loads. Found this on another stackoverflow question but it's not working.
$.getJSON("https://www.gamerholic.com/gamerholic_api/db_api_v1.php? user=1&key=6054abe3517a4da6db255e7fa27f4ba001083311&gameid=1&action=authenticate", function () {
alert("aaa");
});
Try adding &callback=? to the end of the url you are constructing. This will enable jsonp that is accepted by cors.
$.getJSON("https://www.gamerholic.com/gamerholic_api/db_api_v1.php?user=1&key=6054abe3517a4da6db255e7fa27f4ba001083311&gameid=1&action=authenticate&callback=?", function () {
alert("aaa");
});
As per cross domain origin policy you cannot access cross domain url using jquery getJson function.
A callback is required to manage cross domain request using json and it needs to be handled on the server as well as the client end.
Also make sure to check the response using firebug or similar tool because as of now it is returning response code as 200.
I am mentioning two threads here which can guide you the right way
Jquery getJSON cross domain problems
http://www.fbloggs.com/2010/07/09/how-to-access-cross-domain-data-with-ajax-using-jsonp-jquery-and-php/
As part of a PHP webapp I have MySQL contacts table. It is integrated throughout the app, allowing you add a contact, edit a contact or add a contact as a relation to another table. However, currently it is self-contained. The company would like it to sync with Exchange, so that contacts added to Exchange will show up on the webapp and contacts added on the webapp will show up through Exchange.
So I have two problems: 1) communicating with Exchange 2) syncing with Exchange.
As far as the basic communication goes, it looks like this library will be able to manage it https://github.com/jamesiarmes/php-ews. However, I am quite lost as to how to manage syncing and don't where to start.
The build-in way to sync items is via function called SyncFolderItems. Basically to Exchange everything, including contacts is a folder, so you'll just pass CONTACTS as DistinguishedFolderId in your sync request.
The sync works by donloading all the items for given account in batches of max 512 elements and after each batch it gives you SyncState as a refernce point for Exchange to know where you left off. So it gives you ability to do incremental sync.
Now, that's one way of course, meaning Exchange -> Your DB. The other way it aeound you should preform atomic updates/request - the moment you change/add/delete item form your db you should issue adequate request to Exchange server to keep data in sync, elese it'll be overwritten on your next SyncFolderItems.
You can read up more on SyncFolderItems # MSDN
If you'd like to see example of SyncFolderItems you can take a look # python version of EWSWrapper, it's been added in recently. Although it's python, you can still get the basic idea how to construct the request / handle response.
Hope this helps :)
I am aware that this topic is pretty old. However, for future reference find a solution below. It is using the above-mentioned library php-ews.
I have also just added this to the official php-ews wiki: https://github.com/jamesiarmes/php-ews/wiki/Calendar:-Synchronization
// Define EWS
$ews = new ExchangeWebServices($host, $username, $password, $version);
// fill with string from last sync
$sync_state = null;
$request = new EWSType_SyncFolderItemsType;
$request->SyncState = $sync_state;
$request->MaxChangesReturned = 512;
$request->ItemShape = new EWSType_ItemResponseShapeType;
$request->ItemShape->BaseShape = EWSType_DefaultShapeNamesType::ALL_PROPERTIES;
$request->SyncFolderId = new EWSType_NonEmptyArrayOfBaseFolderIdsType;
$request->SyncFolderId->DistinguishedFolderId = new EWSType_DistinguishedFolderIdType;
$request->SyncFolderId->DistinguishedFolderId->Id = EWSType_DistinguishedFolderIdNameType::CALENDAR;
$response = $ews->SyncFolderItems($request);
$sync_state = $response->ResponseMessages->SyncFolderItemsResponseMessage->SyncState;
$changes = $response->ResponseMessages->SyncFolderItemsResponseMessage->Changes;
// created events
if(property_exists($changes, 'Create')) {
foreach($changes->Create as $event) {
$id = $event->CalendarItem->ItemId->Id;
$change_key = $event->CalendarItem->ItemId->ChangeKey;
$start = $event->CalendarItem->Start;
$end = $event->CalendarItem->End;
$subject = $event->CalendarItem->Subject;
}
}
// updated events
if(property_exists($changes, 'Update')) {
foreach($changes->Update as $event) {
$id = $event->CalendarItem->ItemId->Id;
$change_key = $event->CalendarItem->ItemId->ChangeKey;
$start = $event->CalendarItem->Start;
$end = $event->CalendarItem->End;
$subject = $event->CalendarItem->Subject;
}
}
// deleted events
if(property_exists($changes, 'Delete')) {
foreach($changes->Delete as $event) {
$id = $event->CalendarItem->ItemId->Id;
$change_key = $event->CalendarItem->ItemId->ChangeKey;
$start = $event->CalendarItem->Start;
$end = $event->CalendarItem->End;
$subject = $event->CalendarItem->Subject;
}
}
Using a PHP implementation of the java android market API I'm attempting to display the top 10 most popular applications.
Using the code below:
<?php
include("local.php");
include("../proto/protocolbuffers.inc.php");
include("../proto/market.proto.php");
include("../Market/MarketSession.php");
$session = new MarketSession();
$session->login(GOOGLE_EMAIL, GOOGLE_PASSWD);
$session->setAndroidId(ANDROID_DEVICEID);
$ar = new AppsRequest();
$ar->setOrderType(AppsRequest_OrderType::POPULAR);
$ar->setStartIndex(0);
$ar->setEntriesCount(10);
//free or paid
//$ar->setViewType(AppsRequest_ViewType::ALL);
//arcade etc
//$ar->setCategoryId("ARCADE");
$reqGroup = new Request_RequestGroup();
$reqGroup->setAppsRequest($ar);
$response = $session->execute($reqGroup);
$groups = $response->getResponsegroupArray();
foreach ($groups as $rg) {
$appsResponse = $rg->getAppsResponse();
$apps = $appsResponse->getAppArray();
foreach ($apps as $app) {
echo $app->getTitle()."<br/>";
}
}
But the results I'm getting aren't exactly what I expected:
Brightest Flashlight Freeâ„¢
LauncherPro
Seesmic (Facebook, Twitter)
Android Assistant(18 features)
Pho.to Lab
US Yellow Pages
Sudoku Free
Color Flashlight
ElectroDroid
Scanner Radio
I expected the list of top 10 apps to contain Gmail, Facebook, Youtube etc
Anyone know why this could be happening? Any alternative APIs we could use? Any other way we can acheive this?