Summary:
Google_Service_Calendar seems to be "force-paginating" results of $service->events->listEvents();
Background:
google calendar API v3,
using php client library
We are developing a mechanism to sync our internal calendar with a user's google calendar.
Please note I will refer below to $x, which represents google's default limit on the number of events, similar to $options['maxResults']; The default value is 250, but it should not matter: we have tested the below with and without explicitly defined request parameters such as 'maxResults', 'timeMin', and 'timeMax' - the problem occurs in all cases.
Another relevant test we did: export this calendar to foobar.ics, created a new gmail user form scratch, import foobar.ics to newuser#gmail.com. DOES NOT REPLICATE THIS ISSUE. We have reviewed/reset various options in the subject calendar (sharing, etc) but cannot find any setting that has any effect.
The Problem:
Normally, when we call this:
$calendar='primary';
$optParams=array();
$events = $this->service->events->listEvents($calendar, $optParams);
$events comes back as a Google_Service_Calendar_Events object, containing $n "items". IF there are more than $x items, the results could be paginated, but the vanilla response (for a 'normal', result set with ( count($items) < $x ) ) is a single object, and $events->nextPageToken should be empty.
One account we are working with (of course, the boss's personal account) does not behave this way. The result of:
$events = $this->service->events->listEvents('primary', []);
is a Google_Service_Calendar_Events object like this:
Google_Service_Calendar_Events Object
(
[accessRole] => owner
[defaultRemindersType:protected] => Google_Service_Calendar_EventReminder
[defaultRemindersDataType:protected] => array
[description] =>
[etag] => "-kakMffdIzB99fTAlD9HooLp8eo/WiDS9OZS7i25CVZYoK2ZLLwG7bM"
[itemsType:protected] => Google_Service_Calendar_Event
[itemsDataType:protected] => array
[kind] => calendar#events
[nextPageToken] => CigKGmw5dGh1Mms4aWliMDNhanRvcWFzY3Y1ZGkwGAEggICA6-L3lrgUGg0IABIAGLig_Zfi278C
[nextSyncToken] =>
[summary] => example#mydomain.com
[timeZone] => America/New_York
[updated] => 2014-07-23T15:38:50.195Z
[collection_key:protected] => items
[modelData:protected] => Array
(
[defaultReminders] => Array
(
[0] => Array
(
[method] => popup
[minutes] => 30
)
)
[items] => Array
(
)
)
[processed:protected] => Array
(
)
)
Notice that $event['items'] is empty, and nextPageToken is not null. If we then do a paginated request like this:
while (true) {
$pageToken = $events->getNextPageToken();
if ($pageToken) {
$optParams = array('pageToken' => $pageToken);
$events = $this->service->events->listEvents($calendar, $optParams);
if(count($events) > 0){
h2("Google Service returned total of ".count($events)." events.");
}
} else {
break;
}
}
The next result set gives us the events. In other words, the google service seems to be paginating the initial results, despite the fact that we are certain the result is less than $x.
To be clear, if we have 5 events on our calendar, we expect 1 result with 5 items. Instead, we get 1 result with 0 items, but the first result of the 'nextPageToken' logic gives us the desired 5 items.
Solution Ideas?:
A. handle paginated results, and/or "Incremental Syncronization'. These are on our list of features to implement, but we consider these to be more 'optimization' than 'necessity'. In other words, I understand that handling/sending nextSyncToken and nextPageToken are OPTIONAL- thus the issue we are having should not depend on our client code doing this.
B. use a different, non-primary calendar for this user. we think this particular primary calendar may corrupt or somehow cached on google's side: to be fair, we did at one point accidentally insert a bunch of junk events on this calendar to the point that google put us in read-only mode as described here: https://support.google.com/a/answer/2905486?hl=en but we understand that was a temporary result of clunky testing.... In other words, we know we HAD screwed this calendar up badly, but this morning we deleted ALL events, added a single test event, and got the same result as above FOR THIS CALENDAR. Cannot replicate for any other user.... including a brand new gmail user.
C. delete the 'primary' calendar, create a new one. Unfortunately, we understand it is not possible to delete the primary CALENDAR, only to delete the CALENDAR EVENTS.
D. make the boss open a brand new google account
Any other suggestions? We are proceeding with A, but even that is a band-aid to the problem, and does not answer WHY is this happening? How can we avoid it in the future? (Please don't say "D")
Thanks in advance for any advice or input!
There is a maximum page size, if you don't specify one yourself there is an implicit one (https://developers.google.com/google-apps/calendar/v3/pagination). Given this it's necessary to implement pagination for your app to work correctly.
As you noticed, a page does not always contain the maximum number of results so pagination is important even if the number of events does not exceed the page size. Just keep following the page tokens and it will eventually give you all the results (there will be a page with no nextPageToken)
TL;DR A :)
Related
I'm trying to use this php library along with Sheetsu to pull single bits of data from a Google spreadsheet for output on a web page. My php skills are minimal and spotty, I'm afraid, and so I'm missing a final crucial component.
When I set up a test file and run my query, the code dumps everything into a $collection object. If I output print_r($collection); I get this:
Sheetsu\Collection Object (
[models:Sheetsu\Collection:private] => Array (
[0] => Sheetsu\Model Object (
[id] => 2.05.1
[title] => The Mead of Poetry
[answer] => Kvasir was created from the spit of the Aesir and Vanir. He was very wise.
)
)
)
My data's there, everything's working as expected, but I've never seen a structure like that before.
How can I just echo, say, just the [answer] value? I'm not sure what syntax to use to drill into that.
Thanks!
try
$collection->get(0)
or
$collection->getFirst()
or
$collection->getAll()[0]
etc.
models is a private property, you can't access it directly
so to access answer it would be something like
$collection->get(0)->answer
Try echo $collection[0]->answer;
I have a variable called $graph that carries OpenGraph data scraped from a URL. When I print_r the variable, this is what it looks like:
Array (
[OpenGraph_values] => Array (
[url] => http://www.theguardian.com/sport/blog/2017/jun/05/lebron-james-cleveland-cavaliers-nba-finals
[description] => The Cleveland star’s greatness created an arms race in basketball, and it has reached its apotheosis in the Golden State Warriors
[image] => https://i.guim.co.uk/img/media/3746052fc5b7f0a1ffb9ee2d5cd79585c31149a8/562_880_4049_2429/master/4049.jpg?w=1200&h=630&q=55&auto=format&usm=12&fit=crop&crop=faces%2Centropy&bm=normal&ba=bottom%2Cleft&blend64=aHR0cHM6Ly91cGxvYWRzLmd1aW0uY28udWsvMjAxNi8wNS8yNS9vdmVybGF5LWxvZ28tMTIwMC05MF9vcHQucG5n&s=71512608e33d4fda4fcf6ca52ccbd2bf
[type] => article
[title] => LeBron James created the modern NBA superteam. Now it will destroy him
[site_name] => the Guardian
)
[OpenGraph_position] => 0
)
I cannot for the life of me figure out how to access the data in there. I've tried $graph['image'] and $graph['OpenGraph_values']['image'], but no luck. Both come out null. There has to be something really simple that I'm missing, but I'm pretty new to PHP so I don't even know how to search for my problem.
EDIT: Some additional information:
I'm creating the variable by doing this:
$graph = (array)OpenGraph::fetch( $permalink );
where $permalink is a string with a URL.
Which uses this tool to fetch the OpenGraph data: https://github.com/scottmac/opengraph
Let me know if anything else in particular might be useful. This is running in a WordPress site, if that's relevant at all.
I'm integrating a paypal payment method on my website, I got it all running just fine, I'm stuck at the point were paypal sends me to my return URL with information about the customer and items purchased.
I get this following structure on the confirmation array
Array
(
some customer info
...
[L_NAME0] => Frame%20Rojo
[L_NAME1] => External%20Hard%20Disk
[L_NUMBER0] => PD1002
[L_NUMBER1] => PD1003
[L_QTY0] => 1
[L_QTY1] => 1
[L_TAXAMT0] => 0%2e00
[L_TAXAMT1] => 0%2e00
[L_AMT0] => 29%2e00
[L_AMT1] => 100%2e00
...
)
What I'm interested is in saving the whole item list, quantities and prices to my database so I can later keep track of what's been sent and what not.
My issue here is that as you can see, paypal returns to me a set of values that are names "something+n" (L_NUMBER0 and so on), so, I can't just set up a table on my DDBB as I don't know how many items would an user get. I could save it on 2 tables: purchase and items_per_purchase like structure, but I still face the issue of parsing that array.
What would be the best way to run through it and see how many items per purchase there are to save?
I thought of some kind of bucle wich sees:
while(if(isset($_GET['L_NUMBER'.$cont]))) {
// save to ddbb
L_NAME.$cont
L_NUMBER.$cont
...
cont++
}
... and increment some counter but I would like to know if there's a better solution.
I think your solution is fine, though you don't need an if inside the while test...
$cont = 0;
while (isset($_GET['L_NUMBER' . $cont])) {
// save to database after assembling array keys as follows...
// L_NAME . $cont
// L_NUMBER . $cont
// etc.
cont++;
}
And you should never trust input from $_GET. I recommend using parameterized queries with PDO.
I’ve been scratching my head for days on this one. Any help or a push in the right direction would be greatly appreciated.
I’ve extended the sales report under Reports->Sales->Orders and created my own custom filters to filer the report by channels.
Each order has a ‘channel_name’ attribute to identify whether the order came from eBay, Amazon, etc.
Now I cannot for the life of me figure out how sales/order_aggregated_created table that is used to generate the reports is created. Where does the magic happen? I want to add ‘channel_name’ to the order_aggregated_created table to be able to filter by this attribute, but I cannot figure out how to this.
Diagram for order_aggregated_created table with its attributes:
http://www.magento-exchange.com/wp-content/uploads/2010/11/MAGENTO-SALES-ORDER-ER.png
Mage_Sales_Model_Resource_Report_Order_Collection is where the magic starts in retrieving the sales totals, particularly if I understood this correctly inside
protected function _getSelectedColumns(){...}
if (!$this->isTotals()) {
$this->_selectedColumns = array(
'period' => $this->_periodFormat,
'orders_count' => 'SUM(orders_count)',
'total_qty_ordered' => 'SUM(total_qty_ordered)',
'total_qty_invoiced' => 'SUM(total_qty_invoiced)',
'total_income_amount' => 'SUM(total_income_amount)',
'total_revenue_amount' => 'SUM(total_revenue_amount)',
'total_profit_amount' => 'SUM(total_profit_amount)',
'total_invoiced_amount' => 'SUM(total_invoiced_amount)',
'total_canceled_amount' => 'SUM(total_canceled_amount)',
'total_paid_amount' => 'SUM(total_paid_amount)',
'total_refunded_amount' => 'SUM(total_refunded_amount)',
'total_tax_amount' => 'SUM(total_tax_amount)',
'total_tax_amount_actual' => 'SUM(total_tax_amount_actual)',
'total_shipping_amount' => 'SUM(total_shipping_amount)',
'total_shipping_amount_actual' => 'SUM(total_shipping_amount_actual)',
'total_discount_amount' => 'SUM(total_discount_amount)',
'total_discount_amount_actual' => 'SUM(total_discount_amount_actual)',
);
}
If would be awesome if I can just ‘channel_name’ =>$this->_channelName, and be on my merry way.
After a lot and lot of research it turns out that sales/order_aggregated_created table gets generated in here:
Mage_Sales_Model_Resource_Report_Order_Createdat
now I've looked here before seemed like exactly what I needed, but any changes I made would not reflect in Magento reports especially inside the sales/order_aggregated_created table.
I found out that Mage_Sales_Model_Resource_Report_Order_Createdat only gets called when you refresh the statistics inside Reports->Sales->Orders, only then a NEW sales/order_aggregated_created table is generated! So for anyone looking to filter the order sales report by a custom attribute, look inside: /app/code/core/Mage/Sales/Model/Resource/Report/Order/Createat.php
I'm doing some integrations towards MS based web applications which forces me to fetch the data to my php application via SOAP which is fine.
I got the structure of a file system in an xml which I convert to an object. All documents have an ID and it's path. To be able to place the documents in a tree view I've built some methods to calculate the documents whereabouts through the files and folder structure. This works fine until I started to try with large file lists.
What I need is a faster method (or way to do things) than a foreach loop.
The method below is the troublemaker.
/**
* Find parent id based on path
* #param array $documents
* #param string $parentPath
* #return int
*/
private function getParentId($documents, $parentPath) {
$parentId = 0;
foreach ($documents as $document) {
if ($parentPath == $document->ServerUrl) {
$parentId = $document->ID;
break;
}
}
return $parentId;
}
// With 20 documents nested in different folders this method renders in 0.00033712387084961
// With 9000 documents nested in different folders it takes 60 seconds
The array sent to the object looks like this
Array
(
[0] => testprojectDocumentLibraryObject Object
(
[ParentID] => 0
[Level] => 1
[ParentPath] => /Shared Documents
[ID] => 163
[GUID] => 505d70ea-51d7-4ef0-bf79-8e912553249e
[DocIcon] =>
[FileType] =>
[Title] => Folder1
[BaseName] => Folder1
[LinkFilename] => Folder1
[ContentType] => Folder
[FileSizeDisplay] =>
[_UIVersionString] => 1.0
[ServerUrl] => /Shared Documents/Folder1
[EncodedAbsUrl] => http://dev1.example.com/Shared%20Documents/Folder1
[Created] => 2011-10-08 20:57:47
[Modified] => 2011-10-08 20:57:47
[ModifiedBy] =>
[CreatedBy] =>
[_ModerationStatus] => 0
[WorkflowVersion] => 1
)
...
A bit bigger example of the data array is available here
http://www.trikks.com/files/testprojectDocumentLibraryObject.txt
Thanks for any help!
=== UPDATE ===
To illustrate the time different stuff takes I've added this part.
Packet downloaded in 8.5031080245972 seconds
Packet decoded in 1.2838368415833 seconds
Packet unpacked in 0.051079988479614 seconds
List data organized in 3.8216209411621 seconds
Standard properties filled in 0.46236896514893 seconds
Custom properties filled in 40.856066942215 seconds
TOTAL: This page was created in 55.231353998184 seconds!
Now, this is a custom property action that im describing, the other stuff is already somewhat optimized. The data sent from the WCF service is compressed and encoded ratio 10:1 (like 10mb uncompressed : 1mb compressed).
The current priority is to optimize the custom properties part, where the getParentId method takes 99% of the execution time!
You may see faster results by using XMLReader or expat instead of simplexml. Both of these reqd the xml sequentially and won't store the entire document in memory.
Also make sure you have the APC extension on, for the actual loop it's a big big difference. Some benchmarks on the actual loop would be nice.
Lastly, if you cannot make it faster.. rather than trying to optimize reading the large xml document, you should look into ways where this 'slowness' is not an issue. Some ideas include an asynchronous process, proper caching, etc..
Edit
Are you actually calling getParentId for every document? This just occurred to me. If you have a 1000 documents then this would imply already 1000*1000 loops. If this is truly the case, you need to rewrite your code so it becomes a single loop.
How are you populating the array in the first place? Perhaps you could arrange the items in a hierarchy of nested arrays, where each key relates to one part of the path.
e.g.
['Shared Documents']
['Folder1']
['Yet another folder']
['folderA']
['folderB']
Then in your getParentId() method, extract the various parts of the path and just search that section of data:
private function getParentId($documents, $parentPath) {
$keys = explode('/', $parentPath);
$docs = $documents;
foreach ($keys as $key) {
if (isset($docs[$key])) {
$docs = $docs[$key];
} else {
return 0;
}
}
foreach $docs as $document) {
if ($parentPath == $document->ServerUrl) {
return $document->ID;
}
}
}
I haven't fully checked that will do what you're after, but it might help set you on a helpful path.
Edit: I missed that you're not populating the array yourself initially; but doing some sort of indexing ahead of time might still save you time overall, especially if getParentId is called on the same data multiple times.
As usual this was a matter of programming design. And there are a few lessons to be learned from this.
In a file system the parent is always a folder, to speed up such a process in php you can put all the folders in a separate array with it's corresponding ID as the key and search that array when you want to find the parent of a file, instead of searching the entire file structure array!
Packet downloaded in 6.9351849555969 seconds
Packet decoded in 1.2411289215088 seconds
Packet unpacked in 0.04874587059021 seconds
List data organized in 3.7993721961975 seconds
Standard properties filled in 0.4488160610199 seconds
Custom properties filled in 0.15889382362366 seconds
This page was created in 11.578738212585 seconds!
Compare the custom properties by the one from my original post
Cheers