I'm building a synchronisation service that keeps track of all the changes in a specific Google Drive folder.
Following this link I built this function:
public function whatChangesAreThere()
{
$response = [];
$pageToken = $this->lastPageToken;
while ($pageToken != null) {
$listChanges = $this->googleDriveService->changes->listChanges($pageToken, [
'spaces' => 'drive'
]);
collect($listChanges->changes)->each(function ($change) use (&$response) {
$fileResponse = [];
$fileResponse["type"] = $change->type;
if (array_has($change, "file.name")) {
$fileResponse["fileName"] = $change->file->name;
}
$fileResponse["kind"] = $change->kind;
$fileResponse["removed"] = $change->removed;
$fileResponse["teamDriveId"] = $change->teamDriveId;
$fileResponse["time"] = $change->time;
$response[$change->fileId] = $fileResponse;
});
$pageToken = $listChanges->nextPageToken;
}
$this->storeNewPageToken($listChanges->newStartPageToken);
return $response;
}
The function seems to work: when I change something in the file I get the name of the file that changed in my application. The problem is that there can be up to 1-2 minutes delay between changing the file and getting the file listed in this function.
I'm wondering if I'm doing something wrong or if this dealy is due to something else.
What do you think?
Related
public function uploadNotaCorte(Request $request, EstadoRepository $estadoRepository)
{
$error = array();
$path = $request->file('file')->getRealPath();
$notasCorte = Excel::load($path, function($reader) {
})->get();
$chunk = $notasCorte->chunk(100);
foreach ($notasCorte as $key => $notaCorte) {
//RULES
}return $error;
}
**
Hi everyone, I'm new to programming and I'm having a hard time implementing the chunk method, so the dodigo above usually works on small files plus larger error files because of the size.
I need to upload a file with 120,000 records and I am trying to use the chunk for this, I do not know what I can do wrong already looked at the documentation more and very simple and I could not solve the problem can anyone help me ??**
Assuming you're using the maatwebsite/excel package, this link should help: http://www.maatwebsite.nl/laravel-excel/docs/import#chunk
You'll want to change your code to something like this:
public function uploadNotaCorte(Request $request, EstadoRepository $estadoRepository)
{
$error = array();
$path = $request->file('file')->getRealPath();
Excel::filter('chunk')->load($path)->chunk(100, function($results)
{
foreach($results as $row)
{
// RULES
}
});
return $error;
}
This isn't tested and I've never used that package (though good to know it exists) so your mileage may vary.
I have used Zend Gdata for several years now.
However, today when my unchanged code executes the following command
$query = $this->gp->newAlbumQuery();
I receive the following error
exception 'Zend_Gdata_App_HttpException' with message 'Expected response code 200, got 403 Authorization required' in /shared/zend/ZendFramework-1.12.13/library/Zend/Gdata/App.php:717 Stack trace: #0 /shared/zend/ZendFramework-1.12.13/library/Zend/Gdata.php(221): Zend_Gdata_App->performHttpRequest('GET', 'https://picasaw...', Array, NULL, NULL, NULL) #1 /shared/zend/ZendFramework-1.12.13/library/Zend/Gdata/App.php(883): Zend_Gdata->performHttpRequest('GET', 'https://picasaw...', Array)
I thought it would be because authentication had failed. I checked and my credentials are all fine and the following authenticates successfully without an exception
$client = Zend_Gdata_ClientLogin::getHttpClient($this->config['username'],
$this->config['password'],
Zend_Gdata_Photos::AUTH_SERVICE_NAME);
I saw that Zend Gdata is still on version 1.12.13. However there is a recent release date of 20/05/2015. So i did an update using this new release. But the error is still the same.
So all i know is that i am authenticated, but the newAlbumQuery method is raising the above exception.
Is anyone else experiencing this problem with Zend Gdata? Has anyone found a fix or workaround?
Zend_Gdata uses ClientLogin which was deprecated as of April 20, 2012 and turned off on April 20 2015. This code will not longer work you need to switch to using Oauth2.
I ran into the same issue. My app is one only I use so here's how I replaced the ClientLogin with OAuth for my own user in my own application.
I wrote a Zend client class that uses OAuth. It's linked here: https://gist.github.com/monkbroc/4251effc7912b9764ca1
The main difficulty is performing the OAuth process the first time to get a refresh token that the Zend OAuthClient will transform to an access token which can be used for requests, but expires after an hour. The Gist above lists the steps I took.
In the end I'm ending with something like this(very raw code so far, but enought for those who's seaching for solution. You need php google spreadsheet client of https://github.com/asimlqt/php-google-spreadsheet-client). Thats tiny example of inserting one row to my spreadsheet (sorry about my code, but showing just working example)
Thanks for bram brambring showing better way to authorization - answer by bram brambring
<?php
/*
* Google Spreadsheet class to work with google spreadsheets obviously ;D [using OAuth 2.0, as Zend Gdata is not anymore working]
*/
require_once('/Google/Spreadsheet/ServiceRequestInterface.php');
require_once('/Google/Spreadsheet/DefaultServiceRequest.php');
require_once('/Google/Spreadsheet/ServiceRequestFactory.php');
require_once('/Google/Spreadsheet/Spreadsheet.php');
require_once('/Google/Spreadsheet/SpreadsheetFeed.php');
require_once('/Google/Spreadsheet/SpreadsheetService.php');
require_once('/Google/Spreadsheet/Exception.php');
require_once('/Google/Spreadsheet/UnauthorizedException.php');
require_once('/Google/Spreadsheet/Spreadsheet.php');
require_once('/Google/Spreadsheet/Util.php');
require_once('/Google/Spreadsheet/Worksheet.php');
require_once('/Google/Spreadsheet/WorksheetFeed.php');
require_once('/Google/Spreadsheet/ListFeed.php');
require_once('/Google/Spreadsheet/ListEntry.php');
require_once('/Google/Spreadsheet/CellFeed.php');
require_once('/Google/Spreadsheet/CellEntry.php');
require_once('/Google/Config.php');
require_once('/Google/Client.php');
require_once('/Google/Auth/Abstract.php');
require_once('/Google/Auth/OAuth2.php');
require_once('/Google/Http/Request.php');
require_once('/Google/Utils.php');
require_once('/Google/IO/Abstract.php');
require_once('/Google/IO/Curl.php');
require_once('/Google/Http/CacheParser.php');
require_once('/Google/Logger/Abstract.php');
require_once('/Google/Logger/Null.php');
require_once('/Google/Exception.php');
require_once('/Google/Auth/Exception.php');
require_once('/Google/Auth/AssertionCredentials.php');
require_once('/Google/Cache/Abstract.php');
require_once('/Google/Cache/File.php');
require_once('/Google/Signer/Abstract.php');
require_once('/Google/Signer/P12.php');
use Google\Spreadsheet\DefaultServiceRequest;
use Google\Spreadsheet\ServiceRequestFactory;
class Google_Spreadsheet
{
private $default = array(
'worksheetCols' => 12,
'worksheetRows' => 25
);
private $spreadsheetKey;
private $spreadsheetName;
private $worksheetName;
private $spreadsheetFeed;
public $initialized = true;
public function __construct($spreadsheetKey, $worksheetName, $spreadsheetName = '')
{
$this->spreadsheetKey = $spreadsheetKey;
$this->worksheetName = $worksheetName;
$this->spreadsheetName = $spreadsheetName;
$this->initialized = $this->initialize();
return true;
}
private function getToken() {
$client_email = '318977712937456456454656563tcfjblgoi#developer.gserviceaccount.com';
$private_key = file_get_contents('API Project-f10e456456b60.p12');
$scopes = array('https://spreadsheets.google.com/feeds');
$credentials = new Google_Auth_AssertionCredentials(
$client_email,
$scopes,
$private_key,
'notasecret', // Default P12 password
'http://oauth.net/grant_type/jwt/1.0/bearer' // Default grant type
);
$client = new Google_Client();
$client->setAssertionCredentials($credentials);
if ($client->getAuth()->isAccessTokenExpired()) {
$client->getAuth()->refreshTokenWithAssertion();
}
$tokenData = json_decode($client->getAccessToken());
return $tokenData->access_token;
}
public function initialize(/*$reInitialized = false*/)
{
// load OAuth2 token data - exit if false
$tokenData = $this->getToken();
$serviceRequest = new DefaultServiceRequest($tokenData);
ServiceRequestFactory::setInstance($serviceRequest);
$spreadsheetService = new Google\Spreadsheet\SpreadsheetService();
try {
$spreadsheetFeed = $spreadsheetService->getSpreadsheets();
} catch (\Google\Spreadsheet\UnauthorizedException $e) {
Google_Spreadsheet::warnAdmin($e->getMessage());
return false;
}
$this->spreadsheetFeed = $spreadsheetFeed;
return true;
}
public function insertRow($rowData, $default_fields = array()) {
$spreadsheetFeed = $this->spreadsheetFeed;
$spreadsheet = $this->spreadsheetKey ? $spreadsheetFeed->getByKey($this->spreadsheetKey) : $spreadsheetFeed->getByTitle($this->spreadsheetName);
if(!$spreadsheet && !empty($this->spreadsheetName)) {
$spreadsheet = $spreadsheetFeed->getByTitle($this->spreadsheetName);
}
if(!$spreadsheet) {
Google_Spreadsheet::warnAdmin('No spreadsheet', serialize($rowData));
return false;
}
$worksheetFeed = $spreadsheet->getWorksheets();
$worksheet = $worksheetFeed->getByTitle($this->worksheetName);
if(!$worksheet) {
//create worksheet if not exist
$worksheet = $spreadsheet->addWorksheet($this->worksheetName, $this->default['worksheetRows'], $this->default['worksheetCols']);
$cellFeed = $worksheet->getCellFeed();
for( $i= 1 ; $i <= $this->default['worksheetCols']; $i++ ) {
if(isset($default_fields[$i])) {
$cellFeed->editCell(1, $i, $default_fields[$i]);
}
else {
$cellFeed->editCell(1, $i, "head");
}
$cellFeed->editCell(2,$i,"content");
}
}
if(!$worksheet) {
Google_Spreadsheet::warnAdmin('No worksheet', serialize($rowData));
return false;
}
$listFeed = $worksheet->getListFeed();
$data = array();
foreach ($listFeed->getEntries() as $entry) {
$values = $entry->getValues();
$data[] = $values;
break; //only first row needed, as we need keys
}
$keys = array();
if(!count($data)) {
Google_Spreadsheet::warnAdmin('No data', serialize($rowData));
return false;
}
foreach ($data[0] as $key => $value) {
$keys[] = $key;
}
$newRow = array();
$count = 0;
foreach($keys as $key) {
if(isset($rowData[$count])) {
$newRow["$key"] = $rowData[$count];
}
else {
$newRow["$key"] = '';
}
$count++;
}
$listFeed->insert($newRow);
return true;
}
static function warnAdmin($reason = '', $content = '') {
//temporal function to warn myself about all the stuff happening wrong :)
}
}
And in main model I'm using:
$spreadsheet = new Google_Spreadsheet("spreadsheet name or ID", $worksheetname, "My spreadsheet name");
if(!$spreadsheet->initialized) {
Google_Spreadsheet::warnAdmin('cannot initialize spreadsheet', serialize($rowValues));
}
if(!$spreadsheet->initialized || !$spreadsheet->insertRow($rowValues, $this->default_fields)) {
Google_Spreadsheet::warnAdmin('failed to insert row ');
}
#Edit, thanks to bram brambring for his token solution, seems more easier than my. Working like a charm now, I hope his way it gonna refresh token normally. THANK YOU DUDE!
So i have the following code:
private function getArtistInfo($artist){
$artisan = json_decode($artist, true);
$artistObj = array();
//fb($artist);
$artistObj['id'] = $artisan['name']['ids']['nameId'];
$memcache = new Memcached($artistObj['id']);
$artistCache = $memcache->getMemcache();
if($artistCache === false){
$artistObj['name'] = $artisan['name']['name'];
$artistObj['image'] = $artisan['name']['images'][0]['url'];
$initArtist = array('id' => $artistObj['id'], 'name' => $artistObj['name'], 'image' => $artistObj['image']);
$artistObj = $this->buildArtist($artisan, $artistObj);
$memcache->setMemcache($artistObj);
}
else{
$initArtist = array('id' => $artistCache['id'], 'name' => $artistCache['name'], 'image' => $artistCache['image']);
}
return $initArtist;
}
Now the code works but it takes getArtistInfo() too long to finish when i just want the $initArtist value; I would like my client to get right away the $initArtist once its constructed, and somehow let the caching of $artistObj runs in the background.
So far i have read up on several different topic i thought might be useful: event delegation, callback function, call_user_func, observer pattern, threading, gearman etc. However, I have no idea which one of them would actually do what i want. Please point me to the right direction.
EDIT:
My Memcached class:
class Memcached {
private static $MEMCACHED_HOST = "localhost";
private static $MEMCACHED_PORT = "11211";
private $id, $key, $memcache, $cacheOK;
function __construct ($id){
$this->id = $id;
$this->key = 'artistID_'. $this->id;
$this->memcache = new Memcache;
$this->cacheOK = $this->memcache->connect(Memcached::$MEMCACHED_HOST, Memcached::$MEMCACHED_PORT);
}
protected function getMemcache(){
$artistInfo = null;
if($this->cacheOK === true){
$artistInfo = $this->memcache->get($this->key);
}
if($artistInfo === false){
return false;
}
return $artistInfo;
}
public function setMemcache($artistInfo){
$this->memcache->set($this->key, $artistInfo, 0, 60);
}
}
My buildArtist() code:
private function buildArtist($artisan, $artistObj){
$artistObj['amgID'] = $artisan['name']['ids']['amgPopId'];
$discography = $artisan['name']['discography'];
foreach($discography as $album){
$albumID = $album['ids']['amgPopId'];
preg_match('/(\d+)/', $albumID, $matches);
$albumObj['amgAlbumID'] = $matches[1];
$albumObj['title'] = $album['title'];
$albumObj['releaseDate'] = $album['year'];
$albumObj['more'] = $this->getMoreMusic($albumObj['title'], $artistObj['name']);
$artistObj['discography'][] = $albumObj;
}
return $artistObj;
}
Well, it's not entirely clearly how long too long is, or which part of this code is what's slowing you down. For all we know, the slow part isn't the part that stores the data in Memcached.
In any case, once you do identify that this is your bottleneck, one thing you can do to accomplish this type of out of order execution is use a brokerless messaging queue like ZeroMQ to accept JSON object that need cached. A separate PHP script can then take on the job of processing and caching these requests asynchronously outside of any web-request. This separate script could be run through a cron-job or some other job manager that handles the caching part in parallel.
You want to use set and get rather than using the memcache persistence ID, i'm not even sure what setMemcache and getMemcache are but they aren't in the extension documentation.
Here's an example from the documentation:
<?php
$m = new Memcached();
$m->addServer('localhost', 11211);
if (!($ip = $m->get('ip_block'))) {
if ($m->getResultCode() == Memcached::RES_NOTFOUND) {
$ip = array();
$m->set('ip_block', $ip);
} else {
/* log error */
/* ... */
}
}
Please show the code of buildArtist for help on optimizing it.
I've just started working with the PHP API for Rackspace Cloud Files. So far so good-- but I am using it as sort of a poor man's memcache, storing key/value pairs of serialized data.
My app attempts to grab the existing cached object by its key ('name' in the API language) using something like this:
$obj = $this->container->get_object($key);
The problem is, if the object doesn't exist, the API throws a fatal error rather than simply returning false. The "right" way to do this by the API would probably be to do a
$objs = $this->container->list_objects();
and then check for my $key value in that list. However, this seems way more time/CPU intensive than just returning false from the get_object request.
Is there a way to do a "search for object" or "check if object exists" in Cloud Files?
Thanks
I sent them a pull request and hope it'll get included.
https://github.com/rackspace/php-cloudfiles/pull/35
My pull-request includes an example, for you it would be similar to this:
$object = new CF_Object($this->container, 'key');
if ($object->exists() === false) {
echo "The object '{$object->name}' does not exist.";
}
I have more general way to check if object exists:
try {
$this->_container->get_object($path);
$booExists = true;
} catch (Exception $e) {
$booExists = false;
}
If you dump the $object, you'll see that content_length is zero. Or, last modified will be a zero length string.
Example:
$object = new CF_Object($container, 'thisdocaintthere.pdf');
print_r($object->content_length);
There is also, deep in the dumped parent object, a 404 that will return, but it's private, so you'd need to some hackin' to get at it.
To see this, do the following:
$object = new CF_Object($container, 'thisdocaintthere.pdf');
print_r($object->container->cfs_http);
You'll see inside that object a response_status that is 404
[response_status:CF_Http:private] => 404
I know I'm a little late to the party, but hopefully this will help someone in the future: you can use the objectExists() method to test if an object is available.
public static function getObject($container, $filename, $expirationTime = false)
{
if ($container->objectExists($filename)) {
$object = $container->getPartialObject($filename);
// return a private, temporary url
if ($expirationTime) {
return $object->getTemporaryUrl($expirationTime, 'GET');
}
// return a public url
return $object->getPublicUrl();
}
// object does not exist
return '';
}
Use like...
// public CDN file
$photo = self::getObject($container, 'myPublicfile.jpg');
// private file; temporary link expires after 60 seconds
$photo = self::getObject($container, 'myPrivatefile.jpg', 60);
If you do not want to import opencloud to perform this check you can use the following:
$url = 'YOUR CDN URL';
$code = FALSE;
$options['http'] = array(
'method' => "HEAD",
'ignore_errors' => 1,
'max_redirects' => 0
);
$body = file_get_contents($url, NULL, stream_context_create($options));
sscanf($http_response_header[0], 'HTTP/%*d.%*d %d', $code);
if($code!='200') {
echo 'failed';
} else {
echo 'exists';
}
I am using CakePHP v1.2 for my web application hosted here: http://lol-land.in
The application was working fine till Yesterday, but it suddenly started to get stuck in some redirection loops. What makes it weirder is the fact that the problem is with just one controller: posts. And even in that most of the functions are working. But http://lol-land.in/posts is redirecting to lol-land.in/posts/view/133 which is in turn redirecting to itself.
Actually 110 of the 117 posts of the form /posts/view/ are stuck in this 302 redirection.
Can anyone please tell me what could have caused this?
[CakePHP 1.3 and PHP5]
Edit: Adding View Logic
function view($id = null) {
$this->log("View Logic Entry", LOG_DEBUG);
// These Log entires are missing
$this->layout = 'postview';
if (!$id) {
$this->Session->setFlash(__('Invalid Post.', true));
$this->log("Redirect due to missing id", LOG_DEBUG);
$this->redirect(array('action'=>'index'));
}
$log = $this->Session->read('Auth.User');
$logid = $log['id'];
$temp = $this->Post->read(null, $id);
$ratings = $temp['Rating'];
$this->set(compact('up', 'down', 'userrated', 'userrateid'));
$coms = $temp['Comment'];
$comuser = array();
for ($i=0; $i<count($coms); $i++) {
$comuser[$i] = $coms[$i]['user_id'];
}
$comuser = $this->Post->User->find('list', array( 'fields'=>array('User.id', 'User.username'),
'conditions'=>array("User.id" => $comuser)
));
$this->set(compact('comuser'));
$this->pageTitle = $temp['Post']['title'];
$this->set('post', $temp);
$alltypes = $this->Post->Type->find('list', array('fields'=> array('Type.id', 'Type.typename')));
$selectedtab = -1;
$this->set(compact('alltypes', 'selectedtab' ));
//Calling updateFBStats
// Removed because unnecessary.
}
Chances are you are either 1) using circular references with the Auth component OR 2) the function in your controller is redirecting do to something within the method. Can you show the code of posts_controller.php function view() ?