Laravel : How do I compare/obtain specific datas from cache? - php

In my normal polling Laravel chat app, I will save new messages sent by a user into a file cache with the key as a string, getting its value from date(current_time) function and the body of the message.
Then, when I want to obtain those messages, I will use the last Poll value $lastPolled = Session::get('lastPolled') and compare with the key in the cache. Keys that are greater than the $lastPolled value will have their data to be taken as new messages and appended into the conversations.
Finally, I will update the last polled session value Session::put('lastPolled',date(Y-m-d H:i:s)
So, how do I compare $lastPolled with all the keys in cache and get each key's values? Something along the lines of:
$latestMessages = array();
foreach(KeysInCache as Key=>value){
if($lastPolled>Key)
array_push($latestMessages,Key=>value);
}
Thank you!
P.s. bonus points for better suggestions. Oh and I can't use memcache/redis/otherSuperCaches for technical reasons, only file/database cache. :(

Why not try some thing like this by creating cache files based on timestamp or key :
Further details and suggestions on same at : http://evertpot.com/107/
// This is the function you store information with function
store($key,$data,$ttl) {
// Opening the file
$h = fopen($this->getFileName($key),'w');
if (!$h) throw new Exception('Could not write to cache');
// Serializing along with the TTL
$data = serialize(array(time()+$ttl,$data));
if (fwrite($h,$data)===false) {
throw new Exception('Could not write to cache');
}
fclose($h);
}
// General function to find the filename for a certain key private
function getFileName($key) {
return '/tmp/s_cache' . md5($key);
}
// The function to fetch data returns false on failure function
fetch($key) {
$filename = $this->getFileName($key);
if (!file_exists($filename) || !is_readable($filename)) return false;
$data = file_get_contents($filename);
$data = #unserialize($data);
if (!$data) {
// Unlinking the file when unserializing failed
unlink($filename);
return false;
}
// checking if the data was expired
if (time() > $data[0]) {
// Unlinking
unlink($filename);
return false;
}
return $data[1];
}
}

Related

Why is my controller so slow?

I'm on an app that retrieve datas (a 7k lines CSV formated string) from an external server to update my own entity. Each row is an item in a stock.
Today the job is nicely done but it's very very very slow: more than 60s (prod env) to retrieve datas, push it in a 2D array, update the BDD, and finally load a page that display the bdd content.
When only displaying the page it's about 20s (still prod).
This the profiler's timeline result while only displaying records : Symfony's profiler timeline
Anymore, i'm not able to profile the "updateAction" cause i't don't appear in the last ten request list.
2 days ago I was checking each row of the CSV file to add it only if needed, I was soft-deleting items to restore it later when back in the stock etc. but with that speed I tried many things to have normal performances.
At the begening everything was in the controler, I moved the function that add/remove in a dedicated service, then in the repository to finally get it back in my controler. To have decent results I tried to empty the database and then refill it without checking. First, using LOAD DATA LOCAL INFILE but it is not compatible with my table pattern (or I mis understood something) and now I'm simply emptying the table before filling it with the CSV (without any control). The time score I gave earlier was with this last try (which is the best one).
But enought talk
here is my controler:
public function majMatosClanAction()
{
$resMaj = $this->majClanCavernes();
if ($resMaj === NULL)
{
$this->get('session')->getFlashBag()->add('alert-danger', 'Unidentified');
return $this->redirect($this->generateUrl('loki_gbl'));
} else if ($resMaj === FALSE)
{
$this->get('session')->getFlashBag()->add('alert-warning','password update required');
return $this->redirect($this->generateUrl('loki_gbl_ST'));
} else
{
$this->get('session')->getFlashBag()->add('alert-success','success');
return $this->redirect($this->generateUrl('loki_gbl_voirMatosClan'));
}
}
here is the function that my controller call:
public function majClanCavernes()
{
$user = $this->get('security.token_storage')->getToken()->getUser();
$outils = $this->container->get('loki_gbl.outils');
if ($user !== NULL)
{
$pwd = $user->getGob()->getPwd();
$num = $user->getGob()->getNum();
if($outils->checkPwd($num, $pwd) !== TRUE) return FALSE;
$em = $this->getDoctrine()->getManager();
//This is a temporary solution
//////////////////////////////////////////////
$connection = $em->getConnection();
$platform = $connection->getDatabasePlatform();
$connection->executeUpdate($platform->getTruncateTableSQL('MatosClan', true ));
//////////////////////////////////////////////
$repository = $em->getRepository('LokiGblBundle:MatosClan');
$urlMatosClan = "http://ie.gobland.fr/IE_ClanCavernes.php?id=".$num."&passwd=".$pwd;
//encode and format the string via a service
$infosBrutes = $outils->fileGetInfosBrutes($urlMatosClan);
//$csv is a 2D array containing the datas
$csv = $outils->getDatasFromCsv($infosBrutes);
foreach($csv as $item)
{
$newItem = new MatosClan;
$newItem->setNum($item[0]);
$newItem->setType($item[1]);
[...]
$em->persist($newItem);
}
$em->flush();
return TRUE;
}
else{
return NULL;
}
}
What is wrong? 7k lines is not that big!
Could it be a lack of hardware issue?
Check out doctrine's batch processing documentation here.
You can also disable logging:
$em->getConnection()->getConfiguration()->setSQLLogger(null);

Check if blob exists in Azure

I'm wondering if there is a way to check if a blob exists in a container?
$blob = $blobRestProxy->getBlob("mycontainer", "myblobname");
if($blob){
return 'exists';
} else {
return 'not exists';
}
I've tried this but im getting this message whenever the blob does not exists:
BlobNotFoundThe specified blob does not exist.
If exists, the code returns 'exists' naturally. I'm not interested in listing all blobs in the container and iterating until I find a match cause I have a lot of blobs.
When the blob does not exist, the function getBlob will raise a ServiceException exception and exit the PHP progress, the following code will not work.
Please try to add the try catch statement in your code, E.G.
try {
$blob = $tableRestProxy->getBlob("mycontainer", "myblobname");
return 'exists';
} catch (ServiceException $e) {
return 'not exists';
}
so:
$exists = $storageClient->blobExists(<container name>, <blob name>);
Should give you what you're after.
Though older post, below is azure java client sample code, in case if it helps.
//getting the info from external configuration
String accountName = getStorageAccountName();
String accessKey = getAccessKey();
String container = getContainerName();
//create connection string
String connString = String.format("DefaultEndpointsProtocol=http;AccountName=%s;AccountKey=%s;EndpointSuffix=core.windows.net",accountName,accessKey);
// create account
CloudStorageAccount account= CloudStorageAccount.parse(connString );
// create client
CloudBlobClient blobClient = account.createCloudBlobClient();
// create container
CloudBlobContainer container = blobClient.getContainerReference(container);
// fetch item from list
Iterable<ListBlobItem> blobItems = container.listBlobs();
blobItems .forEach(item -> {
if (item instanceof CloudBlockBlob){
//additional condition can be added to check leased
// need to check if the blob item exists first
if((CloudBlockBlob) blobItem).exists()){
//custom business logic
}
// to check container exists and perform logic use below
// ((CloudBlobContainer) item.getContainer()).exists
}
});

How to check on Rackspace if the file is uploaded or not?

I'm working with the Rackspace PHP API, where I've got a requirement to check the file, if it exists then do something and if not then do something.
try {
$file = $container->getObject($end_element);
$fileExists = TRUE;
}
catch(Exception $e) {
$fileExists = FALSE;
}
if ($fileExists) {
// File is their, it needs to be rewrite/overwrite
$file->setContent(fopen('sites/default/files/rackspace/' . $end_element, 'r+'));
$file->update();
// I'm getting this http://docs.rackspace.com/sdks/api/php/class-OpenCloud.ObjectStore.Resource.DataObject.html which I printted print_r($file->update());
}
else {
// New file just to upload
$container->uploadObject($end_element, fopen('sites/default/files/rackspace/' . $end_element, 'r+'), array());
}
To see whether an object exists in a remote container, try using the objectExists method like so:
if ($container->objectExists('objectName.txt')) {
// The object exists
} else {
// The object doesn't exist
}
This will perform a HEAD request on that object, wrapping any 404 failure response in a try/catch block for you.
In terms of finding out the date the object was created, the API only tells you the date of last modification. This will be the create date if you haven't modified the object since it was first uploaded.
To find out the last modified datetime, you need to run:
$object = $container->getObject('objectName.txt');
$created = $object->getLastModified();

PHP: Best way to Generate an unique numeric ID

I need to generate unique numeric id.
I could use uniqid, but it generates alphanumeric value.
And again I could use time, but there is no guarantee that it
will be unique always.
And again I could use the auto increment property of a field in a
database to get an unique id, but here we must need a database.
So what can be the best way to generate an unique numeric id?
Nothing can guarantee 100% uniqueness.
You need to know uniqueness comparing with what do you need.
And use any algorythm plus check each value in list of all used values.
In the world of programming what you need is called pseudo random number. So it's name actually explains what I mean.
Database systems use exclusive locking when creating numbers such as MySQL's auto_increment which takes care of concurrency and many other intricate details.
You have to approach the problem you have the same way - acquire a lock from the PHP process that's serving the request, look up the current value within some sort of persistent storage, increment it by 1, return it and release the lock.
The easiest way to do this is to use a good old file and exclusive locking.
I'll illustrate with a class (which should be debugged since it's not complete):
class MyAutoIncrement
{
protected $fh = null;
protected $file_path = '';
protected $auto_increment_offset = 1;
public function __construct($file_path, $offset = 1)
{
$this->file_path = $file_path;
$this->auto_increment_offset = $offset;
}
public function autoincrement()
{
if($this->acquire())
{
$current = (int)fread($this->fh);
$next += $this->auto_increment_offset;
fwrite($this->fh, $next);
$this->release();
return $next;
}
return null;
}
public function acquire()
{
$handler = $this->getFileHandler();
return flock($handler, LOCK_EX);
}
public function release($close = false)
{
$handler = $this->getFileHandler();
return flock($handler, LOCK_UN);
if($close)
{
fclose($handler);
$this->fh = null;
}
}
protected function acquireLock($handler)
{
return flock($handler, LOCK_EX);
}
protected function getFileHandler()
{
if(is_null($this->fh))
{
$this->fh = fopen($this->file_path, 'c+');
if($this->fh === false)
{
throw new \Exception(sprintf("Unable to open the specified file: %s", $this->file_path));
}
}
return $this->fh;
}
}
Usage:
$ai = new MyAutoIncrement('/path/to/counter/file.txt');
try
{
$id = $ai->autoincrement();
if(!is_null($id))
{
// Voila, you got your number, do stuff
}
else
{
// We went wrong somewhere
}
}
catch(\Exception $e)
{
// Something went wrong
}
As mentioned before. Nothing can guarantee 100% uniqueness.
Although this will be fairly unique :)
$iUniqueNumber = crc32(uniqid());
See
uniqid and crc32 polynomial of a string.
You can use a combination of time() and getmypid() to get what you need - a numeric unique ID. You can have multiple php processes launched at a given time, but they will never have the same process ID at that given time (unless the server process counter overlaps in less than a second, which is virtually impossible when kernel.pid_max is set correctly).
<?
function getUniqueID() {
return time() . '.' . getmypid();
}
?>
That function will generate a unique ID per script execution per server. It will fail if you call it multiple times in the same script and expect it to return unique value every time. In that case you can define some static variable inside the function body to keep track of that.
You talked about time, what about microtime?
Even if you create two numbers in a row you'll get a diferent value. You'll need of course to play a little around to make it an unique integer, but it should be able to help.
I suggest to concatenate PHP process ID with microtime(true) to increase possibility of having unique value.
function getserial()
{
$fn='/where_you_want/serial.dat';
$fp = fopen($fn, "r+");
if (flock($fp, LOCK_EX)) { $serial=fgets($fp);$serial++; }
else
{ print('lock error, ABORT'); exit; }
$h=fopen($fn.'.tmp','w'); fwrite($h,$serial);fclose($h);
if (filesize($fn.'.tmp')>0)
{
system('rm -f '.$fn.'.tmp');
fseek ($fp, 0);
fwrite($fp,$serial);
}
flock($fp, LOCK_UN); fclose($fp); #chmod($fn,0777);
return $serial;
}
this example will get you a unique serial number, after this, you can be sure it's existing with only one instance.
please note, to avoid data corruption, you must create first your file and put a number first.
(for example, write 1 without enter or anything else)
this is a really simple function, but it's working for me over 10 years now ...
You can make your own increment value to guarantee 100% uniqueness without use heavy algo:
Session unique id :
session_start();
$_SESSION['increment'] = 5;//i fix 5 but you need to get it in bdd,xml,...
function get_new_id(){
$_SESSION['increment']++;
//store new value of increment
return $_SESSION['increment'];
}
$my_unique_id = get_new_id();
echo $my_unique_id;
Global unique id (dont use this !):
function get_new_id(){
$increment = file_get_contents('increment.txt');
$increment++;
file_put_contents('increment.txt', $increment);
return $increment;
}
$my_unique_id = get_new_id();
echo $my_unique_id;

PHP APC problems when storing objects

I am trying to build an configuration parser for my application I installed APC today, but everytime I try to put an serialized object in the store, it does not get in there and does not. (I am checking with apc.php for my version[3.1.8-dev] on PHP 5.3.16 [My Dev Environment], so I am sure that the data is not in the cache). this is how I pass the data to the cacher:
// The data before the caching
array (
'key' => md5($this->filename),
'value' => serialize($this->cfg)
);
// The caching interface
function($argc){
$key = $argc['key'];
Cache\APC::getInstance()->set($key,$argc['value']);
}
// The caching method described above
public function set($key, $val) {
if (apc_exists($key)) {
apc_delete ($key);
return apc_store($key, $val);
}
else
return false;
}
// the constructor of the configuration class.
// It 1st looks for the configuration in
// the cache if it is not present performs the reading from the file.
public function __construct($filename = '/application/config/application.ini',
$type = self::CONFIG_INI)
{
if (defined('SYSTEM_CACHE') && SYSTEM_CACHE === 'APC'){
$key = md5($filename);
$cfg = APC::getInstance()->get($key);
if (!empty($cfg)) {
print "From Cache";
$this->cfg = unserialize($cfg);
return;
} else {
print "From File";
}
}
}
I did a few tests and there is not a problem with the MD5() key (which I thought while writing this question) nor with APC itself. I am really stuck on this one, nothing odd in the logs, so if anyone can give me at least some directions will be very appreciated.
Thanks in advance!
The problem is was in my code:\
public function set($key, $val) {
/*
*
* If the key exists in the cache delete it and store it again,
* but how could it exist when the else clause is like that...
*/
if (apc_exists($key)) {
apc_delete ($key);
return apc_store($key, $val);
}
// This is very wrong in the current case
// cuz the function will never store and the if will
// never match..
else
return false;
}
NOTE:
Always think and keep your eyes open, if you still can't find anything get off the PC and give yourself a rest. Get back after 10-15 minutes and pown the code. It helps! :D

Categories