I am trying to improve my Fat Free Framework web app by using cached queries, wherever possibile.
I am wondering whether it would make any sense to try caching even queries involving data which changes quite ofter or so, by applying a little 'trick'.
IE, tipically you wouldn't cache this:
SELECT * FROM tasks WHERE status = 'open';
But what if I could instead cache it indefinitely and have external factors clear its cache when a specific event (e.g. a task is closed, in this case) occurs?
If this is my query:
$tasks = $this->db->exec(
array(
"SELECT *
FROM tasks
WHERE status = 'open'"
),
NULL,
86400
);
is there any possible way by which I can force its 24h-long cached version to expire earlier?
Yes:
Solution #1:
Clear the whole cache content: $f3->clear('CACHE')
Solution #2:
Tag your cached query:
$tasks = $this->db->exec(
array(
"SELECT *
FROM tasks
WHERE status = 'open'"
),
NULL,
[86400,'mytag'] // << tag defined here
);
Then you can clear only the cache content matching this tag:
Cache::instance()->reset('mytag');
Related
I am trying to create Facebook Ads cleaner, which deletes completed AdSets in a specified campaign. There's a limit how may non-deleted non-archived AdSets you can have (which is 10000 for bulk accounts), and we are reaching this limit quite quickly.
I am using Marketing API, and Facebook Ads API SDK for PHP:
https://github.com/facebook/facebook-php-ads-sdk
While it is clear how AdSets can be retrieved, I can't find a way to filter them by end time. There is a "since" parameter, but it looks like it doesn't filter on AdSet start and end date, and rather filters stats themselves. Therefore AdSet, which is still active, will get into specified range.
My code:
use FacebookAds\Object\Campaign;
use FacebookAds\Object\Fields\AdSetFields;
use Carbon\Carbon;
// Facebook ID of a Ad Campaign
$external_id = '123456789';
$campaign = new Campaign( $external_id );
// Querying required field only
$fields = [
AdSetFields::END_TIME
];
// Need to query AdSets which have been completed in the past...?
$params = [
];
$adsets = $campaign->getAdSets( $fields, $params );
$adsets->setUseImplicitFetch( true );
while ( $adsets->valid() )
{
/** #var AdSet $adset */
$adset = $adsets->current();
$end_time = Carbon::parse( $adset->{AdSetFields::END_TIME} );
if ( $end_time->isPast() )
{
$adset->delete();
}
$adsets->next();
}
The problem: in case there are lots of AdSets in a Campaign, this will literally mean iterating ALL AdSets inside one-by-one and checking it's end date. Usually, we're speaking about thousands of records, and such approach is highly ineffective. As we have lots of ad accounts, "cleaning" script runs forever, even if it doesn't timeout quickly.
Question: is there a way to filter AdSets by end_time? It looks like it is possible by explicitly proving a value, but I can't find a way to filter value using "less than" or "more than" syntax.
If adset has completed running, the its effective status will be in 'PAUSED' state. This should return a smaller subset of adsets. You can filter adsets in the PAUSED state and then check for their end time and delete after validation.
I'm building a feature into a Laravel 5 app that will allow you to set the content of a status banner that will display across the top of the page. We will be using this banner both to display page-specific things (status messages, etc) and site-wide announcements (every user sees the same thing, banner stays the same for awhile).
Right now, I've implemented this by using Laravel sessions to allow banners to be added by calling a helper method from any controller or middleware:
// Call set_banner from in a controller or middleware (for persistent banners)
function set_banner($banner_text, $banner_class, $banner_persistant=false, $replace=false)
{
$banners = session()->get('banners', []);
// Create new banner
$banner = [
'text' => $banner_text,
'type' => $banner_class,
'persistent' => $banner_persistant
];
// Only put banner in array if it's not already there
if( !in_array($banner, $banners) ) {
// Either override existing banners, or add to queue
if( !$replace ) session()->push('banners', $banner);
else session()->put('banners', [$banner]);
}
}
// Called by default in the master.blade.php template
function get_banners()
{
$banners = session()->pull('banners', Array());
foreach( $banners as $banner ) {
// Print out each banner
print '<div class="col-md-12"><div class="text-center alert alert-block alert-'.$banner['type'].'">';
print $banner['text'];
print '</div></div>';
// Push back into the session if banner is marked as persistent
if ( $banner['persistent'] ) session()->push( 'banners', $banner );
}
}
Banners are created in controllers or middleware like this:
set_banner("<b>Note:</b> This is a sample persistant-scope banner set in a controller", "success", true);
Is there a better way to accomplish storing both page-level and site-wide banners? My concerns is that hitting the session on every pageload may be inefficient, especially for banners that won't be changing for long periods of time. Will this approach mess with Laravel's cache, etc?
As you said the banners do not change that often. Hence for me i would implement it using Cache. This improves performance since we need only one use to have the banners cached. And for the rest its retrieved faster from the Cache rather Session.
Do you want to have to change code to change the banner of a given page?
I would suggest instead creating a "pages" package, where each page route name is entered into a database.
From there, from your page service provider you get Page::getModel()->banner_text or something similar.
The method would look for a db result matching the current route name with a result within db.
when a controller method is triggered you simply call
Page::getBannerText()
That method will pull the current route name, pull the page result related to that page if it exists or create it if it does not exist (easy way to get everything). You cache the db query result for X hours, days or whatever so whenever someone else makes a call, you don't even need to deal with any storage on client side.
This allows you to modify the value from a db fascet. Its the more "proper" way to do it.
Following is the configuration Code for my DB cache in Yii
'dbcache'=>array(
'class'=>'system.db.CDbConnection',
'connectionString'=>'sqlite:/' . str_replace('\\','/',str_replace('C:\\','',getcwd()).'/protected/data/cache.db'),
),
And the following is my code for getting the record set and setting in the cache:
$recordset = Table1::model ()->cache(0)->find ( "primary_id=:id", array (":id" => $id) );
I have used 0 for the cache duration because I need to make it for infinite period.
Now I need to refresh my Cache. on some condition. How can I refresh the query Cache in Yii if its duration is infinite. Please help :)
First of all, setting the value as 0 doesn't mean that you are setting for Infinite Period. It does mean that you have simply Disabled the cache.
Refer the Code: Class Reference - CActiveRecord
If you want to refresh the Schema Cache, use the code as Yii::app()->schema->refresh()
I made a Diary at http://kees.een-site-bouwen.nl/agenda which can have Events for specific dates. You can add an event using the form located here: http://kees.een-site-bouwen.nl/evenementform
As you can see I have 3 fields 'Datum, Van & Tot' (translation: Datum = date, Van = From, Tot = till)
If the time on that specific date expires I would like to run a script which deletes that specific row from the database.
I searched on google and found a few things like MYSQL Trigger and PHP cronjob, but I don't know if there's an easier way to do this? or how to do it exactly.
My database structure looks like this:
agenda // diary
- - - - // not added the whole structure.
idagenda
titel
waar
www
email
activated
....
....
agendadatum // diary dates
- - - - - -
id
datum
van
tot
idagenda
as you can see I'm using a join to add more dates to one event.
How could I trigger the datetime to delete the rows from the db if the date = today?
NOTE: I am using the Codeigniter framework.
You could set a hook. And use a function like
$this->db->where('tot <', date('Y-m-d H:i:s'));
$this->db->delete('agendadatum');
My codeigniter is a bit rusty but that query should remove all "old" entries on EVERY page load. So if you're going for high traffic this will not hold up.
Running a cronjob every hour/day is probably the "cleanest" way. This will require you to set a where condition on all selections of agendadatum that forces the "tot" date to be in the future. Else it's possible you see expired entries.
Edit:
From what I can gather if you define your hook like:
$hook['post_controller_constructor'] = array(
'class' => '',
'function' => 'delete_old_entries',
'filename' => 'agenda.php',
'filepath' => 'hooks',
'params' => array()
);
And create a file named agenda.php in application/hooks which looks like:
<?php
function delete_old_entries() {
$CI =& get_instance();
$CI->load->database();
$query = $CI->db->query(" DELETE FROM agendadatum WHERE tot < NOW(); ");
}
It should work. However this is pieced together from what I could find and untested. So it might work, it might not. But something along these lines should do the trick even if it isn't this.
If I understand correctly:
CREATE TRIGGER deleteRows AFTER UPDATE,INSERT ON myTable
FOR EACH ROW BEGIN
DELETE FROM myTable WHERE Datum = NOW()
END;
MySQL has a built-in Event Scheduler that basically allows you to run arbitrary SQL code at scheduled times. You could purge your database once a day for instance.
However, I know from your other question that you are hosting your project on a shared host, and, unfortunately, shuch hosts often disable the feature. Check out this manual page to find out whether the feature is active on your server.
I have a Yii dropdown which loads a table of cities, states, zipcodes, lat and lon. When loading the state dropdown, it takes forever. Is there a way to speed up the query to cut down on pageload time? I've included my view:
echo $form->dropDownList($model,'State', CHtml::listData(Zipcodes::model()->findAll(),
'State', 'State', 'State'), array('empty'=>'-- Choose State --'));
The table is 41,000 entries. Setting the $groupField in listData() didn't seem to give any noticeable improvements.
I think you can have two ways:
1) Use cache (with long or no expiration time, because this data you have are not dynamic).
First time your loading time will not change, but after that, it will bee much, much faster, because cache will be already saved.
http://www.yiiframework.com/doc/guide/1.1/en/caching.data#query-caching
$zipcodes = Zipcodes::model()->cache(3600*24*7)->findAll(); //cache for a week
Using cache you need to edit your config/main.php file
//...
'components' => array(
//...
'cache'=>array(
'class'=>'system.caching.CFileCache',
//'class'=>'system.caching.CDummyCache',
//other cache class
),
//...
),
//...
2) Maybe consider to try CJuiAutoComplete
http://www.yiiframework.com/doc/api/1.1/CJuiAutoComplete/
It will give you results only on typing and matching your interested data.
Not loading 41k entries and placing them in HTML <option> tags seems to be a very good start... :-)
Only load those you need using AJAX.
Is the state drop-down culling unique states from that table, i.e., you end up with 50-ish results? You might want to construct the query manually and see where the bottleneck is. For complex queries, this is often dramatically faster than using AR, which I'm assuming you are using to create the model. I'm also assuming you are not trying to load 41k records into a drop-down.
We might need a bit more detail about the structure of the table, how you are creating the model, etc.
Using CDbCriteria, I set the "GROUP BY" for the query:
$stateGroupBy = new CDbCriteria; // i got that bitch criteria. bitches love criteria!
$stateGroupBy->group = 'State'; // group by state
echo $form->dropDownList($model, 'State', CHtml::listData(Zipcodes::model()->findAll($stateGroupBy), 'State', 'State'),
array('ajax' =>
array(
'type'=>'POST', //request type
'url'=>CController::createUrl('search/dynamiccities'), #we don't have to use search/, we could just say dynamiccities
'update'=>'#Zipcodes_City',
/*'data'=>'js:jQuery(this).serialize()'*/
),
'empty'=>'-- Choose State --')
);
I flaked out in getting back to this, but this cut down on the load significantly.