We are using get_browser() in PHP using php_browscap.ini but performance is horrible. We pass 100 or so user-agents into get_browser() per page and it takes over 30 seconds to render the page. We need a performant solution, without storing the actual get_browser() results persistently (we only want to store user agents).
We already use memcached, is there a way we can alter get_browser() to cache results, or load the entire php_browscap.ini into memcached.
Ended up rolling our own solution:
////
// This function caches in memcached.
////
public static function get_browser_memcached($user_agent) {
if(empty(MemcacheConnection::$memcache_connection)) {
MemcacheConnection::connect();
}
$memcache_key = preg_replace('/\s+/', '', sha1($user_agent)) . "_user_agent";
$memcache_result = MemcacheConnection::get($memcache_key);
if($memcache_result !== false) {
return $memcache_result;
}
$browser = get_browser($user_agent);
//Store in Memcached (cached for 7 days)
MemcacheConnection::set($memcache_key, $browser, 604800);
return $browser;
}
I know I'm chiming in a little late, but for what it's worth, I'm using the browscap-php library (as mentioned by #AbcAeffchen) in one of my projects and I'm happy so far.
A typical lookup (from my own simple measurements) takes about 20~30ms on a 1 core 512MB cloud instance (which is pretty much the minimum you can find anywhere). I've opted to cache with Redis and this brings the lookup time down to just a couple of ms... so it's possible to optimize if you really need to.
The convenience alone makes it worth a try.
I havn't used the browscap-php library, but the the usage is highly recommend by the Browser Capabilities Project. http://browscap.org/
The libary on GitHub should improve the performance.
Related
I mass produce very similar sites, meaning they all use the same basic components, pages and are all single industry specific. These are my low end deeply discounted site designs. None of these sites ever get more than 20-30 visitors a day, so any extra load on the server isn't an issue.
In the interest of time, being that they all use the same components, though they may be in different locations or in a different order I would like to write one definition file that can be included with every site, so I can just call the defined constant instead of writing out the code a couple hundred times every year on every site I build. Also for editing later purposes this would make my life MUCH easier.
the definition file would look similar to the following:
define('UPCONTACT','<h1>Contact Us</h1>');
define('ULCONTACT','Contact Us');
define('UPABOUTUS','<h1>About Us</h1>');
define('ULABOUTUS','About Us');
Obviously this is a very basic example but I think you get the idea.
So the question is what are the pros and cons of using define() in this manner?
It's pretty much ok. The disadvantage is that, given you are using constants, you can't override them for a single page or site.
Use an array instead:
config.php
return array(
'aboutus' => '<h1>About Us</h1>',
'contactus' => 'Contact Us'
);
include it like this in your site:
$config = include('config.php');
Then you can print it very easily
<?php echo $config['aboutus'] ?>
You can also change a value when you need it:
$config = include('config.php');
$config['aboutus'] = '<h1>About My Company</h1>';
This is probably your best option.
It has upsides and downsides.
The upsides involve that such way is quicker than loading settings from a database (and creating a database; and creating an abstraction layer, ...).
The downsides involve that such way is not customizable by the client. If they need a change, ensure beforehand the website is static and you will charge them by every change.
IMHO it is better to have some stuff as customizable by the client, and other stuff not. But there's no technical issue at all by using define() in that way (except perhaps allowed datatypes).
A better way to use a ini file or something like that.
(and easily editable from a smartphone if it's a recursive task for you :)
Look for a builtin php function, can make simplify your life
http://php.net/manual/fr/function.parse-ini-file.php
or if you would a more stronger and flexible system,
go for templating (looking for smarty, or self made regex templating)
Looking for my first regex function (loong years ago)
Quitting Smarty to do it manually
Note:
Using Constant does not provide you to dynamically modifying them
inline code, and are poor supported type (you cannot store an array without serialize for example)
I would suggest cascaded ini files:
$conf_dir = dirname(__FILE__);
$config = array_merge_recursive(
parse_ini_file($conf_dir.'base.ini'),
parse_ini_file($conf_dir.'client.ini')
);
The benefits are readability, inability of execution (I like to lock things down that can be), and you can track the base ini in git (or whatever you use) and not the client one. There are some downsides, but such is life. The just feel cleaner, but they are not faster than .php, to be sure.
And if you wanted to eliminate any redundant execution (listen, any "performance benefit" still has "benefit" in it), serialization:
<?php
define('CACHE_DIR', '/tmp/');
// where 'http' is a path part that directly follows the app root, and will always
// be below where this file is called from.
$ini_cache = CACHE_DIR.'config.ser';
if(!file_exists($ini_cache)) {
// Build your config in any way you wish.
$conf_dir = dirname(__FILE__);
$config = array_merge_recursive(
parse_ini_file($conf_dir.'base.ini'),
parse_ini_file($conf_dir.'client.ini')
);
// Store it serialized
file_put_contents($ini_cache, serialize($config));
} else {
$config = deserialize(file_get_contents($ini_cache));
}
You can get more creative with this, but essentially, this allows you to store/generate your configuration in any way you wish. If you wanted to not have to delete the serialized cache on every change, you could add an atime check:
<?php
define('CACHE_DIR', '/tmp/');
// where 'http' is a path part that directly follows the app root, and will always
// be below where this file is called from.
$ini_cache = CACHE_DIR.'config.ser';
$conf_dir = dirname(__FILE__);
$config = array();
if(file_exists($ini_cache)) {
$client_stat = stat($conf_dir.'client.ini');
$cache_stat = stat($ini_cache);
if($client_stat['atime'] < $cache_stat['atime']) {
$config = deserialize(file_get_contents($ini_cache));
}
}
if(empty($config)) {
// Build your config in any way you wish.
$config = array_merge_recursive(
parse_ini_file($conf_dir.'base.ini'),
parse_ini_file($conf_dir.'client.ini')
);
// Store it serialized
file_put_contents($ini_cache, serialize($config));
}
With either serialization method, you can use what ever $config generation scheme you prefer, and if you use PHP, you can even get real creative/complicated with it, and the cached hit to the page will be negligible.
I'm currently coding one of my first php applications.
The application has to connect to a LDAP server and change some user attributes in the directory.
That application has some parameters to read in a mySQL Database in order to run.
Some examples of these parameters could be:
-LDAP Address
-LDAP Service Account
-LDAP Password
there are much more parameters, which rule, for example, the way users authenticate to my application,...
Currently, the database is read at each user session initialization, but, it doesn't have any sense because parameters do not vary from a session to another.
So, i'm looking for a way to load these parameters from the database, only one time (for example, at the php service initialization), and access to these parameters in the "normal" php code through variables.
What would be the best way to do this?
Thank you in advance.
You are looking for a persistent cross-request storage. There are many options for this.
The simplest is APCu (which can be used in conjunction with Zend OpCache, or for PHP < 5.5, APC).
Simply:
if (apc_exists('mykey')) {
$data = apc_fetch('mykey');
} else {
// create it from scratch
apc_store('mike', $data);
}
$data can be most any PHP type, arrays, objects, or scalars.
You can even put this code in the auto_prepend_file INI setting so it is run automatically on every request.
However: this is per server (and per SAPI, so mod_php/php-fpm/cli don't share the cache) so you will have to create it once per server.
Alternatively, for a multi-server setup you can use something like memcached or redis. These are stand-alone daemons that will let you store arbitrary key/value pairs of string data (so you may need to serialize()/unserialize() on the values).
I personally prefer memcache, which has two extensions for PHP, pecl/memcached and pecl/memcache (I prefer pecl/memcached, it has more features).
Both of them are pretty simple.
pecl/memcached:
$memcache = new Memcached();
$memcache->addServer('localhost', '11211');
$data = $memcache->get('mykey');
if (empty($data)) {
// Create data
$memcache->set('mykey', $data);
}
pecl/memcache:
$memcache = new Memcache();
$memcache->connect(); // uses localhost:11211, the default memcache host/port
$data = $memcache->get('mykey');
if (empty($data)) {
// Create data
$memcache->set('mykey', $data);
}
Both extensions support storage of arrays and objects without serialization.
You can of course store multiple keys with any of these solutions and just pull them all, instead of using one, or one with an array/object.
You can use Memcache do cache database requests. See here how to use.
Another way is using Php Sessions.
<?php
session_start(); // need to be before any html code
$_SESSION['something'] = 'Something here...';
echo $_SESSION['something']; // will show "Something here..."
And you can remove using...
unset($_SESSION['something']);
You also can use cookies, using the function setcookie. See here.
And you can get cookies using...
echo $_COOKIE['something'];
Production mode
In a production mode, this will work as set_transient of Wordpress. You will do the first db request to get the value and will cache this value using cookies, sessions or memcache.
If you want to show this values inside of your page, you can use a standard caching library.
My understanding of the question is that you have some SQL data that is more or less constant and you don't want to have to read that in from the SQL connection on every request.
If that is the case you can use memcache to store the data:
http://php.net/manual/en/book.memcache.php
The data will still be persistent and you will only need to go to the database if the cached data isn't there or needs to be refreshed.
If the data is specific to a particular user you can just use a session.
http://php.net/manual/en/book.session.php
http://php.net/manual/en/session.examples.basic.php
If this is only to be used when starting up your server (so once and done) and you don't want to bother to with memcached/xcache (as they would be over kill) you can still use environment variables. See get_env
I need to know if I can improve the way I cache my api calls from the inside of my CodeIgniter app.The way I do it right now is like this, in a hmvc pattern:
Controller HOME == calls to => module app/application/modules/apis/controllers/c_$api == loads library => app/application/libraries/$api ==> Library returns response to module's controller_X, the controller invokes the view with the data it has
//Note: My app does not use twitter api, but others
Inside the apis module is where the all the apc caching is happening, like so:
// Load up drivers
$this->load->library('driver');
$this->load->driver('cache', array('adapter' => 'apc'));
// Get Tweets from Cache
$tweets = $this->cache->get('my_tweets');
if ( ! $tweets)
{
// No tweets in the cache, so get new ones.
$url = 'http://api.twitter.com/1/statuses/user_timeline.json?screen_name=gaker&count=5';
$tweets = json_decode(file_get_contents($url));
$this->cache->save('my_tweets',$tweets, 300);
}
return $tweets;
as explained in this article: http://www.gregaker.net/2011/feb/12/codeigniter-reactors-caching-drivers/
So I was wondering:
Having 3 scenarios: home, query, result; in each module apis's controller, do you think it would be a good idea to implement cache for each controller with all the scenarios? example:
//for each api1, api2 ... apiX, apply this:
//home
$this->cache->save('api_home',$api_home, 300);
//query
$this->cache->save("api_$query", $api_{$query}, 300); // I don't know for sure if $api_{$query} works or not, so don't hang me because I haven't tried it.
//result
$this->cache->save("api_$queryId", $api_{$queryId}, 300);
Even though I cached the api call, do you think I should cache the result in the controller that is calling the api module controller, with the same 3 scenarios (home, query and result)? Like so:
//modules/{home,fetch,article}/controllers/{home,fetch,article}.php
//home
$homeData['latest'][$api] = modules::run("apis/c_$api/data", array('action'=>'topRated'));
$this->cache->save('home_data', $home_data, 300);
//query
$searchResults[$api] = modules::run("apis/c_$api/data", $parameters);
$this->cache->save("search_results_$query", $search_results_{$query}, 300);
//article page
$result = modules::run("apis/c_$api/data", $parameters);
$this->cache->save("$api_article_$id", ${$api}_article_{$id}, 300);
So, what do you think? Is it a good practice the mentioned above, or just an awful stupid one?
//Note, the suggested caching ideas were not tested... so, I don't know if ${$api}_article_{$id} will work or not (even though I suppose it will)
IMHO It is a good idea to cache api results if you don't need real time data. If you don't care that you won't see new data for an hour, then by all means cache it for an hour. So for your first question, you just need to ask yourself: "How fresh does the content need to be for my application?" and implement caching accordingly.
For the second question: I don't see a lot of value in caching content if it's only been manipulated in simple ways. At that point you're using up space in your cache and not getting a lot of value. But if there are database, or other api calls being made using that data, then yes they should be cached using a technique similar to the above.
If you're that worried about processor load (the only reason to cache content after manipulation) you're best bet is to look at something like Varnish or CloudFront.
I'm migrating from memcache to memcached and one of the differences I've noticed is that the compression flag seems to be set for all set() calls, not on an individual basis like with memcache.
Is this a per-connection setting, or can I toggle this on and off? If it's the latter, what would be the best way to go about setting the compression flag for each set() call?
Currently I'm thinking of doing something like this:
<?php
if ($compress) $mc->setOption(Memcached::OPT_COMPRESSION,true);
$return = $mc->set($key,$object,$expiration);
if ($compress) $mc->setOption(Memcached::OPT_COMPRESSION,false);
Is this the right approach?
I'm also setting up $mc like this, if it makes any difference:
<?php
$mc = new Memcached("cl");
$mc->setOption(Memcached::OPT_LIBKETAMA_COMPATIBLE,true);
$mc->setOption(Memcached::OPT_BINARY_PROTOCOL,true);
if (count($mc->getServerList()) === 0) {
$mc->addServers(array (
array ($GLOBALS["mc"]["host"],$GLOBALS["mc"]["port"]),
));
}
Thanks!
Taking a look at the signature of the two methods and also reading the source of Memcached, I think it is not possible to set it for each set.
See php-memcached-dev on github, look for php_memcached.c
(What a stupid name for the client by the way. Memcached used to mean the server daemon, now it means server and client, you just can't google it any more.)
My Google-fu hasn't revealed what I'm looking for, so I'm putting this one out to the crowd.
Coming from an ASP.NET development background, I'm used to having the Application and Cache collections available for me to stash rarely-modified but often-used resources (such as lookup rows from a database or the contents of static XML documents) in the memory of the web server, so I don't have to reload these often-used items during every request.
Does PHP have an equivalent? I've read up briefly on the memcache extension, but this won't work for me (as I don't have control over the server configuration.) I'm tempted to implement something that would allow me to pre-parse or pre-select the resources and generate a sort of PHP cache "file" that would construct the cached object from literals stored in the file, but this seems like a very hacky solution to me.
Is there something in PHP (or, alternatively, a helper library of some sort) that will allow me to accomplish this using best practices?
In short, no, such a thing is not available natively in PHP. To understand why, you have to understand that PHP has its entire environment built for each request, and it is subsequently torn down at the end of the request. PHP does give you $_SESSION to store per session variables, but after digging into the docs you will see that that variable is built during each request also. PHP (or mod php to be more specific) is fundamentally different from other "application servers". Basically, it is not an application server. It is a per request script runner.
Now, don't get me wrong, PHP lets you do application level data store, but you will have to go to a database, or to disk to get it. Remember this though, don't worry about optimizing for performance until it is shown that preformance is a problem. And I will guess that 99 times out of 100, by the time performance is an issue that isn't due to some poor code you wrote, you will have the resources to build your own pretty little memcached server.
Take a look at Zend_Cache library, for example. It can cache in multiple backends.
This is a bit of a hack but but works in php 7+
Basically you cache your data to a temp file and then use include to read the file, which is cached in memory by the php engine’s in-memory file caching (opcache)
function cache_set($key, $val) {
$val = var_export($val, true);
// HHVM fails at __set_state, so just use object cast for now
$val = str_replace('stdClass::__set_state', '(object)', $val);
// Write to temp file first to ensure atomicity
$tmp = "/tmp/$key." . uniqid('', true) . '.tmp';
file_put_contents($tmp, '<?php $val = ' . $val . ';', LOCK_EX);
rename($tmp, "/tmp/$key");
}
And here’s how we “get” a value from the cache:
function cache_get($key) {
#include "/tmp/$key";
return isset($val) ? $val : false;
}
from https://medium.com/#dylanwenzlau/500x-faster-caching-than-redis-memcache-apc-in-php-hhvm-dcd26e8447ad