I have a Facebook application with 250K DAU. I use Memcached to cache the files and MySql selects because without that my server is going crazy.
I have set up the code like this:
$memcache = new Memcache;
$memcache->connect('127.0.0.1', 11211) or die ("Could not connect");
// FOR FILES
$doc = $memcache->get('mem_index_html');
if ($doc == null)
{
$doc = new Document( HTML.'index.html' );
$memcache->set('mem_index_html', $doc, 0, 86400);
}
// FOR SQL
$sql=sprintf('count(qtn_id) FROM tb_questions where qtn_lang="EN"));
$total_pages = $memcache->get($sql);
if ($total_pages == null)
{
$query_id = DB::query($sql);
list( $total_pages ) = DB::nextRow( $query_id );
DB::free( $query_id );
$memcache->set($sql, $total_pages, 0, 86400);
}
The problem is that after some time – usually 24 hours, errors are starting to show. No log is created but I receive a timeouts if I try to reload the page. I assume that there is a lots of traffic in that moments, the Memcached is not working correct and everything is slowing down.
My temporary solution is to Flush the memcache - $memcache->flush(), but that is not good solution.
Am I doing something wrong? Is there something to optimize or I don’t know. I found searching thru web this http://code.google.com/p/memcached/wiki/TutorialCachingStory about making more that one Memecached servers. Is that the best solution?
Thanks for your answers.
Darko
The fourth parameter in the $memcache->set command represents the expiration timeout. Once this is exceeded, the object stored will no longer be available. Since you set it to 24 hours they would not be available after that period unless you updated them.
If the expiry parameter is set to 0, the item will never be expire, but it can still be deleted if memcached is filled up and need to free some space.
If the value is less than 30 days (30 * 24 * 3600 = 2592000), the value provided is treated as a offset from the current time.
If the value is larger than that it will be interpreted as absolute time.
Related
I have a simple script that counts from 1 to 5000 with a for loop. It flushes output in real time to browser and shows a progress bar with %.
What I have: If I leave the page, the process interrupts. If I come back, it starts from 0.
What I want to achieve: If I leave the page, the process continues and, If I come back , it shows the right percentage.
Example: I run the process, it counts till 54, I leave the page for 10 seconds, when I come back it shows me 140 and continues to flush.
Is it possible?
I would suggest you to use server workers - scripts which are intended to run independently from webserver context.
The most common way of doing it - usage of message queues (RabbitMQ, Qless, etc). Event should be initiated by the script in web context, but the actual task should be executed by queue listener in a different context.
What you have asked seems quite simple to do with a session. (Purely assuming on the use case given). This is not running any process in the background, it just simply keep track of the time and show the progress. That's why I said "based on what you asked". If you want to keep track of any real background tasks, then I believe the case would be totally different, and you will have to change the wordings of your question as well ;)
Something like this would do.
<?php
session_start();
$s = &$_SESSION;
$sleep = 1; //seconds
//check if we have a value set in session before, if not set default = 0.
if(!isset($s['last'])){
$s['last'] = 0;
}
//check if we have a last time set in session before. if not set a default = curret time.
if(!isset($s['time'])){
$s['time'] = time();
}
//get the idle time of the user.
$idle = time() - $s['time'];
//start the loop..and set starting point.
$start = $s['last'] + ($idle / $sleep);
for( $i = $start; $i < 100; $i++){
echo $i . '<br />';
$s['last']++;
$s['time'] = time();
flush();
sleep($sleep);
}
Hope it helps!!
I've been searching for a suitable PHP caching method for MSSQL results.
Most of the examples I can find suggest storing the results in an array, which would then get included to page. This seems great unless a request for the content was made at the same time as it being updated/rebuilt.
I was hoping to find something similar to ASP's application level variables, but far as I'm aware, PHP doesn't offer this functionality?
The problem I'm facing is I need to perform 6 queries on page to populate dropdown boxes. This happens on the vast majority of pages. It's also not an option to combine the queries. The cached data will also need to be rebuilt sporadically, when the system changes. This could be once a day, once a week or a month. Any advice will be greatly received, thanks!
You can use Redis server and phpredis PHP extension to cache results fetched from database:
$redis = new Redis();
$redis->connect('/tmp/redis.sock');
$sql = "SELECT something FROM sometable WHERE condition";
$sql_hash = md5($sql);
$redis_key = "dbcache:${sql_hash}";
$ttl = 3600; // values expire in 1 hour
if ($result = $redis->get($redis_key)) {
$result = json_decode($result, true);
} else {
$result = Db::fetchArray($sql);
$redis->setex($redis_key, $ttl, json_encode($result));
}
(Error checks skipped for clarity)
I am wondering how I would use Cache (MemCache) to display how many registered and active users I have on my website. I currently use:
return mysql_result(mysql_query("SELECT COUNT(`user_id`) FROM `users` WHERE `active` = 1"), 0);
And this isn't good to use because on each page load, it's going to query the database. Which isn't good. So how would I go upon making a "memcache" or something of that sort, where it would update that number to all users, after say 10 minutes or so?
I googled how but I couldn't really get a good how to on it.
Thanks!
I'll give you the guidelines rather than a full documentation/tutorial.
Before trying to install Memcached, you should use MySQLi or PDO instead of mysql_* functions. Checkout the warning message in this documentation.
You should install Memcached and its PHP extension.
Once it is done, make sure it is enabled by calling the function phpinfo(). If Memcached isn't there the installation failed.
Here is an explicit code snippet
$mc = new Memcached();
$mc->addServer('localhost', 11211); // add a new server to the pool
// you will want to vary this key according to the query
// if two different queries use the same key your functions will return unexpected data
$cacheKey = 'a-custom-key-that-defines-the-data-within-it';
// how long will it take for the cache to expire in seconds : an hour
$cacheLifetime = 3600;
$data = $mc->get($cacheKey); // we get the data from cache
// check if Memcached was able to retrieve the data
if ($mc->getResultCode() === Memcached::RES_FAILURE) {
// if for some reasons the data is not there we get it and reinject into memcached
$data = mysql_result(mysql_query("SELECT COUNT(`user_id`) FROM `users` WHERE `active` = 1"), 0);
$mc->set($cacheKey, $data, $cacheLifetime); // save the data in the defined $cacheKey
}
return $data;
I just designed an admin console for a social networking website. My boss now wants me to cache the results of several MySQL queries that build these results (for 24 hours). The site uses Memcached (with Wordpress W3 total cache) and XCache. I wanted to know what's the best way to do this.
Here is an example of one such query and how I am getting the results (basically I am returning aggregate stats on users, which means my results are fairly simple, eg:
//users who registered in last 365 days
$users_reg_365 = "select ID
from wp_users
where user_registered > NOW() - interval 365 day";
then use the wpdb query class to get the results:
$users_reg_365 = $wpdb->get_results($users_reg_365);
then display the result in the dashboard:
<li><?php echo "Total users who registered within last 365 days: <span class='sqlresult'>" . sizeof($users_reg_365) . "</span>"; ?></li>
My understanding of Memcached/XCache is that it basically stores strings, so would it make sense to just cache sizeof($users_reg_365)?
The last wrinkle is that our Wordpress site uses W3 total cache, which leverages Memcached, and the boss asked me not to use Memcached but XCache instead, but I find the docs a bit confusing. What's the best way to solve this problem? Can SQL itself be told to 'remember' certain queries like this, or is memory caching the way to go?
Thanks!
You can find more about the differences of both here:
Difference between Memcache, APC, XCache and other alternatives I've not heard of
An example how you could
<?php
$m = new Memcached();
$m->addServer('localhost', 11211);
// cache 24hrs
$cache_expire = 86400;
// users is your key
$users_reg_365 = $m->get('users_reg_365');
if (empty($users_reg_365)) {
$users_reg_365 = "select ID from wp_users where user_registered > NOW() - interval 365 day";
$m->set('users_reg_365', $cache_expire);
}
If you need to exactly refresh the cache at middle night change the value of $cache_expire.
You can refer to the full reference of memcached at
http://www.php.net/manual/en/memcached.get.php
Hm. I'm not sure how your caching is configured inside of WordPress. If you have WordPress's object cache set up to use Memcache(d)/XCache for persistent caching, you could do something like this:
$key = 'custom-key-to-look-up-later';
$data = sizeof( $users_reg_365 );
$group = 'group-id';
$expire = 60 * 60 * 24 // time in seconds before expiring the cache.
wp_cache_set( $key, $data, $group, $expire );
Later you can look up that value like this:
$data = wp_cache_get( $key, $group );
if( ! is_wp_error( $data ) )
// the data is good. do something with it.
You can find the docs on these functions here.
Before you begin, though, make sure WordPress is set up to work with Memcache(d) or XCache. :)
IF you want to store the results of any array just serialize/jsonencode it and store as that..
Hey guys i'm making a website where you submit a server for advertising. When the user goes to the index page of my website it grabs the ip's of all the servers submitted and then tests to see if it is online using fsockopen() like so:
while($row = mysql_fetch_assoc($rs)) {
$ip = $row['ip'];
$info = #fsockopen($ip, 25565, $errno, $errstr, 0.5);
if($info) {
$status = "<div><img width='32px' height='32px'
title='$name is online!' src='images/online.png'/></div>";
$online = true;
} else {
$status = "<div><img width='32px' height='32px'
title='$name is offline!' src='images/offline.png'/></div>";
$online = false;
}
}
}
This way works fine, but the only downside is when you load the site it takes a good 2-4 seconds to start loading the website due to the fsockopen() methods being called. I want to know if there is a better way to do this that will reduce the amount of wait time before the website loads.
Any information will be appreciated, thanks.
Store the online status and last check time in a database, if the last check time is longer than 15 minutes for example, update it. I am pretty sure you don't need to get the status on EVERY pageload? It's the time it takes to connect to each server that slows down the website.
Then again, you would probably wanna move the update process to a cronjob instead of relying on someone visiting your website to update the server statuses.
Looking at your example, I'd make all the $status bits be javascript calls to another php page that checks that individual server.
However, the idea to move the status checks to cron job or use some kind of status caching is very good too. Maybe store statuses in a database only only check the ones that have expired (time limit set by you).