I came across a bug using Memcached in PHP. Here's my piece of code:
<?php
$mc = new \Memcached();
$mc->setOption(\Memcached::OPT_BINARY_PROTOCOL, true);
$mc->addServer("127.0.0.1", 11211);
$mc->touch("key", time() + 600);
$touchResult = $mc->getResultCode();
$mc->set("key", 1, time() + 600);
$setResult = $mc->getResultCode();
echo "<pre>";
echo "Touch result: $touchResult\n";
echo "Set result: $setResult\n";
echo "</pre>";
When you run this for the first time, this is the output:
Touch result: 16
Set result: 0
And for the second time forth:
Touch result: 0
Set result: 5
Correct me if I'm wrong but this is a bug right? Does anyone know a workaround for this?
Here are the versions I use:
Ubuntu 12.04 64bit
PHP 5.3.14
memcached 2.1.0 (PECL module)
libmemcached 1.0.8
Memcached sever 1.4.13
PS. If you wonder what the result codes mean, here they are:
0 RES_SUCCESS
5 RES_WRITE_FAILURE
16 RES_NOTFOUND
[UPDATE]
I played a little more with the code and found something even more interesting. This bug happens regardless of the key that touch and set are working on. As long as the touch operation returns 0 (which means it was successful) the set operation will fail.
[UPDATE]
I managed to produce some other errors as well. e.g. acquiring some key from server and then adding some other will also lead to nasty problems (RES_END code). I believe all these problems are somehow related to binary protocol. It seems to me as if binary protocol's implementation is hardly near stable. Operations which can work without binary protocol will do just fine but once the protocol is set to binary, they will result in blocking problems.
All right.
In first time, you touch not existed key - result is RES_NOTFOUND. When you do set - you write value success - RES_SUCCESS.
In next time you touch existed key (you set it in first linch) and get result of operation RES_SUCCESS, next you try set value for existed key - result false. All right.
If you want change existing value you must use Memcached::replace() method instead of "set"
Related
I've started using the increment() method of the PHP Memcached client, and with that switched to the binary protocol. Apparently, increment() is only supported on the binary protocol. Occasionally, I'm seeing garbage results come back from incremented keys. For example:
$memcached = new \Memcached();
$memcached->setOption(\Memcached::OPT_BINARY_PROTOCOL, TRUE);
…
$this->cache->increment($key,1,1);
…
$this->cache->get($key);
Output:
"1\u0000ants1 0 1\r\n1\r\n1\r\n25\r"
Given that the key did not exist before it was incremented at first, and an initial value of 1 was given to the increment() call, I'd expect the value returned to be an integer. Instead, the strings returned look like left-over junk, e.g. the ants part of that string has no relevance.
Other (possibly) pertinent info:
I'm seeing this on a range of different keys
Our Memcached server is an AWS Elasticache instance
Other clients using the same cache node are not using the binary protocol.
All clients are running the same OS (CentOS), PHP and Memcached versions.
tl;dr;
This is a bug in the PHP extension code...
I dug into the PHP extension code that wraps libmemcached and the libmemcached API code itself, but I think I've found the possible underlying cause of your problem...
If you take a look at the PHP Memcached::increment() implementation you'll see on line 1858 of php_memcached.c
status = memcached_increment_with_initial(m_obj->memc, key, key_len, (unsigned int)offset, initial, expiry, &value);
The problem here is that offset may or may not be 64 bits wide. The libmemcached API tells us that the memcached_increment_with_initial function signature expects a uint64_t for offset whereas here offset is declared long and then cast to unsigned int.
So if we were do something like this...
$memcached = new memcached;
$memcached->addServer('127.0.0.1','11211');
$memcached->setOption(\Memcached::OPT_BINARY_PROTOCOL, TRUE);
$memcached->delete('foo'); // remove the key if it already exists
$memcached->increment('foo',1,1);
var_dump($memcached->get('foo'));
You'd see something like...
string(22) "8589934592
"
as the output from that script. Note this only works if the key foo does not already exist on that memcached server. Also note the length of that string at 22 characters, when clearly it's not supposed to be anywhere near that.
If you look at the hex representation of that string....
var_dump(bin2hex($memcached->get('foo')));
The result is clear garbage at the end...
string(44) "38353839393334353932000d0a000000000000000000"
The object that was being stored was clearly corrupted between the casts. So you may end up getting the same result as me or you may end up getting completely broken data as you have demonstrated above. It depends on how the cast effected the chunk of memory being stored at the time (which is falling into undefined behavior here). Also the only seemingly root cause for this is using an initial value with increment (using increment subsequently after this does not demonstrate that problem or if the key already exists).
I guess the problem of this stems from the fact that the libmemcached API has two different size requirements for offset parameter between memcached_increment and memcached_increment_with_initial
memcached_increment(memcached_st *ptr, const char *key, size_t key_length, uint32_t offset, uint64_t *value)
The former takes uint32_t whereas the later takes uint64_t and PHP's extension code casts both to unsigned int, which would be equivalent to uint32_t pretty much.
This discrepency in the width of the offset parameter is likely what causes the key to be corrupted somehow between the calling PHP extension code and the API code.
I got a sample Script which looks nearly like this:
$memcache = new Memcache;
$memcache->addServer('memcache_host', 11211);
$memcache->addServer('memcache_host2', 11211);
If i use now
$memcache->set('var_key', 'some really big variable');
The key get/value get stored (only on one server), of course. But when i reorder the Memcache Serverlist, its recreating the cache. I noticed it always use the second Server... But why?
I just want to know and understand, how the hash for storage gets generated.
And which factors are relevant for the hash and server selection?
I could not find anything about.
Regards!
The Answer is:
crc32($KEY) % Server.length
So in my case, i got 2 Servers with the Sample Key "var_key".
crc32('var_key') % 2
The result is 0. So the first Server is the lucky one!
crc32('var_key_bit_longer_and_longer') % 2
In this case the second server (Result is 1) gets chosen.
As we can see, the following factors are relevant:
The Key
Server count
Order of Server
That's the way how the server gets chosen, for standard hash strategy.
Am not sure how the hashing algorithm works but client library may be implemented to run a CRC on the key and do a modulus over the number of instances in the list to select an instance from the list for the set/get
Better still you can increase the changes of each server been selected at 50-50 by using weightoption
$memcache = new Memcache;
$memcache->addServer('memcache_host', 11211,true,50);
$memcache->addServer('memcache_host2', 11211,true,50);
Look at Cache::Memcached's _hashfunc(), it uses the key as its only parameter.
sub _hashfunc {
return (crc32($_[0]) >> 16) & 0x7fff;
}
I am including the same "random.inc" in foo.php and bar.php. For each, I want reproducible "random" results.
So in foo.php I always want one set of numbers and/or keywords. In bar.php another. Which shouldn't change on reload. That's what I mean by contant pseudo-random. And that's why I seeding on the url. However I still get different results for individual numbers as well as for array pickson every reload. This is the full php file:
<?
header('Content-Type: text/plain');
$seed = crc32( $_SERVER['REQUEST_URI'] );
echo "phpversion: ".phpversion()."\nseed: $seed\n";
srand( $seed ); // (seed verified to be contant as expected)
// neither single values nor array pics turn out deterministic
echo ''.rand(0,100).' '.rand(0,100).' '.rand(0,100)."\n";
$values = array( '0'=>21,'1'=>89,'2'=>96,'3'=>47,'4'=>88 );
print_r( array_rand( $values, 3 ) );
?>
In the days of PHP4.1 it was (verified) possible to achieve constant pseudo-random like this. array_rand API documentation describes as a feature that since 4.2 initialization happens automatically. Perhaps this is overriding any explicit seeding? (if so, perhaps explicit seeding should raise an internal PHP flag, preventing automatic seeding?). Btw: mt_srand() and srand() are equally not working.
I would really like to get my deterministic / constant pseudo-random back...
Update: Solution below (Windows and/or version 5.2 's fault)
Works for me (PHP/5.3.6):
<?php
$data = range(1, 100);
srand(1);
print_r(array_rand($data, 3));
... always prints:
Array
(
[0] => 21
[1] => 89
[2] => 95
)
... in my machine. Apparently, the exact numbers differ depending on the exact environment but they're reproducible.
Guys, you are all correct! (Sorry, I answer it myself now)
my web hoster runs 5.2.17 under Linux 2.6.36, and above problem exists.
under Win x64 5.3.0 everything works as expected.
So it's everyone's guess if that's an OS related bug and/or a PHP bug, fixed in 5.3.0.
Given that random constant seeding worked before, I am guessing they fixed in 5.3 the bug that came with the autoseed feature enhancement in 4.2. Anyway, Thanx again, at least there's clarity now.
The seeding functions are still available, and should still work; it's just since PHP 4.2 they are automatically seeded with the time on page load; but you can still call them to reset the random sequence to a known starting point.
[edit] I have just done a quick test program to make sure I wasn't imagining it!
mt_srand(50000);
print "rand="+mt_rand(0,10000);
Using PHP 5.2, this always results in the same value being printed (1749).
[EDIT]
As noted by #cwd and in the accepted answer to this question, there appears to be a discrepancy in PHP 5.2's behaviour with random number seeding between the Linux and Windows versions. In PHP 5.2 on Linux, the above technique does not appear to work.
Fortunately, the bug seems to have been fixed in PHP 5.3, so the solution to this problem is simply to upgrade. (PHP 5.2 is not supported any longer, so you should upgrade anyway)
Btw, if anyone else wants "constant windows-pre-5.3 pseudo-random" (of low quality, e.g. for stuff like SEO buzzwording) this is a tested workaround:
$r = abs(crc32($URL))%20; // a number between 0 and 19, based on URL
In PHP 5.2.17 and probably on all versions of PHP 5.2, (not sure about windows), we lose the capability of generating random numbers based on a seed as PHP changes the algorithm used for random numbers.
rand and mt_rand are "broken" not only because they will not give one random number, but they will also not give a same sequence of random numbers - even when using a seed!
At first the PHP developers tried to argue that this is the way that it "should" work, but we can guess they caught enough grief about the problem that they have reverted the way that it works with PHP 5.3.
See the php mt_rand page and the bug tracker to learn about this issue.
I have been using mt_rand(10,100) to get a random number between 10 and 100 but it gives me 74 every time.
Heres what im working with
$success=mt_rand(10,100);
Any ideas ? Do i need to seed somehow ? Thanks in advance.
---EDIT---
I have just tried this
srand(microtime());
$success=rand(10,100);
Still gives same answer every time, 47.
---EDIT---
this is the whole thing
srand(microtime());
$success=rand(10,100);
echo $success;
if ($success == 100) {
$displayline="You succeeded";
session_register("displayline");
header("location:userhome.php");
}
What version of PHP are you running? In 4.2.0, mt_rand() was changed to be seeded automatically.
You can seed it with mt_srand($seed), providing a seed value. I use the output of the microtime() function.
edit: Since you are on 5.2, check your code to see if mt_srand() is being called and comment out all such calls.
That's odd. If you do just mt_rand() what do you get? What's the value for mt_getrandmax()?
Try seeding it with mt_srand()
It seeds on itself.
http://www.suspekt.org/2008/08/17/mt_srand-and-not-so-random-numbers/
Guess you should check your PHP version
Remove all the seeding functions and don't add them back in. Then restart your server.
Also, I found this PHP bug, which occurs on Windows systems with some versions of the php.ini file.
Okay, so I'm getting my MySQL Version like so:
preg_replace('#[^0-9\.]#', '', mysql_get_server_info());
Which gives me a number like: 5.1.36
That's all good. What I need to do, is compare that version with another version. I was about to attempt to write my function to compare them, when I thought of version_compare(). However, upon testing I became unsure, but perhaps I'm just not sure how MySQL Version Numbers work.
This is what I tested:
version_compare('5.1.36', '5.1.4', '<');
or
5.1.36 < 5.1.4
I assumed that this would return true, that 5.1.36 is less than 5.1.4. My reason for that is, I figure 5.1.4 is actually 5.1.40 not 5.1.04. Perhaps I'm wrong there.
So am I thinking wrong, or is the function returning the incorrect result?
The function is correct. The numbering system is M.m.r where each "number" is a decimal number.
M is the major version number
m is the minor version number
r is the revision number
So 5.1.36 would be revision 36 of the 5.1 minor version... Therefore, 5.1.4 would be revision 4 (and hence 36 > 4)...
http://php.net/version_compare :)
http://www.php.net/manual/en/mysqli.get-server-version.php
mysqli's get_server_version() method will return the version as an integer.
main_version * 10000 + minor_version
* 100 + sub_version (i.e. version 4.1.0 is 40100).
I assumed that this would return true,
that 5.1.36 is less than 5.1.4. My
reason for that is, I figure 5.1.4 is
actually 5.1.40 not 5.1.04. Perhaps
I'm wrong there.
Yes, 5.1.36 is greater than 5.1.4.