I seem to have problems with memcached keys that have spaces, though I can't pinpoint exactly what.
A more explicit answer (referred to by Dustin, but not referenced):
Keys
Data stored by memcached is identified with the help of a key. A key
is a text string which should uniquely identify the data for clients
that are interested in storing and retrieving it. Currently the
length limit of a key is set at 250 characters (of course, normally
clients wouldn't need to use such long keys); the key must not include
control characters or whitespace.
Source: protocol.txt (Specific Version)
No. Memcached keys cannot contain spaces.
Memcached clients seem not to validate keys in favor of performance.
What I usually do is create a method named createWellFormedKey($key) and pass the returned result to the set() and get() methods of the memcached client.
I do not use md5 and sha1 hashing unless the base64 version exceeds 250 characters. This is because md5 and sha1 are more expensive operations performance wise.
A sample PHP code looks like this:
/**
* Generates a well formed key using the following algorithm:
* 1. base64_encode the key first to make sure all characters are valid
* 2. Check length of result, less than 250 then return it
* 3. Length of result more than 250 then create a key that is md5($validKey).sha1($validKey).strlen($validKey)
*/
private function createWellFormedKey($key) {
// Get rid of all spaces, control characters, etc using base64
$validKey = base64_encode($key);
$validKeyLength = strlen($validKey);
// 250 is the maximum memcached can handle
if (strlen($validKey) < 250) {
return $validKey;
}
$validKey = md5($validKey).sha1($validKey).$validKeyLength;
return $validKey;
}
At the moment I'm playing around with memcached with PHP and the Problem described IMHO can be easily solved by using hash algorithms like md5 and sha1 (or any other).
I'm using a combination of a md5-hash, sha1-hash and sha256 + the length of the key given.
Obviously this method can be reduced to two hash-methods + length of the key, so you can easily avoid using space or other characters that should not be in the key.
In my opinion the hash-collions are avoided because the chance that both hash algorithms have a collision is nearly 0. By additionally using the key length in the key the problem of a collision is 0.
Applications using the memcached binary protocol can use whitespace-containing keys, though there is still a 250-byte length limit.
Related
Ok so here's the sample code from the page http://phpseclib.sourceforge.net/crypt/examples.html
<?php
include('Crypt/AES.php');
include('Crypt/Random.php');
$cipher = new Crypt_AES(); // could use CRYPT_AES_MODE_CBC
// keys are null-padded to the closest valid size
// longer than the longest key and it's truncated
//$cipher->setKeyLength(128);
$cipher->setKey('abcdefghijklmnop');
// the IV defaults to all-NULLs if not explicitly defined
$cipher->setIV(crypt_random_string($cipher->getBlockLength() >> 3));
$size = 10 * 1024;
$plaintext = str_repeat('a', $size);
echo $cipher->decrypt($cipher->encrypt($plaintext));
?>
Before anything, why is the line //$cipher->setKeyLength(128); is being commented out?
And if I want to set my key to '1234568790', is there anything I should do? Because it's much shorter than they length of the key in the example above (abcdefghijklmnop).
And finally if the plaintext that I want to encrypt is short, something like "my name is james bond", is there anything extra that I should do? Because from the code above, it seems the plaintext's length should be 10 x 1024. (Why is that?)
Before anything, why is the line //$cipher->setKeyLength(128); is being commented out?
Say you pass a 17 byte key to phpseclib via setKey. What do you suppose ought to happen? In the 1.0 and 2.0 versions if setKeyLength isn't called then the key is null-padded to 24 bytes (eg. 192-bits). But if setKeyLength is called it'll be truncated to 16 bytes. In the master branch (which, as I understand it, will eventually become the 3.0 branch), an exception is thrown if the key length isn't valid. In that version it may still be desirable to call setKeyLength if, for example, if you want to set the key length as being 128-bits in one part of the code and then actually set the key in another part of the code. ie. you could do the length checking or phpseclib could do the length checking.
And if I want to set my key to '1234568790', is there anything I should do? Because it's much shorter than they length of the key in the example above (abcdefghijklmnop).
1234568790 isn't technically long enough to be a valid key. It's 80 bytes long. The smallest key AES supports is 128-bits. Maybe you should consider doing something like setPassword('1234568790') instead of setKey('1234568790'). setPassword will use a key-derivation function to generate a key from the password.
In phpseclib 1.0 and 2.0 setKey will, none-the-less, accept that as a key. It'll just null pad the key. The master branch otoh will throw an exception since the key isn't actually long enough.
And finally if the plaintext that I want to encrypt is short, something like "my name is james bond", is there anything extra that I should do? Because from the code above, it seems the plaintext's length should be 10 x 1024. (Why is that?)
10 * 1024 is just an example. phpseclib, by default, will use PKCS7 padding to make strings long enough. eg. your string is 21 bytes long (if I counted correctly). So if you call $cipher->encrypt('my name is james bond') then chr(11) will be appending to the plaintext 11 times, by phpseclib and then the string will be encrypted. This is because block algorithms need to have the the plaintext be a multiple of the block length. So the ciphertext will be 32 bytes long. When you then call $cipher->decrypt('...') you'll get the orig 21-byte string back. I could elaborate further on PKCS7 padding, but I think I've answered your immediate question.
In a web application I'm working on, i need to generate unique id's with an excessive length. Longer than typical UUID's. Another similar web app uses keys that look like this:
cb745abbc635c03f0c259b65y5da57c06e12ef51
What are these called? and how can i create unique ones in PHP? I've tried the UID method, however they are kinda short.
The example you posted is a 40 character hex string, which therefore looks suspiciously like a SHA1 hash. PHP's built-in sha1() function will hash an input string into just such a hash.
If you pass microtime(true) (to get the current time with microseconds as a float) as input, you'll get a value unique in time. Concatenate it with a hostname for a 40 character globally unique value.
echo sha1(microtime(true) . $hostname));
Note that while this type of value is probably satisfactory as a unique identifier for a database object, user ID, etc, it should not be considered cryptographically secure, as its sequence could be easily guessed.
This might be a hash generated from sha1 which is widely used:
From the PHP documentation:
If the optional raw_output is set to TRUE, then the sha1 digest is instead returned in raw binary format with a length of 20, otherwise the returned value is a 40-character hexadecimal number.
echo (sha1("whatever"));
Note that is not certain since it exists multiple other hashing algorithms that will give you a 40 characters length:
echo (hash("ripemd160", "whatever"));
echo (hash("tiger160,3", "whatever"));
echo (hash("tiger160,4", "whatever"));
echo (hash("haval160,3", "whatever"));
echo (hash("haval160,4", "whatever"));
echo (hash("haval160,5", "whatever"));
I'm trying to use Laravel's Crypt functionality, to simply store a value in a database and grab it later on to use. However I noticed that I was unable to decrypt this value.
My application key is a random, 32 character string. My cipher is MCRYPT_RIJNDAEL_128.
From the PHP info, MCRYPT is installed, and RIJNDAEL_128 is supported.
To test, I do the following on a GET rou:
$t = "123456789";
var_dump(Crypt::encrypt($t));
See: http://laravel.io/bin/2e9Xr#
On each page refresh, the output is a different value, which is obviously incorrect - however I have no idea why.
I'm using an EasyPHP as my dev server. However one thing I have noticed is that the application requests are significantly slow on this environment as compared to the production, Apache web server.
This makes me wonder if there is some sort of environment refresh going on each time, potentially resetting the "resources" MCRYPT is using to encrypt, and thus is different each time.
Any clues?
That is normal behavior. Every Crypt::encrypt call should produce a different output for security reasons.
Crypt is incredibly inefficient for small strings. For example, Crypt::encrypt("Hello World") outputs something like the following: eyJpdiI6Imhnb2hRazVabUNZUnVRVzFBSEExVkE9PSIsInZhbHVlIjoiTHJ4c05zcjdJZkZwWU1vRVVRMEcwZE5nTUdjQnhyM2RKWTMzSW04b1cxYz0iLCJtYWMiOiIyZjRmNDc3NGEyNGQyOGJjZGQ4MWQxYWViYzI1MjNjZTU0MmY4YTIxYTEyNWVjNDVlZDc4ZWEzNzRmN2QwM2ZiIn0=
Immediately recognizable as a base 64 string. When decoded, it becomes {"iv":"hgohQk5ZmCYRuQW1AHA1VA==","value":"LrxsNsr7IfFpYMoEUQ0G0dNgMGcBxr3dJY33Im8oW1c=","mac":"2f4f4774a24d28bcdd81d1aebc2523ce542f8a21a125ec45ed78ea374f7d03fb"}
Using Crypt, you can encrypt and decrypt large plaintexts easily without worrying about the details. But if you want to store or transmit a lot of separately encrypted entities, then you might want to consider a different approach.
So why is it like this?
(Note: the directory structures are valid for Laravel 4.2).
For one, most secure block cipher modes of operation require an IV (initialization vector), which is a bunch of random bytes with length matching the block size. Using a different IV for every ciphertext is important for thwarting cryptanalysis and replay attacks. But let's look a bit at the actual Crypt code.
Starting with the config/app.php aliases array, we see 'Crypt' => 'Illuminate\Support\Facades\Crypt'
So we check the vendor/laravel/framework/src/Support/Facades directory, and we find Crypt.php which says the module accessor name is actually "encrypter". Checking the config/app.php providers array shows 'Illuminate\Encryption\EncryptionServiceProvider'.
vendor/laravel/framework/src/Illuminate/Encryption has several files of interest: Encrypter.php and EncryptionServiceProvider.php. The service provider binds the accessor with a function that creates, configures, and returns an instance of Encrypter.
In the Encrypter class, we find the encrypt method:
public function encrypt($value)
{
$iv = mcrypt_create_iv($this->getIvSize(), $this->getRandomizer());
$value = base64_encode($this->padAndMcrypt($value, $iv));
// Once we have the encrypted value we will go ahead base64_encode the input
// vector and create the MAC for the encrypted value so we can verify its
// authenticity. Then, we'll JSON encode the data in a "payload" array.
$mac = $this->hash($iv = base64_encode($iv), $value);
return base64_encode(json_encode(compact('iv', 'value', 'mac')));
}
And there you have it. Each time you call Crypt::encrypt, it generates a new IV, encrypts the value, creates a MAC of the IV and ciphertext, and then returns a base 64 encoded JSON string of an associative array of the IV, MAC, and ciphertext. Each IV will be different, which means every ciphertext and MAC will also be different--even for the same value. Really smart if all plaintexts are large, but pretty impractical for a lot of smaller plaintexts where MACs are unnecessary overhead.
tl;dr version:
16 bytes of randomness is generated for every encrypt call, and it cascades into the ciphertext and MAC, all of which is returned in a base 64 encoded JSON associative array. Thus, every Crypt::encrypt call produces different output.
That's how mcrypt works - http://mnshankar.wordpress.com/2014/03/29/laravel-hash-make-explained/
$test = 'test';
$crypted = Crypt::encrypt($test);
echo $crypted.'<br />'; // encrypted string
echo Crypt::decrypt($crypted); // "test"
I need to generate a strong unique API key.
Can anyone suggest the best solution for this? I don't want to use rand() function to generate random characters. Is there an alternative solution?
As of PHP 7.0, you can use the random_bytes($length) method to generate a cryptographically-secure random string. This string is going to be in binary, so you'll want to encode it somehow. A straightforward way of doing this is with bin2hex($binaryString). This will give you a string $length * 2 bytes long, with $length * 8 bits of entropy to it.
You'll want $length to be high enough such that your key is effectively unguessable and that the chance of there being another key being generated with the same value is practically nil.
Putting this all together, you get this:
$key = bin2hex(random_bytes(32)); // 64 characters long
When you verify the API key, use only the first 32 characters to select the record from the database and then use hash_equals() to compare the API key as given by the user against what value you have stored. This helps protect against timing attacks. ParagonIE has an excellent write-up on this.
For an example of the checking logic:
$token = $request->bearerToken();
// Retrieve however works best for your situation,
// but it's critical that only the first 32 characters are used here.
$users = app('db')->table('users')->where('api_key', 'LIKE', substr($token, 0, 32) . '%')->get();
// $users should only have one record in it,
// but there is an extremely low chance that
// another record will share a prefix with it.
foreach ($users as $user) {
// Performs a constant-time comparison of strings,
// so you don't leak information about the token.
if (hash_equals($user->api_token, $token)) {
return $user;
}
}
return null;
Bonus: Slightly More Advanced Use With Base64 Encoding
Using Base64 encoding is preferable to hexadecimal for space reasons, but is slightly more complicated because each character encodes 6 bits (instead of 4 for hexadecimal), which can leave the encoded value with padding at the end.
To keep this answer from dragging on, I'll just put some suggestions for handling Base64 without their supporting arguments. Pick a $length greater than 32 that is divisible by both 3 and 2. I like 42, so we'll use that for $length. Base64 encodings are of length 4 * ceil($length / 3), so our $key will be 56 characters long. You can use the first 28 characters for selection from your storage, leaving another 28 characters on the end that are protected from leaking by timing attacks with hash_equals.
Bonus 2: Secure Key Storage
Ideally, you should be treating the key much like a password. This means that instead of using hash_equals to compare the full string, you should hash the remainder of the key like a password, store that separately than the first half of your key (which is in plain-text), use the first half for selection from your database and verify the latter half with password_verify.
using mcrypt:
<?php
$bytes = mcrypt_create_iv(4, MCRYPT_DEV_URANDOM);
$unpack = unpack("Nint", $bytes);
$id = $unpack['int'] & 0x7FFFFFFF;
PHP has uniqid function http://php.net/manual/en/function.uniqid.php with optional prefix and you can even add additional entropy to further avoid collision. But if you absolutely possitevily need something unique you should not use anything with randomness in it.
This is the best solution i found.
http://www.php.net/manual/en/function.uniqid.php#94959
I'm using this code:
$url = "http://www.webtoolkit.info/javascript-base64.html";
print base64_encode($url);
But the result is very long: "aHR0cDovL3d3dy53ZWJ0b29sa2l0LmluZm8vamF2YXNjcmlwdC1iYXNlNjQuaHRtbA=="
There is a way to transform long string to short encryption and to be able to transform?
for example:
new_encrypt("http://www.webtoolkit.info/javascript-base64.html")
Result: "431ASDFafk2"
encoding is not encrypting. If you're depending on this for security then you're in for a very nasty shock in the future.
Base 64 encoding is intended for converting data that's 8 bits wide into a format that can be sent over a communications channel that uses 6 or 7 bits without loss of data. As 6 bits is less than 8 bits the encoded string is obviously going to be longer than the original.
This q/a might have what you're looking for:
An efficient compression algorithm for short text strings
It actually links here:
http://github.com/antirez/smaz/tree/master
I did not test it, just found the links.
First off, base64 is an encoding standard and it is not meant to encrypt data, so don't use that. The reason your data is so much longer is that for every 6 bits in the input string, base64 will output 8 bits.
There is no form of encryption that will directly output a shortened string. The result will be just as long in the best case.
A solution to that problem would be to gzip your string and then encrypt it, but with your URL the added data for the zip format will still end up making your output longer than the input.
There are a many different algorithms for encrypting/decryption. You can take a look at the following documentation: http://www.php.net/manual/en/function.mcrypt-list-algorithms.php (this uses mcrypt with different algorithms).
...BUT, you can't force something to be really small (depends on the size you want). The encrypted string needs to have all the information available to be able to decrypt it. Anyways, a base64-string is not that long (compared with really secure salted hashes for example).
I don't see the problem.
Well... you could try using md5() or uniqid().
The first one generate the md5 hash of your string.
md5("http://www.webtoolkit.info/javascript-base64.html");
http://php.net/manual/en/function.md5.php
The second one generates a 13 unique id and then you can create a relation between your string and that id.
http://php.net/manual/en/function.uniqid.php
P.S. I'm not sure of what you want to achieve but these solutions will probably satisfy you.
You can be creative and just do some 'stuff' to encrypt the url so that it is not easy quess able but encode / decode able..
like reverse strings...
or have a random 3 letters, your string encoded with base64 or just replace letters for numbers or numbers for letters and then 3 more random letters.. once you know the recipe, you can do and undo it.
$keychars = "abcdefghijklmnopqrstuvwxyz0123456789";
$length = 2;
$randkey = "";
$randkey2 = "";
for ($i=0;$i<$length;$i++) $randkey .= substr($keychars, rand(1, strlen($keychars) ), 1);