I have some strings that have been encrypted using the PHP function crypt().
The outputs look something like this:
$1$Vf/.4.1.$CgCo33ebiHVuFhpwS.kMI0
$1$84..vD4.$Ps1PdaLWRoaiWDKCfjLyV1
$1$or1.RY4.$v3xo04v1yfB7JxDj1sC/J/
While I believe crypt() is using the MD5 algorithm, the outputs are not valid MD5 hashes.
Is there a way of converting the produced hashes into valid MD5 hashes (16-byte hex values)?
Update:
Thanks for the replies so answers so far. I'm pretty sure the crypt function used is using some sort of MD5 algorithm. What I'm looking to do is convert the ouput that I have into an MD5 hash that looks something like the following:
9e107d9d372bb6826bd81d3542a419d6
e4d909c290d0fb1ca068ffaddf22cbd0
d41d8cd98f00b204e9800998ecf8427e
(taken from Wikipedia)
Is there a way of converting from the hashes I have to ones like the above?
OK, so maybe this answer is a year late, but I'll give it a shot. In your own answer, you note that crypt() is using the FreeBSD MD5, which also does some interesting transformations on the salt before running the hash, so the result of what I'm about to give you will never quite match up with the results of a call to md5(). That said, the only difference between the output you are seeing and the format you are used to is that the output you are seeing is encoded as follows
$1$ # this indicates that it is MD5
Vf/.4.1. # these eight characters are the significant portion of the salt
$ # this character is technically part of the salt, but it is ignored
CgCo33eb # the last 22 characters are the actual hash
iHVuFhpw # they are base64 encoded (to be printable) using crypt's alphabet
S.kMI0 # floor(22 * 6 / 8) = 16 (the length in bytes of a raw MD5 hash)
To my knowledge, the alphabet used by crypt looks like this:
./0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz
So, with all of this borne in mind, here is how you can convert the 22 character crypt-base64 hash into a 32 character base16 (hexadecimal) hash:
First, you need something to convert the base64 (with custom alphabet) into a raw 16-byte MD5 hash.
define('CRYPT_ALPHA','./0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz');
/**
* Decodes a base64 string based on the alphabet set in constant CRYPT_ALPHA
* Uses string functions rather than binary transformations, because said
* transformations aren't really much faster in PHP
* #params string $str The string to decode
* #return string The raw output, which may include unprintable characters
*/
function base64_decode_ex($str) {
// set up the array to feed numerical data using characters as keys
$alpha = array_flip(str_split(CRYPT_ALPHA));
// split the input into single-character (6 bit) chunks
$bitArray = str_split($str);
$decodedStr = '';
foreach ($bitArray as &$bits) {
if ($bits == '$') { // $ indicates the end of the string, to stop processing here
break;
}
if (!isset($alpha[$bits])) { // if we encounter a character not in the alphabet
return false; // then break execution, the string is invalid
}
// decbin will only return significant digits, so use sprintf to pad to 6 bits
$decodedStr .= sprintf('%06s', decbin($alpha[$bits]));
}
// there can be up to 6 unused bits at the end of a string, so discard them
$decodedStr = substr($decodedStr, 0, strlen($decodedStr) - (strlen($decodedStr) % 8));
$byteArray = str_split($decodedStr, 8);
foreach ($byteArray as &$byte) {
$byte = chr(bindec($byte));
}
return join($byteArray);
}
Now that you've got the raw data, you'll need a method to convert it to the base-16 format you're expecting, which couldn't be easier.
/**
* Takes an input in base 256 and encodes it to base 16 using the Hex alphabet
* This function will not be commented. For more info:
* #see http://php.net/str-split
* #see http://php.net/sprintf
*
* #param string $str The value to convert
* #return string The base 16 rendering
*/
function base16_encode($str) {
$byteArray = str_split($str);
foreach ($byteArray as &$byte) {
$byte = sprintf('%02x', ord($byte));
}
return join($byteArray);
}
Finally, since the output of crypt includes a lot of data we don't need (and, in fact, cannot use) for this process, a short and sweet function to not only tie these two together but to allow for direct input of output from crypt.
/**
* Takes a 22 byte crypt-base-64 hash and converts it to base 16
* If the input is longer than 22 chars (e.g., the entire output of crypt()),
* then this function will strip all but the last 22. Fails if under 22 chars
*
* #param string $hash The hash to convert
* #param string The equivalent base16 hash (therefore a number)
*/
function md5_b64tob16($hash) {
if (strlen($hash) < 22) {
return false;
}
if (strlen($hash) > 22) {
$hash = substr($hash,-22);
}
return base16_encode(base64_decode_ex($hash));
}
Given these functions, the base16 representation of your three examples are:
3ac3b4145aa7b9387a46dd7c780c1850
6f80dba665e27749ae88f58eaef5fe84
ec5f74086ec3fab34957d3ef0f838154
Of course, it is important to remember that they were always valid, just formatted differently.
$1$ indeed means that this is a MD5 hash, but crypt generates a random salt. This is why you find a different MD5 value. If you include the generated salt you will find the same result.
The salt is base64 encoded in the output, as the hash.
The algorithm used is a system wide parameter. Generally this is MD5, you are right.
I believe the answer to my original question is no, you can't convert from one format to the other.
The hashes generated by php crypt() appear to be generate by a version of the FreeBSD MD5 hash implementation created by Poul-Henning Kamp.
http://people.freebsd.org/~phk/
From the documentation, this depends on the system. You can force the algorithm used by setting the salt parameter. From the docs:
The encryption type is triggered by
the salt argument. At install time,
PHP determines the capabilities of the
crypt function and will accept salts
for other encryption types. If no salt
is provided, PHP will auto-generate a
standard two character salt by
default, unless the default encryption
type on the system is MD5, in which
case a random MD5-compatible salt is
generated.
From http://php.net/crypt:
crypt() will return an encrypted string using the standard Unix DES-based encryption algorithm or alternative algorithms that may be available on the system.
You want the md5() function:
Calculates the MD5 hash of str using the » RSA Data Security, Inc. MD5 Message-Digest Algorithm, and returns that hash.
If the optional raw_output is set to TRUE, then the md5 digest is instead returned in raw binary format with a length of 16.
Defaults to FALSE.
Related
I need to generate a strong unique API key.
Can anyone suggest the best solution for this? I don't want to use rand() function to generate random characters. Is there an alternative solution?
As of PHP 7.0, you can use the random_bytes($length) method to generate a cryptographically-secure random string. This string is going to be in binary, so you'll want to encode it somehow. A straightforward way of doing this is with bin2hex($binaryString). This will give you a string $length * 2 bytes long, with $length * 8 bits of entropy to it.
You'll want $length to be high enough such that your key is effectively unguessable and that the chance of there being another key being generated with the same value is practically nil.
Putting this all together, you get this:
$key = bin2hex(random_bytes(32)); // 64 characters long
When you verify the API key, use only the first 32 characters to select the record from the database and then use hash_equals() to compare the API key as given by the user against what value you have stored. This helps protect against timing attacks. ParagonIE has an excellent write-up on this.
For an example of the checking logic:
$token = $request->bearerToken();
// Retrieve however works best for your situation,
// but it's critical that only the first 32 characters are used here.
$users = app('db')->table('users')->where('api_key', 'LIKE', substr($token, 0, 32) . '%')->get();
// $users should only have one record in it,
// but there is an extremely low chance that
// another record will share a prefix with it.
foreach ($users as $user) {
// Performs a constant-time comparison of strings,
// so you don't leak information about the token.
if (hash_equals($user->api_token, $token)) {
return $user;
}
}
return null;
Bonus: Slightly More Advanced Use With Base64 Encoding
Using Base64 encoding is preferable to hexadecimal for space reasons, but is slightly more complicated because each character encodes 6 bits (instead of 4 for hexadecimal), which can leave the encoded value with padding at the end.
To keep this answer from dragging on, I'll just put some suggestions for handling Base64 without their supporting arguments. Pick a $length greater than 32 that is divisible by both 3 and 2. I like 42, so we'll use that for $length. Base64 encodings are of length 4 * ceil($length / 3), so our $key will be 56 characters long. You can use the first 28 characters for selection from your storage, leaving another 28 characters on the end that are protected from leaking by timing attacks with hash_equals.
Bonus 2: Secure Key Storage
Ideally, you should be treating the key much like a password. This means that instead of using hash_equals to compare the full string, you should hash the remainder of the key like a password, store that separately than the first half of your key (which is in plain-text), use the first half for selection from your database and verify the latter half with password_verify.
using mcrypt:
<?php
$bytes = mcrypt_create_iv(4, MCRYPT_DEV_URANDOM);
$unpack = unpack("Nint", $bytes);
$id = $unpack['int'] & 0x7FFFFFFF;
PHP has uniqid function http://php.net/manual/en/function.uniqid.php with optional prefix and you can even add additional entropy to further avoid collision. But if you absolutely possitevily need something unique you should not use anything with randomness in it.
This is the best solution i found.
http://www.php.net/manual/en/function.uniqid.php#94959
There is an application written in PHP which I am converting to Ruby. When encrypting passwords the PHP app uses the following code:
if($method == 2 && CRYPT_BLOWFISH) return crypt($pass, '$2a$07$xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/xxxxxxxxxxxxxxxxxxx$');
I'm assuming this is using a Blowfish implementation. The x's here are all a-zA-Z0-9 characters.
The Blowfish implementation in Ruby uses the following syntax (taken from http://crypt.rubyforge.org/blowfish.html):
blowfish = Crypt::Blowfish.new("A key up to 56 bytes long")
plainBlock = "ABCD1234"
encryptedBlock = blowfish.encrypt_block(plainBlock)
I don't have a 56 or fewer byte long string and it's not clear what that should be from the PHP version. So how can I write a Ruby function which will encrypt passwords to give the same result as the PHP one?
The PHP code is hashing $pass with the salt $2a$07$xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/xxxxxxxxxxxxxxxxxxx$ if CRYPT_BLOWFISH is set (CRYPT_BLOWFISH == 1). The salt has to follow the format indicated in the PHP documentation ("$2a$", a two digit cost parameter, "$", and 22 digits from the alphabet "./0-9A-Za-z").
I'm not sure if you can do it with the library you are referring, but you can use bcrypt-ruby instead.
For your code it would be something like this, I'm using the same data from the PHP example ( http://php.net/manual/en/function.crypt.php ), I only take the 29 first characters of the salt because beyond that PHP ignores it:
require 'bcrypt-ruby'
pass = "rasmuslerdorf" # Here you should put the $pass from your PHP code
salt = '$2a$07$usesomesillystringfors' # Notice no $ at the end. Here goes your salt
hashed_password = BCrypt::Engine.hash_secret(pass,salt) # => "$2a$07$usesomesillystringfore2uDLvp1Ii2e./U9C8sBjqp8I90dH6hi"
This gives you the same output as on the PHP example. If your salt is too long take the first 29 characters ($2a$07$ plus the next 22 extra characters).
I tested the behavior of PHP, if the salt is too long (beyond 29 characters in total) the rest is ignored, if the salt is too short it will return 0. E.g in PHP:
<?php
crypt('rasmuslerdorf', '$2a$07$usesomesillystringforsalt$')
// returns $2a$07$usesomesillystringfore2uDLvp1Ii2e./U9C8sBjqp8I90dH6hi
crypt('rasmuslerdorf', '$2a$07$usesomesillystringfors')
// returns $2a$07$usesomesillystringfore2uDLvp1Ii2e./U9C8sBjqp8I90dH6hi
crypt('rasmuslerdorf', '$2a$07$usesomesilly')
// returns 0 because the salt is not long enough
?>
PHP's "shm_get" function requires an integer semaphore key, which I realise to be a restriction of the underlying OS.
I am using the "sha1" function to hash some user input and using the hash to uniquely identify a number of resulting files and and background processes.
Is there a way to convince shm_get to accept an alphanumeric key or to convert a sha1 hash to an acceptable integer?
You can convert a hexadecimal number into a decimal number by using hexdec()
However if you have got a large number in your hash, this won't return an integer. But you need an integer. So you might want to cut it apart and only use a part of the hash.
$hash = sha1('http://www.hashcat.net/');
$hash = substr($hash, 0, 15); // ok on 64bit systems
$number = (int) hexdec($hash); // cap to PHP_INT_MAX anyway
var_dump($hash, $number);
I need to generate a string using PHP, it need to be unique and need to be from 4 to 8 characters (the value of a variable).
I thought I can use crc32 hash but I can't decide how many characters, but sure it will be unique. In the other hand only create a "password generator" will generate duplicated string and checking the value in the table for each string will take a while.
How can I do that?
Thanks!
Maybe I can use that :
function unique_id(){
$better_token = md5(uniqid(rand(), true));
$unique_code = substr($better_token, 16);
$uniqueid = $unique_code;
return $uniqueid;
}
$id = unique_id();
Changing to :
function unique_id($l = 8){
$better_token = md5(uniqid(rand(), true));
$rem = strlen($better_token)-$l;
$unique_code = substr($better_token, 0, -$rem);
$uniqueid = $unique_code;
return $uniqueid;
}
echo unique_id(4);
Do you think I'll get unique string each time for a goood while?
In short, I think you'll get a pretty good random value. There's always the chance of a collision but you've done everything you can to get a random value. uniqid() returns a random value based on the current time in microseconds. Specifying rand() (mt_rand() would be better) and the second argument as true to uniqid() should make the value even more unique. Hashing the value using md5() should also make it pretty unique as even a small difference in two random values generated should be magnified by the hashing function. idealmachine is correct in that a longer value is less likely to have a collision than a shorter one.
Your function could also be shorter since md5() will always return a 32 character long string. Try this:
function unique_id($l = 8) {
return substr(md5(uniqid(mt_rand(), true)), 0, $l);
}
The problem with randomness is that you can never be sure of anything. There is a small chance you could get one number this time and the same number the next. That said, you would want to make the string as long as possible to reduce that probability. As an example of how long such numbers can be, GUIDs (globally unique identifiers) are 16 bytes long.
In theory, four hex characters (16 bits) give only 16^4 = 65536 possibilities, while eight hex characters (32 bits) give 16^8 = 4294967296. You, however, need to consider how likely it is for any two hashes to collide (the "birthday problem"). Wikipedia has a good table on how likely such a collision is. In short, four hex characters are definitely not sufficient, and eight might not be.
You may want to consider using Base64 encoding rather than hex digits; that way, you can fit 48 bits in rather than just 32 bits.
Eight bytes is 8 * 8 = 64 bits.
Reliable passwords You can only make from ascii characters a-zA-Z and numbers 0-9. To do that best way is using only cryptographically secure methods, like random_int() or random_bytes() from PHP7. Rest functions as base64_encode() You can use only as support functions to make reliability of string and change it to ASCII characters.
mt_rand() is not secure and is very old.
From any string You must use random_int(). From binary string You should use base64_encode() to make binary string reliable or bin2hex, but then You will cut byte only to 16 positions (values).
See my implementation of this functions.
I have a script to convert to base 62 (A-Za-z0-9) but how do I get a number out of MD5?
I have read in many places that because the number from an MD5 is bigger than php can handle as an integer it will be inaccurate... As I want a short URL anyway and was not planning on using the whole hash, maybe just 8 characters of it....
So my question is how to get part of the number of an MD5 hash?
Also is it a bad idea to use only part of the MD5 hash?
I'm going to suggest a different thing here.. Since you are only interested in using a decimal chunk of the md5 hash why don't you use any other short numeric hash like CRC32 or Adler? Here is an example:
$hash = sprintf('%u', crc32('your string here'));
This will produce a 8 digit hash of your string.
EDIT: I think I misunderstood you, here are some functions that provide conversions to and from bases up to 62.
EDIT (Again): To work with arbitrary length numbers you must use either the bc_math or the GMP extension, here is a function that uses the bc_math extension and can also convert from base 2 up to base 62. You should use it like this:
echo bc_base_convert(md5('your url here'), 16, 62); // public base 62 hash
and the inverse:
echo bc_base_convert('base 62 encoded value here', 62, 16); // private md5 hash
Hope it helps. =)
If it's possible, I'd advise not using a hash for your URLs. Eventually you'll run into collisions... especially if you're truncating the hash. If you go ahead and implement an id-based system where each item has a unique ID, there will be far fewer headaches. The first item will be 1, the second'll be 2, etc---if you're using MySQL, just throw in an autoincrement column.
To make a short id:
//the basic example
$sid = base_convert($id, 10, 36);
//if you're going to be needing 64 bit numbers converted
//on a 32 bit machine, use this instead
$sid = gmp_strval(gmp_init($id, 10), 36);
To make a short id back into the base-10 id:
//the basic example
$id = base_convert($id, 36, 10);
//if you're going to be needing 64 bit numbers
//on a 32 bit machine, use this instead
$id = gmp_strval(gmp_init($shortid, 36));
Hope this helps!
If you're truly wanting base 62 (which can't be done with gmp or base_convert), check this out:
http://snipplr.com/view/22246/base62-encode--decode/
You can do this like this: (Not all steps are in php, it's been a long time that I've used it.)
Create a md5 hash of the script like this:
$hash = md5(script, raw_output=true);
Convert that number to base 62.
See the questions about base conversion of arbitrary sized numbers in PHP
Truncate the string to a length you like.
There's no risk in using only a few of the bits of a md5. All that changes is danger of collisions.
There actually is a Java implementation which you could probably extract. It's an open-source CMS solution called Pulse.
Look here for the code of toBase62() and fromBase62().
http://pulse.torweg.org/javadoc/src-html/org/torweg/pulse/util/StringUtils.java.html
The only dependency in StringUtils is the LifeCycle-class which provides a way to get a salted hash for a string which you might even omit all together or just copy the method over to your copy StringUtils. Voilá.
You can do something like this,
$hash = md5("The data to be hashed", true);
$ints = unpack("L*num", $hash);
$hash_str = base62($ints['num1']) . base62($ints['num2']) . base62($ints['num3']) . base62($ints['num4'])
As of PHP 5.3.2, GMP supports bases up to 62 (was previously only 36), so brianreavis's suggestion was very close. I think the simplest answer to your question is thus:
function base62hash($source, $chars = 22) {
return substr(gmp_strval(gmp_init(md5($source), 16), 62), 0, $chars);
}
Converting from base-16 to base-62 obviously has space benefits. A normal 128-bit MD5 hash is 32 chars in hex, but in base-62 it's only 22. If you're storing the hashes in a database, you can convert them to raw binary and save even more space (16 bytes for an MD5).
Since the resulting hash is just a string representation, you can just use substr if you only want a bit of it (as the function does).
You may try base62x to get a safe and compatible encoded representation.
Here is for more information about base62x, or simply -base62x in -NatureDNS.
shell> ./base62x -n 16 -enc 16AF
1Ql
shell> ./base62x -n 16 -dec 1Ql
16AF
shell> ./base62x
Usage: ./base62x [-v] [-n <2|8|10|16|32>] <-enc|dec> string
Version: 0.60
Here is an open-source Java library that converts MD5 strings to Base62 strings
https://github.com/inder123/base62
Md5ToBase62.toBase62("9e107d9d372bb6826bd81d3542a419d6") ==> cbIKGiMVkLFTeenAa5kgO4
Md5ToBase62.fromBase62("4KfZYA1udiGCjCEFC0l") ==> 0000bdd3bb56865852a632deadbc62fc
The conversion is two-way, so you will get the original md5 back if you convert it back to md5:
Md5ToBase62.fromBase62(Md5ToBase62.toBase62("9e107d9d372bb6826bd81d3542a419d6")) ==> 9e107d9d372bb6826bd81d3542a419d6
Md5ToBase62.toBase62(Md5ToBase62.fromBase62("cbIKGiMVkLFTeenAa5kgO4")) . ==> cbIKGiMVkLFTeenAa5kgO4
```
You could use a slightly modified Base 64 with - and _ instead of + and /:
function base64_url_encode($str) {
return strtr(base64_encode($str), array('+'=>'-', '/'=>'_'));
}
function base64_url_decode($str) {
return base64_decode(strtr($str, array('-'=>'+', '_'=>'/')));
}
Additionally you could remove the trailing padding = characters.
And to get the raw MD5 value (binary string), set the second parameter (named $raw_output in the manual) to true:
$raw_md5 = md5($str, true);