I am trying to find a way to encode a database ID into a short URL, e.g. 1 should become "Ys47R". Then I would like to decode it back from "Ys47R" to 1 so I can run a database search using the INT value. It needs to be unique using the database ID. The sequence should not be easily guessable such as 1 = "Ys47R", 2 = "Ys47S". It should be something along the lines of YouTube or bitly's URL's. I have read up on hundreds of different sources using md5, base32, base64 and `bcpow but have come up empty.
This blog post looked promising but once I added padding and a passkey, short ID's such as 1 became SDDDG, 2 became "SDDDH" and 3 became "SDDDI". It is not very random.
base32 used only a-b 0-9
base64 had characters such as == on the end.
I then tried this:
function getRandomString($db, $length = 7) {
$validCharacters = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ";
$validCharNumber = strlen($validCharacters);
$result = "";
for ($i = 0; $i < $length; $i++) {
$index = mt_rand(0, $validCharNumber - 1);
$result .= $validCharacters[$index];
}
Which worked but meant I had to run a database query every time to make sure there were no collisions and it did not exist in the database.
Is there a way I can create short ID's that are 4 characters minimum with a charset of [a-z][A-Z][0-9] that can be encoded and decoded back, using increment unique ID in a database where each number is unique. I can't get my head around advance techniques using base32 or base64.
Or am I looking into this too much and there is an easier way to do it? Would it be best to do the random string function above and query the database to check for uniqueness all the time?
You could use function from comments: http://php.net/manual/en/function.base-convert.php#106546
$initial = '11111111';
$dic = '0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ';
var_dump($converted = convBase($initial, '0123456789', $dic));
// string(4) "KCvt"
var_dump(convBase($converted, $dic, '0123456789'));
// string(8) "11111111"
function convBase($numberInput, $fromBaseInput, $toBaseInput)
{
if ($fromBaseInput==$toBaseInput) return $numberInput;
$fromBase = str_split($fromBaseInput,1);
$toBase = str_split($toBaseInput,1);
$number = str_split($numberInput,1);
$fromLen=strlen($fromBaseInput);
$toLen=strlen($toBaseInput);
$numberLen=strlen($numberInput);
$retval='';
if ($toBaseInput == '0123456789')
{
$retval=0;
for ($i = 1;$i <= $numberLen; $i++)
$retval = bcadd($retval, bcmul(array_search($number[$i-1], $fromBase),bcpow($fromLen,$numberLen-$i)));
return $retval;
}
if ($fromBaseInput != '0123456789')
$base10=convBase($numberInput, $fromBaseInput, '0123456789');
else
$base10 = $numberInput;
if ($base10<strlen($toBaseInput))
return $toBase[$base10];
while($base10 != '0')
{
$retval = $toBase[bcmod($base10,$toLen)].$retval;
$base10 = bcdiv($base10,$toLen,0);
}
return $retval;
}
If you want some symmetric obfuscation, then base_convert() is often sufficient.
base_convert($id, 10, 36);
Will return strings like 1i0g and convert them back.
Before and after that base conversion you can add:
To get a minimum string length, I'd suggest just adding 70000 to your $id. And on the receiving end just subtract that again.
A minor multiplication $id *= 3 would add some "holes" in the generated alphanumeric ID range, yet not exhaust the available string space.
For some appearance of arbitrariness, a bit of nibble moving:
$id = ($id & 0xF0F0F0F) << 4
| ($id & 0x0F0F0F0) >> 4;
Which works for generating your obfuscated ID strings, and getting back the original ones.
Just to be crystal clear: this is no encryption of any sort. It just shifts numeric jumps between consecutive numbers, and looks slightly more arbitrary.
You still may not like the answer, but generating random IDs in your database is the only approach that really hinders ID guessing.
I wonder if there is a way in PHP to generate a unique alphanumeric(case sensitive) tokens that can be unique forever without any collision. If we derive them from the time stamp string which is 10 characters like: 1394452319, that might be possible but I am not sure if we can make the token short up to 4 characters? If not possible then 5, 6, 7 and max is 8 characters. Because I want to generate short tokens to be readable by users.
Tokens should look like: 1aYc, ZoXq, 3iU9, etc.
I don't want to show the users any sequence.
One more thing, my application will be used by more than one user, so in case two users clicked at same time to generate the token, will the PHP application generate the same token (I assume we use the timestamp to generate the token)? How can we prevent from this problem?
Thank you for your help!
this is the another function that you can use also
<?php
function generateRandomString($length = 8) {
$characters = '0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ';
$randomString = '';
for ($i = 0; $i < $length; $i++) {
$randomString .= $characters[rand(0, strlen($characters) - 1)];
}
return $randomString;
}
echo generateRandomString();
?>
One approach is to have an incremental (i.e. auto_update) id that you keep hidden internally. From that, you generate a hash, representing the id to hide the sequence. The incremented id gets rid of collision problems (i.e. MySQL has an integrated solution for this).
The trick you need to use now is a random hash table consinsting of two columns, both having the values n to m but with the second column being randomized. i.e.
col1 | col2
1 | 2
2 | 4
3 | 5
4 | 1
5 | 3
if you have the randomly sorted number for your incremented number, it is easy to create a hash from that. Just think about your possible chars as numbers. You get it righgt?
Assuming you have a good algorithm for random numbers, you can make a pretty good hash table. However, there also is a way to find an algorithm, providing you with the numbers as they increase. So in this example it would give you col2 = fn(col1) so i.e. 4 = fn(2).
All you have to do is take the result and re-enginer it into a formular :D
Otherwise you have to fill the table initially.
To give you a glimpse insight into the math of it, think of a function that uses odd/even characteristics of the number and combines it with addition.
With n digits using a range of 62 possibilitys (case sensitive letters and numbers) per char you have 62^n possibilities.
For 4 digits that makes 14776336 possibilities (62^4).
Thou that might sound just wonderfull, you can imagine that having a table, prefilled with 14776336 id's is not the cleanest solution.
Still, i hope this at least leads into the right direction.
EDIT:
We started a discussion on math.stackexchange.com. IT has some additional information on how to create a function for our needs.
You can use something like following
<?php
// chars
$chars = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789!##$%^&*()-+';
// convert to array
$arr = str_split($chars, 1);
// shuffle the array
shuffle($arr);
// array to chars with 8 chars
echo substr(implode('', $arr), 0, 8);
?>
You can use this function :
// RETRUN 24 digit of UNIX ID :
public function getComplexIDTicket(){ // duplicate method on Rest.php
$arrAZ1 = range('A','Z');
$arrAZ2 = range('A','Z');
$arrAZ3 = range('A','Z');
$arrs1 = range('A','Z');
$arrs2 = range('A','Z');
$arrs3 = range('A','Z');
$a1 = $arrAZ1[rand(0,25)];
$a2 = $arrAZ2[rand(0,25)];
$a3 = $arrAZ3[rand(0,25)];
$s1 = $arrs1[rand(0,25)];
$s2 = $arrs2[rand(0,25)];
$s3 = $arrs3[rand(0,25)];
$s = $s1.$s2.$s3;
$t = microtime(true);
$micro = sprintf("%07d",($t - floor($t)) * 10000000);
$id = date('ymdHis').strtoupper(dechex(substr($micro,0,7)));
$id = str_pad($id, 24, $a3.$a2.$a1.$s, STR_PAD_RIGHT);
// 151106214010 3DDBF0 L D C SM4
return $id;
}
I want to generate a unique 4-6 char long AlphaNumeric string to save in db with each record(user). The db field has a unique index, so trying to save a pre-existing string generates an error. Right now I am generating a random string and using try-catch, so when adding a new record if it throws an exception, I generate another random string and attempt to save again, and the code keep trying until it adds a record successfully. This whole solution not only looks heavy but also ugly, so I want to change it. I am interested in an elegant solution, so any help/guidance is welcome.
With the given information :
id must be unique
id must not be numeric
id must not represent a sequential series
id will not be input by the user
The PHP function uniqid is exactly what you need. Though it returns a 13 character long hexadecimal value.
** Edit **
Yes, uniqid will return a seamingly sequential number, but we can get around this easily. Consider this code
class IDGenerator {
//const BIT_MASK = '01110011';
static public function generate() {
$id = uniqid();
$id = base_convert($id, 16, 2);
$id = str_pad($id, strlen($id) + (8 - (strlen($id) % 8)), '0', STR_PAD_LEFT);
$chunks = str_split($id, 8);
//$mask = (int) base_convert(IDGenerator::BIT_MASK, 2, 10);
$id = array();
foreach ($chunks as $key => $chunk) {
//$chunk = str_pad(base_convert(base_convert($chunk, 2, 10) ^ $mask, 10, 2), 8, '0', STR_PAD_LEFT);
if ($key & 1) { // odd
array_unshift($id, $chunk);
} else { // even
array_push($id, $chunk);
}
}
return base_convert(implode($id), 2, 36);
}
}
echo IDGenerator::generate();
Which will give results like
ivpa493xrx7
d173barerui
evpoiyjdryd
99ej19mnau2
Since there is nothing added or modified, except shuffling the bits around, there should not be any duplicated values and everything seems random. Voilà!
** Update (2014-02-24) **
I update this piece of code since the time it was originally posted. You may find the revised version here
What I am trying to do is create a 12 character id for articles on my website similar to how youtube handles their video id (http://www.youtube.com/watch?v=53iddd5IcSU). Right now I am generating an MD5 hash and then grabbing 12 characters of it like this:
$ArticleId = substr(MD5("Article".$currentID),10,12)
where $currentID is the numeric ID from the database (eg 144)
I am slightly paranoid that I will run into a duplicate $ArticleId, but realistically what are the chances that this will happen? And also, being that the column in my database is unique, how can I handle this rare scenario without having an ugly error thrown?
P.S. I made a small script to check for duplicates within the first 5000 $ArticleId's and there were none.
EDIT: I don't like the way the base64_encode hashes look so I did this:
function retryAID($currentID)
{
$AID = substr(MD5("Article".$currentID*2),10,12);
$setAID = "UPDATE `table` SET `artID` = '$AID' WHERE `id` = $currentID ";
mysql_query($setLID) or retryAID($currentID);
}
$AID = substr(MD5("Article".$currentID),10,12);
$setAID = "UPDATE `table` SET `artID` = '$AID' WHERE `id` = $currentID ";
mysql_query($setAID) or retryAID($currentID);
Since the AID column is unique the mysql_query will throw an error and the retryAID function will find a unique id...
What's wrong with using a sequential id? The database will handle this for you.
That aside, 12 characters is still 96 bits. 296 = 79228162514264337593543950336 possible hashes. Even though MD5 is known to have collision vulnerabilities, there's a world of difference between the possibility of a collision and the probability of actually seeing one.
Update:
Based on the return value of the PHP md5 function you're using, my numbers above aren't quite right.
Returns the hash as a 32-character hexadecimal number.
Since you're taking 12 characters from a 32-character hexadecimal number (and not 12 bytes of the 128-bit hash), the actual number of possible hashes you could end up with is 1612 = 281474976710656. Still quite a few.
<?php
function get_id()
{
$max = 1679615; // pow(36, 4) - 1;
$id = '';
for ($i = 0; $i < 3; ++$i)
{
$r = mt_rand(0, $max);
$id .= str_pad(base_convert($r, 10, 36), 4, "0", STR_PAD_LEFT);
}
return $id;
}
?>
Returns a 12 character number in base-36, which gives 4,738,381,338,321,616,896 possibilities. (The probability of collision depends on the distribution of the random number generator.)
To ensure no collisions, you'll need to loop:
<?php
do {
$id = get_id();
} while ( !update_id($id) );
?>
No not very unique.
Why not base64 encode it if you need it shorter?
How about UUID ?
http://php.net/manual/en/function.uniqid.php
I want to create a unique id but uniqid() is giving something like '492607b0ee414'. What i would like is something similar to what tinyurl gives: '64k8ra'. The shorter, the better. The only requirements are that it should not have an obvious order and that it should look prettier than a seemingly random sequence of numbers. Letters are preferred over numbers and ideally it would not be mixed case. As the number of entries will not be that many (up to 10000 or so) the risk of collision isn't a huge factor.
Any suggestions appreciated.
Make a small function that returns random letters for a given length:
<?php
function generate_random_letters($length) {
$random = '';
for ($i = 0; $i < $length; $i++) {
$random .= chr(rand(ord('a'), ord('z')));
}
return $random;
}
Then you'll want to call that until it's unique, in pseudo-code depending on where you'd store that information:
do {
$unique = generate_random_letters(6);
} while (is_in_table($unique));
add_to_table($unique);
You might also want to make sure the letters do not form a word in a dictionnary. May it be the whole english dictionnary or just a bad-word dictionnary to avoid things a customer would find of bad-taste.
EDIT: I would also add this only make sense if, as you intend to use it, it's not for a big amount of items because this could get pretty slow the more collisions you get (getting an ID already in the table). Of course, you'll want an indexed table and you'll want to tweak the number of letters in the ID to avoid collision. In this case, with 6 letters, you'd have 26^6 = 308915776 possible unique IDs (minus bad words) which should be enough for your need of 10000.
EDIT:
If you want a combinations of letters and numbers you can use the following code:
$random .= rand(0, 1) ? rand(0, 9) : chr(rand(ord('a'), ord('z')));
#gen_uuid() by gord.
preg_replace got some nasty utf-8 problems, which causes the uid somtimes to contain "+" or "/".
To get around this, you have to explicitly make the pattern utf-8
function gen_uuid($len=8) {
$hex = md5("yourSaltHere" . uniqid("", true));
$pack = pack('H*', $hex);
$tmp = base64_encode($pack);
$uid = preg_replace("#(*UTF8)[^A-Za-z0-9]#", "", $tmp);
$len = max(4, min(128, $len));
while (strlen($uid) < $len)
$uid .= gen_uuid(22);
return substr($uid, 0, $len);
}
Took me quite a while to find that, perhaps it's saves somebody else a headache
You can achieve that with less code:
function gen_uid($l=10){
return substr(str_shuffle("0123456789abcdefghijklmnopqrstuvwxyz"), 0, $l);
}
Result (examples):
cjnp56brdy
9d5uv84zfa
ih162lryez
ri4ocf6tkj
xj04s83egi
There are two ways to obtain a reliably unique ID: Make it so long and variable that the chances of a collision are spectacularly small (as with a GUID) or store all generated IDs in a table for lookup (either in memory or in a DB or a file) to verify uniqueness upon generation.
If you're really asking how you can generate such a short key and guarantee its uniqueness without some kind of duplicate check, the answer is, you can't.
Here's the routine I use for random base62s of any length...
Calling gen_uuid() returns strings like WJX0u0jV, E9EMaZ3P etc.
By default this returns 8 digits, hence a space of 64^8 or roughly 10^14,
this is often enough to make collisions quite rare.
For a larger or smaller string, pass in $len as desired. No limit in length, as I append until satisfied [up to safety limit of 128 chars, which can be removed].
Note, use a random salt inside the md5 [or sha1 if you prefer], so it cant easily be reverse-engineered.
I didn't find any reliable base62 conversions on the web, hence this approach of stripping chars from the base64 result.
Use freely under BSD licence,
enjoy,
gord
function gen_uuid($len=8)
{
$hex = md5("your_random_salt_here_31415" . uniqid("", true));
$pack = pack('H*', $hex);
$uid = base64_encode($pack); // max 22 chars
$uid = ereg_replace("[^A-Za-z0-9]", "", $uid); // mixed case
//$uid = ereg_replace("[^A-Z0-9]", "", strtoupper($uid)); // uppercase only
if ($len<4)
$len=4;
if ($len>128)
$len=128; // prevent silliness, can remove
while (strlen($uid)<$len)
$uid = $uid . gen_uuid(22); // append until length achieved
return substr($uid, 0, $len);
}
Really simple solution:
Make the unique ID with:
$id = 100;
base_convert($id, 10, 36);
Get the original value again:
intval($str,36);
Can't take credit for this as it's from another stack overflow page, but I thought the solution was so elegant and awesome that it was worth copying over to this thread for people referencing this.
You could use the Id and just convert it to base-36 number if you want to convert it back and forth. Can be used for any table with an integer id.
function toUId($baseId, $multiplier = 1) {
return base_convert($baseId * $multiplier, 10, 36);
}
function fromUId($uid, $multiplier = 1) {
return (int) base_convert($uid, 36, 10) / $multiplier;
}
echo toUId(10000, 11111);
1u5h0w
echo fromUId('1u5h0w', 11111);
10000
Smart people can probably figure it out with enough id examples. Dont let this obscurity replace security.
I came up with what I think is a pretty cool solution doing this without a uniqueness check. I thought I'd share for any future visitors.
A counter is a really easy way to guarantee uniqueness or if you're using a database a primary key also guarantees uniqueness. The problem is it looks bad and and might be vulnerable. So I took the sequence and jumbled it up with a cipher. Since the cipher can be reversed, I know each id is unique while still appearing random.
It's python not php, but I uploaded the code here:
https://github.com/adecker89/Tiny-Unique-Identifiers
Letters are pretty, digits are ugly.
You want random strings, but don't want "ugly" random strings?
Create a random number and print it in alpha-style (base-26), like the reservation "numbers" that airlines give.
There's no general-purpose base conversion functions built into PHP, as far as I know, so you'd need to code that bit yourself.
Another alternative: use uniqid() and get rid of the digits.
function strip_digits_from_string($string) {
return preg_replace('/[0-9]/', '', $string);
}
Or replace them with letters:
function replace_digits_with_letters($string) {
return strtr($string, '0123456789', 'abcdefghij');
}
You can also do it like tihs:
public static function generateCode($length = 6)
{
$az = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ';
$azr = rand(0, 51);
$azs = substr($az, $azr, 10);
$stamp = hash('sha256', time());
$mt = hash('sha256', mt_rand(5, 20));
$alpha = hash('sha256', $azs);
$hash = str_shuffle($stamp . $mt . $alpha);
$code = ucfirst(substr($hash, $azr, $length));
return $code;
}
You can do that without unclean/costy stuff like loops, String concatenations or multiple calls to rand(), in a clean and easy to read way. Also, it is better to use mt_rand():
function createRandomString($length)
{
$random = mt_rand(0, (1 << ($length << 2)) - 1);
return dechex($random);
}
If you need the String to have the exact length in any case, just pad the hex number with zeros:
function createRandomString($length)
{
$random = mt_rand(0, (1 << ($length << 2)) - 1);
$number = dechex($random);
return str_pad($number, $length, '0', STR_PAD_LEFT);
}
The "theoretical backdraw" is, that you are limited to PHPs capabilities - but this is more a philosophical issue in that case ;) Let's go through it anyways:
PHP is limited in what it can represent as a hex number doing it like this. This would be $length <= 8 at least on a 32bit system, where PHPs limitation for this should be 4.294.967.295 .
PHPs random number generator also has a maximum. For mt_rand() at least on a 32bit system, it should be 2.147.483.647
So you are theoretically limited to 2.147.483.647 IDs.
Coming back to the topic - the intuitive do { (generate ID) } while { (id is not uniqe) } (insert id) has one drawback and one possible flaw that might drive you straight to darkness...
Drawback: The validation is pessimistic. Doing it like this always requires a check at the database. Having enough keyspace (for example length of 5 for your 10k entries) will quite unlikely cause collisions as often, as it might be comparably less resource consuming to just try to store the data and retry only in case of a UNIQUE KEY error.
Flaw: User A retrieves an ID that gets verified as not taken yet. Then the code will try to insert the data. But in the meantime, User B entered the same loop and unfortunately retrieves the same random number, because User A is not stored yet and this ID was still free. Now the system stores either User B or User A, and when attempting to store the second User, there already is the other one in the meantime - having the same ID.
You would need to handle that exception in any case and need to re-try the insertion with a newly created ID. Adding this whilst keeping the pessimistic checking loop (that you would need to re-enter) will result in quite ugly and hard to follow code. Fortunately the solution to this is the same like the one to the drawback: Just go for it in the first place and try to store the data. In case of a UNIQUE KEY error just retry with a new ID.
Take a lookt at this article
Create short IDs with PHP - Like Youtube or TinyURL
It explains how to generate short unique ids from your bdd ids, like youtube does.
Actually, the function in the article is very related to php function base_convert which converts a number from a base to another (but is only up to base 36).
10 chars:
substr(uniqid(),-10);
5 binary chars:
hex2bin( substr(uniqid(),-10) );
8 base64 chars:
base64_encode( hex2bin( substr(uniqid(),-10) ) );
function rand_str($len = 12, $type = '111', $add = null) {
$rand = ($type[0] == '1' ? 'abcdefghijklmnpqrstuvwxyz' : '') .
($type[1] == '1' ? 'ABCDEFGHIJKLMNPQRSTUVWXYZ' : '') .
($type[2] == '1' ? '123456789' : '') .
(strlen($add) > 0 ? $add : '');
if(empty($rand)) $rand = sha1( uniqid(mt_rand(), true) . uniqid( uniqid(mt_rand(), true), true) );
return substr(str_shuffle( str_repeat($rand, 2) ), 0, $len);
}
If you do like a longer version of unique Id use this:
$uniqueid = sha1(md5(time()));
Best Answer Yet: Smallest Unique "Hash Like" String Given Unique Database ID - PHP Solution, No Third Party Libraries Required.
Here's the code:
<?php
/*
THE FOLLOWING CODE WILL PRINT:
A database_id value of 200 maps to 5K
A database_id value of 1 maps to 1
A database_id value of 1987645 maps to 16LOD
*/
$database_id = 200;
$base36value = dec2string($database_id, 36);
echo "A database_id value of 200 maps to $base36value\n";
$database_id = 1;
$base36value = dec2string($database_id, 36);
echo "A database_id value of 1 maps to $base36value\n";
$database_id = 1987645;
$base36value = dec2string($database_id, 36);
echo "A database_id value of 1987645 maps to $base36value\n";
// HERE'S THE FUNCTION THAT DOES THE HEAVY LIFTING...
function dec2string ($decimal, $base)
// convert a decimal number into a string using $base
{
//DebugBreak();
global $error;
$string = null;
$base = (int)$base;
if ($base < 2 | $base > 36 | $base == 10) {
echo 'BASE must be in the range 2-9 or 11-36';
exit;
} // if
// maximum character string is 36 characters
$charset = '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ';
// strip off excess characters (anything beyond $base)
$charset = substr($charset, 0, $base);
if (!ereg('(^[0-9]{1,50}$)', trim($decimal))) {
$error['dec_input'] = 'Value must be a positive integer with < 50 digits';
return false;
} // if
do {
// get remainder after dividing by BASE
$remainder = bcmod($decimal, $base);
$char = substr($charset, $remainder, 1); // get CHAR from array
$string = "$char$string"; // prepend to output
//$decimal = ($decimal - $remainder) / $base;
$decimal = bcdiv(bcsub($decimal, $remainder), $base);
} while ($decimal > 0);
return $string;
}
?>