I am trying to find a way to encode a database ID into a short URL, e.g. 1 should become "Ys47R". Then I would like to decode it back from "Ys47R" to 1 so I can run a database search using the INT value. It needs to be unique using the database ID. The sequence should not be easily guessable such as 1 = "Ys47R", 2 = "Ys47S". It should be something along the lines of YouTube or bitly's URL's. I have read up on hundreds of different sources using md5, base32, base64 and `bcpow but have come up empty.
This blog post looked promising but once I added padding and a passkey, short ID's such as 1 became SDDDG, 2 became "SDDDH" and 3 became "SDDDI". It is not very random.
base32 used only a-b 0-9
base64 had characters such as == on the end.
I then tried this:
function getRandomString($db, $length = 7) {
$validCharacters = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ";
$validCharNumber = strlen($validCharacters);
$result = "";
for ($i = 0; $i < $length; $i++) {
$index = mt_rand(0, $validCharNumber - 1);
$result .= $validCharacters[$index];
}
Which worked but meant I had to run a database query every time to make sure there were no collisions and it did not exist in the database.
Is there a way I can create short ID's that are 4 characters minimum with a charset of [a-z][A-Z][0-9] that can be encoded and decoded back, using increment unique ID in a database where each number is unique. I can't get my head around advance techniques using base32 or base64.
Or am I looking into this too much and there is an easier way to do it? Would it be best to do the random string function above and query the database to check for uniqueness all the time?
You could use function from comments: http://php.net/manual/en/function.base-convert.php#106546
$initial = '11111111';
$dic = '0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ';
var_dump($converted = convBase($initial, '0123456789', $dic));
// string(4) "KCvt"
var_dump(convBase($converted, $dic, '0123456789'));
// string(8) "11111111"
function convBase($numberInput, $fromBaseInput, $toBaseInput)
{
if ($fromBaseInput==$toBaseInput) return $numberInput;
$fromBase = str_split($fromBaseInput,1);
$toBase = str_split($toBaseInput,1);
$number = str_split($numberInput,1);
$fromLen=strlen($fromBaseInput);
$toLen=strlen($toBaseInput);
$numberLen=strlen($numberInput);
$retval='';
if ($toBaseInput == '0123456789')
{
$retval=0;
for ($i = 1;$i <= $numberLen; $i++)
$retval = bcadd($retval, bcmul(array_search($number[$i-1], $fromBase),bcpow($fromLen,$numberLen-$i)));
return $retval;
}
if ($fromBaseInput != '0123456789')
$base10=convBase($numberInput, $fromBaseInput, '0123456789');
else
$base10 = $numberInput;
if ($base10<strlen($toBaseInput))
return $toBase[$base10];
while($base10 != '0')
{
$retval = $toBase[bcmod($base10,$toLen)].$retval;
$base10 = bcdiv($base10,$toLen,0);
}
return $retval;
}
If you want some symmetric obfuscation, then base_convert() is often sufficient.
base_convert($id, 10, 36);
Will return strings like 1i0g and convert them back.
Before and after that base conversion you can add:
To get a minimum string length, I'd suggest just adding 70000 to your $id. And on the receiving end just subtract that again.
A minor multiplication $id *= 3 would add some "holes" in the generated alphanumeric ID range, yet not exhaust the available string space.
For some appearance of arbitrariness, a bit of nibble moving:
$id = ($id & 0xF0F0F0F) << 4
| ($id & 0x0F0F0F0) >> 4;
Which works for generating your obfuscated ID strings, and getting back the original ones.
Just to be crystal clear: this is no encryption of any sort. It just shifts numeric jumps between consecutive numbers, and looks slightly more arbitrary.
You still may not like the answer, but generating random IDs in your database is the only approach that really hinders ID guessing.
I'm generating a 6 digit code from the following characters. These will be used to stamp on stickers.
They will be generated in batches of 10k or less (before printing) and I don't envisage there will ever be more than 1-2 million total (probably much less).
After I generate the batches of codes, I'll check the MySQL database of existing codes to ensure there are no duplicates.
// exclude problem chars: B8G6I1l0OQDS5Z2
$characters = 'ACEFHJKMNPRTUVWXY4937';
$string = '';
for ($i = 0; $i < 6; $i++) {
$string .= $characters[rand(0, strlen($characters) - 1)];
}
return $string;
Is this a solid approach to generating the code?
How many possible permutations would there be? (6 Digit code from pool of 21 characters). Sorry math isn't my strong point
21^6 = 85766121 possibilities.
Using a DB and storing used values is bad. If you want to fake randomness you can use the following:
Reduce to 19 possible numbers and make use of the fact that groups of order p^k where p is an odd prime are always cyclic.
Take the group of order 7^19, using a generator co-prime to 7^19 (I'll pick 13^11, you can choose anything not divisible by 7).
Then the following works:
$previous = 0;
function generator($previous)
{
$generator = pow(13,11);
$modulus = pow(7,19); //int might be too small
$possibleChars = "ACEFHJKMNPRTUVWXY49";
$previous = ($previous + $generator) % $modulus;
$output='';
$temp = $previous;
for($i = 0; $i < 6; $i++) {
$output += $possibleChars[$temp % 19];
$temp = $temp / 19;
}
return $output;
}
It will cycle through all possible values and look a little random unless they go digging. An even safer alternative would be multiplicative groups but I forget my math already :(
There is a lot of possible combination with or without repetition so your logic would be sufficient
Collision would be frequent because you are using rand see str_shuffle and randomness.
Change rand to mt_rand
Use fast storage like memcached or redis not MySQL when checking
Total Possibility
21 ^ 6 = 85,766,121
85,766,121 should be ok , To add database to this generation try:
Example
$prifix = "stamp.";
$cache = new Memcache();
$cache->addserver("127.0.0.1");
$stamp = myRand(6);
while($cache->get($prifix . $stamp)) {
$stamp = myRand(6);
}
echo $stamp;
Function Used
function myRand($no, $str = "", $chr = 'ACEFHJKMNPRTUVWXY4937') {
$length = strlen($chr);
while($no --) {
$str .= $chr{mt_rand(0, $length- 1)};
}
return $str;
}
as Baba said generating a string on the fly will result in tons of collisions. the closer you will go to 80 millions already generated ones the harder it will became to get an available string
another solution could be to generate all possible combinations once, and store each of them in the database already, with some boolean column field that marks if a row/token is already used or not
then to get one of them
SELECT * FROM tokens WHERE tokenIsUsed = 0 ORDER BY RAND() LIMIT 0,1
and then mark it as already used
UPDATE tokens SET tokenIsUsed = 1 WHERE token = ...
You would have 21 ^ 6 codes = 85 766 121 ~ 85.8 million codes!
To generate them all (which would take some time), look at the selected answer to this question: algorithm that will take numbers or words and find all possible combinations.
I had the same problem, and I found very impressive open source solution:
http://www.hashids.org/php/
You can take and use it, also it's worth it to look in it's source code to understand what's happening under the hood.
Or... you can encode username+datetime in md5 and save to database, this for sure will generate an unique code ;)
I've found this code on the web, can't remember the exact source but the same code is on various sites.
private function pageRankUrl($q) {
$host='toolbarqueries.google.com';
$seed = "Mining PageRank is AGAINST GOOGLE'S TERMS OF SERVICE. Yes, I'm talking to you, scammer.";
$result = 0x01020345;
$len = strlen($q);
for ($i=0; $i<$len; $i++) {
$result ^= ord($seed{$i%strlen($seed)}) ^ ord($q{$i});
$result = (($result >> 23) & 0x1ff) | $result << 9;
}
$ch=sprintf('8%x', $result);
$url='http://%s/tbr?client=navclient-auto&ch=%s&features=Rank&q=info:%s';
$url=sprintf($url,$host,$ch,$q);
return $url;
}
The code works fine, but I just want help understanding it. I need to understand what the $seed is, and also what's happening in the for loop.
It looks to me like the $seed and the for-loop reduce the query-string, i.e. compute a shorter version. This can be used for a hash-key or something. It looks to me like a reduction function from a rainbow-table when it's compute a password-hash chain.
I want to create a unique id but uniqid() is giving something like '492607b0ee414'. What i would like is something similar to what tinyurl gives: '64k8ra'. The shorter, the better. The only requirements are that it should not have an obvious order and that it should look prettier than a seemingly random sequence of numbers. Letters are preferred over numbers and ideally it would not be mixed case. As the number of entries will not be that many (up to 10000 or so) the risk of collision isn't a huge factor.
Any suggestions appreciated.
Make a small function that returns random letters for a given length:
<?php
function generate_random_letters($length) {
$random = '';
for ($i = 0; $i < $length; $i++) {
$random .= chr(rand(ord('a'), ord('z')));
}
return $random;
}
Then you'll want to call that until it's unique, in pseudo-code depending on where you'd store that information:
do {
$unique = generate_random_letters(6);
} while (is_in_table($unique));
add_to_table($unique);
You might also want to make sure the letters do not form a word in a dictionnary. May it be the whole english dictionnary or just a bad-word dictionnary to avoid things a customer would find of bad-taste.
EDIT: I would also add this only make sense if, as you intend to use it, it's not for a big amount of items because this could get pretty slow the more collisions you get (getting an ID already in the table). Of course, you'll want an indexed table and you'll want to tweak the number of letters in the ID to avoid collision. In this case, with 6 letters, you'd have 26^6 = 308915776 possible unique IDs (minus bad words) which should be enough for your need of 10000.
EDIT:
If you want a combinations of letters and numbers you can use the following code:
$random .= rand(0, 1) ? rand(0, 9) : chr(rand(ord('a'), ord('z')));
#gen_uuid() by gord.
preg_replace got some nasty utf-8 problems, which causes the uid somtimes to contain "+" or "/".
To get around this, you have to explicitly make the pattern utf-8
function gen_uuid($len=8) {
$hex = md5("yourSaltHere" . uniqid("", true));
$pack = pack('H*', $hex);
$tmp = base64_encode($pack);
$uid = preg_replace("#(*UTF8)[^A-Za-z0-9]#", "", $tmp);
$len = max(4, min(128, $len));
while (strlen($uid) < $len)
$uid .= gen_uuid(22);
return substr($uid, 0, $len);
}
Took me quite a while to find that, perhaps it's saves somebody else a headache
You can achieve that with less code:
function gen_uid($l=10){
return substr(str_shuffle("0123456789abcdefghijklmnopqrstuvwxyz"), 0, $l);
}
Result (examples):
cjnp56brdy
9d5uv84zfa
ih162lryez
ri4ocf6tkj
xj04s83egi
There are two ways to obtain a reliably unique ID: Make it so long and variable that the chances of a collision are spectacularly small (as with a GUID) or store all generated IDs in a table for lookup (either in memory or in a DB or a file) to verify uniqueness upon generation.
If you're really asking how you can generate such a short key and guarantee its uniqueness without some kind of duplicate check, the answer is, you can't.
Here's the routine I use for random base62s of any length...
Calling gen_uuid() returns strings like WJX0u0jV, E9EMaZ3P etc.
By default this returns 8 digits, hence a space of 64^8 or roughly 10^14,
this is often enough to make collisions quite rare.
For a larger or smaller string, pass in $len as desired. No limit in length, as I append until satisfied [up to safety limit of 128 chars, which can be removed].
Note, use a random salt inside the md5 [or sha1 if you prefer], so it cant easily be reverse-engineered.
I didn't find any reliable base62 conversions on the web, hence this approach of stripping chars from the base64 result.
Use freely under BSD licence,
enjoy,
gord
function gen_uuid($len=8)
{
$hex = md5("your_random_salt_here_31415" . uniqid("", true));
$pack = pack('H*', $hex);
$uid = base64_encode($pack); // max 22 chars
$uid = ereg_replace("[^A-Za-z0-9]", "", $uid); // mixed case
//$uid = ereg_replace("[^A-Z0-9]", "", strtoupper($uid)); // uppercase only
if ($len<4)
$len=4;
if ($len>128)
$len=128; // prevent silliness, can remove
while (strlen($uid)<$len)
$uid = $uid . gen_uuid(22); // append until length achieved
return substr($uid, 0, $len);
}
Really simple solution:
Make the unique ID with:
$id = 100;
base_convert($id, 10, 36);
Get the original value again:
intval($str,36);
Can't take credit for this as it's from another stack overflow page, but I thought the solution was so elegant and awesome that it was worth copying over to this thread for people referencing this.
You could use the Id and just convert it to base-36 number if you want to convert it back and forth. Can be used for any table with an integer id.
function toUId($baseId, $multiplier = 1) {
return base_convert($baseId * $multiplier, 10, 36);
}
function fromUId($uid, $multiplier = 1) {
return (int) base_convert($uid, 36, 10) / $multiplier;
}
echo toUId(10000, 11111);
1u5h0w
echo fromUId('1u5h0w', 11111);
10000
Smart people can probably figure it out with enough id examples. Dont let this obscurity replace security.
I came up with what I think is a pretty cool solution doing this without a uniqueness check. I thought I'd share for any future visitors.
A counter is a really easy way to guarantee uniqueness or if you're using a database a primary key also guarantees uniqueness. The problem is it looks bad and and might be vulnerable. So I took the sequence and jumbled it up with a cipher. Since the cipher can be reversed, I know each id is unique while still appearing random.
It's python not php, but I uploaded the code here:
https://github.com/adecker89/Tiny-Unique-Identifiers
Letters are pretty, digits are ugly.
You want random strings, but don't want "ugly" random strings?
Create a random number and print it in alpha-style (base-26), like the reservation "numbers" that airlines give.
There's no general-purpose base conversion functions built into PHP, as far as I know, so you'd need to code that bit yourself.
Another alternative: use uniqid() and get rid of the digits.
function strip_digits_from_string($string) {
return preg_replace('/[0-9]/', '', $string);
}
Or replace them with letters:
function replace_digits_with_letters($string) {
return strtr($string, '0123456789', 'abcdefghij');
}
You can also do it like tihs:
public static function generateCode($length = 6)
{
$az = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ';
$azr = rand(0, 51);
$azs = substr($az, $azr, 10);
$stamp = hash('sha256', time());
$mt = hash('sha256', mt_rand(5, 20));
$alpha = hash('sha256', $azs);
$hash = str_shuffle($stamp . $mt . $alpha);
$code = ucfirst(substr($hash, $azr, $length));
return $code;
}
You can do that without unclean/costy stuff like loops, String concatenations or multiple calls to rand(), in a clean and easy to read way. Also, it is better to use mt_rand():
function createRandomString($length)
{
$random = mt_rand(0, (1 << ($length << 2)) - 1);
return dechex($random);
}
If you need the String to have the exact length in any case, just pad the hex number with zeros:
function createRandomString($length)
{
$random = mt_rand(0, (1 << ($length << 2)) - 1);
$number = dechex($random);
return str_pad($number, $length, '0', STR_PAD_LEFT);
}
The "theoretical backdraw" is, that you are limited to PHPs capabilities - but this is more a philosophical issue in that case ;) Let's go through it anyways:
PHP is limited in what it can represent as a hex number doing it like this. This would be $length <= 8 at least on a 32bit system, where PHPs limitation for this should be 4.294.967.295 .
PHPs random number generator also has a maximum. For mt_rand() at least on a 32bit system, it should be 2.147.483.647
So you are theoretically limited to 2.147.483.647 IDs.
Coming back to the topic - the intuitive do { (generate ID) } while { (id is not uniqe) } (insert id) has one drawback and one possible flaw that might drive you straight to darkness...
Drawback: The validation is pessimistic. Doing it like this always requires a check at the database. Having enough keyspace (for example length of 5 for your 10k entries) will quite unlikely cause collisions as often, as it might be comparably less resource consuming to just try to store the data and retry only in case of a UNIQUE KEY error.
Flaw: User A retrieves an ID that gets verified as not taken yet. Then the code will try to insert the data. But in the meantime, User B entered the same loop and unfortunately retrieves the same random number, because User A is not stored yet and this ID was still free. Now the system stores either User B or User A, and when attempting to store the second User, there already is the other one in the meantime - having the same ID.
You would need to handle that exception in any case and need to re-try the insertion with a newly created ID. Adding this whilst keeping the pessimistic checking loop (that you would need to re-enter) will result in quite ugly and hard to follow code. Fortunately the solution to this is the same like the one to the drawback: Just go for it in the first place and try to store the data. In case of a UNIQUE KEY error just retry with a new ID.
Take a lookt at this article
Create short IDs with PHP - Like Youtube or TinyURL
It explains how to generate short unique ids from your bdd ids, like youtube does.
Actually, the function in the article is very related to php function base_convert which converts a number from a base to another (but is only up to base 36).
10 chars:
substr(uniqid(),-10);
5 binary chars:
hex2bin( substr(uniqid(),-10) );
8 base64 chars:
base64_encode( hex2bin( substr(uniqid(),-10) ) );
function rand_str($len = 12, $type = '111', $add = null) {
$rand = ($type[0] == '1' ? 'abcdefghijklmnpqrstuvwxyz' : '') .
($type[1] == '1' ? 'ABCDEFGHIJKLMNPQRSTUVWXYZ' : '') .
($type[2] == '1' ? '123456789' : '') .
(strlen($add) > 0 ? $add : '');
if(empty($rand)) $rand = sha1( uniqid(mt_rand(), true) . uniqid( uniqid(mt_rand(), true), true) );
return substr(str_shuffle( str_repeat($rand, 2) ), 0, $len);
}
If you do like a longer version of unique Id use this:
$uniqueid = sha1(md5(time()));
Best Answer Yet: Smallest Unique "Hash Like" String Given Unique Database ID - PHP Solution, No Third Party Libraries Required.
Here's the code:
<?php
/*
THE FOLLOWING CODE WILL PRINT:
A database_id value of 200 maps to 5K
A database_id value of 1 maps to 1
A database_id value of 1987645 maps to 16LOD
*/
$database_id = 200;
$base36value = dec2string($database_id, 36);
echo "A database_id value of 200 maps to $base36value\n";
$database_id = 1;
$base36value = dec2string($database_id, 36);
echo "A database_id value of 1 maps to $base36value\n";
$database_id = 1987645;
$base36value = dec2string($database_id, 36);
echo "A database_id value of 1987645 maps to $base36value\n";
// HERE'S THE FUNCTION THAT DOES THE HEAVY LIFTING...
function dec2string ($decimal, $base)
// convert a decimal number into a string using $base
{
//DebugBreak();
global $error;
$string = null;
$base = (int)$base;
if ($base < 2 | $base > 36 | $base == 10) {
echo 'BASE must be in the range 2-9 or 11-36';
exit;
} // if
// maximum character string is 36 characters
$charset = '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ';
// strip off excess characters (anything beyond $base)
$charset = substr($charset, 0, $base);
if (!ereg('(^[0-9]{1,50}$)', trim($decimal))) {
$error['dec_input'] = 'Value must be a positive integer with < 50 digits';
return false;
} // if
do {
// get remainder after dividing by BASE
$remainder = bcmod($decimal, $base);
$char = substr($charset, $remainder, 1); // get CHAR from array
$string = "$char$string"; // prepend to output
//$decimal = ($decimal - $remainder) / $base;
$decimal = bcdiv(bcsub($decimal, $remainder), $base);
} while ($decimal > 0);
return $string;
}
?>