I'm making a website which people can publish posts. In my database each post has an ID like 1, 2, 3 etc. but I would like to change them, like using a hash like Youtube does.
For example instead of http://localhost/post/1
They would go to http://localhost/post/hu9NA827z
Is there a method like hashing the numbers and decoding it?
Well, while you could encrypt/decrypt it doesn't make much sense (you're gonna make it slower without any real benefit).
Waht you can do is to have the primary key in your DB to be a string and generate a hash for the id or add a new column with a unique index, save the hash there and search the posts by the hash column (and maybe keep the id for internal purposes). You can use complex algorithms or just md5(uniqid()), since this is not for security i wouldn't worry too much. Make sure that when creating a new post, the uniqueness is not being violated. Now you have another reason for an insertion to fail (the hash not being unique) so prepare for that.
Check:
http://php.net/manual/en/function.md5.php
http://php.net/manual/en/function.uniqid.php
Since there is no need for this hash to be secure, you can just use the PHP built-in hash function, md5(). I suggest using the timestamp as input:
$id = md5(time());
Just truncate it to make it shorter. I suggest you keep the original primary key an autoincrement integer and add this hash as a new column.
The sequence hu9NA827z is BASE64. Decoding it, you get a binary sequence of 6 bytes.
For instance:
base64_encode('123456') // = 'MTIzNDU2'
base64_decode('MTIzNDU2') // = '123456'
However, on YouTube, BASE64 is not being used to protect the information, its purpuse is just to serialize it into a human-readable ASCII format. The real message behind it is a 48-bit binary sequence.
This binary sequence is probably the encrypted version of what would be the video ID on a database, but what it really is only YouTube developers knows for sure and they certainly expect it to remain that way.
In your case, you could simply implement a similar system using one of the many two-way encryption methods offered in PHP like MCrypt that supports a lot of encryption algorithms of your choice including the very safe AES.
Related
What would be useful solutions for hiding true database object ID in URL for security purposes? I found that one of the solutions would be:
1) Using hashids open source project
2) Using something like same old md5 on creation of the object to generate hash and store it in database, then use it in url's and querying by them, but the drawback is that querying by auto-incremented primary keys (IDs) is faster than hashes. So I believe the possibility to hash/unhash would be better?
Also as I'm on Symfony, are there maybe bundles that I could not find or built in functionalities that would help?
Please tell me what you found useful based on your experiences.
This question has been asked a lot, with different word choice (which makes it difficult to say, "Just search for it!"). This fact prompted a blog post titled, The Comprehensive Guide to URL Parameter Encryption in PHP .
What People Want To Do Here
What People Should Do Instead
Explanation
Typically, people want short random-looking URLs. This doesn't allow you much room to encrypt then authenticate the database record ID you wish to obfuscate. Doing so would require a minimum URL length of 32 bytes (for HMAC-SHA256), which is 44 characters when encoded in base64.
A simpler strategy is to generate a random string (see random_compat for a PHP5 implementation of random_bytes() and random_int() for generating these strings) and reference that column instead.
Also, hashids are broken by simple cryptanalysis. Their conclusion states:
The attack I have described is significantly better than a brute force attack, so from a cryptographic stand point the algorithm is considered to be broken, it is quite easy to recover the salt; making it possible for an attacker to run the encoding in either direction and invalidates property 2 for an ideal hash function.
Don't rely on it.
Quote from the site:
Do you have a question or comment that involves "security" and "hashids" in the same sentence? Don't use Hashids.
I'd use true encryption algorithm, like function openssl_encrypt (for example), or something like this. And encrypt ids when passing outside, decrypt when using in your code (like for db queries).
And I won't recommend storing ids in a base like any kind of encrypted "garbage", in my opinion its very inconvenient to hash your real ids. Keep it clean and pretty inside and encrypt for external display only.
Following your idea, you just need to cipher your IDs before writing the URL to HTML page and decipher them when processing those URLs.
If you want just security by obscurity, which is sufficient for, maybe 99% of curious people out there who likes to iterate over IDs in URLs, you use something simple like base64 or rot13. Of course, you can also precalculate those "public IDs" and store in the database, not encrypting each time the URL is being shown to end user.
If you want true security you have to encrypt them with some serious asymmetric cypher, storing both keys at your side, as you essentially talking with yourself and don't want a man-in-the-middle attack. This you will not be able to precalculate as at each encrypting there'll be different cyphertext, which is good for this cause.
In any case, you need something two-way, so if I were you I'd forget about word "hash", hashes are for purposes different from yours.
EDIT:
But the solution which every blog out there uses for this task for several years already is just to utilize URL rewriting, converting, in your case, URLs like http://example.com/book/5 to URLs like http://example.com/rework-by-37signals. This will completely eradicate any sign of database ID from your URL.
Ideologically, you will need something which will uniquely map the request URL to your database content anyway. If you hide MySQL database IDs behind any layer of URL rewriting, you'll just make this rewritten URL a new ID for the same content. All you gain is protection from enumeration attacks and maybe SEF URLs.
What would be useful solutions for hiding true database object ID in URL for security purposes? I found that one of the solutions would be:
1) Using hashids open source project
2) Using something like same old md5 on creation of the object to generate hash and store it in database, then use it in url's and querying by them, but the drawback is that querying by auto-incremented primary keys (IDs) is faster than hashes. So I believe the possibility to hash/unhash would be better?
Also as I'm on Symfony, are there maybe bundles that I could not find or built in functionalities that would help?
Please tell me what you found useful based on your experiences.
This question has been asked a lot, with different word choice (which makes it difficult to say, "Just search for it!"). This fact prompted a blog post titled, The Comprehensive Guide to URL Parameter Encryption in PHP .
What People Want To Do Here
What People Should Do Instead
Explanation
Typically, people want short random-looking URLs. This doesn't allow you much room to encrypt then authenticate the database record ID you wish to obfuscate. Doing so would require a minimum URL length of 32 bytes (for HMAC-SHA256), which is 44 characters when encoded in base64.
A simpler strategy is to generate a random string (see random_compat for a PHP5 implementation of random_bytes() and random_int() for generating these strings) and reference that column instead.
Also, hashids are broken by simple cryptanalysis. Their conclusion states:
The attack I have described is significantly better than a brute force attack, so from a cryptographic stand point the algorithm is considered to be broken, it is quite easy to recover the salt; making it possible for an attacker to run the encoding in either direction and invalidates property 2 for an ideal hash function.
Don't rely on it.
Quote from the site:
Do you have a question or comment that involves "security" and "hashids" in the same sentence? Don't use Hashids.
I'd use true encryption algorithm, like function openssl_encrypt (for example), or something like this. And encrypt ids when passing outside, decrypt when using in your code (like for db queries).
And I won't recommend storing ids in a base like any kind of encrypted "garbage", in my opinion its very inconvenient to hash your real ids. Keep it clean and pretty inside and encrypt for external display only.
Following your idea, you just need to cipher your IDs before writing the URL to HTML page and decipher them when processing those URLs.
If you want just security by obscurity, which is sufficient for, maybe 99% of curious people out there who likes to iterate over IDs in URLs, you use something simple like base64 or rot13. Of course, you can also precalculate those "public IDs" and store in the database, not encrypting each time the URL is being shown to end user.
If you want true security you have to encrypt them with some serious asymmetric cypher, storing both keys at your side, as you essentially talking with yourself and don't want a man-in-the-middle attack. This you will not be able to precalculate as at each encrypting there'll be different cyphertext, which is good for this cause.
In any case, you need something two-way, so if I were you I'd forget about word "hash", hashes are for purposes different from yours.
EDIT:
But the solution which every blog out there uses for this task for several years already is just to utilize URL rewriting, converting, in your case, URLs like http://example.com/book/5 to URLs like http://example.com/rework-by-37signals. This will completely eradicate any sign of database ID from your URL.
Ideologically, you will need something which will uniquely map the request URL to your database content anyway. If you hide MySQL database IDs behind any layer of URL rewriting, you'll just make this rewritten URL a new ID for the same content. All you gain is protection from enumeration attacks and maybe SEF URLs.
I have a website where users can upload blog posts. This website is built entirely by me, no framework documentation to look through.
I've been using Mysql's Primary key as the page ID in the url, but i don't like this as it gives away too much information to the user.
The id appears somewhat like this
www.website.com/view?post=97
Youtube uses an 11 letter combination and looks somewhat like
watch?v=wEoFhRCUEs8 // *Not a plug*
I was thinking of encrypting the ID in an MD5, but 1) this is far too long, 2) Not the intended use of MD5.
Any ideas on how sites like facebook, stackoverflow, youtube etc encrypt each ID whilst ensuring that it is unique?
I'm also unsure if it is best to save a secondary unique ID in mysql, or just parse it through a function that converts it every time I need it.
Thanks
I can provide you the logic:
first there is a post //
then it is tried to be inserted into the database //
Before inserting into the database //
Generate a random string //
As soon as you generate the random string, check in the database if it is taken //
if it is taken, generate a new string //
else utilize that string //
now insert all the necessary data into the database //
Done //
You are actually on the right track with MD5. Basically what you need to do is create a simple encyption based on the post id. As long as it is not security related I'd write a quick 2 way hashing algorithm that allows you to convert 97 to "wEoFhRCUEs8" and vice versa. That way you can look up posts later.
Probably something that just bit shifts the number and xors it into a "secret" string.. etc...
If you want to go a little more secure try the mcrypt lib, depending on algorithm you can limit the output size.
If you don't do a reversable encryption then you have no choice but to store your unique string with your post so you can look it up later.
I need to create truly unique token when inserting records in CakePHP. The table can contain millions of rows so I cant just base on some randomly generated strings. I do not want to use a microtime() as well, because there is, though very small probability that two records can be submitted exactly at the same moment.
Of course the best solution would be to use String::uuid(), but as from cakephp documentation
The uuid method is used to generate unique identifiers as per RFC 4122. The uuid is a 128bit string in the format of 485fc381-e790-47a3-9794-1337c0a8fe68.
So, as far as I understood it does not use cake's security salt for its generation. So, I decided to hash it by security component's hash function (or Auth Password function), because I need it to be unique and very, really very secure at the same time. But then I found the question, saying that it is not a good idea, but for php uniqid and md5.
Why is MD5'ing a UUID not a good idea?
And, also I think the string hashed by security component is much harder to guess - because, for example String::uuid() in for loop has an output like this
for ($i = 0; $i < 30; $i++) {
echo String::uuid()."<br>";
}
die;
// outputs
51f3dcda-c4fc-4141-aaaf-1378654d2d93
51f3dcda-d9b0-4c20-8d03-1378654d2d93
51f3dcda-e7c0-4ddf-b808-1378654d2d93
51f3dcda-f508-4482-852d-1378654d2d93
51f3dcda-01ec-4f24-83b1-1378654d2d93
51f3dcda-1060-49d2-adc0-1378654d2d93
51f3dcda-1da8-4cfe-abe4-1378654d2d93
51f3dcda-2af0-42f7-81a0-1378654d2d93
51f3dcda-3838-4879-b2c9-1378654d2d93
51f3dcda-451c-465a-a644-1378654d2d93
51f3dcda-5264-44b0-a883-1378654d2d93
So, after all the some part of the string is similar, but in case of using hash function the results are pretty different
echo Security::hash('stackoverflow1');
echo "<br>";
echo Security::hash('stackoverflow2');
die;
// outputs
e9a3fcb74b9a03c7a7ab8731053ab9fe5d2fe6bd
b1f95bdbef28db16f8d4f912391c22310ba3c2c2
So, the question is, can I after all hash the uuid() in Cake? Or what is the best secure way to get truly unique and hashed (better according to my security salt) secure token.
UPDATE
Saying secure token, I mean how difficult it is for guessing. UUID is really unique, but from the example above, some records have some similarity. But hashed results do not.
Thanks !!
I don't think you need to worry about the UUIDs overlapping.
To put these numbers into perspective, the annual risk of someone being hit by a meteorite is estimated to be one chance in 17 billion,[38] which means the probability is about 0.00000000006 (6 × 10−11), equivalent to the odds of creating a few tens of trillions of UUIDs in a year and having one duplicate. In other words, only after generating 1 billion UUIDs every second for the next 100 years, the probability of creating just one duplicate would be about 50%. Or, to put it another way, the probability of one duplicate would be about 50% if every person on earth owns 600 million UUIDs.
http://en.wikipedia.org/wiki/Universally_unique_identifier#Random_UUID_probability_of_duplicates
Continue to use String::uuid() and rest easy :)
A UUID is unique
I need to create truly unique token when inserting records in cakphp
That is exactly what a UUID is. It is normally used in distributed systems to prevent collisions (multiple sources inserting data, possibly out of sync, into a datasource).
A UUID is not a security measure
I need it to be unique and very, really very secure at the same time
Not sure in what way hashing a uuid is supposed to enhance security - it won't. Relying on security by obscurity is more or less guaranteed to fail.
If your need is random tokens of some form - use a hash function (Hashing a uuid is simply hashing a random seed), if you need guaranteed-unique identifiers use UUIDs. They aren't the same thing and a UUID is a very poor mechanism of generating random, non-sequential "un-guessable" (or whatever the purpose is) strings.
Generating a random string suitable for cryptographic purposes was answered well here:
Secure random number generation in PHP
The code sample fills the string $pr_bits with random binary data, so the characters are unprintable. To use this in a URL, you could convert the binary data to printable characters a couple ways. None of them enhance the security but make them ready for URLs.
convert bytes to hex: bin2hex($pr_bits)
convert bytes to base64: base64_encode($pr_bits)
hash the bytes (because the output is conveniently in hex, not for added security): string hash ('md5' , $pr_bits)
I include the last one because you will see people use hash functions for other reasons, like to guarantee the output is 16bytes/128bits for md5. In PHP people use it to convert a value into HEX.
I have come up with the following solution
to use a string as a result of concatenating current time in microseconds and random string's hash
$timeStr = str_replace("0.", "", microtime());
$timeStr = str_replace(" ", "", $timeStr);
echo Security::hash('random string').'_'.$timeStr;
// 5ffd3b852ccdd448809abb172e19bbb9c01a43a4_796473001379403705
So, the first part(hash) of the string will contribute for the unguessability of the token, and the second part will guarantee its uniquenes.
Hope, this will help someone.
My PHP Application uses URLs like these:
http://domain.com/userid/120
http://domain.com/userid/121
The keys and the end of the URL are basically the primary key of the MySQL database table.
I don't want this increasing number to be public and I also don't want that someone will be able to crawl the user profiles just by interating the Id.
So I want to encrypt this Id for display in a way I can easily decrypt it again. The string shouldn't get much longer.
What's the best encryption method for this?
Simple Obscuring: Base64 encode them using base64_encode.
Now, your http://domain.com/userid/121 becomes: http://domain.com/userid/MTIx
Want more, do it again, add some letters around it.
Tough Obscuring: Use any encryption method using MCrypt library.
A better approach (from a usability and SEO perspective) would be to use a unique phrase rather than an obscured ID. In this instance the user's user name would seem an ideal solution, and would also be un-guessable.
That said, if you don't want to use this approach you could just use a hash (perhaps md5) of the user's user name which you'd store in the database along with their other details. As such, you can just do a direct lookup on that field. (i.e.: Having encrypt and decrypt part of the URL is probably overkill.)
You have a variety of choices here:
Generate and store an identifier in the database. It's good because you can then have readable keys that are guaranteed to be unique. It's bad because it causes a database schema change, and you have to actually query that table every time you want to generate a link.
Run an actual key-based encryption, for instance based on PHP's MCrypt. You have access to powerful cryptographic algorithms, but most secure algorithms tend to output strings that are much longer than what you expect. XOR does what you want, but it does not prevent accessing sequential values (and the key is pretty simple to determine, given the a priori knowledge about the numbers).
Run a hash-based verification: instead of using 121 as your identifier, use 121-a34df6 where a34df6 are the first six characters of the md5 (or other HMAC) of 121 and a secret key. Instead of decoding, you extract the 121 and recompute the six characters, to see if they match what the user sent. This does not hide the 121 (it's still right there before the hyphen) but without knowing the secret key, the visitor will not be able to generate the six characters to actually view the document numbered 121.
Use XOR with shuffling: shuffle the bits in the 30-bit identifier, then apply the XOR. This makes the XOR harder to identify because the shuffle pattern is also hidden.
Use XOR with on-demand keys: use fb37cde4-37b3 as your key, where the first part is the XOR of 121 and md5('37b3'.SECRET) (or another way of generating an XOR key based on 37b3 and a secret).
Don't use base64, it's easy to reverse engineer: if MTIx is 121, then MTIy is 122 ...
Ultimately, you will have to accept that your solution will not be secure: not only is it possible for users to leak valid urls (through their browser history, HTTP referer, or posting them on Twitter), but your requirement that the identifier fits in a small number of characters means a brute-force attack is possible (and becomes easier as you start having more documents).
Simplest but powerful encryption method: XOR with a secret Key. http://en.wikipedia.org/wiki/XOR_cipher
No practical performance degradation.
Base64 representation is not an encryption! It's another way to say the same.
Hope this helps.
Obscuring the URL will never secure it. It makes it harder to read, but not much harder to manipulate. You could use a hexadecimal number representation or something like that to obscure it. Those who can read hex can change your URL in a few seconds, anyway:
$hexId = dechex($id); // to hex
$id = hexdec($hexId); // from hex
I'd probably say it's better indeed to just create a random string for each user and store that in your database than to get one using hash. If you use a common hash, it's still very easy to iterate over all pages ;-)
I would write this in comments, but don't have the rep for it (yet?).
When user click on a link you should not use primary key, You can use the pkey in a session and get it from that session. Please do not use query string....
generate an unique string for each user and use it in your urls
http://domain.com/user/ofisdoifsdlfkjsdlfkj instead of http://domain.com/userid/121
you can use base64_encode and base64_decode function for encrypt and decrypt your URLS