What would be useful solutions for hiding true database object ID in URL for security purposes? I found that one of the solutions would be:
1) Using hashids open source project
2) Using something like same old md5 on creation of the object to generate hash and store it in database, then use it in url's and querying by them, but the drawback is that querying by auto-incremented primary keys (IDs) is faster than hashes. So I believe the possibility to hash/unhash would be better?
Also as I'm on Symfony, are there maybe bundles that I could not find or built in functionalities that would help?
Please tell me what you found useful based on your experiences.
This question has been asked a lot, with different word choice (which makes it difficult to say, "Just search for it!"). This fact prompted a blog post titled, The Comprehensive Guide to URL Parameter Encryption in PHP .
What People Want To Do Here
What People Should Do Instead
Explanation
Typically, people want short random-looking URLs. This doesn't allow you much room to encrypt then authenticate the database record ID you wish to obfuscate. Doing so would require a minimum URL length of 32 bytes (for HMAC-SHA256), which is 44 characters when encoded in base64.
A simpler strategy is to generate a random string (see random_compat for a PHP5 implementation of random_bytes() and random_int() for generating these strings) and reference that column instead.
Also, hashids are broken by simple cryptanalysis. Their conclusion states:
The attack I have described is significantly better than a brute force attack, so from a cryptographic stand point the algorithm is considered to be broken, it is quite easy to recover the salt; making it possible for an attacker to run the encoding in either direction and invalidates property 2 for an ideal hash function.
Don't rely on it.
Quote from the site:
Do you have a question or comment that involves "security" and "hashids" in the same sentence? Don't use Hashids.
I'd use true encryption algorithm, like function openssl_encrypt (for example), or something like this. And encrypt ids when passing outside, decrypt when using in your code (like for db queries).
And I won't recommend storing ids in a base like any kind of encrypted "garbage", in my opinion its very inconvenient to hash your real ids. Keep it clean and pretty inside and encrypt for external display only.
Following your idea, you just need to cipher your IDs before writing the URL to HTML page and decipher them when processing those URLs.
If you want just security by obscurity, which is sufficient for, maybe 99% of curious people out there who likes to iterate over IDs in URLs, you use something simple like base64 or rot13. Of course, you can also precalculate those "public IDs" and store in the database, not encrypting each time the URL is being shown to end user.
If you want true security you have to encrypt them with some serious asymmetric cypher, storing both keys at your side, as you essentially talking with yourself and don't want a man-in-the-middle attack. This you will not be able to precalculate as at each encrypting there'll be different cyphertext, which is good for this cause.
In any case, you need something two-way, so if I were you I'd forget about word "hash", hashes are for purposes different from yours.
EDIT:
But the solution which every blog out there uses for this task for several years already is just to utilize URL rewriting, converting, in your case, URLs like http://example.com/book/5 to URLs like http://example.com/rework-by-37signals. This will completely eradicate any sign of database ID from your URL.
Ideologically, you will need something which will uniquely map the request URL to your database content anyway. If you hide MySQL database IDs behind any layer of URL rewriting, you'll just make this rewritten URL a new ID for the same content. All you gain is protection from enumeration attacks and maybe SEF URLs.
Related
I want to let users share data outside of their account and have put together a proof of concept that essentially generates a unique URL with relevant URL parameters to display the data.
Obviously, a normal URL would simply let you modify the parameters, modify the query and extract any data you wished. So, with this, when the user generates a link to share data, I take the parameters, add a complex salt, hash the combined string (sha-2) and then use that as a key. So the URL might look like:
mydomain.com/app/shared.php?function=form&account=1&form=a19481e78dd87f5eb04afe94c85ea4f3&key=7dcaa38baa19e0f70262d8775582300346f5c544
When the URL is entered, the server recompiles the parameters and the secret salt and validates the key. If the key is not valid, no data is displayed.
I did think about further securing this by storing parameters in a DB, so the URL looks more like mydomain.com/app/h6Hs52ff2a, and the parameters never directly included in the URL, but equally I quite like the idea of generating sharable URLs on the fly with no DB backend.
I get the sense that the above approach might be a little frowned upon, but equally, unless you know the salt stored on the server (which is itself complex) I can't see any way of bypassing such a system.
Thoughts most welcome.
That's an entirely feasible approach, essentially a signed URL. The only weakness of that system is the secrecy of the salt/key; if you're using a fast hashing/encryption algorithm and a weak salt/key, it is feasible to brute-force the secret offline. So you'll need to use a strong (read: slow) enough algorithm to prevent that (a plain SHA2 is too fast!), and you need to ensure your key doesn't leak. You also need to ensure you don't lose your key accidentally, as that would reset all shared URLs. If this is done properly, it's a nice, stateless way to do things.
I'd look into JWTs as an alternative to your homegrown method, as they basically incorporate all your requirements already (they are essentially arbitrary signed data bags).
The advantage of the database approach is that it has no attack surface, and that you're able to invalidate shared URLs selectively. The drawback is that it uses database storage, which may have operational overhead.
One more deciding factor here would be URL-length, which you may or may not care about.
If you need reassurance, it is the same concept employed by Google Drive when you share document with a link.
Some comments:
Shared link readability. If the shared data is always about the same kind of model (for example account data), the link is OK. But if other types of data can be shared, like user details, I would include it in the URL so that the person getting the shared link can make some sense of the link.
If you are concerned with adding more security, you can enforce a key per user. You would need to keep the key in the database for each user.
When generating the url, add an expiry parameter and hash. When reading hashed link, you can then see if the link expired.
Make sure to use URL safe characters
Looking at what you are trying to achieve makes me wonder, why don't you simply:
map in your database a 128 bit random value to a set form parameters every time a user wishes to share a URL. It's as secure as the random number generator you use and the form values remain a secret (as they never leave the server).
encrypt the parameters using a cipher with message authentication (authenticated encryption with modes like GCM or OCB). Secure unless your keys leak. Hides the form parameters from the eyes of the users.
use only a part of 'authenticated encryption' like a message authentication code (MAC; HMAC). Does not hide the form parameters, adds a code that needs to be verified.
What would be useful solutions for hiding true database object ID in URL for security purposes? I found that one of the solutions would be:
1) Using hashids open source project
2) Using something like same old md5 on creation of the object to generate hash and store it in database, then use it in url's and querying by them, but the drawback is that querying by auto-incremented primary keys (IDs) is faster than hashes. So I believe the possibility to hash/unhash would be better?
Also as I'm on Symfony, are there maybe bundles that I could not find or built in functionalities that would help?
Please tell me what you found useful based on your experiences.
This question has been asked a lot, with different word choice (which makes it difficult to say, "Just search for it!"). This fact prompted a blog post titled, The Comprehensive Guide to URL Parameter Encryption in PHP .
What People Want To Do Here
What People Should Do Instead
Explanation
Typically, people want short random-looking URLs. This doesn't allow you much room to encrypt then authenticate the database record ID you wish to obfuscate. Doing so would require a minimum URL length of 32 bytes (for HMAC-SHA256), which is 44 characters when encoded in base64.
A simpler strategy is to generate a random string (see random_compat for a PHP5 implementation of random_bytes() and random_int() for generating these strings) and reference that column instead.
Also, hashids are broken by simple cryptanalysis. Their conclusion states:
The attack I have described is significantly better than a brute force attack, so from a cryptographic stand point the algorithm is considered to be broken, it is quite easy to recover the salt; making it possible for an attacker to run the encoding in either direction and invalidates property 2 for an ideal hash function.
Don't rely on it.
Quote from the site:
Do you have a question or comment that involves "security" and "hashids" in the same sentence? Don't use Hashids.
I'd use true encryption algorithm, like function openssl_encrypt (for example), or something like this. And encrypt ids when passing outside, decrypt when using in your code (like for db queries).
And I won't recommend storing ids in a base like any kind of encrypted "garbage", in my opinion its very inconvenient to hash your real ids. Keep it clean and pretty inside and encrypt for external display only.
Following your idea, you just need to cipher your IDs before writing the URL to HTML page and decipher them when processing those URLs.
If you want just security by obscurity, which is sufficient for, maybe 99% of curious people out there who likes to iterate over IDs in URLs, you use something simple like base64 or rot13. Of course, you can also precalculate those "public IDs" and store in the database, not encrypting each time the URL is being shown to end user.
If you want true security you have to encrypt them with some serious asymmetric cypher, storing both keys at your side, as you essentially talking with yourself and don't want a man-in-the-middle attack. This you will not be able to precalculate as at each encrypting there'll be different cyphertext, which is good for this cause.
In any case, you need something two-way, so if I were you I'd forget about word "hash", hashes are for purposes different from yours.
EDIT:
But the solution which every blog out there uses for this task for several years already is just to utilize URL rewriting, converting, in your case, URLs like http://example.com/book/5 to URLs like http://example.com/rework-by-37signals. This will completely eradicate any sign of database ID from your URL.
Ideologically, you will need something which will uniquely map the request URL to your database content anyway. If you hide MySQL database IDs behind any layer of URL rewriting, you'll just make this rewritten URL a new ID for the same content. All you gain is protection from enumeration attacks and maybe SEF URLs.
We were making kind of a simple game,
in which:
Users receive the next number of play as an encrypted string Before they play
After they play, the encryption password is provided to them to check the play number was correct.
Each encrypted string is only valid for 1-2 hours and number of play , verificating string and encrypted string is regenerated again after that time
The encrypted string includes a verification (5 char) code so both users and we can make sure Decryption process was successful
Sample Character to get Encrypted (QQ9LU is random verification code provided to user before the play):
Next Play Number: 8 - Verify String: QQ9LU
Sample Encrypted String (provided to user before play):
NXRykKOv3B6kuu4Ke3svp7HH3enNiqIZrJSXJiF54QkHHjtXgqpUXxyuP7YUNICeFLg==
Sample Password (provided after play):
Please note this is generated randomly for each encryption
FA00RDjA77hlOzcOzH6kuGcc29CyM7Hw
We use CodeIgniter 2.2.2 Encryption Class to encrypt/decrypt strings
Encryption Method Info:
Function Used: $this->encrypt->encode($msg, $pass); with random pass each time
Cipher is CodeIgniter 2 default MCRYPT_RIJNDAEL_256
Mcrypt mode is MCRYPT_MODE_CBC
My Questions are:
Can i trust that users cannot break the encrypted string (and know the number of play before they get the password) in 1-2 hours (aside from getting lucky)
Is placing random verification code Verify String: T3YH4 in there good or bad? does is affect security? (this is to verify decryption result was successful, also we added it because the only variable in each string was a single digit, for example only number 8 changes to 7, so we wanted to add more variable characters to the string to possibly have a better security)
Any other suggestion is appreciated
Short answers:
From a technical POV, what you're doing is unsafe, although it might be enough for just a 2-hour timeframe.
What you're trying to do here is called "message authentication", but that's not how it should be done, which in turn does impact security. You should use a HMAC instead.
My advice would be to upgrade to CodeIgniter 3 (CI2 will stop receiving even security updates in a few months) as soon as possible, and use its new Encryption library instead. That will make it safe for years, not hours.
Long answer:
The encryption library should do both encryption and authentication for you, but unfortunately the CI_Encrypt class itself is badly written and lacking a lot of functionality (such as authentication), which is why it was DEPRECATED and is being replaced by a new (CI_Encryption) library in CodeIgniter 3.
Explaining all the flaws in here would be quite the task, so I'd rather link you to an external article (not self-promoting, don't worry), which does that quite nicely if you're interested in the low-level details.
No matter which library you use however, one thing must be noted - a password is not the same thing as an encryption key.
Passwords have a varying length and are used by humans, which means that they must be readable by humans, and that in turn limits them to a certain set of characters.
Encryption keys on the other hand have a fixed length (each encryption algorithm is designed to work with a specific key length; for Rijndael-256 that's 32 bytes, which you seem to match) and are not limited to human-readable characters (which means more entropy and therefore more security) - they represent raw binary data.
Anything else can be controlled (and therefore automatically done) by a library, but if you pass a password instead of a key - that's what the library will use, so you should take care of that.
The best and simple way to do that is to use the filesystem functions to create a simple text file for each user in non public path with two lines, the first of them is a unique random string (long string varied in length) and the second is the number.
Then using sha1_file get the hash value of the file then store it in the database related to its path and creating time, then send this hash to the user.
After the user has played, check the value by another script that get the value of the hash from the database, then read the file and parse its second line to display the number.
By this way, you have gave the user a hash not for a string, but it for a file and cracking it to get the file back is not simple as to be done in two hours.
You are giving your Encryption/Decryption logic to client side. Hacker will easily identify how your password and encryption strings are being match.
Many framework have their own password creationg and compare logics. Yii using SALT and other features like SHA1 etc...
Keep it simple and keep all things at your end. Generate your encryption things and store at your end. Follow simple steps,
Generate encryption password (using SALT and/or other encryption tools) and store at your end
Ask client (user) to enter their password (key) and get at server side
Convert your password (key) to encryption password and compare
CPasswordHelper will be helpful for you. Try to download Yii source code and put out their logic for you.
Hope that helps !!
Sounds like a fun game!
I am assuming you are creating these strings in files on a filesystem. If you were hosting them on some web application that would assume different techniques to break the string.
Adding a code to the end of the string is called salting the string. While this makes the string harder to guess, if you are adding a hardcoded salt instead of a randomly generated salt it can still be easily broken by brute force methods.
I would try using a one-way hashed string for the password and storing that in a database. The user is unable to decrypt the string and has to just provide a matching password to gain access to your string. It is possible for programs to break one-way hashed strings but I find it unlikely someone will be smart enough to do that if they are in college and only have two hours. It takes alot of domain knowledge and experience to start generating one-way hashed strings to brute force it.
In addition you are probably safe with the method you are doing currently, students will not likely be able to break a string in 2 hours unless they are familiar with advanced encryption hacking scripts that take some work to find. I am guessing they will do trial and error, using different decryption libraries similar to the example you provide and hoping they get lucky with the library of strings they are trying to match against yours.
Also information is important with any type of encryption. Telling someone you are adding a 5 code salt to your string, will give them some insight into how your encryption algorithm works. They can then try methods of breaking it based on the information you give them. Try the same thing with your own algorithm and leave the students in the dark, I doubt anyone will break anything in the time alotted. Alot of hacking techniques involve going through an information gathering process where the hacker scopes out or maps a system before trying to attack it.
My PHP Application uses URLs like these:
http://domain.com/userid/120
http://domain.com/userid/121
The keys and the end of the URL are basically the primary key of the MySQL database table.
I don't want this increasing number to be public and I also don't want that someone will be able to crawl the user profiles just by interating the Id.
So I want to encrypt this Id for display in a way I can easily decrypt it again. The string shouldn't get much longer.
What's the best encryption method for this?
Simple Obscuring: Base64 encode them using base64_encode.
Now, your http://domain.com/userid/121 becomes: http://domain.com/userid/MTIx
Want more, do it again, add some letters around it.
Tough Obscuring: Use any encryption method using MCrypt library.
A better approach (from a usability and SEO perspective) would be to use a unique phrase rather than an obscured ID. In this instance the user's user name would seem an ideal solution, and would also be un-guessable.
That said, if you don't want to use this approach you could just use a hash (perhaps md5) of the user's user name which you'd store in the database along with their other details. As such, you can just do a direct lookup on that field. (i.e.: Having encrypt and decrypt part of the URL is probably overkill.)
You have a variety of choices here:
Generate and store an identifier in the database. It's good because you can then have readable keys that are guaranteed to be unique. It's bad because it causes a database schema change, and you have to actually query that table every time you want to generate a link.
Run an actual key-based encryption, for instance based on PHP's MCrypt. You have access to powerful cryptographic algorithms, but most secure algorithms tend to output strings that are much longer than what you expect. XOR does what you want, but it does not prevent accessing sequential values (and the key is pretty simple to determine, given the a priori knowledge about the numbers).
Run a hash-based verification: instead of using 121 as your identifier, use 121-a34df6 where a34df6 are the first six characters of the md5 (or other HMAC) of 121 and a secret key. Instead of decoding, you extract the 121 and recompute the six characters, to see if they match what the user sent. This does not hide the 121 (it's still right there before the hyphen) but without knowing the secret key, the visitor will not be able to generate the six characters to actually view the document numbered 121.
Use XOR with shuffling: shuffle the bits in the 30-bit identifier, then apply the XOR. This makes the XOR harder to identify because the shuffle pattern is also hidden.
Use XOR with on-demand keys: use fb37cde4-37b3 as your key, where the first part is the XOR of 121 and md5('37b3'.SECRET) (or another way of generating an XOR key based on 37b3 and a secret).
Don't use base64, it's easy to reverse engineer: if MTIx is 121, then MTIy is 122 ...
Ultimately, you will have to accept that your solution will not be secure: not only is it possible for users to leak valid urls (through their browser history, HTTP referer, or posting them on Twitter), but your requirement that the identifier fits in a small number of characters means a brute-force attack is possible (and becomes easier as you start having more documents).
Simplest but powerful encryption method: XOR with a secret Key. http://en.wikipedia.org/wiki/XOR_cipher
No practical performance degradation.
Base64 representation is not an encryption! It's another way to say the same.
Hope this helps.
Obscuring the URL will never secure it. It makes it harder to read, but not much harder to manipulate. You could use a hexadecimal number representation or something like that to obscure it. Those who can read hex can change your URL in a few seconds, anyway:
$hexId = dechex($id); // to hex
$id = hexdec($hexId); // from hex
I'd probably say it's better indeed to just create a random string for each user and store that in your database than to get one using hash. If you use a common hash, it's still very easy to iterate over all pages ;-)
I would write this in comments, but don't have the rep for it (yet?).
When user click on a link you should not use primary key, You can use the pkey in a session and get it from that session. Please do not use query string....
generate an unique string for each user and use it in your urls
http://domain.com/user/ofisdoifsdlfkjsdlfkj instead of http://domain.com/userid/121
you can use base64_encode and base64_decode function for encrypt and decrypt your URLS
I am producing a script that others will put in their websites. It is designed for people with limited knowledge of PHP so that all they have to do is include() the script and set a few configuration variables. This means headers will probably have been sent already and so using sessions may not work. I recommend they call session_start in their own scripts, but I want a fallback option as well.
I already have a hidden input to deal with forms, but I also need links to have a query string appended to their URIs to identify the session. But if the hash is based only on the password+salt then there is a security risk: the logged-in user could click an external link and the owner of the external site could see the URI with the hash in their referrer logs. All they'd have to do is used that hash and they'd be logged in.
Therefore I want to salt the hash in a time-sensitive manner, limiting a session to 10 minutes. I can't figure out how to do this. Of course I can use time() to salt it, but how do I check how old the session is based only on the hash?
Expiring sessions after 10 minutes does not protect your users against session hijacking attacks. It only succeeds in annoying your users by forcing a login every 10 minutes. Your proposed scheme still has all kinds of vulnerabilities. Your salted hashed passwords can still leak to the outside world through many other channels; packet sniffing, intermediate proxies, users' emailing each other links to pages or even the saved html, just to name a few. I advise you not to homegrow you're own security framework without being an expert in the area. Even then, this is a solved problem. Just go with a known trusted solution. There are many subtleties in web security that are easy to mess up.
This sounds like a real bad idea. And complicated too. I would definitely not recommend it. You probably can use something like this:
if (session_id() == "") session_start();
The above will basically check if session has been started or not, and otherwise start session.
Since you planning to distribute to users, the whole approach seems a bit off to me. Am not sure what you are looking to achieve out of this, but you could try using a JS which calls your PHP file per page instead. This will make it easier for you. If you could elaborate on what kind of application you are developing, then I could probably help you out better. I have a lot of experience in mass consumer software apps similar to what you are doing.
This is not necessarily a bad idea, but it is dangerous if not done correctly. In fact, it is a fairly common implementation of multi-domain single sign-on using Hash-based Message Authentication Codes. Some ground rules:
Never ever include the password as part of the hash (even as the salt)
Require a timestamp generation as part of the hash that must be passed ALONG with the hash.
Each site to use this hash should have their own 32 or 64 byte guid to be used as a unique salt.
Pass specific data in the query string such as username, timestamp, anything else, and the HMAC. So it would look something like ?user=steve×tamp=66343532233&otherdata=otherdata&HMAC=AB3445-1234144-AFBBDEDD (you get the idea)
When site authentication is made cross-site, the HTTP_REFERER should be used (if possible) to get the key to generate the comparing HMAC.
Use a solid hashing algorithm (SHA1 is preferred) for generating the HMAC. Generate the the private site keys as randomly as possible. Do not use a standard derivation method, simply make sure that the end result is large enough/unique enough.