This is theoretical question but I am curious about it. What if I do this (code in PHP, but the language isn't really matter in this case):
$value = ''; //starting value
$repeat = false;
while(true)
{
$value = md5($value);
/*Save values in database, one row per value*/
/*Check for repeated hash value in db, and set $repeat flag true if there is one*/
if($repeat)break;
}
As you can see I suspect that there will be repeated hash values. I think there is no way that every existing text has its own value as it should mean that every hash value has its own and that doesn't make sense.
My questions are: Is there any article about this "problem" out there? It can happen I got the same value in one system for example when I hash files for check if they are valid? Can this caused problems anywhere in any system?
If you care about multiple texts hashing to the same value, don't use MD5. MD5 has fast collision attacks, which violated the property you want. Use SHA-2 instead.
When using a secure hash function, collisions for 128 hashes are extremely difficult to find, and by that I mean that I know of no case where it happened. But if you want to avoid that chance, simply use 256 bit hashes. Then finding a collision using brute-force is beyond the computational power of all humanity for now. In particular there is no known message pair for which SHA-256(m1) == SHA-256(m2) with m1 != m2.
You're right that hashed can't be unique(See Pidgeonhole principle), but the chances of you actually finding such a case are extremely low. So don't bother with handling that case.
I typically aim for a 128 bit security level, so when I need a collision free hash function, I use a 256 bit hash function, such as SHA-256.
With your hash chain you won't find a collision, unless you're willing to wait for a long time. Collisions become likely once you have around 2^(n/2) times, which is 2^64 in the case of 128 bit hashes such as md5. I know of no brute-force collisions against a 128 bit hash. The only collisions I know are carefully crafted messages that exploit weaknesses in the hashing scheme you use (those exist against md5).
Hash it multiple times by same method or different method, Then it would be nearly impossible to repeat its self, Also check if they repeat then repeat the hash function until the values are different, Then save in database or use it where ever you like...
Related
So I know that MD5's are technically a no-no in new applications, but I randomly had a thought of this:
Since
md5($password);
is insecure, wouldn't
md5(md5($password))
be a better alternative?
would it keep getting more secure the more I use it? Say if I made a function like this
function ExtremeEncrypt($password)
{
$encryptedpass = md5(sha1(md5(md5($pass))));
return $encryptedpass;
}
Would this function be a good alternative to say using a random salt for every account like vbulletin does.
Double hashing a string does nothing except limit your key space and make collisions more likely. Please don't do this. Double md5 hashing is actually less secure than a single hash with some attack vectors.
A better option would be to use the password_hash function in php 5.5 or ircmaxell's password_compat library for earlier php versions.
First of: hash and encryption are not the same. Hash is a one-way function while encryption expects data could be decrypted.
You should not try to invent your own solution when it comes to security. In PHP, since 5.5 version, there is native solution called Password Hashing. md5() is insecure and you should be aware of that.
If you have PHP below 5.5 version, you should use salt to hash & store your passwords.
You have lots of answers here and they are accurate but they don't really explain why.
MD5 is a hashing algorithm. What a Hashing algorithm does, is take a long piece of data and analyse it cryptographically in a way that creates a smaller piece of data. So from ABCDEFGHIJKLMNOPQRSTUVWXYZ with my custom hash algorithm I might create a single digit hash 5.
When that is done, you lose information - ABCDEFGHIJKLMNOPQRSTUVWXYZ contains far more information than 5 and there is no way to make the translation the other way.
The problem with hashing in a way that only allows an outcome of 0-9 ( this is effectively a Checksum ) is that if you take two pieces of text, the chances are quite high that they will have the same hash. So maybe with my algorithm ZZZZZZZZZ will also produce a hash of 5. This is what is termed a Hash Collision.
Now what happens if I take the hash of my hash? Well, my starting point is already very low information - the most it can possibly be is one of ten digits, so the chance of a collision is now exceedingly high. Supposing when my hash algorithm runs on numbers it returns 1 if it is odd and 0 if it is even- so if I have a hash of ABCDEFGHIJKLMNOPQRSTUVWXYZ which comes to 5 then I have a 10% chance of a collision. But if I make a hash of that hash, I will now have a 50% chance of a collision.
The trick of cryptography is hiding information in such an enormous possible space that it is unbelievably hard to find. The more you shrink that possible space, the less well hidden your information is.
Short answer: No.
md5 is easy to break using brute-force. Adding additional layers of hashing only slows down a brute-force attack linearly.
First of all md5 isn't really encryption, because there isn't a decryption method to it. It's called hashing.
The standard practice is to salt your passwords:
$salt = [some random/unique number, people usually use user_id or timestamp]
$hashed_password = sha1($salt . $password)
Remember that you need to know the salt, hence usually it means storing it along with the hashed password.
You can have multiple salts, and arrange them however you like.
Suppose i assume if hash collision occur while i am using sha1 function in php .
Will this code avoids it permanently or do i have to use any other way
$filename=sha1($filename.'|'.microtime());
OR
$filename=sha1($filename.'|'.rand());
If no this code doesn't provide protection from hash collision .
What should i do to avoid any type of hash collision if i assume there can be more than 100,000 entries in db.
Its very unlikely that a hash collision will happen for sha1.
Probability of sha1 collision is negligible
And hash collision risk is not practical. No one has found sha1 collision till yet . So you are safe to use it.
Using a salt like microtime or random number may decreases the chances of probability but you simply can't avoid it.
And what you are using is sha1(string) whether string is a mixed value or single string.so using microtime and rand function wont affect anything to probability of hash collision.
Therefore there can be possibility that sha1(mixedvalue) collision might be equal or greater than collision of sha1(filename) so certainly that is of no use.
So dont worry and use this or simple way if you like to, it wont create problem in future, Thinking about hash collision is waste of time when the chances are very very very less.
Just to be clear, you can't completely avoid hash collisions. It's an infinite number of inputs to a finite number of outputs, but you can take into account things like the file's size, the current system time and other data to use as a salt which will increase the entropy of your message digests.
Just sha1() the entire file path, not only the file name.
Filename xy.png can be only one in a directory, therefore your hash will be unique for that filename.
Also, this has the advantage that you will not have duplicate files (while with rand()/microtime() you can get same file 10 times in same dir, and if it's a 1GB file can cause problems)
Neither of these avoid hash collision.
Hash collisions happen when you have an algorithm that generates a hash of a specific size, regardless of the starting value.
A hash collision is when two different values, like "mypassword" and "dsjakfuiUIs2kh-1jlks" end up generating the same hash because of the mathematical operations performed on them.
You can't write code to prevent hash collisions, how often that happens is dependent on the hashing algorithm you are using.
I know MD5's safety is under question lately and this is the reason a lot of people are using salt (I dont understand this at all btw) but I was wondering if you wanted to easily implement a safe system in php can you just md5 something twice?
like test > 098f6bcd4621d373cade4e832627b4f6 > fb469d7ef430b0baf0cab6c436e70375
So basically:
$val = 'test';
$val = md5($val);
$val = md5($val);
Would that solve the whole rainbow security stuff? Is there an easy/noob proof way of making secure database passwords in php?
Hashing twice makes little real sense and doesn't accomplish much. In general, however, multiple hashing can make some sense. For example, if you hash enough times to take something like 100 ms (or so, depending on hardware) it can help a little. The basic idea of it is pretty simple: adding 100 ms to a normal login is a barely noticeable delay -- but if you're trying to build something like a table for a dictionary attack, multiplying the time by something like a thousand (or whatever exactly it works out to) starts to make a real difference -- a table that you could normally compute in (say) a day, takes a few years instead. That's enough difference that anything but really serious attackers will often give up (or just get bored) long before they finish the job.
Salt is an entirely separate tool. Using it does not make up for weakness in the underlying hash function. The idea here is that the size of a table for a dictionary attack becomes substantially larger (e.g., for a one-byte salt, 256 times larger). The salt is not normally kept secret, but it's relatively random, so an attacker who's doing a dictionary attack can't just hash each word as-is, but has to take each possible salt value into account. At the risk of repetition: it deals with a weakness in how (most) people pick passwords, not any weakness in the hash function itself.
If you don't believe in MD5, you can try a higher algorithm by using the hash() function:
$hash1 = hash('sha1', 'The string to hash by SHA-1');
$hash2 = hash('sha256', 'The string to hash by SHA-256');
$hash3 = hash('sha512', 'The string to hash by SHA-512');
$hash4 = hash('ripemd160', 'The string to hash by RIPEMD-160');
In my opinion it does not make sense to hash twice.
EDIT: Fixed typo in last line of code.
Whether or not you use the MD5 algorithm...
No, an attacker can always have two rainbow tables (one for the extra level of hashes, and one for the passwords). And from another answer of mine:
[...] it still just requires the password and nothing more to crack. In other words, you are just applying the hashing functions to the same thing a few times more.
You use a salt to make it more difficult for the attacker to get at your passwords, because then he would need to know the salt so that he can use it in computing the hashes for your passwords.
Storing passwords securely is tricky, most the advice posted here is not accurate. So I will defer to Thomas Ptacek's widely cited post on the subject: http://chargen.matasano.com/chargen/2007/9/7/enough-with-the-rainbow-tables-what-you-need-to-know-about-s.html
For the record, I evaluated that
$val = 'test';
$salt='somerandom!!aa##9900';
$val = md5($salt.$val);
$val = md5($val);
Its pretty safe. The secret is in the salt.
However, md5 is short so the chances of concurrences are "high" (one in 1.208.925.819.614.629.174.706.176 = 32^16, 32 words with an hexadecimal each one)
I have used unsalted md5/sha1 for long time, but as this method isn't really secure (and is getting even less secure as time goes by) I decided to switch to a salted sha512. Furthermore I want to slow the generation of the hash down by using many iterations (e.g. 100).
My question is whether I should append the salt on every iteration or only once at the beginning. Here are the two possible codes:
Append every time:
// some nice big salt
$salt = hash($algorithm, $salt);
// apply $algorithm $runs times for slowdown
while ($runs--) {
$string = hash($algorithm, $string . $salt, $raw);
}
return $string;
Append once:
// add some nice big salt
$string .= hash($algorithm, $salt);
// apply $algorithm $runs times for slowdown
while ($runs--) {
$string = hash($algorithm, $string, $raw);
}
return $string;
I first wanted to use the second version (append once) but then found some scripts appending the salt every time.
So, I wonder whether adding it every time adds some strength to the hash. For example, would it be possible that an attacker found some clever way to create a 100timesSha512 function which were way faster than simply executing sha512 100 times?
In short: Yes. Go with the first example... The hash function can lose entropy if feed back to itself without adding the original data (I can't seem to find a reference now, I'll keep looking).
And for the record, I am in support of hashing multiple times.
A hash that takes 500 ms to generate is not too slow for your server (considering that generating hashes are typically not done the vast majority of requests). However a hash that takes that long will significantly increase the time it will take to generate a rainbow table...
Yes, it does expose a DOS vulnerability, but it also prevents brute force attacks (or at least makes them prohibitively slow). There is absolutely a tradeoff, but to some the benefits exceed the risks...
A reference (more like an overview) to the entire process: Key Strengthening
As for the degenerating collisions, the only source I could find so far is this discussion...
And some more discussion on the topic:
HEKS Proposal
SecurityFocus blog on hashing
A paper on Oracle's Password Hashing Algorithms
And a few more links:
PBKDF2 on WikiPedia
PBKDF2 Standard
A email thread that's applicable
Just Hashing Is Far From Enough Blog Post
There are tons of results. If you want more, Google hash stretching... There's tons of good information out there...
In addition to re-hashing it multiple times, I would use a different salt for each password/user. Though I think 5000 iterations is a bit too much, try a lower number. There's a trade-off here; you'll have to tweak it according to your needs and hardware.
With different salts for each password, an attacker would be forced to bruteforce each password individually instead of constructing a rainbow table, which increases the workload considerably.
As always, here's a recommended read for this: Just hashing is far from enough
EDIT: Iterative hashing is a perfectly valid tactic. There are trade-offs, but everything has them. If you are worried about computation time, why not just store the plaintext password?
Please please please do not roll your own crypto. This is what libraries like OpenSSL are for. Here's few good examples of how you would use it to make salted hashes.
Salted Hashes in OpenSSL
The reason for iterative hashing is to make process as slow as possible. So you can do even better: use different salts for each iteration. It can be done by encrypting you original data again and again on each iteration with fixed key and XORing with salt value.
I prefer to go with a double sha1 with two different salts and prevent DoS delaying the answer incrementally (with a simple usleep) for every invalid password check.
I am going to generate a random password for my users. Probably I will use salt and sha1 method. But I found that there are other version of sha1, like sha256, sha512 etc.
What is the differences between the sha versions? Which one better, more secure, and faster (performances)? Which one should I use??
EDIT:
I am php user, thanks :)
Password generation is not as straightforward as you may think. For instance, if you're generating a password for a site which can be accessed from a mobile phone (or any device with only a numerical keyboard) you should ensure that no two consecutive characters in your password are on the same key since that would be a huge UI flaw.
An example: consider dfe4Pl7 as a password. In order to type it in, a used should press 3 once, than wait, then press 3 three times, then wait again and press '3' two more times. This sucks.
In more general terms, password should be generated so that misinterpretation is avoided as much as possible. For example, I'd personally avoid including 1's and l's, 0's and O's, etc.
The bottomline is: hashing algorithms will be of a very little help here. What you need is a good random-number generator and a UI-wise solid generation algorithm.
You are confusing two issues here. Random password generation, which is basically picking random symbols of a certain (minimum) length, and password storage, which in practical terms involve storing a hash of the password with a salt so that the plaintext password can't be discover just by reading the database.
Generation
To pick random symbols, you have an array of all the symbols, get a random number and subscript into the array using that number as index to return a symbol. This is what Kane Wallmann suggested earlier. However, for security related application such as password generation, one need a cryptographically strong pseudo-random number generation, which rand() is not.
Storage
As noted earlier, password shouldn't be stored in plaintext in the database. Otherwise, anyone with access to it can just read it and know what the password of any and all particular user password are. What happen is a hash of the password is store instead. When a user attempt to login, their password is hash and compare to the value in the database to see if it's the same. The hashing function is what's call a one-way function. You can hash the password to get a hashed value, but you can't get the password from the hash value without trying all possible combination to see if it matches. (At least, that's the idea anyway.) It's more complicated than that of course, since hash value output is fixed length, whereas the possible input while in practical terms isn't infinite is definitely more than the possible number of output.
Now, where salt comes in is because simple hashing of the password alone isn't secure either. For one, such a method would give the same hashed value for any two users with the same password. Compromise of one account would result in the compromise of the other. Secondly, what an attacker can do is to build what is called a rainbow table ahead of time. While this take time, it only have to be done once for any storage algorithm, and he or she don't have to do it themselves. The work can be spilt over many computers, and there's in fact websites on the internet where you can download such rainbow tables for weak password hashing system such as LM and NTLM. After that, the attacker can just look up any particular hash value against the table and determine the plaintext password. So to protect against that, a random (per user) salt value is added to the password before it is hashed. This makes the input different even for the same password, so prevent the first problem. It also mitigate against the second problem if the salt & password combined is long enough as the length of the input is such that it would become computationally infeasible to brute force it.
As to the question of which particular SHA, or for that matter, other hashing algorithms to use. SHA is a US NIST standard, and are acknowledged to be pretty good. There's been a little theoretical breakthrough into SHA-1, but in practice it's still secure enough for most purpose. The SHA-2 algorithms are better than SHA-1, with no known breakthrough. Which variant to choose are down to various things including size. They produce different length output, and different amount to calculate because of the size difference. PHP have native implementation of SHA-1, SHA-256, 384, and 512, among a number of others hashing algorithms.
After all that, in practice, which of the SHA algorithms you choose probably doesn't matter as the weak point in your system are likely elsewhere. Users writing down their passwords. Users using the same password across different systems. Programming flaw that allow things like XSS and SQL injection. Etc. etc.
Encryption algorithm security doesn't mean anything if you are simply using them to make random letters and numbers for a password. You are better off making a random password function, it would be MUCH more efficient.... Why are you salting anyway?
Here is a SIMPLE password generator written in PHP.
function RandomPassword( $length )
{
$characters = "abcdefghjklmnpqrstuvwxyz23456789";
$result = "";
for( $i=0; $i < $length; $i++ )
{
$result .= $characters[ rand( 0, strlen( $characters )-1 ) ];
}
return $result;
}
NOTE: i,1,0 and o are intentionally missing as they are easily mistaken as each other.
here's the wiki you can read about sha
there it talks about sha-2 family being algorithmically similar, however sha-1 is more popular.