I'm trying to safely store email addresses in a database, but need to check if a specific email address is already in that database. My current encrypting with openssl_encrypt (I used the example on https://www.php.net/manual/en/function.openssl-encrypt.php) will output a different output every time I refresh the page. That way I cannot check if the given email address (stored as encrypted string in the database) is already in that database; I don't want to decrypt them all and use a foreach loop to check existence.
Is there a secure way of storing the email addresses, but where I'm able to quickly check the existance? Thanks!
Edit:
I think a bit more context is needed. I added the underneath additionaly:
MD5 is not secure for a while now. If my database is hacked, I don't want those email addresses saved in a unsecure hash method.
The application is to register yourself when a product is not in stock. When the product is back in stock, all the registered persons should receive an email. So I do need to decrypt the email addresses again to send the email to.
Now the issue is that when a person is already registered, I want to prevent registering twice and give feedback that they already registered. To prevent decrypting all registered email addresses and then check if one of those is trying to register, I want to do something like "select * from table where emailhashed = 'HashedEmail'". With openssl-encrypt I get different HashedEmail outputs so that doesn't work. The result will be zero and the registering continues resulting in a duplicate (triple, etcetera) registration.
I'm making a website where you can create an account. As part of the sign up process, I want to send an activation email to the users email address. I was thinking that this email would include a link that the users clicks on which will activate the account, the problem is I'm not sure how I would go about creating the link and how it would work. How would I dynamically generate this link and how would it activate the account?
Thanks
When you are preparing to send the email, after creating the account, you will have to add into your database a record to validate that account, which should include the ID for that account and an activation string (I would generate a 10 to 15 character random hex or alphanumeric string). This record should also have a boolean true or false whether it's been activated or a numeric (or Binary) 1 or 0. Set this to false before sending the email. Send this activation string and account ID in a link in the email.
For any links that you create, you're going to want a single page for them to go to, i.e. activateaccount.php
From inside that file, you can receive a $_GET parameter, maybe $_GET['activation'] and $_GET['account_id']. From this page, you will validate that the activation and account_id are both valid and that they match each other. If not, then the script should NOT attempt to fix the mismatch by writing to the database, as this could allow undesired attacks. If they are both valid and match, then the account may be activated.
Also as Pingbeat suggested, have a go at Swiftmailer. I have used it before in many projects and can confirm that it is extremely versatile, fast, very well documented and incredibly easy to use.
I hope this suggestion helps.
I'm writing a web application and security is very important to me. When writing the 'forgot password' function I have taken the following advises into account:
Generate secure random auth token
Send user an email with this token
Send user an SMS with a pin code to enter in combination with the email link
Do not leak account info, the site will not mention an unknown email address when the user enters an unknown address. This way accounts cannot be linked to the webapp.
But this made me thinking. The website requires the user email address to be unique. Therefore the register form notices the user when the email already exists.
This means my attempts to not leak account existence information to an attacker in the 'password recovery' part are useless because I leak the same information in the register part of the site.
I have been looking at ways to prevent this and best practices around this, but not much information is written about it.
I have also noticed the same issue in the stackoverflow account system. Stackoverflow does not leak this information in the 'forgot password' function but it does leak account existance information in the registration process.
Can anyone point me in the right direction on this one?
Kind regards,
Daan
Update
A few minutes after typing this I had the following idea on how not to leak information to an attacker:
Do not present the new user with the message 'email address is in use', but accept the registration as if it would be a new one.
When the email address already exists, just send an email with the message there is already an account for that email address.
What do you think?
if you do password recovery, always send an alert that the email has been send when the email is legit, but only send when he is in your database.
In the registration part it's required to say if the e-mail exists in the database, so leakage is possible but for people to guess e-mail addresses without knowing them they need to bruteforce it which can be avoided by limiting the amount of posts the users can do to prevent spamming
There is always the possibility to move towards Login Names for logging in. You don't quite need your email address to be unique unless that is the sole method of logging in.
The answer her would just be more obfuscation if you realistically want to be more secure. What you could have:
Login name (the actual unique identifier to login to your web application, has to be unique)
Display name (the name visible to other users, does not have to be unique)
This could make guessing login names more difficult, and it would be slightly more secure. Email addresses would not need to be unique necessarily. But then the user must always keep track of what login name they signed up with. This process could ostensibly hide this information from attackers. Since both login and password recovery would ask for the username only.
I don't think this problem has a great answer. Like Joachim pointed out, you are always prone to leaking when you require the email to be unique. Since checking has to be done upon registration. Password recovery can be managed simply by always giving the message "Instructions have been sent to the email you provided" whether or not the email exists in your database. And failure or success would be handled internally but never be shown. Best way to make timing attacks impossible is defer the whole sending process to be asynchronous from your HTTP response.
So options are either, move to a login name system (with the same security holes ostensibly) or just accept that with unique email addresses you will leak during the registration process.
There might not be a best answer here. Part of security is getting users to be clever. And signing up with just an email address that is not really public, tends to help. But if an attacker has access to their sign up email, things are out of your control. And there is not much you can do (unless you have two step auth with SMS or something)
I am struggling these days to prevent spam from my signup form. I do not want just to prevent it from bots (with honeypots etc) but when a real human writes a script designed for my website to fill my database with dummy registrations(i do not want to use captcha). I have the following things in my mind to implement about it:
Check if email addresses exist (not only valid).I have read that you may be banned if there are lots of requests.Moreover it is possible that the script can contain valid email addresses (for instance when a university provides students email that are slightly different).
The other solution is to make a comparison between IP/Time_of_request and in case the same IP gives lots of requests for signup, consider user as spammer. For this you can set a threshold that you can consider signup request as spam. The problem here is that the script may find the threshold (e.g. 1 second ) and send request every 1.1 seconds. Moreover someone may use onion routing(?) and i will not be able to ban him.
3.What do you think about random input names ?
So what practices here are considered to be good enough to come through this situation?
Thanks!
UPDATE
I send email with confirmation link for activation,but i give users the feature to use the site for about 4 days without activating their accounts! I do not want in first step my database to have spam accounts!
SOLUTION
For everyone interested in, i used honeypots combined with a temporary database ! It seems to work fine!Thanks!
One more thing you can create a temporary registration database, and if someone verify email in 24 hours of registration, his/her data will moved to main registration database. and every entry will be deleted in 24 hours if email is not verified by user.
To validate the email address you could send a confirmation email with a validation link and the user should click on that link to confirm that is his email address and is not a boot.
In Response to OP Update: If you need to give users the option of using the site without clicking the activation link for X days, perhaps you could also send a set of two 4-digit (or just 6, but not secure) PIN numbers in the email (or separately) and have them use that as their temporary password until the account is activated via the original email link. In your database you'd notice if the PIN was used or not, indicating if it was a spam account. It could even be a one-time-use PIN.
Most of the examples I see on the web create user accounts in this sequence: user comes to the site, they choose a username and password and enter their email. A confirmation email to sent to this email and if they click the link, the account gets "verified". If they don't verify, the account gets deleted after a while.
I was told about another way: get the user to verify the email first, and when they click the verification link in their email they can start to create a username and password.
Does anyone see any problems with the second way, whether a security concern or anything else? It's not common and I personally cannot find a totally obvious problem with it, but I'd prefer to use it only after many people confirm they don't see problems or loopholes with it either.
Personally I do see an issue that can be inconveniencing for the user:
When most people register with a web site, they expect that they will have to answer quite a few questions, spend some time reading the FAQ and the terms of service and then spend some more time setting up some preliminary aspects of their profile.
The traditional flow allows the user to choose the time to go through that process. Afterwards, the user only receives a verification link, which normally is a 3-second process to use and can be done at practically any time.
Your proposed flow forces the potential user of your site to spend time reading your documentation, then wait until they receive the message and then find some more time, potentially after a few days, to fill in the forms. I, for one, would find that at least slightly annoying - if not outright discouraging - especially if the mail takes its sweet time to arrive, as it's often bound to do.
I also don't like the inherent implication of such a scheme:
Traditional flow: "Oh nice, you filled in our forms, just give us an address to send you a proper verification". The user here is merely waiting to complete what is essentially a done deal.
Mail-first flow: "Oh it's you. Well, wait for a while and we will send you an invitation if we want you". Here, on the other hand, the user is left in a limbo of subconscious uncertainty until they receive your message.
I believe that the first approach is far more open and friendly to the user. It's also the current standard flow for these cases, which should be enough of an incentive to use on its own - you should avoid forcing your users into processes they are not used to, unless there is no other way.
Getting an email from a friend with an invite link to access a site is exciting - it feels exclusive and new and fun. I'm being given something - so I gladly sign up.
Being required to enter an email address in order to start using a site feels draconian and restrictive and annoying. I'm being asked to give something up as the first step then possibly (maybe?) get something of value down the road.
It's not logical - in both cases, my email address is must be verified before an account can be created. In fact, the first case requires my friend to actively SPAM me with an offer I never requested.
Do you know why I first created this StackOverflow account? Because when I wanted to contribute an answer I could click on the Google logo on the login page and start using the site immediately. No username, password, first name, last name, DOB, or other B.S.
Do you know why I never created an Experts Exchange account? Because the first time I tried to access an answer I was prompted to enter a credit card number, billing address and phone number. Before I could even sample what the site had to offer, I had to give something up.
The point is this: barriers to entry make your site suck. Account creation should be as seamless and painless as possible. Being able to access a site immediately after filling out a single-page signup form and a CAPTCHA is awesome, even if access to other features is restricted until email verification is completed. Maybe I'll even tell you my DOB and favorite color if it unlocks more features.
Personally I don't see a problem with it - its a matter of choice. I think the key point though is making it clear to the user that they must
1) enter their email address
2) wait for a confirmation email before they can get to step 3
3) sign up for the account.
It potentially removes the amount of data held and time invested by the user if they only have to enter a single piece of information (their email address) before filling in the rest of the information you require.
Personally, I'd keep it standard so users don't get confused. The amount of work is the same - get a username/password/email address - wait for users to click the link before they can login to your site.
So how many times would you allow to use link send in email?
If only once, user can't create an account if he close browser before selecting username.
If multiple times, a lot of people can create accounts using same link. Publishing this link and using password recovery feature can be nice phishing trick.
And if you check for this email in your database and allow it only once, user would not be able to create two legitimate accounts.
I could see this method being slightly simpler - when the user clicks the verify link in their email, you send them to a form with a hidden pre-generated id number inserted, and then assign a username and password to it afterwards. Blank accounts, with just and id and no other information, are easy to periodically filter out and you're not storing any details whatsoever until the account is successfully created.
However, there's probably a reason why most sites collect username and password before email - you're getting a user invested before you ask for a more personal bit of information. The account is created - now just verify your email. The other way around ask for an email address first and an account second - even though functionally it's the same, perceptually it's not. Also, the advantage of the standard "flow" is that users know what to expect - following conventions mean users feel like they know what's happening and don't get confused or lose interest.
I want to share some thoughts about second approach.
First of all, it is very similar to invite system, but IT IS NOT the same.
You have to allow to send more than one registration request for a single e-mail address. If you don't - potential user might get it accidentally deleted and there will be no way to repeat the procedure. If you do allow that some angry dudes might use this as spam tool (send as many mails as possible to one(maybe even more) e-mail address. Imagine how would you company/site look for a person who got 10k registration requests...
Standard way has one serious advantage: it allows user name reservation without confirming e-mail (user might want to register, but don't want or has no access to the e-mail server/account).
You MUST consider that your server might delay email sending for pretty long time. Possible reasons: out of memory, DoS attack, email server failure and etc. If you choose mail first approach and user don't receive that mail in 5 minutes (for ANY reason), 3 of 4 potential users will course you company/site and never complete registration.
There is a reason why it is called a standard way, as a lot of small details are considered.
Both approaches are OK - but if you're going defer creation of the account, then you're going to have to embed all the required details into the URL - expiry date, username, password and email address and then encrypt it all to prevent tampering - which makes it rather large.
Actually - you couldn't allow people to pick their own usernames - since you'd have no way of checking whether the username had already previously been requested and not verified. And if you're going to publish usernames, then you'd therefore be publishing email addresses.....not such a good idea?
Here would be my concerns with this approach.
Email delivery is not guaranteed and can be slow. If the user doesn't get the email right away, they may not complete the registration process. What if they mistype their email address or if the email gets marked as SPAM?
In my experience, it is always better to keep record of the users that try to register to a site.
The problem is that more then often the users do not get the confirmation e-mail.
When that happens they often forget the site and do not come back.
What I do is to retry sending the confirmation e-mail after a while, say one week. Often they receive the second e-mail and you end up recovering a registered user that otherwise would be lost.
As a matter of fact, I retry sending the confirmation e-mail once every week until the user confirms or it passed 30 days since the registration attempt.
Even if the user does not confirm after 30 days, I do not delete the account. Often the user comes back trying to register again. Then I just send him again the account confirmation once again and encourage the user to contact the site if he does not get it again.
All this is to maximize the chances of recovering a registered user that otherwise could be lost.
I would suggest the second option. Let the users verify themselves by clicking the link in their email. Then they can choose their preferred username and password. I hope the usernames are unique in the site.
It would be helpful in the situation where some users forget to verify the link in their emails for a long time and so their usernames are locked. Others cannot choose those usernames (until that record is deleted later). Also this can eradicate spammers from picking their own usernames and locking them for use by others.
Hence i would suggest to go with the second option. Let the user first verify his email and his existence before he picks a username and locks it for use by others.
There are actually some sites that do that.
You enter your mail
you get a
verification mail with an initial
password and verification link
once you click the link your account is
active you're directed to a form with
additional details (full name, etc.)
but you may skip them and fill them
any time in the future.
This minimal registration process will help you avoid the loss of potential customers who don't want to bother with filling to many forms and supplying data before they really need to.
What it comes down to is convenience for the user. If the only reason for them the check their email is to verify the account then it may seem like an inconvenience. Instead have the system generate a password for them, email it to them, and instruct them to check their email to get their password. You can allow them to change the password after they log in if they want. This method also help to make sure "strong" passwords are out there initially.