I am trying to implement a web service and need some (very) simple Authenticate to restrict access to the service.
I found out about HMAC and I think I understand how to implement it. But I have a couple of questions in mind.
Let's say I have this HTML Form on the consumer side. When making a GET/POST request to my server.
Is is enough to create a hash of: public_key using the secret_key?
OR, do I need to create a hash of the entire POST variables/array?
I'm thinking it would be enough to send the hash of the public_key only but just wanted to make sure and ask you guys.
I am planning to do this:
Create a hash of the public_key
Put the hash in a hidden field or in the URL as a param together with the public_key (or client_id) and other POST/GET variables.
Receive on my server and verify the hash against the database by recreating the hash of the public_key using the secret_key.
If the hash matches, I accept the POST/GET requests.
Your thoughts?
Clarification: public_key is like the client unique id where I can use to identify what secret key to use to generate the hash on the server.
The pubkey is just used as an alternative way to recognize the user. It could be the user email as well, by the way since you don't likely want to expose your user data to their programmer (or to potential sniffers) you create a unique identifier for each user. It's all it means. Then you need a private key to sign your hash.
Of course to make it worth it you have to sign all unique request data, otherwise someone could alter your request body and you wouldn't be able to detect it (MITM attack).
You also should care of creating a timestamp that must be included in the HMAC itself, then pass it alongside with the request. This way you can make the signature expirable and so you are not exposed to replay attacks (someone steals the request and without modifying it replies it against the server, operating multiple times the same action... think what a problem if it's a request to pay for your service, your user would be very very angry with you).
Also remember (nobody does) to encrypt also the Request-URI inside the HMAC itself and also the HTTP method (aka verb) if you're using a RESTful webservice, otherwise malicious users will be able to send the request to other URIs or (using RESTful services) change the meaning of your request, so a valid GET can become a potential DELETE.
An example could be: user wants to see all its data, makes a GET request, a Man in the Middle reads the request and changes GET with DELETE. You are not given the opportunity to detect that something has been changed if it's not inside your HMAC you can check about, so you receive a DELETE request and boom! you destroy all user data.
So always remember: everything is essential to your request must be validable
And if you rely on a HMAC then you must encrypt everything you need to trust the request.
Also always remember to start designing your system by denying all request, then if you can validate them perform requested actions. This way you always fall back on denied requests. It's better to have a user email telling you that he cannot do something that have your user data propagated on the net.
Use TLS. It fixes this and a host of problems you haven't even thought of yet.
Related
I've been reading up on API communication securities and trying to figure out the best way to build a secure API. I know that OAuth and such exist, but I'm also trying to educate myself in the process and not rely on libraries.
Basically I have a Web Service and in that web service users can register for API. They will be provided a Profile ID and secret key which they have to use to build the API request from another web system.
API request is built similarly to the way banks do it, all input data sent to API has to be sorted, hash calculated and then the hash sent to the server, like this:
// Profile data
$apiProfile='api123';
$apiSecret='this-is-a-good-day-to-be-a-secret-key';
// Input
$input=array();
$input['name']='Thomas Moore';
$input['profession']='Baker';
// To ensure that the order of variables checked and received is the same on both ends:
ksort($input);
// Using serialize() for simplifying things
// http_build_query() is another option, or just placing values in order
$input['hash']=sha1(serialize($input).$apiSecret);
// Making a request to URL:
// Using file_get_contents() as an example, would use cURL otherwise
$result=file_get_contents('http://www.example.com/api.php?'.http_build_query($input));
// SERVER CALCULATES COMPARISON HASH BASED ON KNOWN SECRET KEY AND INPUT DATA
This is really good and works. But! My problem is the potential replay attack. If someone snatches this request URL, they can send it to the server again, even though they cannot change the data itself.
Now I've read some things about it that you should also either check the time or add a one-time-use token to the request, but I am unsure how exactly should I do that? Is sending a timestamp with the request really secure enough? (Receiving server would make sure that the request has originated few seconds within the time the request was made, if the clocks are somewhat in sync).
I could also add IP validations to the mix, but these can change and can be spoofed somewhat and are more of a hassle for the user.
I would love this one-time-token type of system, but I am unsure how to do this without exposing token generation to the exact same replay attack problem? (Last thing I need is allowing to give out secure tokens for middle-men).
Opinions and articles would be really welcome, I've been unable to find material that answers my specific concerns. I want to say that my API is secure, without it being just marketing speak.
Thank you!
You need to only allow token exchange via a secure channel (https), and you should have a unique hash per message. Include things like a timestamp and the ip of the client. If you don't use https, you are vulnerable to a firesheep-style attack.
Other than that, you are doing the token generation and exchange correctly.
Sending the time (and including it into the cache) is really an option.
The other option would be 2-phase algorithm when you first request for the session token or a session key, then use it for the session, and its TTL is stored on the server (which can be time or number of requests allowed)
As for the session keys idea look at schemes like http://en.wikipedia.org/wiki/Diffie%E2%80%93Hellman_key_exchange
Example of 1-time token algorithm:
1) client composes a request for the 1-time token, signs this request with the secret key and sends it to the server.
2) server generates the key, signs it with the same key and sends it to the client (together with the signature)
3) client verifies the token using the secret key
4) client composes the request, including the token, and signs the whole request body with the secret key, then sends to the server
5) server checks whole body integrity and the token validity, then sends the response (again it can be signed with the secret key for integrity and authorship verification)
I'm modifying an Android app that utilizes a webapp via a webview. Currently the the code base for the webapp is written in ColdFusion - so all the session management is done in CF. There are certain hooks in the webapp that force the Android app to do native functions and sometimes call external scripts in PHP.
These php scripts get data posted to them (userid, friendid, etc) - currently the php scripts just make sure there is valid data being posted, then process the request if the data is present and valid.
I am looking for ways to increase the security of these php scripts to prevent bots / malicious users from posting false data to these pages - at this point nothings stopping anyone sending a correct userid/friendid and having the script from executing.
Session management would be the first line of defense, but since the webapp is in a different language I can't use that - and sometimes the php scripts are on a different domain completely (same server though).
The other method I considered was on sign up creating a user token to associate with a user, and saving this on the Android side of things - then when requesting these php scripts send the userid and token. And verify the token for that user matches in the remote database - this would make it harder to guess posting credentials for malicious user. Clearly not the best because the token is stored locally and going over the wire, but I digress.
Question are there any better methods to use in order to protect these lone php scripts from being executed, with out the use of session management? Does my token idea make any sense?
Note: I can use SSL on any / all requests.
I know exactly what you need, if you're up to the task. Your API needs to impliment OAuth2.0.
What OAuth can provide you is a secure way to pass information to and from your service while making sure that all secret information is kept private and that only the correct people can access that information. It gives each user a unique signature.
OAuth is used by Facebook, Google, Twitter and more to give developers a secure way to access information while keeping everyone from doing things they shouldn't be doing.
OAuth has support for ColdFusion, Java, PHP, C#, dotNet, VB.net, LIST, Javascript, Perl, Python, Ruby, and more.
http://oauth.net/
Session management or OAuth are the best solutions, but not the easiest. An easier way is implementing a hashing algorithm in both your app and the PHP scripts. When the app prepares a request, you hash some of the values that are being sent to the server using your secret method. This hash is being sent with the request. The server does the same and compares the two hashes. When they're the same, it knows the request is from the app (or someone who cracked your algorithm). When they're not, the server can simply ignore the request.
An example:
Data: userid = 2042; name = JohnDoe; email = john-doe#someprovider.com
Hash (in PHP, but you should implement it in the app as well):
<?php
$userid = 2042;
$name = 'JohnDoe'
$email = 'john-doe#someprovider.com';
// Remove some letters with other letters
$name = str_replace(array('a', 'd', 'g'), array('E', 'x', '9'), $name);
// Reverse a string
$email = strrev($email);
// Make a super secret hash (with salt!)
$hash = sha1('fnI87' . $useris . '87bk;s.' . $name . 'unf%gd' . $email);
// Some more fun
$hash = str_rot13($hash);
?>
Request: http://www.your-server.com/script.php?userid=2042&name=JohnDoe&email=john-doe#someprovider.com&hash=YOUR-GENERATED-HASH
Now the server can apply the same hashing method and compare the resulting hash with the hash sent with the request.
I'd like to suggest a more abstract approach, but similar to Jonathan's.
I make the following assumptions:
You have a PHP-script that anyone can call (if they know the URL / sniff the network packets).
Your android app is closed source; meaning that if you have a hash algorithm no-one but you will know what it is.
You want to prevent anyone from directly calling the PHP scripts - circumventing the app and any security you might have built in there.
What you need is way to identify that your app is sending the requests, and not someone else.
The idea is that you generate a signature for each request that only your app can make (ie. a salt + a hash).
$input = array(
"userid" => 1234,
"friendid" => 2345
"etc" => "..."
);
$salt = "s3kr4tsal7"; // this is essentially your app signature
$signature = md5($salt . serialize($input)); // you could also use json_encode or any other to-string serialization
// pick whichever is easy to do in PHP and in your app
$request = array(
"input" => $input,
"signature" => $signature
); // send this
Then in your PHP script check if the signature matches the calculated signature. This is similar to Jonathan's solution but it allows for any input, it's not dependent on $email or any other property. I also don't think you need an overly complex hashing algorithm, just md5 with a salt is 'hard enough'.
There is another type of attack you should be aware of and that is a replay-attack.
If you look at the RAW data going over the line, you could capture it and simply play it again. If you know what action has what output you can simply repeat the output.
The typical solution for a replay-attack is a trial-and-response. SSL does this for you but you could also make a custom implementation (but that is significantly more complex).
As usual, it depends on what level of protection you need, and how much you are willing to invest. Since you cannot use sessions, you need some sort of a stateless way to authenticate. There are generally two ways to do this: post credentials each time (e.g., basic authentication) or send some sort of a token (BTW, the session ID is exactly that, a token that links to a live session on the server).
When you generate the token, it is a good idea to use a standard and proven algorithm, instead of inventing your own and/or relying on obscurity. Even if it looks mostly secure, it might not be. For example, there are known attacks against the MD5 idea above (it is easy to append data to the message without knowing the key and obtain another valid MAC). HMAC-SHA1 is designed to avoid those.
First thing first: if you can, do use SSL for all requests. This would accomplish a few things at the same time:
users (your app) can be sure that they are posting their data to the right place (i.e., your webapp). SSL server authentication takes care of this.
it would make sure any crednetials/tokens you post are automatically encrypted.
replay attack become practically impossible
It seems you already have authenticated users, so issuing tokens should be relatively easy. You might want to think about the protocol to implement, but as you consider more cases, you will be getting closer to re-inventing OAuth and friends. Some things to consider:
an expiration period on tokens: so that even if someone gets a hold of one, they cannot use it indefinitely.
a way to revoke tokens
maybe have different tokens for different parts (services) of the webapp, so you can grant/revoke access to only the necessary services
To make sure you (i.e., your webapp(s)) are the only one that can issue said tokens you would want to sign them with a key only you have. Since the signer and verifier are the same (you) you don't have to use public key cryptography, HMAC should do. You could, for example, concatenate the username, issue time and any other relevant information, and use them as input to HMAC. Pack those parameters along with the signature (HMAC output) to create a token, and have the app send it with each request. Verify on the server and allow access if valid, require re-login (new token) if expired, deny access otherwise.
Alternatively, if you want to authenticate just the app, and not get user info mixed up in this, you could use a similar approach to sign requests on the client (app side). If you choose this way, do use a standard algorithm. This would, of course, require the signing key (in some form) to be in the app, so if someone gets hold of it (by reverse engineering, etc.) they could issue as many requests as they want. There are ways to mitigate this though:
implement the signing logic in native code
don't store the raw key, but derive it at runtime from bits and pieces stored in different places
And of course, the easiest way of all would be to require basic or digest authentication at the server (over SSL, of course), and embed the username and password in the app (sufficiently obfuscated). On the server side that would require only a change in server configuration, a few lines added on the client side. The downside is that there is really no way to change those credentials if they get compromised (short of releasing a new version and blocking access from the old one to force people to update; not pretty).
I am researching ways to secure a javascript application I am working on. The application is a chat client which uses APE (Ajax Push Engine) as the backend.
Currently, anyone can access the page and make a GET/POST request to the APE server. I only want to serve the chat client to registered users, and I want to make sure only their requests will be accepted. I can use username/password authentication with PHP to serve a user the page. But once they have the page, what's to stop them from modifying the javascript or letting it fall into the wrong hands?
This method for securing a client/server application looks promising: http://abhinavsingh.com/blog/2009/12/how-to-add-content-verification-using-hmac-in-php/
I have another source that says this is ideal for a javascript client since it doesn't depend on sending the private key. But how can this be? According to to the tutorial above, the client needs to provide the private key. This doesn't seem very safe since anyone who has the javascript now has that user's private key. From what I understand it would work something like this:
User logs in with a username and password
PHP validates the username and password, looks up the user's private key and inserts it into the javascript
Javascript supplies a signature (using the private key), and the public key with all APE requests
APE compares the computed signature to the received signature and decides whether to handle the requests.
How is this secure if the javascript application needs to be aware of the private key?
Thanks for the help!
The answer: You technically cannot prevent the user from modifying the JavaScript. So don't worry about that because you can do nothing about it.
However, the attack you do need to prevent is Cross-Site Request Forgery (CSRF). Malicious scripts on different domains are capable of automatically submitting forms to your domain with the cookies stored by the browser. To deal with that, you need to include an authentication token (which should be sufficiently random, not related to the username or password, and sent in the HTML page in which the chat client resides) in the actual data sent by the AJAX request (which is not automatically filled in by the browser).
How is this secure if the javascript application needs to be aware of the private key?
Why not? It's the user's own private key, so if he is willing to give it out to someone else, it's his problem. It's no different from giving out your password and then saying someone else has access to your account.
If you think about this a bit, you'll realize that you don't need to implement public-key encryption, HMAC or anything like that. Your normal session-based authentication will do, provided the communication channel itself is secure (say using HTTPS).
HMAC authentication is better served for an API that third parties are going to connect to. It seems like your app would be better served by writing a cookie to the client's browser indicating that they've been authenticated. Then with each ajax request you can check for that cookie.
Edit: I take back a bit of what I said about HMAC being better served for third party APIs. Traditionally with HMAC each user gets their own private key. I don't think this is necessary for your application. You can probably get away with just keeping one master private key and give each user a unique "public" key (I call it a public key, but in actuality the user would never know about the key). When a user logs in I would write two cookies. One which is the combination of the user's public key + time stamp encrypted and another key stating what the time stamp is. Then on the server side you can validate the encrypted key and check that the time stamp is within a given threshold (say 10-30 minutes in case they're sitting around idle on your app). If they're validated, update the encrypted key and time stamp, rinse and repeat.
I have an Android application from which I want to upload some data to a database on my web server. As the MySql java library has a size of about 5 mb, I don't want to include it with the application.
So I'll make a HTTP request for a php script and send the data with the URL as parameters. How do I make sure that only I can call this? I don't want people to sniff up the URL and call it outside my application.
Thanks
Use a simple static token to identify the client is yourself or in an advance way, first authenticate with a username/password, generate a token and use this token for further transactions .This token can expire after some time.
option1: http://[your request url]&key=xyz
where xyz is known only to you
option 2: first ping server with username password and upon successful validation get a dynamic token [dKey], store it locally.
then for further requests.
http://[your request url]&key=dKey.
option 2 is the one normally being followed.
The short answer: you cannot prevent sniffing.
But you can make sniffer's life harder by implement a some sort of internal authentication, GET/POST predefined parameters (or dynamic, but calculated by algorithm you only know how) exchange, hidden header fields, etc.
But all this could also be sniffed/reverse engineered.
A possible Overkill Way would be using some sort of asymmetric private/public key encryption/signature. Such as RSA. Your app will only include public key, and sign the request data with it. And your server-side will have a secret private key, it will use it to check validity of client requests.
I know very little about android - but it's not really relevant to the question.
If you want to prevent someone from sniffing the URL (and authentication details?) then the only option is to use SSL. On the other hand if you merely want to prevent other people from accessing the URL, its simply a question of authentication. If you're not using SSL, then that means you need to use sessions and a challenge-based authentication to avoid people sniffing the traffic. You could do this via digest authentication or roll your own code.
C.
I'm asked to write a Web API for an application (pc executable, not web-app) that will allow sending emails.
A user clicks something, the app communicates with the API which generates an email and sends it out.
I have to make sure noone unauthorised will have access to the API, so I need to make some kind of authentication and I haven't got an idea how to do it correctly.
There will be more applications accessing the API.
First thought was - send username and password, but this doesn't solve the problem really. Because if someone decompiles the application, they'll have the request url and variables including user/password or simply it can just be sniffed.
so... what options do I have?
I'm fairly sure secure connection (SSL) is not available to me at the moment, but still, this won't help me against the decompiling problem, will it?
EDIT
I haven't said that initially, but the user will not be asked for the username/password. It's the application(s) that will have to be authenticated, not users of the application(s).
The distribution of your software is really the crux of the problem. Hashing user names and passwords and storing them in the software isn't any more useful than storing un-hashed values, as either one would work to access the API server. If you're going to implement usernames and passwords for your users, I think you can use that as a pre-cursor to API control without storing the values in the software itself. Let me describe this in two parts.
Request Signatures
The most common method in use for API request verification is request signatures. Basically, before a request is sent to an API server, the parameters in the request are sorted, and a unique key is added to the mix. The whole lot is then used to produce a hash, which is appended to the request. For example:
public static function generateRequestString(array $params, $secretKey)
{
$params['signature'] = self::generateSignature($params, $secretKey);
return http_build_query($params,'','&');
}
public static function generateSignature($secretKey, array $params)
{
$reqString = $secretKey;
ksort($params);
foreach($params as $k => $v)
{
$reqString .= $k . $v;
}
return md5($reqString);
}
You could create an API request query string using the above code simply by calling the generateRequestString() method with an array of all the parameters you wanted to send. The secret key is something that is provided uniquely to each user of the API. Generally you pass in your user id to the API server along with the signature, and the API server uses your id to fetch your secret key from the local database and verify the request in the same way that you built it. Assuming that the key and user id are correct, that user should be the only one able to generate the correct signature. Note that the key is never passed in the API request.
Unfortunately, this requires every user to have a unique key, which is a problem for your desktop app. Which leads me to step two.
Temporal Keys
So you can't distribute keys with the application because it can be decompiled, and the keys would get out. To counter-act that, you could make very short-lived keys.
Assuming that you've implemented a part of the desktop app that asks users for their username and password, you can have the application perform an authentication request to your server. On a successful authentication, you could return a temporal key with the response, which the desktop app could then store for the lifetime of the authorized session, and use for API requests. Because you mentioned that you can't use SSL, this initial authentication is the most vulnerable part, and you have to live with some limitations.
The article Andy E suggested is a good approach (I voted it up). It's basically a handshake to establish a short-lived key that can be used to authenticate. The same key could be used for signature hashing. You could also take your chances and just send the username/password unencrypted and get a temporal key (it would only happen once), but you'd have to be aware that it could be sniffed.
Summary
If you can establish a temporal session key, you won't have to store anything in the client program that can be decompiled. A username/password sent once to your server should be enough to establish that. Once you have that key, you can use it to create requests in the desktop apps, and verify requests on the API server.
I would recommend you check out OAuth. it should definitely help you out in sorting out the security issues with authorizing tools to access your API.
http://oauth.net
Someone is always going to be able to decompile and hunt for the variables. An obfuscator might be able to hide them a little better. Sniffing is also easy without SSL unless you use a private and public keyset to encrypt the request data client side and decrypt server side (but obviously this key will be stored in the client application).
The best thing to do is provide as many layers of protection as you think you will need, creating a secure connection and obfuscating your code. You could look at the following article, which demonstrates a secure connection without using SSL:
http://www.codeproject.com/KB/security/SecureStream.aspx
As mattjames mentioned, you should never store passwords in plain text format. When the user enters their password into the application, store a hash of the password. The same hash should be stored on the server. That way, if the hash is seen by an interceptor they at least wouldn't see the user's original password.
You will need to use SSL if you need to prevent people from seeing the plain text password that is sent from the app over the network to the API.
For the decompilation issue, you would want to store the hash of the password in the API, not the original password. See explanation here: http://phpsec.org/articles/2005/password-hashing.html.