My Requirement
I am making a website which will have mobile version as well. So, I am making it API centric. Now I want to make my API secure without the complexities of OAuth because the security I need is quite simple. I don't want anyone with access to the api links to be able to access my data.
So, I came across this article http://www.thebuzzmedia.com/designing-a-secure-rest-api-without-oauth-authentication/ which is quite amazing and cleared most of my doubts.
Right now, I am trying to recreate whatever is there in the article. I am using Laravel 5 framework for PHP for development.
I want to make sure that the API is being used by the mobile app and the web version only and no one else.
I have seen api links like
example.com/fetchallinformation&publicKey=<something>&Hashkey?<some_hash_key>
Now, I understand that this key is generated by using hash_hmac() function in php.
My Approach
I have a table where I am storing the publicKey and privateKey for my api users
That HashKey in the URL is generated by hashing the privateKey and the publicKey in the client side and then sent to the server. So, i send the generated Hash along with the publicKey to the server.
In the server side, I take the publicKey and the Hash. I retrieve the private key from the table corresponding to the publicKey and has them and check if the resulting hash is same as the hash send by the client
If it is same, then I give permission to them, else, I don't.
My Confusion
I am not sure if this is the right way to do this.
Can we get the data that has been used to generate the hash using hash_hmac() by decrypting the hash?
That HashKey in the URL is generated by hashing the privateKey and the publicKey in the client side and then sent to the server. So, i send the generated Hash along with the publicKey to the server.
Close, but not quite. As you just described it, a user with a given public key would send the same hmac with every request. That's no better than "username and password."
Side note: if you aren't using https, you're already insecure and whatever else you do to secure the site is of relatively little value.
The point of generating an hmac signature is that it not only authenticates the user as being in possession of the secret key, it also authenticates the specific request as being made by that user and being made during a specific window of time. Two different requests back to back should have a different hmac. One request today and an identical request tomorrow should also have a different hmac. Otherwise, you're in for replay attacks. This means information about the current time or expiration time of the signature, and information about the request itself, must be included in the information that's passed through the hmac algorithm or you're not accomplishing much.
For any given request, by a specific user, at a specific time, there can only be one possible valid signature. HMAC is not reversible, so you can't take the signature apart at the server end and figure out the attributes of the request.
Of course, of you're thinking about embedding that secret key in your app, remember that such tactics can be relatively trivial to reverse-engineer.
Is it a viable authentication mechanism? Of course. As the article points out, Amazon Web Services uses hmac signatures on their APIs, and they have a massive potential attack surface... but does that mean you will implement it in a meaningfully secure fashion? Not necessarily. There is always someone more clever, devious, and determined than you can imagine.
Even Amazon apparently realizes that their Signature Version 2 is not as strong as it could be, so they now have Signature Version 4, which has a much more complex algorithm, including several rounds of hashing and generation of an intermediate "Signing Key" that is derived from your secret, the current date, the specific AWS service, AWS region, and other attributes. Regions where Amazon S3 was first deployed in 2014 or later don't have support for the original Sig V2 at all -- and it seems like it can only be security-consciousness that drove that decision, since the old algorithm is computationally less expensive, by far.
Use caution in rolling your own security mechanisms.
If you are primarily trying to avoid the learning curve with OAuth, which I agree is quite annoying at first, you could be on a fool's errand.
If this method works for you it should definitely be fine, and undoubtedly it is secure.
Regarding decryption - HMAC is not supposed to be decrypted due to its nature (hash). HMAC is considered to be very secure and you should have no problems with it. You can read a bit more about How and when do I use HMAC? [SE Security]
I want to make sure that the API is being used by the mobile app and the web version only and no one else.
This is a problem that neither OAuth nor AWS-style signature authentication really help with. Both are about authenticating users, not applications. You can certainly implement either approach if you have a bunch of time to sink into it, but in both cases you're going to need to embed a "secret" in your apps, and once you give that app to a user your secret's not really a secret any more...
There's no great way to do what you're looking for. If someone's going to take the time to reverse-engineer your app to learn about how to directly hit the underlying API, anything else you do client-side to "authenticate" the calling application can be reverse-engineered as well.
I'd recommend not even bothering, and spending the time you save polishing your app so no one wants to bypass it and hit your API directly. :)
Related
Summary:
I've created an app in PHP. It's lead management system for a call center. I now need to allow a partner to be able to add new leads to the app by integrating our app with their proprietary CRM. In short, I guess I need to build an API for my app.
The easiest approach I can think of is a simple HTML post. Would this be considered too insecure? If so, what would be best approach for this type of situation?
Thanks for any help,
Andrew.
Through your quest to build an API, you'll most likely come across some of these. I'm going to outline the concepts that might come in very handy to actually build an API that is usable, and that follows open standards (which, in turn, makes it trivial for third-party to adapt existing code to interact with it).
API dispositions
The first keyword is: SSL. Don't ever think of not using it. This provides a secure socket layer on which communication can happen in a secure fashion, and consequently makes eavesdropping and MitM attacks significantly more difficult to conceive.
No matter what, do not skip on this. Certificates cost less than $60/year, so it is not exactly costly, and can save you a lot in the long run.
In terms of server techs, use what you want. Your main requirement is a webserver that can handle the four common HTTP verbs: GET, POST, PUT, DELETE. I'll explain why in a moment.
API authorisation
This one is the contentious field, as lots of people "think they have a secure way to do so". The answer is simply not true. The point of your authentication is to allow a client to easily authenticate with their credentials, but to prevent a third-party who is not privileged from doing so.
Simply adding an API key to the feed will just lead to someone eventually getting hold of it. I have seen this specific thing so many times that I strongly advise against it, especially as there are significantly easier options.
I'll go over a couple of things, labelling them as (A) or (S), respectively for Authentication and Signature. Signing is the method used to render your request tamper-proof. Authentication proves who you are.
HMAC-SHA512 signing (A) (S)
This technique is used by Amazon for all their S3/AWS APIs, and is a very lightweight method of signing and authenticating a request. I personally find it relatively ingenious.
The basic idea:
Round up all the GET and POST fields (including your public key)
Sort them alphabetically
Concatenate them using URLEncode or equivalent
Perform a HMAC hashing cipher on the data, with your private key as the key of the HMAC.
Append the result of 4 to your request.
This is simple and ingenious. What it guarantees:
You cannot change the request without knowing the private and public keys
You cannot change the key without changing the request
This neatly wraps both issues using the same HTTP request at the cost of one reserved GET/POST field. Amazon also requires the presence of a Timestamp in the request, which prevents replay attacks. Neat!
(For the reference: HMAC-ALGO = ALGO( (key XOR PAD) concat ALGO(key XOR PAD2) concat message). ALGO can be any hash cipher - SHA256 is preferred for its lightweight nature)
OAuth (A)
You've probably heard of it. the idea is simple: you get given a key and secret. This allows you to queue up for a temporary token. This token is then used to perform requests.
The main advantage of this is that lots of libraries exist to handle it, both client-side and server-side. The other advantage is that OAuth has two modes of operation: two-legged (server->server without client interaction) and three-legged (client->server->server).
The main drawback is 2 HTTP requests to get a token.
Simply sending private keys through (A)
... Leads to replay attacks. don't consider it.
A mixture of methods is a possible things. The HMAC signage is awesome when combined with OAuth, for example!
API conception
API endpoints these days follow two main standards: SOAP (XML-RPC), or REST. If you are building an endpoint to post leads, you may as well build the corresponding API to read leads and to delete them for the future.
Your API would therefore take the form:
/my/endpoint/
- GET: gets a list of leads
- POST: creates a new lead
/my/endpoint/ID/
- GET: get lead info
- PUT: modifies lead
- DELETE: deletes the lead
This allows you to future-proof your API conveniently as well.
A HTML post will suffice, that's not a problem. It would be even better if you're able to use HTTPS to ensure the transferred data is encoded, but this isn't critical.
The most common way of securing this kind of API is to provide a shared 'secret' or 'key', which is used to encode a hash. You'll then be able to verify that the request came from a trusted source, but it's up to the user to ensure that they keep the shared key a secret.
e.g. Users of your API will need to:
// build hash string to be sent with API POST request (use a sensible combination of values)
$string = sprintf('%s.%d.%d.%d', $username, $orderId, $currentTimestamp, $price);
// hash
$encodedString = sha1($string);
// concatenate with shared key
$stringWithKey = sprintf('%s.%s', $encodedString, $sharedKey); // GET KEY FROM SECURE PLACE
// hash again to get hash that will be sent with the POST request
$hash = sha1($stringWithKey);
Then you'll perform the same logic at your end from the POST values provided and verify that their hash matches the hash that you build with the user's shared key.
This is exactly what an API is for. I'd make a unique key per external account and require that API key for each $_GET or $_POST transaction that is sent to your server.
Might want to build an API management console while you're at it. Oh and don't forget the separate DB table for the API keys.
When you're done it'll be something like:
https://api.mysite.com/index.php?key=r328r93fuh3u4h409890fj34klj&other=something&another=somethingelse
You get the idea.
I am working on an application who's front end is in javascript (backbone) and backend in codeigniter(php). I want a way where I can encrypt a message in backend using a secret key and decrypt that message from javascript using same secret key. Basically I need this flow for user authentication.
Its turning out to be a hard luck as I am not able to find such a combination over internet. What way can be used for this workflow?
Thanks
using HTTPS is pretty much the only answer that makes sense. Relying on Javascript cryptography is bad because malicious users can do all sorts of nasty stuff to inject scripts to make your front end instead send passwords out in plaintext, or send them to a server controlled by the attackers.
HTTPS and SSL/TLS has it's problems, but it is the standard mechanism for securing web application communication channels. It has been reviewed extensively by experts in the field, and has been secured against various failure modes and attack scenarios.
For instance: It sounds like in your comment that you are going to use a single static key for your encryption. Since you need to send the key to the client, how are you going to do this securely? What prevents a user from getting the key legitimately by visiting your login page, then using it to decrypt the communications of any other users? What about when enough messages have been sent with one key, what is your mechanism for moving to a new key?
These sorts of scenarios are covered by HTTPS. You might have to pay a few bucks for a certificate if you are doing something "real" with your webapp (and it really is just a few bucks these days), but it is really the way to go.
I wrote a Web Application and I would like to allow other developers to get the information from it.
The server Im working on is not that awsome and cant handle that many request, so the idea is to generate and assign api keys to everyone that wants to query our information. With Api keys I can limit the daily requests and perhaps collect some statistics to see what information is really useful for the other developers.
The thing is, Im concerned about the security aspect of it. Since the Api key is going to be sent to our server (via GET/POST etc), someone could sniff the request with wireshark and get the developers API key without much effort, right?
I thought about generating a secret key and an API key. I could then instruct the other developers to concatenate them and send a hash of it to our API. I would then validate that the hash is valid and allow the request... But then the same problem would persist. A hacker could still sniff that hash and make requests on behalf of the other developer's app.
So my questions are
How can I avoid this problem?
Or better yet, Is my concern even valid? Is that a real problem?
Is there a better and secure way to allow access to my information without making it too complicated for the other developers?
What do you guys think?
I think you're trying to solve a bunch of different questions here.
If your objective is to limit the number of requests to your server, you should create a throttling mechanism. This is non-trivial, but I'd base it on IP address rather than a license key - a single licensed user might swamp your system with lots of requests. You can throttle by IP address without implementing a license mechanism.
If you want to create a licensing scheme, you need to understand cryptography etc. - it's a non-trivial problem. For instance, how do you stop a legitimate user sharing their license key with all their friends? How do you stop a hacker from stealing your key and sharing it with all of his friends?
There are a number of solutions to this - they all impose some degree of pain on your users. For instance, you can run your service on HTTPS; this stops snooping, but reduces performance. You can issue "tokens" for your service which expire after a certain number of uses; getting new tokens requires a cryptographic exchange (which might check your IP address). You might require a "challenge/response" type of logic (including an IP address validation). All these steps make life harder for your users; they probably won't thank you much for the extra work they have to do.
With respect to sniff, your problem can be solved with HTTPS on your server.
it definitely makes sense to put some authentication on the API if you want to limit access + potential some usage rate limits. If you use an API key and want to avoid sniffing then HTTPS is definitely the way to go. If that's not an option then you can also use a hash-style auth like oAuth 1.0 (http://oauth.net/core/1.0/) or Amazon AWS authentication. These work by issuing your API users with an ID and a Secret. They use the secret on the client side by inserting it into the message body, computing a hash and including the hash (not the secret) in the request. On the incoming side you compare the hash with the same operation done on the message content with their specific secret included.
This means that you can verify the sender without having to send the secret over the wire (note that the content still isn't secure - but you avoid passing the key over the wire with every request). The downside is that it's complex for developers to implement. Even if you use the oAuth 1.0 pattern which there are libraries for it's a bit of an overhead.
I work at 3scale and some of our tools might be helpful also - our systems provide API Keys, oAuth Secret sharing and also API rate limits out of the box (http://www.3scale.net and the PHP libraries are here: https://github.com/3scale/3scale_ws_api_for_php).
I'm modifying an Android app that utilizes a webapp via a webview. Currently the the code base for the webapp is written in ColdFusion - so all the session management is done in CF. There are certain hooks in the webapp that force the Android app to do native functions and sometimes call external scripts in PHP.
These php scripts get data posted to them (userid, friendid, etc) - currently the php scripts just make sure there is valid data being posted, then process the request if the data is present and valid.
I am looking for ways to increase the security of these php scripts to prevent bots / malicious users from posting false data to these pages - at this point nothings stopping anyone sending a correct userid/friendid and having the script from executing.
Session management would be the first line of defense, but since the webapp is in a different language I can't use that - and sometimes the php scripts are on a different domain completely (same server though).
The other method I considered was on sign up creating a user token to associate with a user, and saving this on the Android side of things - then when requesting these php scripts send the userid and token. And verify the token for that user matches in the remote database - this would make it harder to guess posting credentials for malicious user. Clearly not the best because the token is stored locally and going over the wire, but I digress.
Question are there any better methods to use in order to protect these lone php scripts from being executed, with out the use of session management? Does my token idea make any sense?
Note: I can use SSL on any / all requests.
I know exactly what you need, if you're up to the task. Your API needs to impliment OAuth2.0.
What OAuth can provide you is a secure way to pass information to and from your service while making sure that all secret information is kept private and that only the correct people can access that information. It gives each user a unique signature.
OAuth is used by Facebook, Google, Twitter and more to give developers a secure way to access information while keeping everyone from doing things they shouldn't be doing.
OAuth has support for ColdFusion, Java, PHP, C#, dotNet, VB.net, LIST, Javascript, Perl, Python, Ruby, and more.
http://oauth.net/
Session management or OAuth are the best solutions, but not the easiest. An easier way is implementing a hashing algorithm in both your app and the PHP scripts. When the app prepares a request, you hash some of the values that are being sent to the server using your secret method. This hash is being sent with the request. The server does the same and compares the two hashes. When they're the same, it knows the request is from the app (or someone who cracked your algorithm). When they're not, the server can simply ignore the request.
An example:
Data: userid = 2042; name = JohnDoe; email = john-doe#someprovider.com
Hash (in PHP, but you should implement it in the app as well):
<?php
$userid = 2042;
$name = 'JohnDoe'
$email = 'john-doe#someprovider.com';
// Remove some letters with other letters
$name = str_replace(array('a', 'd', 'g'), array('E', 'x', '9'), $name);
// Reverse a string
$email = strrev($email);
// Make a super secret hash (with salt!)
$hash = sha1('fnI87' . $useris . '87bk;s.' . $name . 'unf%gd' . $email);
// Some more fun
$hash = str_rot13($hash);
?>
Request: http://www.your-server.com/script.php?userid=2042&name=JohnDoe&email=john-doe#someprovider.com&hash=YOUR-GENERATED-HASH
Now the server can apply the same hashing method and compare the resulting hash with the hash sent with the request.
I'd like to suggest a more abstract approach, but similar to Jonathan's.
I make the following assumptions:
You have a PHP-script that anyone can call (if they know the URL / sniff the network packets).
Your android app is closed source; meaning that if you have a hash algorithm no-one but you will know what it is.
You want to prevent anyone from directly calling the PHP scripts - circumventing the app and any security you might have built in there.
What you need is way to identify that your app is sending the requests, and not someone else.
The idea is that you generate a signature for each request that only your app can make (ie. a salt + a hash).
$input = array(
"userid" => 1234,
"friendid" => 2345
"etc" => "..."
);
$salt = "s3kr4tsal7"; // this is essentially your app signature
$signature = md5($salt . serialize($input)); // you could also use json_encode or any other to-string serialization
// pick whichever is easy to do in PHP and in your app
$request = array(
"input" => $input,
"signature" => $signature
); // send this
Then in your PHP script check if the signature matches the calculated signature. This is similar to Jonathan's solution but it allows for any input, it's not dependent on $email or any other property. I also don't think you need an overly complex hashing algorithm, just md5 with a salt is 'hard enough'.
There is another type of attack you should be aware of and that is a replay-attack.
If you look at the RAW data going over the line, you could capture it and simply play it again. If you know what action has what output you can simply repeat the output.
The typical solution for a replay-attack is a trial-and-response. SSL does this for you but you could also make a custom implementation (but that is significantly more complex).
As usual, it depends on what level of protection you need, and how much you are willing to invest. Since you cannot use sessions, you need some sort of a stateless way to authenticate. There are generally two ways to do this: post credentials each time (e.g., basic authentication) or send some sort of a token (BTW, the session ID is exactly that, a token that links to a live session on the server).
When you generate the token, it is a good idea to use a standard and proven algorithm, instead of inventing your own and/or relying on obscurity. Even if it looks mostly secure, it might not be. For example, there are known attacks against the MD5 idea above (it is easy to append data to the message without knowing the key and obtain another valid MAC). HMAC-SHA1 is designed to avoid those.
First thing first: if you can, do use SSL for all requests. This would accomplish a few things at the same time:
users (your app) can be sure that they are posting their data to the right place (i.e., your webapp). SSL server authentication takes care of this.
it would make sure any crednetials/tokens you post are automatically encrypted.
replay attack become practically impossible
It seems you already have authenticated users, so issuing tokens should be relatively easy. You might want to think about the protocol to implement, but as you consider more cases, you will be getting closer to re-inventing OAuth and friends. Some things to consider:
an expiration period on tokens: so that even if someone gets a hold of one, they cannot use it indefinitely.
a way to revoke tokens
maybe have different tokens for different parts (services) of the webapp, so you can grant/revoke access to only the necessary services
To make sure you (i.e., your webapp(s)) are the only one that can issue said tokens you would want to sign them with a key only you have. Since the signer and verifier are the same (you) you don't have to use public key cryptography, HMAC should do. You could, for example, concatenate the username, issue time and any other relevant information, and use them as input to HMAC. Pack those parameters along with the signature (HMAC output) to create a token, and have the app send it with each request. Verify on the server and allow access if valid, require re-login (new token) if expired, deny access otherwise.
Alternatively, if you want to authenticate just the app, and not get user info mixed up in this, you could use a similar approach to sign requests on the client (app side). If you choose this way, do use a standard algorithm. This would, of course, require the signing key (in some form) to be in the app, so if someone gets hold of it (by reverse engineering, etc.) they could issue as many requests as they want. There are ways to mitigate this though:
implement the signing logic in native code
don't store the raw key, but derive it at runtime from bits and pieces stored in different places
And of course, the easiest way of all would be to require basic or digest authentication at the server (over SSL, of course), and embed the username and password in the app (sufficiently obfuscated). On the server side that would require only a change in server configuration, a few lines added on the client side. The downside is that there is really no way to change those credentials if they get compromised (short of releasing a new version and blocking access from the old one to force people to update; not pretty).
I'm asked to write a Web API for an application (pc executable, not web-app) that will allow sending emails.
A user clicks something, the app communicates with the API which generates an email and sends it out.
I have to make sure noone unauthorised will have access to the API, so I need to make some kind of authentication and I haven't got an idea how to do it correctly.
There will be more applications accessing the API.
First thought was - send username and password, but this doesn't solve the problem really. Because if someone decompiles the application, they'll have the request url and variables including user/password or simply it can just be sniffed.
so... what options do I have?
I'm fairly sure secure connection (SSL) is not available to me at the moment, but still, this won't help me against the decompiling problem, will it?
EDIT
I haven't said that initially, but the user will not be asked for the username/password. It's the application(s) that will have to be authenticated, not users of the application(s).
The distribution of your software is really the crux of the problem. Hashing user names and passwords and storing them in the software isn't any more useful than storing un-hashed values, as either one would work to access the API server. If you're going to implement usernames and passwords for your users, I think you can use that as a pre-cursor to API control without storing the values in the software itself. Let me describe this in two parts.
Request Signatures
The most common method in use for API request verification is request signatures. Basically, before a request is sent to an API server, the parameters in the request are sorted, and a unique key is added to the mix. The whole lot is then used to produce a hash, which is appended to the request. For example:
public static function generateRequestString(array $params, $secretKey)
{
$params['signature'] = self::generateSignature($params, $secretKey);
return http_build_query($params,'','&');
}
public static function generateSignature($secretKey, array $params)
{
$reqString = $secretKey;
ksort($params);
foreach($params as $k => $v)
{
$reqString .= $k . $v;
}
return md5($reqString);
}
You could create an API request query string using the above code simply by calling the generateRequestString() method with an array of all the parameters you wanted to send. The secret key is something that is provided uniquely to each user of the API. Generally you pass in your user id to the API server along with the signature, and the API server uses your id to fetch your secret key from the local database and verify the request in the same way that you built it. Assuming that the key and user id are correct, that user should be the only one able to generate the correct signature. Note that the key is never passed in the API request.
Unfortunately, this requires every user to have a unique key, which is a problem for your desktop app. Which leads me to step two.
Temporal Keys
So you can't distribute keys with the application because it can be decompiled, and the keys would get out. To counter-act that, you could make very short-lived keys.
Assuming that you've implemented a part of the desktop app that asks users for their username and password, you can have the application perform an authentication request to your server. On a successful authentication, you could return a temporal key with the response, which the desktop app could then store for the lifetime of the authorized session, and use for API requests. Because you mentioned that you can't use SSL, this initial authentication is the most vulnerable part, and you have to live with some limitations.
The article Andy E suggested is a good approach (I voted it up). It's basically a handshake to establish a short-lived key that can be used to authenticate. The same key could be used for signature hashing. You could also take your chances and just send the username/password unencrypted and get a temporal key (it would only happen once), but you'd have to be aware that it could be sniffed.
Summary
If you can establish a temporal session key, you won't have to store anything in the client program that can be decompiled. A username/password sent once to your server should be enough to establish that. Once you have that key, you can use it to create requests in the desktop apps, and verify requests on the API server.
I would recommend you check out OAuth. it should definitely help you out in sorting out the security issues with authorizing tools to access your API.
http://oauth.net
Someone is always going to be able to decompile and hunt for the variables. An obfuscator might be able to hide them a little better. Sniffing is also easy without SSL unless you use a private and public keyset to encrypt the request data client side and decrypt server side (but obviously this key will be stored in the client application).
The best thing to do is provide as many layers of protection as you think you will need, creating a secure connection and obfuscating your code. You could look at the following article, which demonstrates a secure connection without using SSL:
http://www.codeproject.com/KB/security/SecureStream.aspx
As mattjames mentioned, you should never store passwords in plain text format. When the user enters their password into the application, store a hash of the password. The same hash should be stored on the server. That way, if the hash is seen by an interceptor they at least wouldn't see the user's original password.
You will need to use SSL if you need to prevent people from seeing the plain text password that is sent from the app over the network to the API.
For the decompilation issue, you would want to store the hash of the password in the API, not the original password. See explanation here: http://phpsec.org/articles/2005/password-hashing.html.