Summary:
I've created an app in PHP. It's lead management system for a call center. I now need to allow a partner to be able to add new leads to the app by integrating our app with their proprietary CRM. In short, I guess I need to build an API for my app.
The easiest approach I can think of is a simple HTML post. Would this be considered too insecure? If so, what would be best approach for this type of situation?
Thanks for any help,
Andrew.
Through your quest to build an API, you'll most likely come across some of these. I'm going to outline the concepts that might come in very handy to actually build an API that is usable, and that follows open standards (which, in turn, makes it trivial for third-party to adapt existing code to interact with it).
API dispositions
The first keyword is: SSL. Don't ever think of not using it. This provides a secure socket layer on which communication can happen in a secure fashion, and consequently makes eavesdropping and MitM attacks significantly more difficult to conceive.
No matter what, do not skip on this. Certificates cost less than $60/year, so it is not exactly costly, and can save you a lot in the long run.
In terms of server techs, use what you want. Your main requirement is a webserver that can handle the four common HTTP verbs: GET, POST, PUT, DELETE. I'll explain why in a moment.
API authorisation
This one is the contentious field, as lots of people "think they have a secure way to do so". The answer is simply not true. The point of your authentication is to allow a client to easily authenticate with their credentials, but to prevent a third-party who is not privileged from doing so.
Simply adding an API key to the feed will just lead to someone eventually getting hold of it. I have seen this specific thing so many times that I strongly advise against it, especially as there are significantly easier options.
I'll go over a couple of things, labelling them as (A) or (S), respectively for Authentication and Signature. Signing is the method used to render your request tamper-proof. Authentication proves who you are.
HMAC-SHA512 signing (A) (S)
This technique is used by Amazon for all their S3/AWS APIs, and is a very lightweight method of signing and authenticating a request. I personally find it relatively ingenious.
The basic idea:
Round up all the GET and POST fields (including your public key)
Sort them alphabetically
Concatenate them using URLEncode or equivalent
Perform a HMAC hashing cipher on the data, with your private key as the key of the HMAC.
Append the result of 4 to your request.
This is simple and ingenious. What it guarantees:
You cannot change the request without knowing the private and public keys
You cannot change the key without changing the request
This neatly wraps both issues using the same HTTP request at the cost of one reserved GET/POST field. Amazon also requires the presence of a Timestamp in the request, which prevents replay attacks. Neat!
(For the reference: HMAC-ALGO = ALGO( (key XOR PAD) concat ALGO(key XOR PAD2) concat message). ALGO can be any hash cipher - SHA256 is preferred for its lightweight nature)
OAuth (A)
You've probably heard of it. the idea is simple: you get given a key and secret. This allows you to queue up for a temporary token. This token is then used to perform requests.
The main advantage of this is that lots of libraries exist to handle it, both client-side and server-side. The other advantage is that OAuth has two modes of operation: two-legged (server->server without client interaction) and three-legged (client->server->server).
The main drawback is 2 HTTP requests to get a token.
Simply sending private keys through (A)
... Leads to replay attacks. don't consider it.
A mixture of methods is a possible things. The HMAC signage is awesome when combined with OAuth, for example!
API conception
API endpoints these days follow two main standards: SOAP (XML-RPC), or REST. If you are building an endpoint to post leads, you may as well build the corresponding API to read leads and to delete them for the future.
Your API would therefore take the form:
/my/endpoint/
- GET: gets a list of leads
- POST: creates a new lead
/my/endpoint/ID/
- GET: get lead info
- PUT: modifies lead
- DELETE: deletes the lead
This allows you to future-proof your API conveniently as well.
A HTML post will suffice, that's not a problem. It would be even better if you're able to use HTTPS to ensure the transferred data is encoded, but this isn't critical.
The most common way of securing this kind of API is to provide a shared 'secret' or 'key', which is used to encode a hash. You'll then be able to verify that the request came from a trusted source, but it's up to the user to ensure that they keep the shared key a secret.
e.g. Users of your API will need to:
// build hash string to be sent with API POST request (use a sensible combination of values)
$string = sprintf('%s.%d.%d.%d', $username, $orderId, $currentTimestamp, $price);
// hash
$encodedString = sha1($string);
// concatenate with shared key
$stringWithKey = sprintf('%s.%s', $encodedString, $sharedKey); // GET KEY FROM SECURE PLACE
// hash again to get hash that will be sent with the POST request
$hash = sha1($stringWithKey);
Then you'll perform the same logic at your end from the POST values provided and verify that their hash matches the hash that you build with the user's shared key.
This is exactly what an API is for. I'd make a unique key per external account and require that API key for each $_GET or $_POST transaction that is sent to your server.
Might want to build an API management console while you're at it. Oh and don't forget the separate DB table for the API keys.
When you're done it'll be something like:
https://api.mysite.com/index.php?key=r328r93fuh3u4h409890fj34klj&other=something&another=somethingelse
You get the idea.
Related
I am developing an Android and iOS application that will use a RESTful API that I developed on a separate server. The applications will simply poke at the endpoints to update data within the database and to retrieve information when needed. My problem is not doing that, the architecture of the API is already setup I just need to think about the most secure way to implement it.
I have an API Key system that requires any uses of the API to be paired with the requests which will prevent anyone without the key from accessing the API. This works well, but I am worried about people who are more determined to break through the system. Specifically on the Android side, applications can be decompiled and (some) of the code can be viewed. I am pretty sure that strings can be viewed so I am worried that if someone decompiles the application they will get ahold of the API Key and be able to access the API.
I am looking for suggestions to make the API more secure. I have already considered pairing another unique value with the API Key when updating information but I am not sure if this will work that well either.
Any suggestions are very appreciated it.
You are right about being worried that your application can be decompiled. Code and strings are stored in the clear. In fact this is rather trivial to do.
I've heard that someone looked at two applications: one contained email service password and username in clear text, very useful for spammers. Another had its "random" number algorithm exposed allowing hackers to generate today's "random" number and ride for free using public transport facilities.
You can raise the bar by:
Obfuscating your code with tools. Some tools will offer capability to encrypt strings.
Manually encrypt sensitive strings in your application using new encryption key per new install, split your encryption key and hide it in several different places, e.g. part in code, part in a database, part in OS and apply ACL to external locations
Use transport security (e.g TLS) to reduce risk of man-in-the-middle attacks
Use API rate limiter on the server side
Introduce geo location services to see if same API key is used from different locations at the same time
My Requirement
I am making a website which will have mobile version as well. So, I am making it API centric. Now I want to make my API secure without the complexities of OAuth because the security I need is quite simple. I don't want anyone with access to the api links to be able to access my data.
So, I came across this article http://www.thebuzzmedia.com/designing-a-secure-rest-api-without-oauth-authentication/ which is quite amazing and cleared most of my doubts.
Right now, I am trying to recreate whatever is there in the article. I am using Laravel 5 framework for PHP for development.
I want to make sure that the API is being used by the mobile app and the web version only and no one else.
I have seen api links like
example.com/fetchallinformation&publicKey=<something>&Hashkey?<some_hash_key>
Now, I understand that this key is generated by using hash_hmac() function in php.
My Approach
I have a table where I am storing the publicKey and privateKey for my api users
That HashKey in the URL is generated by hashing the privateKey and the publicKey in the client side and then sent to the server. So, i send the generated Hash along with the publicKey to the server.
In the server side, I take the publicKey and the Hash. I retrieve the private key from the table corresponding to the publicKey and has them and check if the resulting hash is same as the hash send by the client
If it is same, then I give permission to them, else, I don't.
My Confusion
I am not sure if this is the right way to do this.
Can we get the data that has been used to generate the hash using hash_hmac() by decrypting the hash?
That HashKey in the URL is generated by hashing the privateKey and the publicKey in the client side and then sent to the server. So, i send the generated Hash along with the publicKey to the server.
Close, but not quite. As you just described it, a user with a given public key would send the same hmac with every request. That's no better than "username and password."
Side note: if you aren't using https, you're already insecure and whatever else you do to secure the site is of relatively little value.
The point of generating an hmac signature is that it not only authenticates the user as being in possession of the secret key, it also authenticates the specific request as being made by that user and being made during a specific window of time. Two different requests back to back should have a different hmac. One request today and an identical request tomorrow should also have a different hmac. Otherwise, you're in for replay attacks. This means information about the current time or expiration time of the signature, and information about the request itself, must be included in the information that's passed through the hmac algorithm or you're not accomplishing much.
For any given request, by a specific user, at a specific time, there can only be one possible valid signature. HMAC is not reversible, so you can't take the signature apart at the server end and figure out the attributes of the request.
Of course, of you're thinking about embedding that secret key in your app, remember that such tactics can be relatively trivial to reverse-engineer.
Is it a viable authentication mechanism? Of course. As the article points out, Amazon Web Services uses hmac signatures on their APIs, and they have a massive potential attack surface... but does that mean you will implement it in a meaningfully secure fashion? Not necessarily. There is always someone more clever, devious, and determined than you can imagine.
Even Amazon apparently realizes that their Signature Version 2 is not as strong as it could be, so they now have Signature Version 4, which has a much more complex algorithm, including several rounds of hashing and generation of an intermediate "Signing Key" that is derived from your secret, the current date, the specific AWS service, AWS region, and other attributes. Regions where Amazon S3 was first deployed in 2014 or later don't have support for the original Sig V2 at all -- and it seems like it can only be security-consciousness that drove that decision, since the old algorithm is computationally less expensive, by far.
Use caution in rolling your own security mechanisms.
If you are primarily trying to avoid the learning curve with OAuth, which I agree is quite annoying at first, you could be on a fool's errand.
If this method works for you it should definitely be fine, and undoubtedly it is secure.
Regarding decryption - HMAC is not supposed to be decrypted due to its nature (hash). HMAC is considered to be very secure and you should have no problems with it. You can read a bit more about How and when do I use HMAC? [SE Security]
I want to make sure that the API is being used by the mobile app and the web version only and no one else.
This is a problem that neither OAuth nor AWS-style signature authentication really help with. Both are about authenticating users, not applications. You can certainly implement either approach if you have a bunch of time to sink into it, but in both cases you're going to need to embed a "secret" in your apps, and once you give that app to a user your secret's not really a secret any more...
There's no great way to do what you're looking for. If someone's going to take the time to reverse-engineer your app to learn about how to directly hit the underlying API, anything else you do client-side to "authenticate" the calling application can be reverse-engineered as well.
I'd recommend not even bothering, and spending the time you save polishing your app so no one wants to bypass it and hit your API directly. :)
I wrote a Web Application and I would like to allow other developers to get the information from it.
The server Im working on is not that awsome and cant handle that many request, so the idea is to generate and assign api keys to everyone that wants to query our information. With Api keys I can limit the daily requests and perhaps collect some statistics to see what information is really useful for the other developers.
The thing is, Im concerned about the security aspect of it. Since the Api key is going to be sent to our server (via GET/POST etc), someone could sniff the request with wireshark and get the developers API key without much effort, right?
I thought about generating a secret key and an API key. I could then instruct the other developers to concatenate them and send a hash of it to our API. I would then validate that the hash is valid and allow the request... But then the same problem would persist. A hacker could still sniff that hash and make requests on behalf of the other developer's app.
So my questions are
How can I avoid this problem?
Or better yet, Is my concern even valid? Is that a real problem?
Is there a better and secure way to allow access to my information without making it too complicated for the other developers?
What do you guys think?
I think you're trying to solve a bunch of different questions here.
If your objective is to limit the number of requests to your server, you should create a throttling mechanism. This is non-trivial, but I'd base it on IP address rather than a license key - a single licensed user might swamp your system with lots of requests. You can throttle by IP address without implementing a license mechanism.
If you want to create a licensing scheme, you need to understand cryptography etc. - it's a non-trivial problem. For instance, how do you stop a legitimate user sharing their license key with all their friends? How do you stop a hacker from stealing your key and sharing it with all of his friends?
There are a number of solutions to this - they all impose some degree of pain on your users. For instance, you can run your service on HTTPS; this stops snooping, but reduces performance. You can issue "tokens" for your service which expire after a certain number of uses; getting new tokens requires a cryptographic exchange (which might check your IP address). You might require a "challenge/response" type of logic (including an IP address validation). All these steps make life harder for your users; they probably won't thank you much for the extra work they have to do.
With respect to sniff, your problem can be solved with HTTPS on your server.
it definitely makes sense to put some authentication on the API if you want to limit access + potential some usage rate limits. If you use an API key and want to avoid sniffing then HTTPS is definitely the way to go. If that's not an option then you can also use a hash-style auth like oAuth 1.0 (http://oauth.net/core/1.0/) or Amazon AWS authentication. These work by issuing your API users with an ID and a Secret. They use the secret on the client side by inserting it into the message body, computing a hash and including the hash (not the secret) in the request. On the incoming side you compare the hash with the same operation done on the message content with their specific secret included.
This means that you can verify the sender without having to send the secret over the wire (note that the content still isn't secure - but you avoid passing the key over the wire with every request). The downside is that it's complex for developers to implement. Even if you use the oAuth 1.0 pattern which there are libraries for it's a bit of an overhead.
I work at 3scale and some of our tools might be helpful also - our systems provide API Keys, oAuth Secret sharing and also API rate limits out of the box (http://www.3scale.net and the PHP libraries are here: https://github.com/3scale/3scale_ws_api_for_php).
I want to build an API for users to build applications that easily interact with a site, and I was wondering what the best way to authenticate users would be.
Taking a look at other API's a lot of them have the user send the username and password as a GET parameter over a HTTPS connection. Is this the best way to go about it? Or are there other methods that I should look into or consider?
I've seen OAuth been tossed around and it looks like a good solution, but just for a simple API is it overkill?
You can use API key's. Generate a unique hash tied to an account upon request. Then check that the key is a valid key. As long as the API doesn't have any major security issues with someone using someone else's key then Authorization isn't needed. If there is a problem with someone using someone else's key then Authentication would be justified.
This is usually achieved with cookies.
The client sends their username and password with a POST request to your API (do not use GET, that's insecure). If the credentials are acceptable, then generate a random, unique session key, store it on your side and send it in a cookie back to the client (see setcookie()).
When the client now makes further requests, they send the session key cookie with the request. Check $_COOKIE for the session key if it matches a stored key on your side; if yes, that means the user authenticated.
Take note that this minimal example is vulnerable to brute-force attacks trying to guess valid session keys. You need to log invalid keys that clients send in their cookies and block their IP address for some period of time to prevent this.
Username / password in a GET isn't a great way to do this because you're potentially exposing the whole user account for hijacking even if the API has more limited functionality than logging into the site. So it's good practice to separate concerns between Web-site login and API access.
I'm not sure which case you're in but:
If the users are business customers of somekind who are embedding some type of widget or code in another website then it's probably best to use an API key which is scoped to the referrer domain (much like Google Maps does).
If they are end-users who won't know anything about the API but are going to be using Apps built by third parties then oAuth is likely to be your best bet, otherwise your users might literally be giving their usernames/passwords to unknown third parties. It's more complex but likely to be worth it in the long run.
To get a bunch of this stuff out of the box you can use something like 3scale (http://www.3scale.net) and it'll handle most of it for you (disclaimer, I work there so adjust for bias!) or there are open source libraries for oAuth in most languages (in PHP Zend-OAuth component might do the job for you).
I'm modifying an Android app that utilizes a webapp via a webview. Currently the the code base for the webapp is written in ColdFusion - so all the session management is done in CF. There are certain hooks in the webapp that force the Android app to do native functions and sometimes call external scripts in PHP.
These php scripts get data posted to them (userid, friendid, etc) - currently the php scripts just make sure there is valid data being posted, then process the request if the data is present and valid.
I am looking for ways to increase the security of these php scripts to prevent bots / malicious users from posting false data to these pages - at this point nothings stopping anyone sending a correct userid/friendid and having the script from executing.
Session management would be the first line of defense, but since the webapp is in a different language I can't use that - and sometimes the php scripts are on a different domain completely (same server though).
The other method I considered was on sign up creating a user token to associate with a user, and saving this on the Android side of things - then when requesting these php scripts send the userid and token. And verify the token for that user matches in the remote database - this would make it harder to guess posting credentials for malicious user. Clearly not the best because the token is stored locally and going over the wire, but I digress.
Question are there any better methods to use in order to protect these lone php scripts from being executed, with out the use of session management? Does my token idea make any sense?
Note: I can use SSL on any / all requests.
I know exactly what you need, if you're up to the task. Your API needs to impliment OAuth2.0.
What OAuth can provide you is a secure way to pass information to and from your service while making sure that all secret information is kept private and that only the correct people can access that information. It gives each user a unique signature.
OAuth is used by Facebook, Google, Twitter and more to give developers a secure way to access information while keeping everyone from doing things they shouldn't be doing.
OAuth has support for ColdFusion, Java, PHP, C#, dotNet, VB.net, LIST, Javascript, Perl, Python, Ruby, and more.
http://oauth.net/
Session management or OAuth are the best solutions, but not the easiest. An easier way is implementing a hashing algorithm in both your app and the PHP scripts. When the app prepares a request, you hash some of the values that are being sent to the server using your secret method. This hash is being sent with the request. The server does the same and compares the two hashes. When they're the same, it knows the request is from the app (or someone who cracked your algorithm). When they're not, the server can simply ignore the request.
An example:
Data: userid = 2042; name = JohnDoe; email = john-doe#someprovider.com
Hash (in PHP, but you should implement it in the app as well):
<?php
$userid = 2042;
$name = 'JohnDoe'
$email = 'john-doe#someprovider.com';
// Remove some letters with other letters
$name = str_replace(array('a', 'd', 'g'), array('E', 'x', '9'), $name);
// Reverse a string
$email = strrev($email);
// Make a super secret hash (with salt!)
$hash = sha1('fnI87' . $useris . '87bk;s.' . $name . 'unf%gd' . $email);
// Some more fun
$hash = str_rot13($hash);
?>
Request: http://www.your-server.com/script.php?userid=2042&name=JohnDoe&email=john-doe#someprovider.com&hash=YOUR-GENERATED-HASH
Now the server can apply the same hashing method and compare the resulting hash with the hash sent with the request.
I'd like to suggest a more abstract approach, but similar to Jonathan's.
I make the following assumptions:
You have a PHP-script that anyone can call (if they know the URL / sniff the network packets).
Your android app is closed source; meaning that if you have a hash algorithm no-one but you will know what it is.
You want to prevent anyone from directly calling the PHP scripts - circumventing the app and any security you might have built in there.
What you need is way to identify that your app is sending the requests, and not someone else.
The idea is that you generate a signature for each request that only your app can make (ie. a salt + a hash).
$input = array(
"userid" => 1234,
"friendid" => 2345
"etc" => "..."
);
$salt = "s3kr4tsal7"; // this is essentially your app signature
$signature = md5($salt . serialize($input)); // you could also use json_encode or any other to-string serialization
// pick whichever is easy to do in PHP and in your app
$request = array(
"input" => $input,
"signature" => $signature
); // send this
Then in your PHP script check if the signature matches the calculated signature. This is similar to Jonathan's solution but it allows for any input, it's not dependent on $email or any other property. I also don't think you need an overly complex hashing algorithm, just md5 with a salt is 'hard enough'.
There is another type of attack you should be aware of and that is a replay-attack.
If you look at the RAW data going over the line, you could capture it and simply play it again. If you know what action has what output you can simply repeat the output.
The typical solution for a replay-attack is a trial-and-response. SSL does this for you but you could also make a custom implementation (but that is significantly more complex).
As usual, it depends on what level of protection you need, and how much you are willing to invest. Since you cannot use sessions, you need some sort of a stateless way to authenticate. There are generally two ways to do this: post credentials each time (e.g., basic authentication) or send some sort of a token (BTW, the session ID is exactly that, a token that links to a live session on the server).
When you generate the token, it is a good idea to use a standard and proven algorithm, instead of inventing your own and/or relying on obscurity. Even if it looks mostly secure, it might not be. For example, there are known attacks against the MD5 idea above (it is easy to append data to the message without knowing the key and obtain another valid MAC). HMAC-SHA1 is designed to avoid those.
First thing first: if you can, do use SSL for all requests. This would accomplish a few things at the same time:
users (your app) can be sure that they are posting their data to the right place (i.e., your webapp). SSL server authentication takes care of this.
it would make sure any crednetials/tokens you post are automatically encrypted.
replay attack become practically impossible
It seems you already have authenticated users, so issuing tokens should be relatively easy. You might want to think about the protocol to implement, but as you consider more cases, you will be getting closer to re-inventing OAuth and friends. Some things to consider:
an expiration period on tokens: so that even if someone gets a hold of one, they cannot use it indefinitely.
a way to revoke tokens
maybe have different tokens for different parts (services) of the webapp, so you can grant/revoke access to only the necessary services
To make sure you (i.e., your webapp(s)) are the only one that can issue said tokens you would want to sign them with a key only you have. Since the signer and verifier are the same (you) you don't have to use public key cryptography, HMAC should do. You could, for example, concatenate the username, issue time and any other relevant information, and use them as input to HMAC. Pack those parameters along with the signature (HMAC output) to create a token, and have the app send it with each request. Verify on the server and allow access if valid, require re-login (new token) if expired, deny access otherwise.
Alternatively, if you want to authenticate just the app, and not get user info mixed up in this, you could use a similar approach to sign requests on the client (app side). If you choose this way, do use a standard algorithm. This would, of course, require the signing key (in some form) to be in the app, so if someone gets hold of it (by reverse engineering, etc.) they could issue as many requests as they want. There are ways to mitigate this though:
implement the signing logic in native code
don't store the raw key, but derive it at runtime from bits and pieces stored in different places
And of course, the easiest way of all would be to require basic or digest authentication at the server (over SSL, of course), and embed the username and password in the app (sufficiently obfuscated). On the server side that would require only a change in server configuration, a few lines added on the client side. The downside is that there is really no way to change those credentials if they get compromised (short of releasing a new version and blocking access from the old one to force people to update; not pretty).