I am trying to pass a JSON string from one web application to another using URL parameters (for an internal SSO server).
What I need to do is be able to encrypt the JSON string (which is a user payload object) with a pre-shared key, forward the user to the service provider application with the payload attached as a URL parameter and then on the service provider application decrypt the payload back into a JSON string to get the required information.
Now this part isn't as much of an issue thanks to all of PHP's built in encryption functions but the next part is the difficulty. I am needing to embed a checksum within the encrypted string which can be checked when decrypting it so that if it has been modified in transit then I can raise an exception.
The purpose of this is to make sure that the user payload has not been modified in transit either accidentally or deliberately.
You want to provide more than a "checksum" (usually defined as "calculable by any party"); you want to provide an authentication tag or message authentication code (MAC). You have a couple options:
Use an "authenticated encryption" (AE) or "authenticated encryption with associated data" (AEAD) cipher to do this. AE(AD) ciphers provide an "authentication tag" over the cipher text, either in a single pass or with a repeated process over the encrypted cipher text. Examples (probably available in whichever PHP cipher library you're using) are GCM, EAX, and CCM. This is recommended, as the decryption operation will fail if the authentication tag is not verified, and only one shared secret (key) is necessary.
You can construct the system yourself using cryptographic primitives. This is less ideal, as you are responsible for more independent pieces, you need to manage more keys (if you have access to an OMAC implementation, you can use the same key), and your individual construction is not vetted by third parties (aka the collective work of the internet). If you follow this path, you need to keep some key details in mind:
Use a strong hash-based message authentication code (HMAC) such as HMAC/SHA-256, -384, or -512. Do not use SHA-1 or MD5, as these are easily brute forced.
Verify the HMAC before decrypting the cipher text. Any HMAC that fails means the entire cipher text should be discarded. You can remember this (on the generating side) as Encrypt Then MAC, and if you search for it, you'll see that not following this advice is the source of many cryptographic vulnerabilities and implementation exploits.
Verify the HMAC with a constant-time algorithm (i.e. do not use a short-circuit string equality comparison, the default in Java). PHP provides hash_equals to do this. Here's a quick explanation of timing attacks and a code review of a PHP example.
For either choice you'll want to encode the resulting cipher text and authentication tag with URL-safe Base64 in order to avoid data loss or corruption. If your message format is not strictly structured with included lengths, you'll have to pre-share the protocol ahead of time (i.e. for message m of length n bytes -> 16 bytes IV | n-48 bytes cipher text | 32 bytes HMAC).
Last note: always use a unique, non-predictable IV for each message that is encrypted with a key. Many people gloss over this, because it's "easy to just use 0x00 * 16", but any stream cipher mode of operation like CTR used as the foundation of GCM and CCM will lose fundamental security if two messages are encrypted with the same IV and key.
Related
One of the columns in a table in my API's database is an encrypted text formed of two pieces of information concatenated with a given delimiter, lets say a colon. The second piece of information is always, say, 12 characters long.
Does knowing this information open up a security flaw and potentially allow attackers to decrypt the information should they get a hold of it?
The information is encrypted using the php method openssl_encrypt() and with the 'aes-128-gcm' cipher.
What you're talking about is known in the literature as a partially known-plaintext attack (partial-KPA).
The information is encrypted using the php method openssl_encrypt() and with the 'aes-128-gcm' cipher.
AES-GCM combines AES-CTR with GHASH (a polynomial MAC over GF(2^128)).
Answering your question further requires delving a bit deeper into the cryptography weeds, but generally:
AES is considered a secure block cipher.
CTR mode turns a block cipher into a stream cipher.
Stream cipher encryption is conceptually equivalent to One-Time Pads, except the keystream is generated from the key and nonce, and thus can only have a finite (in this case, 128-bit) security level.
One of the columns in a table in my API's database is an encrypted text formed of two pieces of information concatenated with a given delimiter, lets say a colon. The second piece of information is always, say, 12 characters long.
The only thing you can infer from an AES-GCM ciphertext is the length of the plaintext.
In order to be able infer the remainder of the keystream from one byte of ciphertext (and a corresponding known plaintext for that byte), AES would need to be a very insecure block cipher. Since we know (thanks to roughly 21 years of cryptanalysis effort) that AES is a secure block cipher, and AES-GCM is a secure authenticated cipher mode, partial-KPAs are not a concern for this construction.
I'm trying to design a small api however I'm a bit stuck on how to secure the api. I have read some articles about how to do this one of them is: Login and retrieving an apikey and then hash some values with this apikey and send the hashed string back along with the request, so it can be done again on server level.
Is this a good way or is this dangerous to do?
If not miss understood, to avoid man in the middle I can add the request url to the variables that will be hashed, or isn't that the appropriate way
Also my brain is stuck on how to use a time stamp to avoid making lots of request to the same url with same data.
I'm sorry if my question have been asked a 1000 times. However I have read some articles now and it's still not clear to me what way to go for my small api.
From what I have read and understand from it this should be the way.
public key is stored in the application to let the user or application login.
server creates private key for this particular user when it's accessed. Or should this be always the same or a static value that has been created by a person?
user makes request sends along with the request a signature that's hash_hmac(some values+private key);
server checks if these value's are correct and does by creating the same hash from the value's that are send.
If server generates the same hash, the request is valid and can then be executed.
Is this they way to go or am I missing some mayor things here.
For hasing the data is the underneath way a good way to create a hash?
$l_sPrivateKey = 'something returned by database when user loged in';
$l_aData = array();
foreach($_POST as $key => $value){
if($key == 'signature') continue;
$l_aData[$key] = $value;
}
//This should then be the same as $_POST['signature'];
hash_hmac('sha256',serialize($l_aData),$l_sPrivateKey, false);
Your input would be appreciated.
Kind regards and thanks in advance
Secure Remote Password Protocol (SRP6a) With HMAC Fits Your Requirement
The following assumes that your API is browser-to-server so JavaScript-to-PHP not server-to-server using only PHP. SRP will work for both scenarios but the answer below discusses browser-to-server libraries.
Use the Secure Remote Password protocol to authenticate the user of the API which has the side effect of creating a strong session key. You can then use the shared strong session key to sign API requests and responses using HMAC.
RFC5054 uses SRP rather than public keys to create a shared session key to encrypt TLS traffic. There is an implementation in OpenSSL. This demonstrates that SRP authentication is a perfectly safe replacement to public keys to create a secure shared secret. IMHO using SRP is more convenient to solve your problem.
The Thinbus SRP library is a JavaScript SRP library which has a demo of authenticated to a PHP server. The PHP demo does not show using the shared session key but it is simply $srp->getSessionKey() on the server and client.getSessionKey() in the browser once the authentication protocol has finished. The default Thinbus configuration results in a 256bit shared key. You can use this with HMAC see the footnote 1 below about using signed JSON.
How It Works
The registration flow would be:
Client API registration form generates a random API password using JavaScript at the client which is not transmitted to the server. This is saved into the browser local storage and shown to the user asking them to print it off and keep a backup.
The password is given to the Thinbus SRP client JS library code which outputs a client salt and password verifier.
The salt and verifier are posted to the server and saved in the database for that client. Normally Thinbus recommends you keep the verifier hidden by using HTTPS to send the verifier to the server to prevent brute force attacks to recover the password. If you are using a random generated password as long as a typical software license key then you can transmit the verifier over HTTP. See footnote 2 below.
The API usage flow would start with an SRP authentication of the client that has the side effect of generating a session key. Note all this is in the Thinbus demo code as "standard usage" but is explained here to give a flavour of how STP authentication works. This authentication protocol is shown in sequence diagram of the thinbus page and is running in the online demos:
Client javascript loads the API password from browser local storage.
Client AJAX fetches from the server the client salt and a server random one-time number B.
Client javascript generates a one-time number A then uses the password, salt, and both one-time numbers to generate a session key K and hashes that with the both one-time numbers to create a password proof M that it posts to the server along with its random A.
Server uses the password verifier saved to the database at registration, the client salt, and the two random numbers to compute the session key K then confirms the client sent password proof M is good. If that is all good it sends its own proof M2back to the client. At this point the client has been authenticated using STP as a zero-knowledge proof of password.
Client checks M2 against its computation. If all is good both sides have a shared secret K which is a one time 256 bit session key derived from the random A and B that no man-in-the-middle can feasibly know.
All API requests and responses can be HMAC signed with the shared secret and verified on the other side.
All of the above is covered in the PHP demo of Thinbus minus actually calling $srp->getSessionKey() at the end to have a key that can be used to sign things with using HMAC.
Give that SRP replaces password authentication with a cryptographic zero-knowledge proof of password it is surprising that not all developers use it by default. The fact that it also generates a shared session key for API signing is simply an added bonus.
Footnote 1: Most APIs would prefer to post one JSON value with all the data in it. This is because JSON is simple yet more powerful with built in API in both PHP and JavaScript to turn objects into strings and back again. As #dbrumann pointed in a comment there is a standard for signing JSON which is JWT. Google suggest that here are libraries for this in both PHP and JavaScript. So if you upgrade to passing one JSON input value and returning one JSON output for every command in your API Ayou can use a JWT library to sign and validate the JSON inputs and outputs of the API. One of the JWS algorithms is "JWSAlgorithm.HS256 - HMAC with SHA-256, 256+ bit secret". The libraries will sort out the mechanics of actually signing and verifying so you don't have to write that code and worry about possible security bugs.
Footnote 2: The recommendation with Thinbus is to transmit the password verifier to the server over HTTPS to keep the verifier secret. This is to prevent interception then an offline dictionary attack against the password verifier to recover the password (i.e. the password is salted into the verifier so you would need to run the 16G crackstation password dictionary through the verifier generation code with the user salt to find a match). With API usage the browser window.crypto API can generate a truly random "API key". Typically windows keys were 16 upper case letters shown to the user formatted as XXXX-XXXX-XXXX-XXXX. Checking the GRC password search space page it says that a random 16 letter upper case password that size would take a government 14 years to exhaustively search. Given that estimation you can safely transmit a password verifier generated for such a long random password over plain HTTP without encryption as no-one will feasibly dedicate many years of computing power to run so many password guesses through the verifier generation algorithm (which uses the random client salt so cannot be pre-computed) to find a match to recover the client API password.
I've just been looking at adding a HMAC to PHP mcrypt encryption.
Is this simply hashing the encrypted data with hash_hmac using the encryption key and appending it to the encrypted data? Then on decryption you split off the HMAC, hash_hmac the rest of the data with the key again and check it matches the HMAC.
I'm confused because in this SO question When authenticating ciphertexts, what should be HMACed? it says:
you have to include in the HMAC input everything that impacts the decryption process, i.e. not only the encryption result per se, but also the IV which was used for that encryption, and, if the overall protocol supports algorithm agility, you should also input the specification of the encryption algorithm (otherwise, an attacker could alter the header of your message to replace the tag which says "AES-256" with the tag which says "AES-128" and you would unknowingly decrypt with the wrong algorithm).
Is this so? If this is true, why isn't using hash_hmac on just the encrypted data enough?
Short Answer: Yes
Long Answer:
HMAC is Hash-based message authentication code. You should HMAC anything which you want to authenticate, or in other words, anything which you want to protect against being modified.
Although the RFC standard is more complicated, it may make sense to think of HMAC as a salted hash.
e.g. hmac(message, key) = hash(message + key)
You can only recreate the same hmac with an identical message and key.
You can't recreate the same hmac if the key is identical but the message differs.
You can't recreate the same hmac if the message is identical but the key differs.
An attacker (who doesn't have the HMAC key) cannot modify part of the HMAC message without invalidating the existing HMAC. It really does depend on your data format and your usage of that data to determine what should be included in the HMAC message and HMAC key. But assuming you are using the HMAC to authenticate the decryption, then you should always include in the HMAC message anything that the decryption depends on. The symmetric key is typically used as the HMAC key.
In your quote, the poster says the IV and the algorithm should also be hashed. Consider a file/database format consisting of
ALGORITHM + IV + CIPHERTEXT + HMAC
If you only HMAC the ciphertext, an attacker would be able to modify the algorithm or IV (corrupting the file) without affecting the validity of the HMAC. This is bad because you can end up with a corrupted encrypted file with a valid HMAC. Decryption will proceed as normal because your software will think everything is ok. The result is a totally garbled decryption, but the point is that your software is broken because it returned the wrong output when decrypting and didn't give any errors. This can be classed as a 'security risk' if your application tries to do something with that erroneous data because it assumes it is correct. It is not a security risk in the sense that it makes the underlying encryption weaker or easier to crack. HMAC and symmetric encryption are two totally different technologies doing different things. The point of using a HMAC is that you can assume that the decryption layer is returning data which is 100% correct.
In the above example the ALGORITHM is a dynamic piece of data which I used to explain "algorithm agility" in the OPs quote. It defines what encryption algorithm was used. The point is that it is dynamic so it needs to be read from somewhere rather than hardcoded. This fact makes it a dependency of the decryption so it should be included in the HMAC message. However, if you always use some static algorithm then it should be assumed by (hardcoded in to) your decryption code and there is no need to store this data anyway. There is no need to include static data in the HMAC message because it has no affect on the decryption.
An example of a file format which uses a static algorithm is the open source AES-256 Crypt File Format. The algorithm is consistent and so it is always assumed. It actually uses 2 HMACs for speed reasons. 1 to authenticate the IV and keys, and the 2nd to authenticate the encrypted data part.
I've been having trouble trying to communicate between PHP and my iOS application using AES encryption.
So far, I've considered two methods of implementation. The first was to use OpenSSL.
On the iOS side, I implemented in a way to mimic the code shown here: http://saju.net.in/code/misc/openssl_aes.c.txt.
On the PHP side, I took the generated key and IV (from the iPhone) and used it as input to the PHP openssl encrypt.
The results differed in terms of the output...
I have also considered: http://iphonedevelopment.blogspot.com/2009/02/strong-encryption-for-cocoa-cocoa-touch.html
but this SO post: AESCrypt decryption between iOS and PHP deterred me.
The project is not tied down to AES, it just seemed like a strong encryption algorithm that wouldn't be too hard to implement.
My basic question is: what is the easiest way to implement a good encryption algorithm that can easily be used to communicate between iOS and PHP?
I just got through this same sort of project. I used the library you referenced in "also considered..."
Here is some example code to decrypt with php:
$iv2 = '';
for($i=0;$i<16;$i++){
$iv2 .= "\0";
}
$plain_text_CBC = mcrypt_decrypt(MCRYPT_RIJNDAEL_128, $key, $encrypted_text, MCRYPT_MODE_CBC, $iv2);
var_dump($plain_text_CBC);
Make sure your keys are both 256-bit (32 characters, I have not yet had any encoding issues, but if you do, remember that you are encrypting bytes, not characters). Note that 128 in MCRYPT_RIJNDAEL_128 is the block size and not the key size, while in the method AES256DecryptWithKey, 256 is a reference to the key size, while the block size is 128. AES256DecryptWithKey runs in CBC mode, but has a null initialization vector (iv).
CBC means that each block depends on the last block, and so it uses a pre-set, usually random, "block -1" called the IV
ECB means that each block is encrypted in the same way, hence it reveals when two blocks in the same message are the same. The library mentioned does not use it, so I mentioned it just for contrast.
The use of a zero iv (0000000000000000 in bytes) is considered insecure. To fix this you would have to create an NSData *iv variable for the IV and modify the CCcrypt argument of NSData+AESCrypt.m to add [iv bytes] for the iv parameter (I have not yet tested this code), and you would need to store this iv and pass it to the php along with you message. But first I would test and have everything working with a zero iv.
As said in the comments, it would probably easiest for you to use HTTPS.
I once set up an iPhone app that had to communicate with a PHP backend over HTTPS, and spent many hours trying to find out why the iPhone wouldn't accept the encrypted connection.
As it turned out, it didn't work because I was using a self-signed certificate on the server side. Buying an SSL certificate from a Certificate Authority solved all issues.
SSL certificates that validate a single domain name without company or extended validation are really cheap, so I suggest you give that a try!
For a direct example, my open source project "Techno Tap" contains PHP and iOS source that uses AES encryption successfully, feel free to take a look here
The encryption on iOS is done in ScoreboardManager.m (using NSData+AES) and decryption is done on the PHP side in Scoreboard.php
Here's a theoretical one that not only applies to PHP, but probably to more languages.
Let's say that I encrypt a string with the mcrypt library using and the AES-256 cipher. The string, encrypted, would now look similar to þøÆ{”ò(ü´îÚÜÇW¹ËŸK¯L‘rø?ª¶!JF£º+Œ’Ú'‚.
If the encryption key would change between the events of decryption and encryption, the result of the decryption would obviously be worthless.
Since an encrypted string contains, at least to me, random chars, It wouldn't be easy to run some sort of test on it to ensure that it is in encrypted/decrypted state.
I've spent some time thinking. How can I test that a string has been properly decrypted?
What if I appended a small prefix to the original string before encrypting it in the first place, and then removed this prefix upon decryption. If this prefix wasn't found, it would be safe to say that the decryption has failed.
Would this be an appropriate way to handle this?
To test data integrity you want a Message Authentication Code (MAC).
There are a few stand-alone MAC algorithms, which look like a hash function with a key. The very standard MAC algorithm is HMAC (which uses a hash function).
Since you also encrypt the data, you will want to use an encryption mode with a builtin MAC; there are a few such modes, such as GCM or EAX. Those modes apply to a block cipher, usually the AES.
Adding a known prefix or suffix to the data before encryption is a homemade MAC. MACs are subtle and error prone. For instance, if you add a CRC32 and then encrypt with a stream cipher (or a block cipher in CTR mode), then you are reproducing one of the seven capital sins of WEP (see section 4 in particular, for the CRC32-as-MAC issue). Basically your integrity check is no longer resistant to active attacks; you are only detecting innocent mistakes, such as using the wrong key.
(Unfortunately, it seems that MCrypt does not support any combined encryption/MAC mode. PHP itself, when compiled with the --with-mhash option, provides the mhash() function which implements both raw hashing, and HMAC.)
How can I test that a string has been properly decrypted?
The "small prefix" idea should be fine; also the excellent idea by #CodeInChaos. Other than that, storing the string in some defined format (like serialize() or json_encode()) and failing to restore it (unserialize(), json_decode()) would be indication of a broken decryption as well.