The problem is the base 64 encoded iv(in PHP) doesn't match when decoded in IOS
The scenario is I am using Blowfish Algorithm CBC mode . Simply server(PHP) generate a random cryptographic IV
and encoded base64 format , send to IOS .
The Problem is here when-time I try to decode , the decoded results are not correct always . Sometime it generates correct and rest of time error.
Here is my PHP code.
public function ivGenerator()
{
$ivSize = mcrypt_get_iv_size(MCRYPT_BLOWFISH, MCRYPT_MODE_CBC);
$iv = mcrypt_create_iv($ivSize, MCRYPT_RAND);
echo base64_encode($iv);
}
Here is two result set of ivGenerator which i will use later for decoding
bTpkpLxMbNo=
aIvJeujxW7w=
Here is my IOS code .
NSString *ecodedIVString = #"bTpkpLxMbNo=";
NSData *ecodedIVData= [[NSData alloc] initWithBase64EncodedString:ecodedIVString options:0];
NSString *decodeIVString = [[NSString alloc] initWithData:ecodedIVData encoding:NSASCIIStringEncoding];
NSLog(#"decodeIVString %#" , decodeIVString); // m:d¤¼LlÚ //its correct
If I use ,
NSString *ecodedIVString = #"aIvJeujxW7w=";
Then the result is hÉzèñ[¼ , but the actual result should be h‹Ézèñ[¼
I have tried
NSWindowsCP1252StringEncoding , NSISOLatin1StringEncoding , NSUTF8StringEncoding
But all of them are unable to generate accurate result (approx. 10 test I have found it).
Thanks in advance !
You should print out the result in hexadecimals and then perform a comparison.
It seems that the encoding on both platforms is the same (something you should not take for granted) so you get the same characters back. However, it isn't sure that the font or font version you are using shows the characters. It could also be that one platform shows an ‹ character for unknown encoding sequences, and the other doesn't show anything at all.
This brings us to the basic issue: if you decode random byte values then you may encounter byte sequences that do not translate to a character. So comparing binary strings by converting them to a printable string is no good. Instead you need to directly compare the bytes (e.g. by saving the IV to iv.bin files) or by translating them into hexadecimals.
It's extremely likely that the IV's are identical, despite the fact that the printout is different.
I need to translate some existing PHP code to Python. This job connects to gravity forms and queries for certain data. In order to make the query, a signature must be calculated in order to verify the connection.
The Gravity Forms web api gives good PHP directions here.
The PHP method is as follows:
function calculate_signature( $string, $private_key ) {
$hash = hash_hmac( 'sha1', $string, $private_key, true );
$sig = rawurlencode( base64_encode( $hash ) );
return $sig;
}
Based on my understanding of Python and the information about hash-hmac and rawurlencoded from php2python.com, I wrote the following:
import hmac, hashlib, urllib, base64
def calculate_signature(string, private_key):
hash_var = hmac.new(private_key, string, hashlib.sha1).digest()
sig = urllib.quote(base64.b64encode(hash_var))
return sig
However, the two signatures are not equivalent, and thus Gravity Forms returns a HTTP 403: Bad Request response.
Am I missing something within my translation?
Update (11/04/15)
I have now matched my php and python urls. However, I still receive a 403 error.
The reason the php and python signatures did not match had nothing to do with their calculate_signature() methods.
The issue was caused by differing expires variables. Php used strtotime("+60 mins") which resulted in a UTC time 60 minutes from now. Whereas Python used datetime.date.now() + timedelta(minutes=60). This is also 60 minutes from now, but in your current timezone.
I always want to calculate the expire variable in UTC so I replaced my Python calculation with datetime.datetime.utcnow() + timedelta(minutes=60).
You're almost there. urllib.quote does not encode slashes, for example, as PHP's rawurlencode does. You can use urllib.quote_plus to achieve the desired effect:
import hmac, hashlib, urllib, base64
def calculate_signature(string, private_key):
hash_var = hmac.new(private_key, string, hashlib.sha1).digest()
sig = urllib.quote_plus(base64.b64encode(hash_var))
return sig
I have an iOS app and PHP web service that take a user submitted "password" and hash it.
The problem arises when you enter an emoji:
PHP hashes the salt + 😍
iOS hashes the salt + [heart-for-eyes smiley emoji]
Is there a way to convert the emoji on iOS to the corresponding HTML Entity (like PHP does automatically) so that the resulting hash values are the same?
stringWithUTF8String does not work
NSNonLossyASCIIStringEncoding converts it to "\ud83d\ude0d"
Here is a solution, perhaps there is a built-in:
NSString *emojiString = #"😍";
NSLog(#"emojiString: %#", emojiString);
NSData *codePointData = [emojiString dataUsingEncoding:NSUTF32LittleEndianStringEncoding];
u_int32_t codePoint = *((u_int32_t *)(codePointData.bytes));
NSString *escapedEmoji = [NSString stringWithFormat:#"&#%d", codePoint];
NSLog(#"escapedEmoji: %#", escapedEmoji);
NSLog output:
emojiString: 😍
escapedEmoji: 😍
This based on the fact that the escaped emoji is the decimal number of the code point. The binary code point is the UTF-32 encoding of the unicode character.
Unfortunately I was not able to find a pre-existing solution.
Preface:
I've a web page with a form and a text field.
1) On submission, the text in the field is sent with ajax to a php script (with GET method).
2) The php script gets the text and passes it as a parameter to a shell tool.
3) The shell C tool parses argc into an array of unichars (actually an NSString in my current implementation)
(4.. 5.. 6.. then the tool does his job, returns a result to stdout that the php script serves back as response to the web page...)
I'm looking for the correct / canonical / "unicode" way to do each step so that: content is properly encoded and preserved, no security issues come out.
What I'm doing now:
1) (JavaScript) the text is retrieved from the form this way
theText = $('#theField').attr('value');
and sent to the server this way
httpReq.open('GET','myScript.php?theText=' + encodeURIComponent(theText),true);
2) (PHP) I get the text
$theText=(isset($_GET["theText"])?$_GET["theText"]:"");
I call the C tool
$cmd = "/usr/bin/thetool -theText ".escapeshellarg($theText);
echo shell_exec( $cmd );
3) (Objective-C) I'm on MacOS X, so I take advantage of NSString and NSUserDefaults classes (but a plain C solution would be good for me as well, assumed that I'll end up with an array of unichars)
int main(int argc, const char * argv[])
{
NSUserDefaults *userDefaults = [NSUserDefaults standardUserDefaults];
NSString *theText = [userDefaults stringForKey: #"theText"];
Question(s)
Is this the good way?
Is escapeshellarg alone safe when invoking shell_exec?
Am I going to lose some characters along the way if the user types something peculiar?
Waiting from a competent reply, I've started making some empiric tests...
First I changed
echo shell_exec( $cmd );
to
echo $cmd;
to see what the command line invocation was turning out to be given various text entered in the form. It seem that escapeshellarg on the PHP side do a good job.
The text passed to the Tool seems to be always properly sealed between single quotes, with "dangerous" character well escaped. I found no way to tamper with the tool invocation.
Then I tested for the text passed to see if something was getting lost somewhere.
I set up the C tool this way and looked for the output
int main(int argc, const char * argv[])
{
NSUserDefaults *userDefaults = [NSUserDefaults standardUserDefaults];
NSString *theText = [userDefaults stringForKey: #"theText"];
int i;
unichar c;
for(i=0;i<[theText length];i++)
{
c = [searchString characterAtIndex:(NSUInteger) i];
printf("%d\n",c);
}
return 0;
}
Made various tries. It seems all ok. As the last test I entered a "MUSICAL SYMBOL G CLEF" in the form
http://www.fileformat.info/info/unicode/char/1d11e/index.htm
It turned out to correctly end into the tool as a couple* of unichars
55348
56606
(* being this a very special character whose code exceeds 65535 it need to be represented with a couple of surrogate unichars. This is the most edge case I found).
Anyway as I stated at the beginning these are just empiric tests. I don't like to assume that sensible code is good just beacuse passes a dozen of test. I'd very happy to receive comments or suggestions (or warnings!).
I tested on Mac OS X - Firefox on the client side - Mac OS X - Mamp on the server side.
I am building a library to communicate with the back end that drives our iPad client.
The basic Structure for all requests is:
{
"metaData":{ JSON Object that is not important for this question },
"requestData":
{
"nonce":"random string",
"params":"JSON string containing request data"
}
"checksum":"hash of JSON string representing value of requestData",
"connectionString":"someIdentifier"
}
For the specific case where I am having trouble is when requestData has the following structure:
{
"requestData":
{
"nonce":"random string",
"params":
"{\"userId\":1,\"formData\":\"encrypted string, then Base 64 encoded on iPad\"}"
}
"checksum":"hashed value of string representation of requestData"
}
An example of the string (as printed by NSLog) used to generate the checksum is:
{"params":"{\"groupId\":3,\"formData\":\"SExvR0J1ZkJSQkhObU5xZkiEXdBede2moVN3LtMDZlxcXYVj7Uz!BFdiQC9SwxIhrrcGv2GtWJzjqMhHzdFDZW568tbnLTKQ9931efrpjtvqlK9mudInXj0FQdBLY0M6f9zBlLu6TcQ7sA6AD15DF0HyUPIi4fnc90ZV7omGqRpyI412aGSpDPJEbCUBSY5WMUFJqRstyK1+Qo0vmN8uMprztDIyEFufP24DHHtYZHVAic8Sg8CxbsUTTYDgDc!0ASQwahEgy1sWkMP!BVpK8VU7quXDdIJrxbSNL7OO4tsJrHIXyhhK7ZUNKMaZX+fBSdw6DbNtTM86K0X4NSRXPVLE0EAklAJ2OpMDBsoz9k!jhCba5gRXY7r48USpsMyyj1v8SsDKn58FsvDxsdCrPY77KmIX3Icy!n3iA!lBfc3ol6c90wkwPSqNvnO7uRDYrfbP1c0zRYSXbLTQvHLLdfAWKariCKtNg6YAXNfgQ6lWFRXce8flHgUz6E7rkt9tjc9i4K+EjcL10H+E3AGkidYPGtQOm1vey!M8oineM!Cgg3VcvNCv!yN90iq3T+tqI0ivvBnh+1aCw2H90tnNm8Gi+XCrIdhORN3QjSkkNbpfoSCLoIkuBmXlNuTskaJ4nnV3kHrmU!4hYMeZIIZ8OnZWPpU47xJi!kh3MDdI2c+WorT+y+M5XwcQO6jGv3tXyRVBy!ne+sSnU!InISm7x1VQjJLmjULMnqxRDoZatBsofxICJysEUaDJvgwZasMJpQk1zyrPraBWBJ0lVVaWhH5OTi6U0!hHNVs5Xf+H23JxmPpUNWqNvsAGfnTfY!kSoiLoSxEocICK8zsJFMc69101DNAanayf!MjFFDeFRlzpKhcRON7cxDfvBdSoc9hL1lcMzFbLemrL1w8jNNMfKlY7QDZ5ebOERJMjY0!o8znlxOa0ViuJ++O7+QrT!mGdSQYGh3NJ3MK1IdJkXuFpY!guyXOgohTsqcD0DZSk84OsI76L18snFvs4qMHw9SUf3l0jWPxbTYimmlM3DVUR7Sn7xOsGmQGcwpGK1tinlIDA+w8Ci+CLWESsjZ5QDQCr\",\"internalFormId\":\"MTN13511759141\",\"userId\":1,\"code\":\"\",\"_queuedSubmission\":false,\"groupName\":\"LDMAdmin\",\"saleType\":\"1\",\"formTableName\":\"myTableName\",\"emailName\":\"default\"}","nonce":"XqfK9Nxwuggw4m"}
When I examine the server logs, I find the checksum generated on the device does not equal the checksum generated on the server (in PHP, case sensitivity is not the issue), thus authentication fails.
An example of the structure that fails is:
PageManager.m - 905 -> Final Request:
{
"checksum":"9D51170D1510C4081936870D11E96C869DB26B895393B9C14B2A6BC3C1F10F23",
"connectionString":"testBed",
"metaData":
{
"accessToken":"myAccessToken",
"appId":"myAppID",
"deviceId":"1X:1X:1X:1X:1X:1X",
"groupId":3,
"groupName":"LDMAdmin",
"timestamp":1351018002.780379,
"userId":1,
"useragent":"iPhone OS",
"username":"admin"
},
"requestData":
{
"nonce":"1iezcBdjbE",
"params":"{\"groupId\":3,\"formData\":\"Tnh3dWdndzRtRmdxSFNmN/NXCIQSukpx3+mhmbNQh0PTGbLlEFoDinyrq3wRJGZ+8sQ/+xcjS4cU7evluipxqQDZIOvp4ZcoDnxTPeqBZJrG/bq5FHR6PVCYK2DaLHfj025z/H3RM8dUEoWcrTLqSUcW+E7Mfl8ZCApqJMxSa8eYYqLT7tm7r1SC+bjXNOQZLTC2laFhihQ5hLKqFFnO/z3AlUYAAUhKKD1lWIipnJUUNoyHdWuuOobMSS1ZZP5f5f+RTFsmGZUDe6qX6h2cjIQ2+VGPIsP//gqwO4iDx/FdHD+xrjCyEgL2Va/m/Z+ANxCr3DN2o2Jnwg8B8QycFN2tGrgusseAgoa9Ng9LRgooZW+KuECWDhorHzvuv2rOlhOskymj4XTu8890ZMJbcr1Ic6zwztm82R1qKaoy1o6gIbUNtVZFSqUlP8TO7mWHKr3Y8Awn7ih9HzSOg1486EDL4OjfOR9J2pw1jbK7ZJb7LxzrWFgoyrwDBAw3q7PrV4Ml9ngI6oXOh3veAq/wulyBOdF46n7evqIkAKg4FYdvzmFKd2bgOpxwBlAI7vL2IiC4v8GXI5977SkPPEKUZHXWmfrgr/VzF79gIxJDqV9N0ceAcgY8bWbBXf7DLd9H82obFa60yZBo5/MBjq9SNuD08vJEEauVGs4wfDr9+xzsr3z+plqxAejODdxKfF48Ra21L8Xozozv5papTP9cpGVU11mCWj+no5gtM0VQKRB7IQcpDWjQgQyThN2aoE06ecA2gY5SSXN0XHVRw5OKM0/rlNIuMiqow5wqHLl41IzDSF2HuJKj06Lv8t5CLLOd9rkOjYw6w8SrbsZeG5jwagJkyQ0UuKu+PIoIc2DJnUWDC5iqlb0TO9nPDNFKad+MYlfgDR0CxR+3ddkqWNBNSW5rsh5QZDlJHDjhQFLkuqiiRAnMvKOcbqAnXIZ9EuAo/DkcmtGPHkEyEaA2cb3mXysBP49jhY0m/qinloza+j3d7Kb/Fu35U929fOxH6+W+5oZv/r+a9KvkDhPoRwiFouVwTtTOwbjVDT+NEg2OUfDaEYbQ/RbM7i6X+XjSkZMLYsRs1Q9CwdBabY860uBNFQ==\",\"internalFormId\":\"MTN13510178111\",\"userId\":1,\"code\":\"\",\"_queuedSubmission\":false,\"groupName\":\"LDMAdmin\",\"saleType\":\"1\",\"formTableName\":\"myTableName\",\"emailName\":\"default\"}"
},
"url":"http://myurl.com/myAction"
}
The formData is generated by creating a JSON string of the desired data, encrypting this string, and encoding the resulting NSData object into a base 64 string. The checksum is generated by taking the hash of the JSON string representing the requestData value (with SHA256).
requestData is converted to a JSON string, and hashed with:
+(NSString *)hmacSHA256:(NSString *)string withKey:(NSString *)key {
NSString *hash = 0;
NSData *hmac = 0;
NSMutableString *temp = [[NSMutableString alloc] initWithString:#""];
#ifdef DEBUG
NSLog( #"%s - %d -> Values:\nString: %#\nKey: %#", __FILE__, __LINE__, string, key );
#endif
const char *cKey = [key cStringUsingEncoding:NSASCIIStringEncoding];
const char *cData = [string cStringUsingEncoding:NSASCIIStringEncoding];
unsigned char cHMAC[CC_SHA256_DIGEST_LENGTH];
unsigned char *digest;
unsigned int dLength;
CCHmac( kCCHmacAlgSHA256, cKey, strlen(cKey), cData, strlen(cData), cHMAC );
hmac = [[NSData alloc] initWithBytes:cHMAC length:sizeof(cHMAC)];
digest = (unsigned char *)[hmac bytes];
dLength = hmac.length;
for ( int i = 0; i < dLength; ++i )
[temp appendFormat:#"%02X", digest[i]];
hash = [[NSString alloc] initWithString:temp];
return hash;
}
The PHP script uses a stock function to generate the hash:
protected static function _generateChecksum($data, $key, $output)
{
$jsonData = json_encode($data);
log_message('error', 'encoded string is: '.$jsonData);
$checksum = hash_hmac('sha256', $jsonData, $key);
//log_message('error', 'Checksum pre encode: '.$checksum);
// if($output == true)
// {
// $checksum = base64_encode($checksum);
// }
return $checksum;
}
The hash is compared to the hash sent in the request.
An example of the server log:
ERROR - 2012-10-23 21:31:16 --> Encoded Checksum recieved is: 14F03A1DCAEA9DBBC7EC8CA1D666D89C391760AA246C91B52164D526AC83E5E5
DEBUG - 2012-10-23 21:31:16 --> Model Class Initialized
DEBUG - 2012-10-23 21:31:16 --> Model Class Initialized
DEBUG - 2012-10-23 21:31:16 --> Database Driver Class Initialized
ERROR - 2012-10-23 21:31:16 --> encoded string is: {"params":"{\"groupId\":3,\"formData\":\"SE5tTnFmWHFmSzlOeHd1ZyYzHN9y\/aJSbsqEn6X1TypGwWcXtXGpW4ODrCDVwZyIjhq3oOeZ4C6hGCDFGqHsa5hhNXxeerWLG5SyvfksCTG1+GCvWFwMx0CzZwOAfJRwSoCBCaeZ\/pivs3dHQS22SEbWn6+2e2vayeap7mxvZZw9Jrl\/c4dGFAiNqQB5pQbNO661AqbJWDAHCS8EWBhXXsd0SbTHlZAip4H0MdlF2rnElCVfHlc01RcuJNXLF3NJvfjY9m4sXmI3BAED0c0C\/i0Uw2M6pe4iDJv\/OvOI0NVS8RKbRbjhTo3oktAmNttfKTG6xp0wMhbANppuoo4QY3XwQ5BKUjqhmr5kx8j0RTmebcTCmxsC9h1dqjHYnf1JnZDFATkVsKnn\/Ela1wSjhGL7uP6jl3r4xDGKGPWDj0E3iAPNN56pmJxzyQrHOOqUzGbmvU3qj7Ul039IGYZTzn74VUkWi3JsxJH+kU9iWSvuC+YoOHcf\/0OFn1PqBoDHjDTbN+3HV8wwSqrVFJ6z9RX4MwRfffVgKl8xL2hHqBnegjvyd65KbZbSd\/3OrEBeL0dAjuARiPioNTpjzwga4chFRA471gweLT+cKweZBXZMYll36sNqulIBzbCmqbndDk63Id9iSrs9\/fQVWUA7RJDudnAxvQPs8gTznp9Dz1SomyY4ONYrJ9EticAEnUEjF2sCdejYlgu61a3Zss19m+MzgEhxkwmRwttsRbFfNK44wP\/wB2FgdfjsY94nHpJ+6lPEZtRmWpYtNVxQMVC6mMde6CbSEem71byIiN424baPImtNIfF+bKl6BKxyEl7BhI3z25NXaKyfaflzxGY8Yvdg0f73SfT3omPP3KxdudFgJrQ6eiO2AXt5L6lPjezjRr17R6hTUNmYwvZ3C5S0zoY7ynmCHebeiNlavVepUBkn0Iu6w\/qKDJ5wr80n7XX3EXuo1ODCC5aCjOSr+gSS9eVm0\/IBQNF\/ec4kjI29LRyrOFOS\/2poHY9XzyagVURiwi101a0yPETRRsC8n4B\/XFmOFQ0VcCQNgTuXute52fsccxB3DkV7ixBbQ8mt6o2XDWGk2HnrDwmRuNX87rBHow==\",\"internalFormId\":\"MTNC13510277491\",\"userId\":1,\"code\":\"\",\"_queuedSubmission\":false,\"groupName\":\"LDMAdmin\",\"saleType\":\"1\",\"formTableName\":\"myTableName\",\"emailName\":\"default\"}","nonce":"gw4m"}
ERROR - 2012-10-23 21:31:16 --> Encoded Checksum expected is: 8d999d76e48907905e701da3ccdbccb4061d05ed5a7c18b58507b6e6352fb1f5
However, other components of the application use the same structure (minus the base 64 encoding) behave as designed and expected. The checksum is generated using the same method and key as the example above. Example:
{
"requestData":
{
"nonce":"random string",
"params":"{\"latitude\":37.7,\"longitude\":-122.4}"
}
"checksum":"hashed value of the string representation of requestData"
}
I have no idea why the PHP script on the server would be generating a different hash in the failed case mentioned above. Has anyone encountered this type of problem before?
UPDATE:
I am using SBJson for encoding and decoding my data structures into JSON strings (link).
UPDATE 2:
Base on the discussion that has occurred so far, there is no guarantee the order in which a JSON object (or in my case an NSDictionary) is serialized to a JSON string. But if that were the case, I do not understand why cases where data is sent in clear text, the same hash is generated on the server, and in the case where some data is in the form of base 64 string, a different hashes are generated between the server and the client.
The only solution, which is probably not right solution, is to remove / from the base 64 alphabet (I replaced it with !). I am pretty sure this will cause problems when trying to decode the string on the server side.
At this point, I would like to understand how a forward slash could be causing this problem.
There's no such thing as "the JSON string representing such-and-such". If you have a data structure in memory and use your platform's default JSON serializer on it, you have no guarantee that it will produce the same string as another platform's default JSON serializer -- or even that it will produce the same string as the JSON serializer you use will produce tomorrow, or on Fridays in leap years.
JSON gives serializers considerable latitude in where to place whitespace, how to format numbers, which order to emit object fields, and so forth. It cannot be used as a basis for hashing the underlying abstract dataset.
It was the forward slash that was in the encoding table, and somehow, this was causing a conflict when generating the hash in the php code.
The original table in my Objective C code:
//64 digit code
static char Encode[] = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
Which I had to replace with:
//64 digit code
static char Encode[] = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+!";
This may be only a temporary solution, since this SO answer states that forward slashes are acceptable in a base 64 encoded string. Simply replacing the forward slash solved the issues for now.