Related
in Facebook validation documentation
Please note that we generate the signature using an escaped unicode
version of the payload, with lowercase hex digits. If you just
calculate against the decoded bytes, you will end up with a different
signature. For example, the string äöå should be escaped to
\u00e4\u00f6\u00e5.
I'm trying to make a unittest for the validation that I have, but I don't seem to be able to produce the signutre because I can't escape the payload. I've tried
mb_convert_encoding($payload, 'unicode')
But this encodes all the payload, and not just the needed string, as Facebook does.
My full code:
// on the unittest
$content = file_get_contents(__DIR__.'/../Responses/whatsapp_webhook.json');
// trim whitespace at the end of the file
$content = trim($content);
$secret = config('externals.meta.config.app_secret');
$signature = hash_hmac(
'sha256',
mb_convert_encoding($content, 'unicode'),
$secret
);
$response = $this->postJson(
route('whatsapp.webhook.message'),
json_decode($content, true),
[
'CONTENT_TYPE' => 'text/plain',
'X-Hub-Signature-256' => $signature,
]
);
$response->assertOk();
// on the request validation
/**
* #var string $signature
*/
$signature = $request->header('X-Hub-Signature-256');
if (!$signature) {
abort(Response::HTTP_FORBIDDEN);
}
$signature = Str::after($signature, '=');
$secret = config('externals.meta.config.app_secret');
/**
* #var string $content
*/
$content = $request->getContent();
$payloadSignature = hash_hmac(
'sha256',
$content,
$secret
);
if ($payloadSignature !== $signature) {
abort(Response::HTTP_FORBIDDEN);
}
For one, mb_convert_encoding($payload, 'unicode') converts the input to UTF-16BE, not UTF-8. You would want mb_convert_encoding($payload, 'UTF-8').
For two, using mb_convert_encoding() without specifying the source encoding causes the function to assume that the input is using the system's default encoding, which is frequently incorrect and will cause your data to be mangled. You would want mb_convert_encoding($payload, 'UTF-8', $source_encoding). [Also, you cannot reliably detect string encoding, you need to know what it is.]
For three, mb_convert_encoding() is entirely the wrong function to use to apply the desired escape sequences to the data. [and good lord are the google results for "php escape UTF-8" awful]
Unfortunately, PHP doesn't have a UTF-8 escape function that isn't baked into another function, but it's not terribly difficult to write in userland.
function utf8_escape($input) {
$output = '';
for( $i=0,$l=mb_strlen($input); $i<$l; ++$i ) {
$cur = mb_substr($input, $i, 1);
if( strlen($cur) === 1 ) {
$output .= $cur;
} else {
$output .= sprintf('\\u%04x', mb_ord($cur));
}
}
return $output;
}
$in = "asdf äöå";
var_dump(
utf8_escape($in),
);
Output:
string(23) "asdf \u00e4\u00f6\u00e5"
Instead of trying to re-assemble the payload from the already decoded JSON, you should take the data directly as you received it.
Facebook sends Content-Type: application/json, which means PHP will not populate $_POST to begin with - but you can read the entire request body using file_get_contents('php://input').
Try and calculate the signature based on that, that should work without having to deal with any hassles of encoding & escaping.
hello fellow developers,
I’m facing an issue with the load callback (and the uninstall callback by extension).
I’m trying to verify the requests authenticity following the algorithm described in the documentation. https://developer.bigcommerce.com/apps/load#signed-payload
I am able to decode the json string and the data is correct, but the signatures never match. I made sure to use the right client secret and tried out different encoding/decoding scenarios with no luck.
An other concern is with the snippet of code (PHP) they provide in example (and in their sample app). They seem to return null when the signatures match and the decoded data when they don’t… (try secureCompare())
Meaning that the security test would pass every time, since in all my attempts the signatures didn’t match.
Am I missing something here ?
Edit: Here is the example in the doc. I can't really give you sample data as the client secret is to remain secret...
function verify($signedRequest, $clientSecret)
{
list($payload, $encodedSignature) = explode('.', $signedRequest, 2);
// decode the data
$signature = base64_decode($encodedSignature);
$data = json_decode(base64_decode($payload), true);
// confirm the signature
$expectedSignature = hash_hmac('sha256', $payload, $clientSecret, $raw = true);
if (secureCompare($signature, $expectedSignature)) {
error_log('Bad Signed JSON signature!');
return null;
}
return $data;
}
function secureCompare($str1, $str2)
{
$res = $str1 ^ $str2;
$ret = strlen($str1) ^ strlen($str2); //not the same length, then fail ($ret != 0)
for($i = strlen($res) - 1; $i >= 0; $i--) {
$ret += ord($res[$i]);
}
return !$ret;
}
You're not missing anything, and it's not a clock sync issue - the 28 lines of sample code provided both here and here has some pretty critical flaws:
The sample code does a hash_hmac of the raw base64-encoded JSON, instead of the base64-decoded JSON. (The hash provided to you by the BigCommerce API is really a hash of the base64-decoded JSON).
Since hash_hmac is called with $raw=true, this means the two strings will always be vastly different: one is raw binary, and the other is hexits.
Bad check of secureCompare logic. The if (secureCompare... part of the verify function expects opposite behavior from the secureCompare function. If the secureCompare function returns true when the strings match, why are we calling error_log?
Put all three of these issues together, and you end up with code that appears to work, but is actually silently failing. If you use the sample code, you're likely allowing any and all "signed" requests to be processed by your application!
Here's my corrected implementation of the verify function:
<?php
function verifySignedRequest($signedRequest, $clientSecret)
{
list($encodedData, $encodedSignature) = explode('.', $signedRequest, 2);
// decode the data
$signature = base64_decode($encodedSignature);
$jsonStr = base64_decode($encodedData);
$data = json_decode($jsonStr, true);
// confirm the signature
$expectedSignature = hash_hmac('sha256', $jsonStr, $clientSecret, $raw = false);
if (!hash_equals($expectedSignature, $signature)) {
error_log('Bad signed request from BigCommerce!');
return null;
}
return $data;
}
This does not work:
$jsonDecode = json_decode($jsonData, TRUE);
However if I copy the string from $jsonData and put it inside the decode function manually it does work.
This works:
$jsonDecode = json_decode('{"id":"0","bid":"918","url":"http:\/\/www.google.com","md5":"6361fbfbee69f444c394f3d2fa062f79","time":"2014-06-02 14:20:21"}', TRUE);
I did output $jsonData copied it and put in like above in the decode function. Then it worked. However if I put $jsonData directly in the decode function it does not.
var_dump($jsonData) shows:
string(144) "{"id":"0","bid":"918","url":"http:\/\/www.google.com","md5":"6361fbfbee69f444c394f3d2fa062f79","time":"2014-06-02 14:20:21"}"
The $jsonData comes from a encrypted $_GET variable. To encrypt it I use this:
$key = "SOME KEY";
$iv_size = mcrypt_get_iv_size(MCRYPT_BLOWFISH, MCRYPT_MODE_ECB);
$iv = mcrypt_create_iv($iv_size, MCRYPT_RAND);
$enc = mcrypt_encrypt(MCRYPT_BLOWFISH, $key, $data, MCRYPT_MODE_ECB, $iv);
$iv = rawurlencode(base64_encode($iv));
$enc = rawurlencode(base64_encode($enc));
//To Decrypt
$iv = base64_decode(rawurldecode($_GET['i']));
$enc = base64_decode(rawurldecode($_GET['e']));
$data = mcrypt_decrypt(MCRYPT_BLOWFISH, $key, $enc, MCRYPT_MODE_ECB, $iv);
some time there is issue of html entities, for example \" it will represent like this \", so you must need to parse the html entites to real text, that you can do using
html_entity_decode()
method of php.
$jsonData = stripslashes(html_entity_decode($jsonData));
$k=json_decode($jsonData,true);
print_r($k);
You have to use preg_replace for avoiding the null results from json_decode
here is the example code
$json_string = stripslashes(html_entity_decode($json_string));
$bookingdata = json_decode( preg_replace('/[\x00-\x1F\x80-\xFF]/', '', $json_string), true );
Most likely you need to strip off the padding from your decrypted data. There are 124 visible characters in your string but var_dump reports 144. Which means 20 characters of padding needs to be removed (a series of "\0" bytes at the end of your string).
Probably that's 4 "\0" bytes at the end of a block + an empty 16-bytes block (to mark the end of the data).
How are you currently decrypting/encrypting your string?
Edit:
You need to add this to trim the zero bytes at the end of the string:
$jsonData = rtrim($jsonData, "\0");
Judging from the other comments, you could use,
$jsonDecode = json_decode(trim($jsonData), TRUE);
While moving on php 7.1 I encountered with json_decode error number 4 (json syntex error). None of the above solution on this page worked for me.
After doing some more searching i found solution at https://stackoverflow.com/a/15423899/1545384 and its working for me.
//Remove UTF8 Bom
function remove_utf8_bom($text)
{
$bom = pack('H*','EFBBBF');
$text = preg_replace("/^$bom/", '', $text);
return $text;
}
Be sure to set header to JSON
header('Content-type: application/json;');
str_replace("\t", " ", str_replace("\n", " ", $string))
because json_decode does not work with special characters. And no error will be displayed. Make sure you remove tab spaces and new lines.
Depending on the source you get your data, you might need also:
stripslashes(html_entity_decode($string))
Works for me:
<?php
$sql = <<<EOT
SELECT *
FROM `students`;
EOT;
$string = '{ "query" : "' . str_replace("\t", " ", str_replace("\n", " ", $sql)).'" }';
print_r(json_decode($string));
?>
output:
stdClass Object
(
[query] => SELECT * FROM `students`;
)
I had problem that json_decode did not work, solution was to change string encoding to utf-8. This is important in case you have non-latin characters.
Interestingly mcrypt_decrypt seem to add control characters other than \0 at the end of the resulting text because of its padding algorithm. Therefore instead of rtrim($jsonData, "\0")
it is recommended to use
preg_replace( "/\p{Cc}*$/u", "", $data)
on the result $data of mcrypt_decrypt. json_decode will work if all trailing control characters are removed. Pl refer to the comment by Peter Bailey at http://php.net/manual/en/function.mdecrypt-generic.php .
USE THIS CODE
<?php
$json = preg_replace('/[[:cntrl:]]/', '', $json_data);
$json_array = json_decode($json, true);
echo json_last_error();
echo json_last_error_msg();
print_r($json_array);
?>
Make sure your JSON is actually valid. For some reason I was convinced that this was valid JSON:
{ type: "block" }
While it is not. Point being, make sure to validate your string with a linter if you find json_decode not te be working.
Try the JSON validator.
The problem in my case was it used ' not ", so I had to replace it to make it working.
In notepad+ I changed encoding of json file on: "UTF-8 without BOM".
JSON started to work
TL;DR Be sure that your JSON not containing comments :)
I've taken a JSON structure from API reference and tested request using Postman. I've just copy-pasted the JSON and didn't pay attention that there was a comment inside it:
...
"payMethod": {
"type": "PBL" //or "CARD_TOKEN", "INSTALLMENTS"
},
...
Of course after deletion the comment json_decode() started working like a charm :)
Use following function:
If JSON_ERROR_UTF8 occurred :
$encoded = json_encode( utf_convert( $responseForJS ) );
Below function is used to encode Array data recursively
/* Use it for json_encode some corrupt UTF-8 chars
* useful for = malformed utf-8 characters possibly incorrectly encoded by json_encode
*/
function utf_convert( $mixed ) {
if (is_array($mixed)) {
foreach ($mixed as $key => $value) {
$mixed[$key] = utf8ize($value);
}
} elseif (is_string($mixed)) {
return mb_convert_encoding($mixed, "UTF-8", "UTF-8");
}
return $mixed;
}
Maybe it helps someone, check in your json string if you have any NULL values, json_decode will not work if a NULL is present as a value.
This super basic function may help you. I made the NULL in an array just in case I need to add more stuff in the future.
function jsonValueFix($json){
$json = str_replace( array('NULL'),'""',$json );
return $json;
}
I just used json_decode twice and it worked for me
$response = json_decode($apiResponse, true);
$response = json_decode($response, true);
Im trying to encrypt data through a custom function i created... (based on base64)
The code works but to a degree... it gives random results (sometimes work and sometimes doesn't).
<?php
set_time_limit(0);
class encryptor{
function encrypt($data){
$all = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789!?#,.&*()$; ";
$chars = str_split($all, 1); // Split $all to array of single characters
$text_chars = str_split($data, 1); // Split $data to array of single characters
// Create array of unique results based on the characters from $all
foreach($chars as $char){
$array[$char] = md5(uniqid(rand(), true));
}
// Replace the input text with the results from $array
foreach($text_chars as $text_char){
$data = str_replace($text_char,$array[$text_char], $data);
}
// Encode and compress $array as $solution
$solution = gzcompress(base64_encode(json_encode($array)),9);
// Return the encoded solution + Breaker + encoded and compressed input text
return $solution."BREAKHERE".gzcompress(base64_encode($data),9);
}
function decrypt($data){
// Break the encrypted code to two parts
$exploded = explode('BREAKHERE', $data);
// Decoding the contents
$contents = base64_decode(gzuncompress($exploded[1]));
// Decoding solution ($array)
$solves = json_decode(base64_decode(gzuncompress($exploded[0])),true);
$fliped = array_flip($solves);
// Replace the encrypted data with the solution
foreach($solves as $solve){
$contents = str_replace($solve,$fliped[$solve], $contents);
}
return($contents); // Return decoded data
}
}
$text = "11 1";
$enc = new encryptor();
$encrypted = $enc->encrypt($text);
$decrypted = $enc->decrypt($encrypted);
echo $decrypted;
?>
Since its just for fun, this change appears to make it work:
// Replace the input text with the results from $array
$encrypted = '';
foreach($text_chars as $text_char){
//$data = str_replace($text_char,$array[$text_char], $data);
$encrypted .= $array[$text_char];
}
Running str_replace in the loop results in substitutions of data that have been previously substituted.
Aside from that, to make transport of the encrypted data easier, I'd change:
return $solution."BREAKHERE".gzcompress(base64_encode($data),9);
to:
return base64_encode($solution."BREAKHERE".gzcompress($data,9);
and then make the appropriate changes in the decryptor. The compress data can have null characters and non-printable characters so if you base64 encode the entire result you can pass it around more easily.
I have been trying to get something called simplesamlphp hooked up to a django app.
I'm almost there… although i need to duplicate, in Python, the functionality of this php script:
I have copied the contents of the $raw variable in php, to the file 64.rtf. However when i run the Python equivalent i get an error stating:
TypeError: Incorrect padding
PHP code:
function getValue($raw) {
$val = $raw;
$url = parse_url($raw, PHP_URL_QUERY);
if (!empty($url)) $val = $url;
$arr = array();
$query = parse_str($val, &$arr);
#echo('<pre>');print_r($arr);
if (array_key_exists('SAMLResponse', $arr)) return $arr['SAMLResponse'];
if (array_key_exists('SAMLRequest', $arr)) return $arr['SAMLRequest'];
if (array_key_exists('LogoutRequest', $arr)) return $arr['LogoutRequest'];
if (array_key_exists('LogoutResponse', $arr)) return $arr['LogoutResponse'];
return rawurldecode(stripslashes($val));
}
function decode($raw) {
$message = getValue($raw);
#echo 'using value: ' . $message; exit;
$base64decoded = base64_decode($message);
$gzinflated = gzinflate($base64decoded);
if ($gzinflated != FALSE) {
$base64decoded = $gzinflated;
}
$decoded = htmlspecialchars($base64decoded);
return $decoded;
}
I have only come up with this in Python so far:
string64 = open("64.rtf", "rU").read()
decodedstring = base64.b64decode(string64,)
What am I not getting? the rawurldecode(stripslashes bit?? or url_parser??
and what exactly does these do thats so essential to the decoding?
I hope you can help. thanks…
Here it is, in all of its glory.
#!/usr/bin/env python
import base64
import zlib
import cgi
import urlparse
def getValue(raw):
args = urlparse.parse_qs(urlparse.urlparse(raw).query)
keys = ['SAMLResponse', 'SAMLRequest', 'LogoutRequest', 'LogoutResponse']
for key in keys:
if key in args: return args[key][0]
def decode(raw):
message = getValue(raw)
message = message + "=" * (4 - len(message) % 4)
base64decoded = base64.b64decode(message)
try:
base64decoded = zlib.decompressobj().decompress('x\x9c' + base64decoded)
except zlib.error:
pass # may want to handle this error
return cgi.escape(base64decoded, True)
data = 'PHNhbWxwOlJlc3BvbnNlIHhtbG5zOnNhbWxwPSJ1cm46b2FzaXM6bmFtZXM6dGM6U0FNTDoyLjA6cHJvdG9jb2wiIHhtbG5zOnNhbWw9InVybjpvYXNpczpuYW1lczp0YzpTQU1MOjIuMDphc3NlcnRpb24iIElEPSJwZnhkYTAxMjkzOC03MDkxLWNjZjQtZTc2Ny0wZWQ4OGVhN2Q1YmYiIFZlcnNpb249IjIuMCIgSXNzdWVJbnN0YW50PSIyMDExLTAxLTIxVDEyOjI4OjI5WiIgRGVzdGluYXRpb249Imh0dHBzOi8vd2F5Zi5teWRvbWFpbi5kay9zaW1wbGVzYW1sL3NhbWwyLW15bG9naW4ucGhwIiBJblJlc3BvbnNlVG89Il82ZDhmNDAxZDUzYTg1NDkzMzY2N2FiNWU5NzE1NWNmMzJjYWExMjBkZDciPjxzYW1sOklzc3Vlcj5odHRwczovL3Rlc3RicmlkZ2Uud2F5Zi5kazwvc2FtbDpJc3N1ZXI'
url = "http://www.google.com?SAMLResponse=" + data
print decode(url)
The reason you were getting an error when trying to b64decode your string, is because the string is not a TRUE base64 encoding. base64 encoded data is always a length that is evenly divisible by 4. Your string is not. In order to make the string length evenly divisible by 4, '=' characters are appended to the end of the string. In your case, the length % 4 == 3. So, we need to add one '=' to the end of the string to get it to decode correctly.