I am trying to unserialize a longtext (utf8_unicode_ci) from a database query, but when I do I get the following error.
unserialize(): Error at offset 6 of 737 bytes
I guessed that my string was invalid so I outputted it to the browser. I noticed the string length was a little longer then what it was actually supposed to be. When I copy and paste the output in the browser and hard code it into the php the sting length changes to a smaller one and the unserialize works.
Database string...
ASCII - string(737) "a:10:{s:2:"id";i:2234950;s:13:"full_describe";s:11:"Brace start";s:6:"person";s:17:"Dr Aalok Y Shukla";s:8:"datetime";s:25:"2014-01-06T09:00:00+00:00";s:8:"duration";i:30;s:11:"on_waitlist";b:0;s:10:"company_id";s:5:"35927";s:8:"attended";b:1;s:6:"_links";a:2:{s:4:"self";a:1:{s:4:"href";s:60:"http://uk.bookingbug.com/api/v1/admin/35927/bookings/2234950";}s:6:"client";a:1:{s:4:"href";s:57:"http://uk.bookingbug.com/api/v1/admin/35927/client/809828";}}s:11:"appointment";a:1:{s:11:"description";s:11:"Brace start";}}"
Copied and pasted string into php...
ASCII - string(517) "a:10:{s:2:"id";i:2234950;s:13:"full_describe";s:11:"Brace start";s:6:"person";s:17:"Dr Aalok Y Shukla";s:8:"datetime";s:25:"2014-01-06T09:00:00+00:00";s:8:"duration";i:30;s:11:"on_waitlist";b:0;s:10:"company_id";s:5:"35927";s:8:"attended";b:1;s:6:"_links";a:2:{s:4:"self";a:1:{s:4:"href";s:60:"http://uk.bookingbug.com/api/v1/admin/35927/bookings/2234950";}s:6:"client";a:1:{s:4:"href";s:57:"http://uk.bookingbug.com/api/v1/admin/35927/client/809828";}}s:11:"appointment";a:1:{s:11:"description";s:11:"Brace start";}}"
As you can see the strings are the same but the string from the database call shows the offset error.
Do I need to do some sort of decoding or formatting on the string returned from the database?
Thanks
The problem was that I was serializing multidimensional arrays which causes problems.
To fix just base64_encode() the string before you use serialize and vise versa when you use unserialize.
//to safely serialize
$safe_string_to_store = base64_encode(serialize($multidimensional_array));
//to unserialize...
$array_restored_from_db = unserialize(base64_decode($encoded_serialized_string));
Article found at http://davidwalsh.name/php-serialize-unserialize-issues
Hope this helps others
Related
I am having a for loop which is iterating through a large array $result. Every array item $row is used to generate a new line of string $line. This string is concatenated into $str. To preserve memory, I am using unset($result[$i]) to clear every array item that has been processed in the original array. This looks like this:
$resultcount = count($result);
for($i=0; $i<$resultcount; ++$i){
$row = $result[$i];
$line = do_something($row);
$str.= '<tr><td>'.$line.'</td></tr>';
unset($result[$i]);
}
return $str;
This (more or less exotic piece of code) works unless the string exceeds a length of approx. 1'000'000 characters. In this case the return value is just empty. This is highly irritating because:
The calculation effort (in this example do_something()) is not a problem. By using echo count($result).' - '.strlen($str)."\n" I can see that the loop finishes properly. There is no web server or php error shown.
The memory_limit is also not a problem. I am working with more data on other parts of the application.
It appears that the problem lies in return $str itself. If I am using return substr($str, 0, 980000) then it just works fine. Further debugging shows that the string gets tainted as soon as it reaches the length of 999'775 bytes.
I can't put longer return value string into another string variable. But I am able to do a strlen() to get a proper result (1310307). So the return value string has a total length of 1'310'307 bytes. But I can't use them properly as a string.
Where does this limitation come from and how may I circumvent it?
You seem to describe return value in contradicting way:
In this case the return value is just empty.
but
But I am able to do a strlen() to get a proper result (1310307). So the return value string has a total length of 1'310'307 bytes.
Something can't be empty and have length at the same time?
I have tested returning a string that long and it works fine for me. Returning larger and larger strings works to a point memory limit is exceeded, which triggers explicit Fatal Error.
On a technical side strings can be much larger than million characters:
Note: As of PHP 7.0.0, there are no particular restrictions regarding the length of a string on 64-bit builds. On 32-bit builds and in earlier versions, a string can be as large as up to 2GB (2147483647 bytes maximum)
http://php.net/manual/en/language.types.string.php
I suspect that the failing operation has to do with how you use the result rather than return from the function.
Well, it took me two and a half year to figure this out ;(
Rarst was right: It was not a problem of the function compiling and delivering the string. Later in the process I was using preg_replace() to do some further replacements. This function is not able to handle long strings. This could be fixed by using ini_set('pcre.backtrack_limit', 99999999999) beforehand as mentioned in How to do preg_replace on a long string
So I am working with this API and using Laravel, and I am trying to build an auth string. This is the documentation I was given, but I am having a little trouble as this is something relatively new to me.
Here are the auth instructions:
The authentication parameter is a string and it can calculated by the
caller or the caller can choose to save this value as a parameter
together with connection ID and API key.
The authentication is a base64 string of a HMAC SHA1 hash. This is
computed by using the binary of API Key in in
########################## format in all lower case and UTF8 encoding as the key and computer HMAC SHA1 hash on the binary of
Connection ID in ################################ format in all lower
case and UTF8 encoding.
The result binary hash is then base64 encoded and the text result is
what should be passed as the authentication parameter. In C# the code
to calculate the authentication may look like:
HMACSHA1 hmac = new HMACSHA1(
UTF8Encoding.UTF8.GetBytes(apiKey.ToString("N").ToLower())
);
string authentication = Convert.ToBase64String(
hmac.ComputeHash(
UTF8Encoding.UTF8.GetBytes(connectionId.ToString("N").ToLower())
)
);
As an example the following credentials:
Connection ID: 5fecbc200f0e4a7cbf41040e11047e56
API Key: 2de51c4fd0f04b9fabeb95225e87da70
Should result in a computed authentication value of
m5/Vc1RzhUETQvEtx/JdIglQpTg=
So what i have been trying is:
$a = strtolower('5fecbc200f0e4a7cbf41040e11047e56');
$b = strtolower('2de51c4fd0f04b9fabeb95225e87da70');
$z = hash_hmac("sha1", utf8_encode(decbin($b)), utf8_encode(decbin($a)), true);
dd(base64_encode($z));
Which outputs QjG3kzUs7U1UukNd++3t24pBWNk=
I have tried a few more variations, but I am just lost on this one. First time really decoding or encoding anything. Would greatly appreciate any tips, ideas, or snippets that can help me figure this out. Already spent a few hours on this and it's bugging me..
First: Get rid of utf8_encode() and just generally don't use it. It assumes that the input string is ISO-88591-1 and if it is anything else it will silently corrupt the data. This function has an incredibly misleading name, and I would go as far as to suggest that no one should ever use it or the corresponding utf8_decode() which will break your data in the same manner, but reversed.
If you need to convert string encodings in PHP use something that explicitly defines the input and output encodings, eg: mb_convert_encoding(). [you still don't need it for this]
Second: Whatever you think decbin() does, you're incorrect. It converts an integer into a literal, capital-S String composed of 0 and 1 characters.
Third: PHP strings have no inherent encoding and are roughly equivalent to byte arrays if you twisted my arm for a description. The bytes you put into them are the bytes you get out of them.
Fourth: I'm not exactly a C# expert [or intermediate, or even beginner] but that example code is horrendous. What even is the significance of the N in connectionId.ToString("N")? I can't find any documentation about this.
Start simple, use meaningful variable names, build up, and read the docs.
$connectionID = strtolower('5fecbc200f0e4a7cbf41040e11047e56');
$apiKey = strtolower('2de51c4fd0f04b9fabeb95225e87da70');
$hash = hash_hmac("sha1", $connectionID, $apiKey, true);
var_dump(base64_encode($hash));
Output:
string(28) "m5/Vc1RzhUETQvEtx/JdIglQpTg="
I'm working on an encrypted database... I have been using m_crypt functions.. I have sucessfully got my method of encryption/decryption.. But a problem lies with creating my OO class to serve this function.. I have the following:
class Encryption {
public function __construct($Hex = null){
if (isset($Hex)){
if (ctype_xdigit($Hex)){
echo "Is Hex";
}
if (preg_match('~^[01]+$~', $Hex)) {
echo "Is Binary";
}
}
}
}
$key = pack('H*', "bcb04b7e103a0cd8b54763051cef08bc55abe029fdebae5e1d417e2ffb2a00a3");
$Class_OO = new Encryption($key);
The echos are for testing purposes.. But I want to validate this as a valid hexidecimal/binary or the datatype this string is.
performing:
print_r($key);
Returns the following:
¼°K~:صGcï¼U«à)ýë®^A~/û*£
But what datatype is this? On the documentation: http://www.php.net/manual/en/function.mcrypt-encrypt.php The line is presented:
convert a string into a key
key is specified using hexadecimal
So my question is what datatype is this? I understand this is in the ASCII range, but that is as far as my knowledge goes.. Furthermore, a successful answer for this will also assist me in creating another key which is not the one specified by the actual documentation
Your $key is the return value from pack, which in this case is a binary string (essentially raw binary values). See the first line in the documentation for the pack() function return value: http://php.net/manual/en/function.pack.php
Pack given arguments into binary string [emphasis added] according to format.
You would normally base64 encode a binary string before attempting any kind of output, because by definition, a binary string may (and often does) include non-printable characters, or worse - terminal control/escape sequences which can hose up your screen.
Think of it like printing a raw Word or Excel file: you'll probably see recognizable values (although in this case occasional alpha-numerics), but lots of garbage too.
Base64 encoding is a technique to inspect these strings in a safe way.
But what your question implies is that you are very much entering this territory new. You should probably take a look at the Matasano crypto tutorial here: http://www.matasano.com/articles/crypto-challenges/. It is an excellent starting point, and completing exercise #1 in it (maybe 20 minutes of work) will shed complete light on your question above.
In response to your question.. The only viable viable datatype this is submitted in is a string. As you said in your comment:
I have figured using the IV functions of mcrypt then using bin2hex,
using this in the second param of the pack function seems to work
without a fail.. BUT, my overall question is how to validate:
¼°K~:صGcï¼U«à)ýë®^A~/û*£ down to a specific datatype
You have answered how to create an acceptable format for the pack('H*') but as far as validation goes:
if (is_string($Var)){
}
Is the way to go, as this is how it's submitted. It's not a bool, hex, binary, int.. So the only valid method of validating is to validate it as a string.
I have stored binary data in Mysql field like this: 0x31 and 0x31303030303332 . this was converted to binary data from string, I don't know that string and I don't know how string had converted to this binary data.
As far as I know the pack() and unpack() PHP functions can help me, BUT I can't convert back this binary data to string because I don't know what format (second argument of unpack() function) had been used.
How can I find correct format to add in unpack('format', $mybinarystring) function?
Or may be this can be converted back via another function or method?
There is no way to definitely find out which format is used to create the binary data, because its just binary data. You can guess, look at the content and find a hint, or just try&error.
I have base 64 encoded string that looks something like this.
cuVrcYvlqYze3OZ8Y5tSqQY205mcquu0GsHkgXe4bPg=
I have tried base64_decode and output is.
råkq‹å©ŒÞÜæ|c›R©6Ó™œªë´Áäw¸lø
I think I may be doing something wrong. I appreciate any help to convert base64 string to binary array.
Thanks
like this
$a = base64_decode("cuVrcYvlqYze3OZ8Y5tSqQY205mcquu0GsHkgXe4bPg=");
$b = array();
foreach(str_split($a) as $c)
$b[] = sprintf("%08b", ord($c));
print_r($b);
You already are getting binary data back from base64_decode (if the encoded data was in fact binary), only this binary data is interpreted as encoding for some text by whatever you're outputting to (browser?). A "0011010110011001" output itself would also only be text, which would be encoded using some (different) binary stream. The computer does not work with 1's and 0's internally, contrary to popular believe. If you want to visualize binary data in the form of 1's and 0's, you'll need to do the binary/text conversion yourself. Usually that's a pretty pointless thing to do, though.
You're probably already doing the right thing. Your mistake is in expecting binary data to be represented as "0100101010".