I have a large 2d array that I serialize and base64_encode and throw into a database. On a different page I pull the array out and when I base64_decode the serialized array I can echo it out and it definitely looks valid.
However, if I try to unserialize(base64_decode($serializedArray)) it just throws the same error to the point of nearly crashing Firefox.
The error is:
Warning: unserialize() [function.unserialize]: Node no longer exists in /var/www/dev/wc_paul/inc/analyzerTester.php on line 24
I would include the entire serialized array that I echo out but last time I tried that on this form it crashed my Firefox.
Does anyone have any idea why this might be happening?
Are you sure you're just serializing an array, and not an object (e.g. DOMNode?) Like resources, not all classes are going to be happy with being unserialized. As an example with the DOM (which your error suggests to me you're working with), every node has a reference to the parentNode, and if the parentNode doesn't exist at the moment a node is being unserialized, it's not able to recreate that reference and problems ensue.
I would suggest saving the dom tree as XML to the database and loading it back later.
Make sure that the database field is large enough to hold the serialized array. Serialized data is very space-inefficient in PHP, and many DBs (like MySQL) will silently truncate field values that are too long.
What type of elements are in your array? serialize/unserialize does not work with built-in PHP objects, and that is usually the cause of that error.
Also, based on your comment this isn't your problem, but to save space in your database don't base64 encode the data, just escape it. i.e. for mysql use mysql_real_escape_string.
Make sure you don't serialize resources, they can't be serialized.
Resources#php.net
Related
I am developing a program to store binary image data, and I've decided to use memcached for the cache layer. I don't actually get any error but when I store the binary data in it doesn't quite work.This is what the output of the array is , Look closely at where it says Profile_Pic, the value is just one character
This is the code I use to update the memcaches array. Note that storing regular
values in memcached works fine.
<?php $unit_array=$this->Memc->getByKey("Unit_Array_Casting_Server",$unit_name);
$unit_array[$uuid][$keyname]=$keyvalue;
$this->Memc->setByKey("Unit_Array_Casting_Server",$unit_name,$unit_array); ?>
Nevermind, it turns out the issue was a internal code issue, I was inputing $keyvalue variable as an array which it is not.
I want to edit some options from a WordPress installation. I located the fields in a serialized field in wp_options:
the value of field is in the following format:
a:93:{s:19:"hide_upgrade_notice";s:5:"3.3.5";s:25:"icl_capabilities_verified";b:1;s:21:"interview_translators";i:1;s:34:"existing_content_language_verified";i:1;s:25:"language_negotiation_type";s:1:"1";s:23:"theme_localization_type";i:1;s:14:"icl_lso_header";i:0;s:18:"icl_lso_link_empty";i:0;s:13:"icl_lso_flags";s:1:"1";s:19:"icl_lso_native_lang";i:0;s:20:"icl_lso_display_lang";i:0;s:18:"sync_page_ordering";i:1;s:16:"sync_page_parent";i:1;s:18:"sync_page_template";i:1;s:16:"sync_ping_status";i:1;s:19:"sync_comment_status";i:1;s:16:"sync_sticky_flag";i:1;s:13:"sync_password";i:1;s:17:"sync_private_flag";i:1;s:16:"sync_post_format";i:1;s:11:"sync_delete";i:0;s:15:"sync_delete_tax";i:0;s:20:"sync_post_taxonomies";i:1;s:14:"sync_post_date";i:0;s:21:"sync_taxonomy_parents";i:0;s:25:"translation_pickup_method";i:0;s:15:"notify_complete";i:1;s:26:"translated_document_status";i:1;s:17:"remote_management";i:0;s:15:"auto_adjust_ids";i:1;s:11:"alert_delay";i:0;s:12:"promo
How is this formatted? I know that this isn't valid JSON.
How can I edit/remove settings manually?
This is PHP serialized data, a format that is largely unused now thanks to the prevalence of JSON. It was quite popular among PHP developers at one stage. It fell into disuse because there are some security concerns with php serialize and unserialize and also because of the rise of JSON. However it's still found in some apps.
use unserialize to convert to a php object
unserialize — Creates a PHP value from a stored representation
Note: You haven't posted a valid serialized string (it's truncated)
I'm currently working with data I'd like to temporarily store in my database as encrypted data. I'm not worried about the database getting hacked into, I just want to ensure the people that had entered the data that it is not reachable by any other than themselves. (and me of course)
The data is not meant to be stored permanently in the database since I'm exporting it to a third party application using their API, but since they have a rate limit I need to store the data in our database until the limit is over and I can upload it. (Assuming the rate limit occurs)
The process:
The request I receive from the form is in an array, so to begin with I serialize() the array to get a long string which I will unserialize() later.
Then I want to use a method that lets me convert the string into numbers and back again without losing information.
The reason I want to turn the data into numbers is because I use the HashIds library, which only encodes numbers. To my knowledge it's an extra layer of security I'm happy to add.
Read more on HashIds here: http://hashids.org/
What I have tried:
I tried converting the string into hex numbers, and then the hex numbers into decimals. Unfortunately the number was too large, and i haven't had any luck using biginteger with it.
base64_encode() which is not going to turn the data into numbers, but then base_converting them is. But I couldn't figure out the base converting in php since apparently it's rather odd.
Conclusion:
How can I convert the data I'm receiving from a form request into a short encoded string which can be converted back into the data without too much hassle? I don't quite know all the options PHP offers yet.
UPDATE:
To conclude this thread, I ended up using OpenSSL to encrypt my serialized array. Only problem I ran into was if the request contained a file I wouldn't be able to serialize it and save the object to the database. I do still need a way around this, since the third party application expects the file to be a multipart/formdata object i can't just save the filepath to the database and upload that. But I guess I will have to figure out that one later.
That link http://hashids.org/ provides a pretty clear example. Lets assume that your integer is 15.
$hashids = new Hashids\Hashids('some random string for a salt. Make sure you use the same salt if you want to be able to decode');
$encoded = $hashids->encode(15);
print_r(['hashedId' => $encoded]);
$decoded = $hashids->decode($hashed);
print_r(['decoded' => $decoded]);
So the value of $decoded should equal 15
Update
Sorry - the hashids bit of your question threw me and as such, I misunderstood what you were asking. I will update my answer:
You should really be using https://secure.php.net/openssl_encrypt and https://secure.php.net/manual/en/function.openssl-decrypt.php
I'm writing a command line application in PHP that accepts a path to a local input file as an argument. The input file will contain one of the following things:
JSON encoded associative array
A serialized() version of the associative array
A base 64 encoded version of the serialized() associative array
Base 64 encoded JSON encoded associative array
A plain old PHP associative array
Rubbish
In short, there are several dissimilar programs that I have no control over that will be writing to this file, in a uniform way that I can understand, once I actually figure out the format. Once I figure out how to ingest the data, I can just run with it.
What I'm considering is:
If the first byte of the file is { , try json_decode(), see if it fails.
If the first byte of the file is < or $, try include(), see if it fails.
if the first three bytes of the file match a:[0-9], try unserialize().
If not the first three, try base64_decode(), see if it fails. If not:
Check the first bytes of the decoded data, again.
If all of that fails, it's rubbish.
That just seems quite expensive for quite a simple task. Could I be doing it in a better way? If so, how?
There isn't much to optimize here. The magic bytes approach is already the way to go. But of course the actual deserialization functions can be avoided. It's feasible to use a verification regex for each instead (which despite the meme are often faster than having PHP actually unpack a nested array).
base64 is easy enough to probe for.
json can be checked with a regex. Fastest way to check if a string is JSON in PHP? is the RFC version for securing it in JS. But it would be feasible to write a complete json (?R) match rule.
serialize is a bit more difficult without a proper unpack function. But with some heuristics you can already assert that it's a serialize blob.
php array scripts can be probed a bit faster with token_get_all. Or if the format and data is constrained enough, again with a regex.
The more important question here is, do you need reliability - or simplicity and speed?
For speed, you could use the file(1) utility and add "magic numbers" in /usr/share/file/magic. It should be faster than a pure PHP alternative.
You can try json_decode() and unserialize() which will return NULL if they fail, then base64_decode() and run that again. It's not fast, but it's infinitely less error prone than hand parsing them...
The issue here is that if you have no idea which it can be, you will need to develop a detection algorithm. Conventions should be set with an extension (check the extension, if it fails, tell whoever put the file there to place the correct extension on), otherwise you will need to check yourself. Most algorithms that detect what type a file actually is do use hereustics to determine it's contents (exe, jpg etc) because generally they have some sort of signature that identifies them. So if you have no idea what the content will be for definate, it's best to look for features that are specific to those contents. This does sometimes mean reading more than a couple of bytes.
So I need to encode an array in PHP and store it in plain text in MySQL database, my question is should I use serialize() or json_encode()? What are the advantages and disadvantages of each of them?
I think either of them would do in this situation. But which one would you prefer and why? If it is for something other than an array?
Main advantage of serialize : it's specific to PHP, which means it can represent PHP types, including instances of your own classes -- and you'll get your objects back, still instances of your classes, when unserializing your data.
Main advantage of json_encode : JSON is not specific to PHP : there are libraries to read/write it in several languages -- which means it's better if you want something that can be manipulated with another language than PHP.
A JSON string is also easier to read/write/modify by hand than a serialized one.
On the other hand, as JSON is not specific to PHP, it's not aware of the stuff that's specific to PHP -- like data-types.
As a couple of sidenotes :
Even if there is a small difference in speed between those two, it shouldn't matter much : you will probably not serialize/unserialize a lot of data
Are you sure this is the best way to store data in a database ?
You won't be able to do much queries on serialized strins, in a DB : you will not be able to use your data in where clauses, nor update it without the intervention of PHP...
I did some analysis on Json Encoding vs Serialization in PHP. And I found that Json is best for plain and simple data like array.
See the results of my experiments at https://www.shozab.com/php-serialization-vs-json-encoding-for-an-array/
Another advantage of json_encode over serialize is the size. I noticed that as I was trying to figure out why our memcache used memory was getting so big, and was trying to find ways to reduce is:
<?php
$myarray = array();
$myarray["a"]="b";
$serialize=serialize($myarray);
$json=json_encode($myarray);
$serialize_size=strlen($serialize);
$json_size=strlen($json);
var_dump($serialize);
var_dump($json);
echo "Size of serialized array: $serialize_size\n";
echo "Size of json encoded array: $json_size\n";
echo "Serialize is " . round(($serialize_size-$json_size)/$serialize_size*100) . "% bigger\n";
Which gives you:
string(22) "a:1:{s:1:"a";s:1:"b";}"
string(9) "{"a":"b"}"
Size of serialized array: 22
Size of json encoded array: 9
Serialize is 59% bigger
Obviously I've taken the most extreme example, as the shorter the array, the more important the overhead with serialize (relative to the initial object size, due to formatting which imposes a minimum number of characters no matter how small the content). Still from a production website I see serialized array that are 20% bigger than their json equivalent.
Well firstly serializing an array or object and storing it in a database is typically a code smell. Sometimes people end up putting a comma separated list into a column and then get into all sorts of trouble when they later find out they need to query on it.
So think very carefully about that if this is that kind of situation.
As for the differences. PHP serialize is probably more compact but only usable with PHP. JSON is cross-platform and possibly slower to encode and decode (although I doubt meaningfully so).
If you data will never has to leave your PHP application, I recommend serialize() because it offers a lot of extra functionality like __sleep() and __wakeup() methods for your objects. It also restores objects as instances of the correct classes.
If you will pass the serialized data to another application, you should use JSON or XML for compatibility.
But storing a serialized objet into a database? Maybe you should think about that again. It can be real trouble later.
First, thanks to Shozab Hasan and user359650 for these tests. I was wondering which choice was the best and now i know:
To encode a simple array, JSON which is OK with both PHP AND javascript, maybe other languages.
To encode a PHP object, serialize is a better choice because of specificity of PHP Objects only instanciable with PHP.
To store datas, either store encoded datas in a file or use MySQL with standard format. It would be much easier to get your datas back. MySQL has great functions to get datas the way you'd like to get them without PHP treatment.
I've never made any test but i think that file storage is the best way to store your datas if system file sorting is enough to get back your files in alphabetical/numeral order.
MySQL is to greedy for this kind of treatment and uses file system too...