I'm trying to store a complex object here and am doing that by serialising the object running a mysql_real_escape_string on it and inserting it into a mysql database.
However when I retrieve it running a sql query - I'm using Zend frameworks Zend_DB_Table here but anyway - and when I try to stripslashes and unserialize I dont get my object back. I've tried to just unserialize without stripping slashes and all but nothings working.
UPDATE
This is weird. I made a simple page which just unserializes a serialised object. If I take the serialized string as it is retrieved from the database and unserialize it via this other page which just has an unserialize() on it - it works perfectly and I get my object back. However in the code where ironically I'm retriving the string and I run the exact same unserialize option there ,its not working!
So basically there is nothing wrong with the serialized string - for some weird reason it won't unserialize it in my application but it unserializes somewhere else, it makes no sense.
You probably need to run it through base64 encoding first:
$safe_string_to_store = base64_encode(serialize($data));
Then to get it back out:
$date = unserialize(base64_decode($safe_string_to_store));
Try that and let us know if it works.
(and dont run stripslashes on it - there is no need to)
You shouldn't run stripslashes on it - the database will give you back the right string to put into unserialize.
Make sure you have notices turned on and echo the string before you unserialize it - does it look right?
You should be able to just do the following:
Assuming MyTable is your instance of Zend_Db_Table_Abstract:
$t = new MyTable();
$n = $t->createRow();
$n->serializedfield = serialize($data);
$n->save();
and let Zend DB take care of the escaping for you.
If you're doing it via an insert(), you shouldnt need to do anything either (the above uses insert())
Otherwise use $db->quoteInto() like
$db->quoteInto('INSERT INTO mytable (serializedfield) values (?)', serialize($data));
I strongly recommend you to use json_encode instead of serialize. Some day you will find yourself trying to use that data from another place that is not PHP and having it stored in JSON makes it readable everywhere; virtually every language supports decoding JSON and is a well stablished standard. And is even worse if you base64 it, you also make the serialized content unreadable from your database console client.
Related
I am using a Yii2 model that is hooked to a PostgresQL database. I have a behavior that encodes and decodes certain attributes of this model to/from json. To encode/decode, I am using the Json helper, the Json::encode and Json::decode methods.
The column in the table is of a json type. An example of what ends up in the database:
"{\"additional_tags\":[\"#здрасте\",\"#кафе\"],\"vk\":\"vk.com\\\/privetik\"}"
When I try to decode it back into a php array, here's what's returned instead:
'{"additional_tags":["#здрасте","#кафе"],"vk":"vk.com\/privetik"}'
EDIT: Come to think of it, the string seems fine, but the behavior of the ::decode method is strange. Essentially, all it does is remove the escape slashes, instead of converting it into a php array or throwing an exception.
What should I do to fix this? Appreciate any feedback.
Looks like the string in your database is encoded twice. Try passing it through Json::decode another time, I bet it will return your array.
I have this value under Items in my DB:
a:1:{i:0;a:9:{s:12:"model_number";s:10:"TT1-W";s:5:"price";s:4:"3810";s:10:"unit_price";d:3135.6300000000001091393642127513885498046875;s:8:"id_price";d:3810;s:9:"sales_tax";d:290.3700000000000045474735088646411895751953125;s:5:"sales";d:3084.6300000000001091393642127513885498046875;s:7:"service";s:2:"51";s:7:"freight";s:3:"384";s:13:"co_cat";s:3:"X4";}}
Making it more reader-friendly:
a:1:
{
i:0;
a:9:
{
s:12:"model_number";
s:10:"TT1-W";
s:5:"price";
s:4:"3810";
s:10:"unit_price";
d:3135.6300000000001091393642127513885498046875;
s:8:"id_price";
d:3810;
s:9:"sales_tax";
d:290.3700000000000045474735088646411895751953125;
s:5:"sales";
d:3084.6300000000001091393642127513885498046875;
s:7:"service";
s:2:"51";
s:7:"freight";
s:3:"384";
s:13:"co_cat";
s:3:"X4";
}
}
I am unable to find out how to decode this string since it can not seem to find reference to it in the php code that displays it on the page. It looks to me to be JSON but i can not seem to find a "standard" format for the above in order to start figuring out where it starts and where it ends.
I am needing this to be decoding using ASP.net. But then again, i need to figure out what it is before i can start decoding it!
Any help to what it is would be great!
Try with unserialize:
function.unserialize
EDIT: If you can use C# libraries:
How to unserialize PHP Serialized array/variable/class and return suitable object in C#
EDIT2:
Visual Studio Tip: Three ways to use C# within a VB.NET project
EDIT3:
i need to figure out what it is
It's standard PHP-solution to store (and restore) arrays and objects (and other types, see manual) in strings.
That appears to be PHP's serialization methodology. You just need to use PHP's unserialize() on it.
This looks a serialized object. PHP's unserialize is probably what you want:
unserialize() takes a single serialized variable and converts it back
into a PHP value.
There is no built in way to turn that into an ASP.Net object, but it is a regular format, so you can build your own parser to create a simple dictionary representation of the attributes of that particular structure.
But if you're trying to de-serialize a PHP object in ASP.Net you're probably doing something wrong!
I have my data stored in a JSON string like these...
a:1:{s:15:"s2member_level1";s:1:"1";}
How can i read this values in mysql?
I need to know if the value "s2member_level1" is 1.
Thanks!!!
That's not JSON but a string resulted from calling serialize() in PHP. You cannot parse it easily in MySQL. If you can use PHP, use the unserialize function:
$obj = unserialize($data_from_mysql);
if ($obj['s2member_level1'] == 1) {
// more code here
}
You can convert data to JSON in PHP using the json_encode function. In a similar way, you construct an object from a JSON string using json_decode.
#Lekensteyn is correct, but you could do a like statement, although its performance would most likely be very poor. My true answer is to change how you store this information to take advantages of best performing queries.
select * from table
where column like '%s:15:"s2member_level1";s:1:"1";%';
#Lekensteyn is right about the type of this particular String, but for others, PHP has json_decode which you can use to convert a JSON object to a PHP object. It would be considerably more difficult to read such an object using MySQL only.
This is no json, but serialized data. It was probably serialized with the 'serialize' function of PHP. Try:
print_r(unserialize('a:1:{s:15:"s2member_level1";s:1:"1";}'));
... to unserialize it.
I'm storing some "unstructured" data (a keyed array) in one field of my table, and i'm currently using serialize() / unserialize() to "convert" back and forth from array to string.
Every now and then, however, I get errors when unserializing the data. I believe these errors happen because of Unicode data in the strings inside the array i'm serializing, although there are some records with Unicode data that work just fine. (DB field is UTF-8)
I'm wondering whether using json_encode instead of serialize will make a difference / make this more resilient. This is not trivial for me to test, since in my dev environment everything works well, but in production, every now and then (about 1% of records) I get an error.
Btw, I know i'm weaseling out of finding an actual explanation for the problem and just blindly trying something, I'm kind of hoping I can get rid of this without spending too much time on it.
Do you think using json_encode instead of serialize will make this more resilient to "serialization errors"? The data format does look more "forgiving" to me...
UPDATE: The actual error i'm getting is:
Notice: unserialize(): Error at offset 401 of 569 bytes in C:\blah.php on line 20
Thanks!
Daniel
JSON has one main advantage :
compatibility with other languages than PHP.
PHP's serialize has one main advantage :
it's specifically designed to store PHP-based data -- most notably, it can store serialized objects, instance of classes, that will be re-instanciated to the right class-type when the string is unserialized.
(Yes, those advantages are the exact opposite of each other)
In your case, as you are storing data that's not really structured, both formats should work pretty well.
And the encoding problem you have should not be related to serialize by itself : as long as everything (DB, connection to the DB, PHP files, ...) is in UTF-8, serialization should work too.
I think unless you absolutely need to preserve php specific types that json_encode() is the way to go for storing structured data in a single field in MySQL. Here's why:
https://dev.mysql.com/doc/refman/5.7/en/json.html
As of MySQL 5.7.8, MySQL supports a native JSON data type defined by RFC 7159 that enables efficient access to data in JSON (JavaScript Object Notation) documents
If you are using a version of MySQL that supports the new JSON data type you can benefit from that feature.
Another important point of consideration is the ability to perform changes on those JSON strings. Suppose you have a url stored in encoded strings all over your database. Wordpress users who've ever tried to migrate an existing database to a new domain name may sympathize here. If it's serialized, it's going to break things. If it's JSON you can simply run a query using REPLACE() and everything will be fine. Example:
$arr = ['url' => 'http://example.com'];
$ser = serialize($arr);
$jsn = json_encode($arr);
$ser = str_replace('http://','https://',$ser);
$jsn = str_replace('http://','https://',$jsn);
print_r(unserialize($ser));
PHP Notice: unserialize(): Error at offset 39 of 43 bytes in /root/sandbox/encoding.php on line 10
print_r(json_decode($jsn,true));
Array
(
[url] => https://example.com
)
json_encode() converts non-ASCII characters and symbols (e.g., “Schrödinger” becomes “Schr\u00f6dinger”) but serialize() does not.
Source: https://www.toptal.com/php/10-most-common-mistakes-php-programmers-make#common-mistake-6--ignoring-unicodeutf-8-issues
To leave UTF-8 characters untouched, you can use the option JSON_UNESCAPED_UNICODE as of PHP 5.4.
Source: https://stackoverflow.com/a/804089/1438029
If the problem is (and I believe it is) in UTF-8 encoding, there is not difference between json_encode and serialize. Both will leave characters encoding unchanged.
You should make sure your database/connection is properly set up for handle all UTF-8 characters or encode whole record into supported encoding before inserting to the DB.
Also please specify what "I get an error" means.
Found this in the PHP docs...
function mb_unserialize($serial_str) {
$out = preg_replace('!s:(\d+):"(.*?)";!se', "'s:'.strlen('$2').':\"$2\";'", $serial_str );
return unserialize($out);
}
I don't quite understand it, but it worked to unserialize the data that I couldn't unserialize before. Moved to JSON now, i'll report in a couple of weeks whether this solved the problem of randomly getting some records "corrupted"
As I'm going through this I'll give my opinion, both serialize and json_encode are good for storing data in DB, but for those looking for performance, I've tested and I get these results, json_encode are a little microsegunds faster tham serialize, i used this script to calculate a the difference time.
$bounced =array();
for($i=count($bounced); $i<9999; ++$i)$bounced[$i]=$i;
$timeStart = microtime(true);
var_dump(serialize ($bounced));
unserialize(serialize ($bounced));
print timer_diff($timeStart) . " sec.\n";
$timeStart = microtime(true);
var_dump(json_encode ($bounced));
json_decode(json_encode ($bounced));
print timer_diff($timeStart) . " sec.\n";
function timer_diff($timeStart)
{
return number_format(microtime(true) - $timeStart, 3);
}
As a design decision, I'd opt for storing JSON because it can only represent a data structure, whereas serialization is bound to a PHP data object signature.
The advantages I see are:
* you are forced to separate the data storage from any logic layer on top.
* you are independent from changes to the data object class (say, for example, that you want to add a field).
I am looking to have a list of arguments passed across in an a URL.
$url['key1']=1;
$url['key2']=2;
$url['key3']=3;
$url['key4']=4;
$url['key5']=5;
$url['key6']=6;
$url['key7']=7;
Please Note I am trying to pass this in the URL in 1 GET variable. I know this would be better done by ?key1=1&key2=2&key3=3...etc but for reasons that are too complicated to try and explain they can't be in this format.
Any suggestions how I can convert this array into something that can be passed as 1 get var in a URL string?
Thanks in advance.
You can use json_encode() or serialize()
$myUrl = 'http://www.example.com/?myKey=' . urlencode(json_encode($url));
or
$myUrl = 'http://www.example.com/?myKey=' . urlencode(serialize($url));
Using json_encode will usually give you a shorter string, but very old PHP version might not have the json_decode function available to decode it again.
The final way would be to create your own custom encoding... it could be as simple a pipe-separated values: key1|1|key2|2|key3|3
This would give you the best option for a short URL, but is the most work.
Try http_build_query:
$url['key1']=1;
$url['key2']=2;
$url['key3']=3;
$url['key4']=4;
$url['key5']=5;
$url['key6']=6;
$url['key7']=7;
echo http_build_query($url);
//echos key1=1&key2=2&key3=3&key...
What it does is converting an array into a query string using the keys and automatically takes care of url-encoding.
EDIT:
Just read your additional requirement that it should be just one variable. So nevermind this answer.
If your problem was the proper encoding though you might want to give this another try.
Hope that helps.
The recommendation to use serialize() is fine. If space is an issue, then use a combination of bzcompress() and serialize().
However, there's a security considering that hasn't been brought up, and that's that the end user (who can see and edit this url) could manipulate the data within it. You may think it's difficult, but most of the PHP-attacking worms in the wild do this to some degree or another.
If letting the user directly manipulate any of the keys or values (or replacing it with an integer, or an object, or anything else), then you should protect your script (and your users) from this attack.
A simple solution is to simply use a shared secret. It can be anything; just so long as it's unique and truly secret (perhaps you should randomly generate it at install-time). Let's say you have in your config file something like this:
define('SECRET', 'unoqetbioqtnioqrntbioqt');
Then, you can digitally sign the serialized data created with: $s=serialize($m) using $k=sha1($s.SECRET) and make the url value $k.$s
Then, before you unserialize() do this:
$v=substr($input,0,40);
$s=substr($input,40);
if ($v != sha1($s.SECRET)) { die("invalid input"); }
$m=unserialize($s);
This way, you know that $m is the same as the original value that you serialized.
If you like, you can use the following drop-in replacements:
define('SECRET','buh9tnb1094tib014'); // make sure you pick something else
function secureserialize($o) {
$s=serialize($o);
return sha1($s.SECRET).$s;
}
function secureunserialize($i) {
$v=substr($i,0,40);$s=substr($i,40);
if ($v!=sha1($s.SECRET)){die("invalid input");}
return unserialize($s);
}
You could serialize them as key-value pairs when constructing the URL, putting the resultant serialized value in a single $_GET variable (e.g. data=sfsdfasdf98sdfasdf), then unserialize the $_GET["data"] variable. You'll need to use urlencode to make sure the resultant serialized values are URL-safe. Make sure you watch out for maximum URL lengths - 2083 characters in IE.
However, unless you really can't use key-value pairs in URLs (per your question), key1=foo&key2=bar... is definitely the way to go.
If you don't mind dropping the key names, you can use
http://example.com?url[]=1&url[]=2&url[]=3
EDIT Keeping the key names:
http://example.com?values[]=1&values[]=2&values[]=3&keys[]=1&keys[]=2&keys[]=3
Then in your PHP script:
$url = array_combine($_GET['keys'], $_GET['values']);
Could you solve your problem by saving the data as a HTML cookie? That way you don't have to modify the URL at all.
If you know the values in advance, you can set them from the server side when you send the user the page with your target link on it.
If you won't know the values until the user fills out a form it can still be done using JavascriptL When the user clicks the form submit you can set multiple cookies by making multiple javascript calls like:
document.cookie = 'key1=test; expires=Mon, 7 Sept 2009 23:47:11 UTC; path=/'
The security model might give you some trouble if you are trying to pass this data from one domain to another though.