I have the following problem. I am retrieving a mysql text field that is a serialized text of a invoice. I am working in 2 different projects. Both have the same version of PHP. The data was exported & imported from db to db. If i var_dump the data from db1 it tells me it's length is x. When I do the same in from db2 i get x+2
string(595)
"a:3:{s:11:"userdetails";a:20:{s:4:"name";s:3:"bas";s:8:"lastname";s:7:"schmitz";s:5:"email";s:17:"email#test.de";s:6:"street";s:11:"f�rstenwall";s:7:"street2";s:0:"";s:7:"company";s:0:"";s:3:"zip";s:5:"40215";s:9:"residence";s:10:"d�sseldorf";s:7:"country";s:7:"Germany";s:5:"phone";s:7:"3033185";s:3:"fax";s:0:"";s:10:"customerID";i:202771;s:2:"nr";s:3:"228";s:6:"region";s:3:"nrw";s:10:"phone_code";s:3:"211";s:8:"fax_code";s:0:"";s:10:"salutation";s:2:"Mr";s:5:"sales";s:0:"";s:12:"country_code";s:0:"";s:10:"vat_number";s:0:"";}s:6:"domain";s:15:"bas-schmitz2.de";s:10:"has_domain";b:1;}"
string(597)
"a:3:{s:11:"userdetails";a:20:{s:4:"name";s:3:"bas";s:8:"lastname";s:7:"schmitz";s:5:"email";s:17:"email#test.de";s:6:"street";s:11:"fürstenwall";s:7:"street2";s:0:"";s:7:"company";s:0:"";s:3:"zip";s:5:"40215";s:9:"residence";s:10:"düsseldorf";s:7:"country";s:7:"Germany";s:5:"phone";s:7:"3033185";s:3:"fax";s:0:"";s:10:"customerID";i:202771;s:2:"nr";s:3:"228";s:6:"region";s:3:"nrw";s:10:"phone_code";s:3:"211";s:8:"fax_code";s:0:"";s:10:"salutation";s:2:"Mr";s:5:"sales";s:0:"";s:12:"country_code";s:0:"";s:10:"vat_number";s:0:"";}s:6:"domain";s:15:"bas-schmitz2.de";s:10:"has_domain";b:1;}"
As I am pasting these I can see that there is a difference when displaying germanic characters
Any idea to why this is happening?
The output of serialize() cannot be handled as plain text:
Return Values
Returns a string containing a byte-stream representation of value that
can be stored anywhere.
Note that this is a binary string which may include null bytes, and
needs to be stored and handled as such. For example, serialize()
output should generally be stored in a BLOB field in a database,
rather than a CHAR or TEXT field.
Thus your data is corrupted in the first place.
If you're unable to change the database design (which would be the proper fix), you need to re-encode serialised data in a plain text encoding such as Base64:
$encoded = base64_encode(serialize($foo));
$decoded = unserialize(base64_decode($encoded));
Related
I am trying to generate a JSON string using php from an array, and I would like to store that string in a csv file, also with PHP. I want to do this, because I am working with a quite large amount of data, and I would like to use MySQL's LOAD DATA LOCAL INFILE to populate and update my database table.
This is the code I have:
$tmpFileProducts = 'path/to/file';
$tmpFileProductsHandler= fopen($tmpFileProducts, 'w');
foreach ($attributeBatch as $productId => $batch) {
fputcsv($tmpFileProductsHandler, array($productId, $batch['title'], $batch['parsed'], json_encode($batch['attributes'])), "|", "\"");
}
My problem is, that when I am creating the CSV file, the JSON double-quotes are not escaped, thus I end up with simmilar lines in my csv file:
43541|"telefon mobil 3l 2020 4g "|"2020-12-05 17:38:19"|"{""color"":""dark chrome"",""memory_value"":4294967296,""storage_value"":68719476736,""sim_slot"":""dual""}"
My first possible solution would be, to change the string enclosure of my CSV file, but what enclosure should I use, to ensure no conflicts could arrise with the inner json column? I am in complete control of the array that is stringified, it will only contain ASCII characters, in it.
Would there be a way, to keep the current string enclosure, and instead escape the JSON string somehow? Later on, I will need to fetch the data that is included to the database, and convert it to an array again.
DISCLAIMER: I am well aware that instead of storing the data as a JSON string, I could store it in a specific relational table (which I am also doing), but I would need quick access to this data, for a background script that is running, and I would like to save on the time of the queries, to the relational table, as when the background script will use this data, it doesn't need to search in it.
Follow up question: as I am explicitly telling the fputcsv function what to use as string enclosure shouldn't it automatically escape all the simmilar inner strings?
In php, json_encode() will encode UTF8 in hex entities, e.g.
json_encode('中'); // become "\u4e2d"
Assume the data "\u4e2d" is now being stored in MySQL, is it possible to convert back from "\u4e2d" to 中 without using PHP, just plain MySQL?
On my configuration, select hex('中'); returns E4B8AD
which is the hex code of the UTF8 bytes. Naturally it
is not the same as the hex of the code point 4e2d, but you can get
that with select hex(cast('中' as char(1) character set utf16));.
Update: The questioner has edited the question, to what looks to me like a completely different question, now it's apparently: how to get '中' given a string containing '\u4e2d' when 4e2d is the code point of 中 and the default character set is utf8. Okay, that is
select cast(char(conv(right('\u4e2d',4),16,10) using utf16) as char(1) character set utf8);
Encoding non-ASCII characters as JavaScript entities is only one of the different things that JSON encoders will do—and it isn't actually mandatory:
echo json_encode('中'), PHP_EOL;
echo json_encode('中', JSON_UNESCAPED_UNICODE), PHP_EOL;
echo json_encode('One "Two" Three \中'), PHP_EOL;
"\u4e2d"
"中"
"One \"Two\" Three \\\u4e2d"
Thus the only safe decoding approach is using a dedicated JSON decoder. MySQL bundles the required abilities since 5.7.8:
SET #input = '"One \\"Two\\" Three \\\\\\u4e2d"';
SELECT #input AS json_string, JSON_UNQUOTE(#input) AS original_string;
json_string original_string
============================ ===================
"One \"Two\" Three \\\u4e2d" One "Two" Three \中
(Demo)
If you have an older version you'll have to resort to more elaborate solutions (you can Google for third-party UDF's).
In any case, I suggest you get back to the design table. It's strange that you need JSON data in a context where you don't have a proper JSON decoder available.
i have a string with special characters, like:
$text = "NÃO";
When i use serialize($text), returns
a:1:{i:0;s:4:"NÃO";}
but when i use a string that i get from my database, like:
$query = SELECT special_text FROM ...
(...)
$text = $row->"special_text"
serialize($text);
returns
a:1:{i:0;s:3:"NÃO";}
, what is crashing my script.
what i have to do when i serialize data from database?
Thx, and sorry for my english
If you don't want to use utf-8, try to encode serialized string before saving it to db, and decode after retrieve it. You may use base64 encoding, just if you use any 8-bit encoding in your db. This method increases data size by 4/3. Or you may use binary (BLOB) field in your db and encode your data with gzip or something like that. Very useful for large amount of text, but increases CPU load.
i have a MySQL field value with a json object containing Hebrew characters like this:
[{"name":"אספנות ואומנות","value":1,"target":null},{"name":"אופניים","value":2,"target":null}]
(the one in the name field)
This field output is giving me some trouble with a certain web interface.
so, looking around in the database i found another field containing json object and its output works fine.
[{"name":"\u05d0\u05e1\u05e4\u05e0\u05d5\u05ea \u05d5\u05d0\u05d5\u05de\u05e0\u05d5\u05ea","value":1,"target":null},{"name":"\u05d0\u05d5\u05e4\u05e0\u05d9\u05d9\u05dd","value":2,"target":null}]
So i would like to convert the first field to this encoding to see if its solves the output issue.
what is this encoding ? is it UTF-8 ? how can i convert it using PHP ?
i tried to isolate the value and convert it to UTF-8 using
echo iconv("Windows-1255","UTF-8",'אספנות ואומנות');
but its just returning an empty value.
Any help would be great
So, in PHP
json_encode('אספנות ואומנות');
did the trick
I have a string I want to save into my DB in an encrypted format. I am using the security utility provided by cake so this is the code I use to encrypt my sensitive data:
// get my encryption key
$encrypt_key = Configure::read('Secret.encrypt_key');
// encrpyt this string to be stored in the database
$this->request->data['User']['message'] = Security::rijndael($this->request->data['User']['message'], $encrypt_key, 'encrypt');
// save this user data
$user_saved = $this->User->save( $this->request->data[ 'User' ] );
This looks like ever guide I have seen for how to do this, but in my case all of the other fields will save and I will get an empty field for message
My question is why is this blank database save happening and how do I fix it. Thank you.
The problem ended up being that the rjindeal function returns a raw binary string 010100100010100101111101010010101 that my database cannot handle. By simply converting the result to hex code via bin2hex($encrypted_message) the data is transformed into a form that my database can handle.