How to convert \u00253A and \u00252F in string using php ?
I treid to use this code but not work.
<?php
$match ="http\u00253A\u00252F\u00252Fwww.google.com"
$str = preg_replace_callback('/\u([0-9a-fA-F]{4})/', function ($match) {
return mb_convert_encoding(pack('H*', $match[1]), 'UTF-8', 'UCS-2BE');
}, $str);
echo $str;
?>
How can i do ?
That string has been URL-encoded, turning : into %3A, which has then been JSON-encoded (presumably?) to turn % into \u0025. This is mightily messed up and should be fixed at the source, this is not something you want to generally deal with as normal.
Here's how to decode it back:
$match = "http\u00253A\u00252F\u00252Fwww.google.com";
echo rawurldecode(json_decode('"' . $match . '"'));
Related
$ser = 'a:2:{i:0;s:5:"héllö";i:1;s:5:"wörld";}'; // fails
$ser2 = 'a:2:{i:0;s:5:"hello";i:1;s:5:"world";}'; // works
$out = unserialize($ser);
$out2 = unserialize($ser2);
print_r($out);
print_r($out2);
echo "<hr>";
But why?
Should I encode before serialzing than? How?
I am using Javascript to write the serialized string to a hidden field, than PHP's $_POST
In JS I have something like:
function writeImgData() {
var caption_arr = new Array();
$('.album img').each(function(index) {
caption_arr.push($(this).attr('alt'));
});
$("#hidden-field").attr("value", serializeArray(caption_arr));
};
The reason why unserialize() fails with:
$ser = 'a:2:{i:0;s:5:"héllö";i:1;s:5:"wörld";}';
Is because the length for héllö and wörld are wrong, since PHP doesn't correctly handle multi-byte strings natively:
echo strlen('héllö'); // 7
echo strlen('wörld'); // 6
However if you try to unserialize() the following correct string:
$ser = 'a:2:{i:0;s:7:"héllö";i:1;s:6:"wörld";}';
echo '<pre>';
print_r(unserialize($ser));
echo '</pre>';
It works:
Array
(
[0] => héllö
[1] => wörld
)
If you use PHP serialize() it should correctly compute the lengths of multi-byte string indexes.
On the other hand, if you want to work with serialized data in multiple (programming) languages you should forget it and move to something like JSON, which is way more standardized.
I know this was posted like one year ago, but I just have this issue and come across this, and in fact I found a solution for it. This piece of code works like charm!
The idea behind is easy. It's just helping you by recalculating the length of the multibyte strings as posted by #Alix above.
A few modifications should suits your code:
/**
* Mulit-byte Unserialize
*
* UTF-8 will screw up a serialized string
*
* #access private
* #param string
* #return string
*/
function mb_unserialize($string) {
$string = preg_replace('!s:(\d+):"(.*?)";!se', "'s:'.strlen('$2').':\"$2\";'", $string);
return unserialize($string);
}
Source: http://snippets.dzone.com/posts/show/6592
Tested on my machine, and it works like charm!!
Lionel Chan answer modified to work with PHP >= 5.5 :
function mb_unserialize($string) {
$string2 = preg_replace_callback(
'!s:(\d+):"(.*?)";!s',
function($m){
$len = strlen($m[2]);
$result = "s:$len:\"{$m[2]}\";";
return $result;
},
$string);
return unserialize($string2);
}
This code uses preg_replace_callback as preg_replace with the /e modifier is obsolete since PHP 5.5.
The issue is - as pointed out by Alix - related to encoding.
Until PHP 5.4 the internal encoding for PHP was ISO-8859-1, this encoding uses a single byte for some characters that in unicode are multibyte. The result is that multibyte values serialized on UTF-8 system will not be readable on ISO-8859-1 systems.
The avoid problems like this make sure all systems use the same encoding:
mb_internal_encoding('utf-8');
$arr = array('foo' => 'bár');
$buf = serialize($arr);
You can use utf8_(encode|decode) to cleanup:
// Set system encoding to iso-8859-1
mb_internal_encoding('iso-8859-1');
$arr = unserialize(utf8_encode($serialized));
print_r($arr);
In reply to #Lionel above, in fact the function mb_unserialize() as you proposed won't work if the serialized string itself contains char sequence "; (quote followed by semicolon).
Use with caution. For example:
$test = 'test";string';
// $test is now 's:12:"test";string";'
$string = preg_replace('!s:(\d+):"(.*?)";!se', "'s:'.strlen('$2').':\"$2\";'", $test);
print $string;
// output: s:4:"test";string"; (Wrong!!)
JSON is the ways to go, as mentioned by others, IMHO
Note: I post this as new answer as I don't know how to reply directly (new here).
This solution worked for me:
$unserialized = unserialize(utf8_encode($st));
Do not use PHP serialization/unserialization when the other end is not PHP. It is not meant to be a portable format - for example, it even includes ascii-1 characters for protected keys which is nothing you want to deal with in javascript (even though it would work perfectly fine, it's just extremely ugly).
Instead, use a portable format like JSON. XML would do the job, too, but JSON has less overhead and is more programmer-friendly as you can easily parse it into a simple data structure instead of having to deal with XPath, DOM trees etc.
One more slight variation here which will hopefully help someone ... I was serializing an array then writing it to a database. On retrieving the data the unserialize operation was failing.
It turns out that the database longtext field I was writing into was using latin1 not UTF8. When I switched it round everything worked as planned.
Thanks to all above who mentioned character encoding and got me on the right track!
I would advise you to use javascript to encode as json and then use json_decode to unserialize.
/**
* MULIT-BYTE UNSERIALIZE
*
* UTF-8 will screw up a serialized string
*
* #param string
* #return string
*/
function mb_unserialize($string) {
$string = preg_replace_callback('/!s:(\d+):"(.*?)";!se/', function($matches) { return 's:'.strlen($matches[1]).':"'.$matches[1].'";'; }, $string);
return unserialize($string);
}
we can break the string down to an array:
$finalArray = array();
$nodeArr = explode('&', $_POST['formData']);
foreach($nodeArr as $value){
$childArr = explode('=', $value);
$finalArray[$childArr[0]] = $childArr[1];
}
Serialize:
foreach ($income_data as $key => &$value)
{
$value = urlencode($value);
}
$data_str = serialize($income_data);
Unserialize:
$data = unserialize($data_str);
foreach ($data as $key => &$value)
{
$value = urldecode($value);
}
this one worked for me.
function mb_unserialize($string) {
$string = mb_convert_encoding($string, "UTF-8", mb_detect_encoding($string, "UTF-8, ISO-8859-1, ISO-8859-15", true));
$string = preg_replace_callback(
'/s:([0-9]+):"(.*?)";/',
function ($match) {
return "s:".strlen($match[2]).":\"".$match[2]."\";";
},
$string
);
return unserialize($string);
}
In my case the problem was with line endings (likely some editor have changed my file from DOS to Unix).
I put together these apadtive wrappers:
function unserialize_fetchError($original, &$unserialized, &$errorMsg) {
$unserialized = #unserialize($original);
$errorMsg = error_get_last()['message'];
return ( $unserialized !== false || $original == 'b:0;' ); // "$original == serialize(false)" is a good serialization even if deserialization actually returns false
}
function unserialize_checkAllLineEndings($original, &$unserialized, &$errorMsg, &$lineEndings) {
if ( unserialize_fetchError($original, $unserialized, $errorMsg) ) {
$lineEndings = 'unchanged';
return true;
} elseif ( unserialize_fetchError(str_replace("\n", "\n\r", $original), $unserialized, $errorMsg) ) {
$lineEndings = '\n to \n\r';
return true;
} elseif ( unserialize_fetchError(str_replace("\n\r", "\n", $original), $unserialized, $errorMsg) ) {
$lineEndings = '\n\r to \n';
return true;
} elseif ( unserialize_fetchError(str_replace("\r\n", "\n", $original), $unserialized, $errorMsg) ) {
$lineEndings = '\r\n to \n';
return true;
} //else
return false;
}
$ser = 'a:2:{i:0;s:5:"héllö";i:1;s:5:"wörld";}'; // fails
$ser2 = 'a:2:{i:0;s:5:"hello";i:1;s:5:"world";}'; // works
$out = unserialize($ser);
$out2 = unserialize($ser2);
print_r($out);
print_r($out2);
echo "<hr>";
But why?
Should I encode before serialzing than? How?
I am using Javascript to write the serialized string to a hidden field, than PHP's $_POST
In JS I have something like:
function writeImgData() {
var caption_arr = new Array();
$('.album img').each(function(index) {
caption_arr.push($(this).attr('alt'));
});
$("#hidden-field").attr("value", serializeArray(caption_arr));
};
The reason why unserialize() fails with:
$ser = 'a:2:{i:0;s:5:"héllö";i:1;s:5:"wörld";}';
Is because the length for héllö and wörld are wrong, since PHP doesn't correctly handle multi-byte strings natively:
echo strlen('héllö'); // 7
echo strlen('wörld'); // 6
However if you try to unserialize() the following correct string:
$ser = 'a:2:{i:0;s:7:"héllö";i:1;s:6:"wörld";}';
echo '<pre>';
print_r(unserialize($ser));
echo '</pre>';
It works:
Array
(
[0] => héllö
[1] => wörld
)
If you use PHP serialize() it should correctly compute the lengths of multi-byte string indexes.
On the other hand, if you want to work with serialized data in multiple (programming) languages you should forget it and move to something like JSON, which is way more standardized.
I know this was posted like one year ago, but I just have this issue and come across this, and in fact I found a solution for it. This piece of code works like charm!
The idea behind is easy. It's just helping you by recalculating the length of the multibyte strings as posted by #Alix above.
A few modifications should suits your code:
/**
* Mulit-byte Unserialize
*
* UTF-8 will screw up a serialized string
*
* #access private
* #param string
* #return string
*/
function mb_unserialize($string) {
$string = preg_replace('!s:(\d+):"(.*?)";!se', "'s:'.strlen('$2').':\"$2\";'", $string);
return unserialize($string);
}
Source: http://snippets.dzone.com/posts/show/6592
Tested on my machine, and it works like charm!!
Lionel Chan answer modified to work with PHP >= 5.5 :
function mb_unserialize($string) {
$string2 = preg_replace_callback(
'!s:(\d+):"(.*?)";!s',
function($m){
$len = strlen($m[2]);
$result = "s:$len:\"{$m[2]}\";";
return $result;
},
$string);
return unserialize($string2);
}
This code uses preg_replace_callback as preg_replace with the /e modifier is obsolete since PHP 5.5.
The issue is - as pointed out by Alix - related to encoding.
Until PHP 5.4 the internal encoding for PHP was ISO-8859-1, this encoding uses a single byte for some characters that in unicode are multibyte. The result is that multibyte values serialized on UTF-8 system will not be readable on ISO-8859-1 systems.
The avoid problems like this make sure all systems use the same encoding:
mb_internal_encoding('utf-8');
$arr = array('foo' => 'bár');
$buf = serialize($arr);
You can use utf8_(encode|decode) to cleanup:
// Set system encoding to iso-8859-1
mb_internal_encoding('iso-8859-1');
$arr = unserialize(utf8_encode($serialized));
print_r($arr);
In reply to #Lionel above, in fact the function mb_unserialize() as you proposed won't work if the serialized string itself contains char sequence "; (quote followed by semicolon).
Use with caution. For example:
$test = 'test";string';
// $test is now 's:12:"test";string";'
$string = preg_replace('!s:(\d+):"(.*?)";!se', "'s:'.strlen('$2').':\"$2\";'", $test);
print $string;
// output: s:4:"test";string"; (Wrong!!)
JSON is the ways to go, as mentioned by others, IMHO
Note: I post this as new answer as I don't know how to reply directly (new here).
This solution worked for me:
$unserialized = unserialize(utf8_encode($st));
Do not use PHP serialization/unserialization when the other end is not PHP. It is not meant to be a portable format - for example, it even includes ascii-1 characters for protected keys which is nothing you want to deal with in javascript (even though it would work perfectly fine, it's just extremely ugly).
Instead, use a portable format like JSON. XML would do the job, too, but JSON has less overhead and is more programmer-friendly as you can easily parse it into a simple data structure instead of having to deal with XPath, DOM trees etc.
One more slight variation here which will hopefully help someone ... I was serializing an array then writing it to a database. On retrieving the data the unserialize operation was failing.
It turns out that the database longtext field I was writing into was using latin1 not UTF8. When I switched it round everything worked as planned.
Thanks to all above who mentioned character encoding and got me on the right track!
I would advise you to use javascript to encode as json and then use json_decode to unserialize.
/**
* MULIT-BYTE UNSERIALIZE
*
* UTF-8 will screw up a serialized string
*
* #param string
* #return string
*/
function mb_unserialize($string) {
$string = preg_replace_callback('/!s:(\d+):"(.*?)";!se/', function($matches) { return 's:'.strlen($matches[1]).':"'.$matches[1].'";'; }, $string);
return unserialize($string);
}
we can break the string down to an array:
$finalArray = array();
$nodeArr = explode('&', $_POST['formData']);
foreach($nodeArr as $value){
$childArr = explode('=', $value);
$finalArray[$childArr[0]] = $childArr[1];
}
Serialize:
foreach ($income_data as $key => &$value)
{
$value = urlencode($value);
}
$data_str = serialize($income_data);
Unserialize:
$data = unserialize($data_str);
foreach ($data as $key => &$value)
{
$value = urldecode($value);
}
this one worked for me.
function mb_unserialize($string) {
$string = mb_convert_encoding($string, "UTF-8", mb_detect_encoding($string, "UTF-8, ISO-8859-1, ISO-8859-15", true));
$string = preg_replace_callback(
'/s:([0-9]+):"(.*?)";/',
function ($match) {
return "s:".strlen($match[2]).":\"".$match[2]."\";";
},
$string
);
return unserialize($string);
}
In my case the problem was with line endings (likely some editor have changed my file from DOS to Unix).
I put together these apadtive wrappers:
function unserialize_fetchError($original, &$unserialized, &$errorMsg) {
$unserialized = #unserialize($original);
$errorMsg = error_get_last()['message'];
return ( $unserialized !== false || $original == 'b:0;' ); // "$original == serialize(false)" is a good serialization even if deserialization actually returns false
}
function unserialize_checkAllLineEndings($original, &$unserialized, &$errorMsg, &$lineEndings) {
if ( unserialize_fetchError($original, $unserialized, $errorMsg) ) {
$lineEndings = 'unchanged';
return true;
} elseif ( unserialize_fetchError(str_replace("\n", "\n\r", $original), $unserialized, $errorMsg) ) {
$lineEndings = '\n to \n\r';
return true;
} elseif ( unserialize_fetchError(str_replace("\n\r", "\n", $original), $unserialized, $errorMsg) ) {
$lineEndings = '\n\r to \n';
return true;
} elseif ( unserialize_fetchError(str_replace("\r\n", "\n", $original), $unserialized, $errorMsg) ) {
$lineEndings = '\r\n to \n';
return true;
} //else
return false;
}
I have this function for converts html entities back to html special characters:
function html_entity_decode_utf8($value, $double_encode = TRUE)
{
return htmlspecialchars( (string) $value, ENT_QUOTES, "UTF-8", $double_encode);
}
Now, I need to Print this:
echo html_entity_decode_utf8('&zwnj');
Output is:
&zwnj
Why htmlspecialchars notwork ?! how i can fix this?
htmlspecialchars converts & into &. To convert back, you need htmlspecialchars_decode:
function html_entity_decode_utf8($value)
{
return htmlspecialchars_decode( (string) $value, ENT_QUOTES);
}
To see more options for using this function, check out the documentation.
I am trying to build a function in php that will take the input of a string and encode it correctly before displaying it. So far I have:
public function encode($str)
{
if(!preg_match('!!u', $str)) // Needs to be encoded
{
$str = "YES: " . $str;
$str = utf8_encode($str);
}
else
{
$str = "NO: " . $str;
}
return $str;
}
The if statement I am using i found here. And it works in some cases but in others it is encoding the string when they shouldn't be.
Perhaps there is an easier way to handle the encoding ?
I wrote an SMPP Server Transceiver in PHP.
I get this SMS string from my SMPP. It's a UTF8 message which is actually at 7Bit. Here is a sample message:
5d30205d30205d3
I know how to convert it. It should be:
\x5d3\x020\x5d3\x020\x5d3
I don't want to write it myself. I guess there is already a function that does that for me. Some hidden iconv or using pack() / unpack() to convert this string to the correct format.
I am trying to achieve this using PHP.
Any ideas?
Thanks.
This should do it :
$message = "5d30205d30205d3";
echo "\x".implode("\x", str_split($message, 3));
// \x5d3\x020\x5d3\x020\x5d3
Here is what i used eventually:
public static function sms__from_unicode($message)
{
$org_msg = str_split(strtolower($message), 3);
for($i = 0;$i < count($org_msg); $i++)
$org_msg[$i] = "\u0{$org_msg[$i]}";
$str = implode(null, $org_msg);
$str = preg_replace_callback('/\\\\u([0-9a-f]{4})/i', 'replace_unicode_escape_sequence', $str);
return $str;
}
function replace_unicode_escape_sequence($match) {
return mb_convert_encoding(pack('H*', $match[1]), 'UTF-8', 'UCS-2BE');
}
10x. all.