i use curl to fetch a page's source, it works great but not the inside html, it's returned with something like this :
the html look like this : t i & # 7 8 7 1; t, #&&#%$^%, etc....
yes, i really want to convert it to normal text, i try php decode functions but there's no luck at all
Thank you
##################3
edit :
thank you sirs,i tried
$fixed_result = html_entity_decode($result, ENT_COMPAT, "UTF-8");
and it works like a charm, but there are some character become "�", as this :
" S�C KH�CH"
i have no idea what is this
thank you sirs
That appears to be HTML entity encoded, you should be able to revert to the normal characters using html_entity_decode with the appropriate character set specified. e.g.:
$fixed_result = html_entity_decode($result, ENT_COMPAT, "UTF-8");
As per ajreal's comment check the manual for html_entity_decode and try the fallback solution:
// For users prior to PHP 4.3.0 you may do this:
function unhtmlentities($string)
{
// replace numeric entities
$string = preg_replace('~&#x([0-9a-f]+);~ei', 'chr(hexdec("\\1"))', $string);
$string = preg_replace('~&#([0-9]+);~e', 'chr("\\1")', $string);
// replace literal entities
$trans_tbl = get_html_translation_table(HTML_ENTITIES);
$trans_tbl = array_flip($trans_tbl);
return strtr($string, $trans_tbl);
}
$c = unhtmlentities($a);
(edit) For multi-byte support try the following version, taken from the PHP comments page,
function unhtmlentities($string)
{
// replace numeric entities
$string = preg_replace('~&#x([0-9a-f]+);~ei', 'code2utf(hexdec("\\1"))', $string);
$string = preg_replace('~&#([0-9]+);~e', 'code2utf("\\1")', $string);
// replace literal entities
$trans_tbl = get_html_translation_table(HTML_ENTITIES);
$trans_tbl = array_flip($trans_tbl);
return strtr($string, $trans_tbl);
}
// Returns the utf string corresponding to the unicode value (from php.net, courtesy - romans#void.lv)
function code2utf($num)
{
if ($num < 128) return chr($num);
if ($num < 2048) return chr(($num >> 6) + 192) . chr(($num & 63) + 128);
if ($num < 65536) return chr(($num >> 12) + 224) . chr((($num >> 6) & 63) + 128) . chr(($num & 63) + 128);
if ($num < 2097152) return chr(($num >> 18) + 240) . chr((($num >> 12) & 63) + 128) . chr((($num >> 6) & 63) + 128) . chr(($num & 63) + 128);
return '';
}
$c = unhtmlentities($a);
Related
I've a string like this :
%d8%b7%d8%b1%d8%a7%d8%ad%db%8c-%d8%a7%d9%be%d9%84%db%8c%da%a9%db%8c%d8%b4%d9%86-%d9%81%d8%b1%d9%88%d8%b4%da%af%d8%a7%d9%87%db%8c
the meta tag of page is set to utf-8
<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
i want to convert this unicode to pure readable utf-8 string
I've tested lots of code ,thie is my last code :
function convertFarsi($str) {
return html_entity_decode(preg_replace('/\\\\u([a-f0-9]{4})/i', '&#x$1;', $str),ENT_QUOTES, 'UTF-8');
}
and it doesn't work.
How can I convert these unicode to utf8 string ?
You can use url_decode to get the following result:
<?php
$string = '%d8%b7%d8%b1%d8%a7%d8%ad%db%8c-%d8%a7%d9%be%d9%84%db%8c%da%a9%db%8c%d8%b4%d9%86-%d9%81%d8%b1%d9%88%d8%b4%da%af%d8%a7%d9%87%db%8c';
$outpout = urldecode($string);
echo $outpout; // طراحی-اپلیکیشن-فروشگاهی
This function doesn't decode unicode characters. I wrote a function that does.
function unicode_urldecode($url)
{
preg_match_all('/%u([[:alnum:]]{4})/', $url, $a);
foreach ($a[1] as $uniord)
{
$dec = hexdec($uniord);
$utf = '';
if ($dec < 128)
{
$utf = chr($dec);
}
else if ($dec < 2048)
{
$utf = chr(192 + (($dec - ($dec % 64)) / 64));
$utf .= chr(128 + ($dec % 64));
}
else
{
$utf = chr(224 + (($dec - ($dec % 4096)) / 4096));
$utf .= chr(128 + ((($dec % 4096) - ($dec % 64)) / 64));
$utf .= chr(128 + ($dec % 64));
}
$url = str_replace('%u'.$uniord, $utf, $url);
}
return urldecode($url);
}
Source
Demo
This seems to do it:
<?php
$s = '%d8%b7%d8%b1%d8%a7%d8%ad%db%8c-%d8%a7%d9%be%d9%84%db%8c%da%a9%db%8c%d8%b4%d9%86-%d9%81%d8%b1%d9%88%d8%b4%da%af%d8%a7%d9%87%db%8c';
$t = urldecode($s);
var_dump($t == 'طراحی-اپلیکیشن-فروشگاهی');
https://php.net/function.urldecode
I want to convert hindi / Devanagari text for example "आए थे पर्यटक, खुद ही बह ग" into Unicode escaped characters like "\u0906\u090f \u0925\u0947 \u092a\u0930\u094d\u092f\u091f\u0915, \u0916\u0941\u0926 \u0939\u0940 \u092c\u0939 \u0917".
I am developing a hindi website and i have seen most of sites are using Escaped Unicode sequence inside their meta tags and schema.org.
So i decided to give it a try.
i can see Hindi AKA Devanagari letters with their Escaped Unicode sequence at http://www.endmemo.com/unicode/devanagari.php
and i have also seen a tool which works the same https://www.mobilefish.com/services/unicode_escape_sequence_converter/unicode_escape_sequence_converter.php
but i cannot find any way to convert these Devanagari letters into Escaped Unicode sequence via php.
I have tried few things but nothing is working and i am not getting much help from google because all articles / forums are talking to decoding unicode escape sequence to unicode but none of them is taking about encoding..
header( 'Content-Type: text/html; charset=utf-8' );
function encode2($str) {
$str = mb_convert_encoding($str , 'UTF-32', 'UTF-8');
$t = unpack("N*", $str);
$t = array_map(function($n) { return "&#$n;"; }, $t);
return implode("", $t);
}
$message = "आए थे पर्यटक, खुद ही बह गए";
$message_convert = encode2($message);
echo $message_convert;
echo "fdfdfdfdfdfdfd<br/>";
echo mb_convert_encoding($message, "HTML-ENTITIES", "auto");
I want this "आए थे पर्यटक, खुद ही बह ग" to "\u0906\u090f \u0925\u0947 \u092a\u0930\u094d\u092f\u091f\u0915, \u0916\u0941\u0926 \u0939\u0940 \u092c\u0939 \u0917"
Please help!
as suggest by #paskl i tried:
$message = "आए थे पर्यटक, खुद ही बह गए";
$unicode = json_encode($message)
echo $unicode;
And i got ""\u0906\u090f \u0925\u0947 \u092a\u0930\u094d\u092f\u091f\u0915, \u0916\u0941\u0926 \u0939\u0940 \u092c\u0939 \u0917\u090f""
I hope it will help others who want to convert devanagari/hindi letters into Escaped Unicode sequence with php on their website.
Thanks to #paskl
Unless you're looking to transmit this data as JSON I wouldn't really recommend using json_encode() as it will wrap your output in literal double quotes that you'd need to strip back off. However there's not an easy way to encode unicode escapes in PHP in a way that is memory-efficient.
That said, here is the not-easy code:
// PHP < 7.2
// https://github.com/symfony/polyfill-mbstring/blob/master/Mbstring.php#L708-L730
if( ! function_exists("mb_ord") ) {
function mb_ord($s) {
if (1 === \strlen($s)) {
return \ord($s);
}
$code = ($s = unpack('C*', substr($s, 0, 4))) ? $s[1] : 0;
if (0xF0 <= $code) {
return (($code - 0xF0) << 18) + (($s[2] - 0x80) << 12) + (($s[3] - 0x80) << 6) + $s[4] - 0x80;
}
if (0xE0 <= $code) {
return (($code - 0xE0) << 12) + (($s[2] - 0x80) << 6) + $s[3] - 0x80;
}
if (0xC0 <= $code) {
return (($code - 0xC0) << 6) + $s[2] - 0x80;
}
return $code;
}
}
function ord2seqlen($ord) {
if($ord < 128){
return 1;
} else if($ord < 224) {
return 2;
} else if($ord < 240) {
return 3;
} else if($ord < 248) {
return 4;
} else {
throw new \Exception("No support for 5 or 6 byte sequences.");
}
}
function utf8_seq_iter($input) {
for($i=0,$c=strlen($input); $i<$c; ) {
$bytes = ord2seqlen(ord($input[$i]));
yield substr($input, $i, $bytes);
$i += $bytes;
}
}
function escape_codepoint($codepoint, $skip_low=true) {
$ord = mb_ord($codepoint);
if( $skip_low && $ord < 128 ) {
return $codepoint;
} else {
return sprintf("\\u%04x", $ord);
}
}
$input = "आए थे पर्यटक, खुद ही बह गए";
$output = '';
foreach( utf8_seq_iter($input) as $codepoint ) {
$output .= escape_codepoint($codepoint);
}
var_dump($output);
Output:
string(121) "\u0906\u090f \u0925\u0947 \u092a\u0930\u094d\u092f\u091f\u0915, \u0916\u0941\u0926 \u0939\u0940 \u092c\u0939 \u0917\u090f"
Edit: I've turned this into a small composer package available here:
https://packagist.org/packages/wrossmann/utf8_escape
I have encoded string like ªÙªÑ à¾ç§Íé¹
Please check below function that i have used for decode(utf-8 to tis620) it.
function utf8_to_tis620($string) {
$str = $string;
$res = "";
for ($i = 0; $i < strlen($str); $i++) {
if (ord($str[$i]) == 224) {
$unicode = ord($str[$i+2]) & 0x3F;
$unicode |= (ord($str[$i+1]) & 0x3F) << 6;
$unicode |= (ord($str[$i]) & 0x0F) << 12;
$res .= chr($unicode-0x0E00+0xA0);
$i += 2;
} else {
$res .= $str[$i];
}
}
return $res;
}
So it will return string like ชูชัย Gงอ้น but it isn't correct in THAI language.
Actually it should return ชูชัย เพ็งอ้น that is returned from http://string-functions.com/encodedecode.aspx.
But there is used windows 874 decoding.
Please let me know how can i decode utf-8 to windows 874 in php?
You can use mb_convert_encoding to change the character encoding:
function utf8_to_tis620($string) {
return mb_convert_encoding($string, 'UTF-8', 'TIS-620');
}
As noted by krasipenkov in their comment,
There is small difference between ISO-8859-11 and TIS-620. ISO-8859-11 is
nearly identical to the national Thai standard TIS-620 (1990). The
sole difference is that ISO/IEC 8859-11 allocates non-breaking space
to code 0xA0, while TIS-620 leaves it undefined. (In practice, this
small distinction is usually ignored.)
Instead of 'TIS-620', you can use 'ISO-8859-11' for the Windows 874 character encoding if needed.
You can use iconv for convert string to request character encoding.
$string= iconv('TIS-620','UTF-8//ignore',$string);
I have this code to decode numeric html entities to the UTF8 equivalent character.
I'm trying to convert this character:
which should output:
However, it just disappears (no output). (i've checked the source code of the page, the page has the correct utf8 character set headers/meta tags).
Does anyone know what is wrong with the code?
function entity_decode($string, $quote_style = ENT_COMPAT, $charset = "UTF-8") {
$string = html_entity_decode($string, $quote_style, $charset);
$string = preg_replace_callback('~&#x([0-9a-fA-F]+);~i', "chr_utf8_callback", $string);
$string = preg_replace('~&#([0-9]+);~e', 'chr_utf8("\\1")', $string);
//this is another method, which also doesn't work..
//$string = preg_replace_callback("/(\&#[0-9]+;)/", "entity_decode_callback", $string);
return $string;
}
function chr_utf8_callback($matches) {
return chr_utf8(hexdec($matches[1]));
}
function chr_utf8($num) {
if ($num < 128) return chr($num);
if ($num < 2048) return chr(($num >> 6) + 192) . chr(($num & 63) + 128);
if ($num < 65536) return chr(($num >> 12) + 224) . chr((($num >> 6) & 63) + 128) . chr(($num & 63) + 128);
if ($num < 2097152) return chr(($num >> 18) + 240) . chr((($num >> 12) & 63) + 128) . chr((($num >> 6) & 63) + 128) . chr(($num & 63) + 128);
return '';
}
function entity_decode_callback($m) {
return mb_convert_encoding($m[1], "UTF-8", "HTML-ENTITIES");
}
echo '=' . entity_decode('');
html_entity_decode already does what you're looking for:
$string = '';
echo html_entity_decode($string, ENT_COMPAT, 'UTF-8');
It will return the character:
’ binary hex: c292
Which is PRIVATE USE TWO (U+0092). As it's private use, your PHP configuration/version/compile might not return it at all.
Also there are some more quirks:
But in HTML (other than XHTML, which uses XML rules), it's a long-standing browser quirk that character references in the range to are misinterpreted to mean the characters associated with bytes 128 to 159 in the Windows Western code page (cp1252) instead of the Unicode characters with those code points. The HTML5 standard finally documents this behaviour.
See: is getting converted as “\u0092” by nokogiri in ruby on rails
Is it possible to input a character and get the unicode value back? for example, i can put ⽇ in html to output "⽇", is it possible to give that character as an argument to a function and get the number as an output without building a unicode table?
$val = someFunction("⽇");//returns 12103
or the reverse?
$val2 = someOtherFunction(12103);//returns "⽇"
I would like to be able to output the actual characters to the page not the codes, and I would also like to be able to get the code from the character if possible.
The closest I got to what I want is php.net/manual/en/function.mb-decode-numericentity.php but I cant get it working, is this the code I need or am I on the wrong track?
function _uniord($c) {
if (ord($c[0]) >=0 && ord($c[0]) <= 127)
return ord($c[0]);
if (ord($c[0]) >= 192 && ord($c[0]) <= 223)
return (ord($c[0])-192)*64 + (ord($c[1])-128);
if (ord($c[0]) >= 224 && ord($c[0]) <= 239)
return (ord($c[0])-224)*4096 + (ord($c[1])-128)*64 + (ord($c[2])-128);
if (ord($c[0]) >= 240 && ord($c[0]) <= 247)
return (ord($c[0])-240)*262144 + (ord($c[1])-128)*4096 + (ord($c[2])-128)*64 + (ord($c[3])-128);
if (ord($c[0]) >= 248 && ord($c[0]) <= 251)
return (ord($c[0])-248)*16777216 + (ord($c[1])-128)*262144 + (ord($c[2])-128)*4096 + (ord($c[3])-128)*64 + (ord($c[4])-128);
if (ord($c[0]) >= 252 && ord($c[0]) <= 253)
return (ord($c[0])-252)*1073741824 + (ord($c[1])-128)*16777216 + (ord($c[2])-128)*262144 + (ord($c[3])-128)*4096 + (ord($c[4])-128)*64 + (ord($c[5])-128);
if (ord($c[0]) >= 254 && ord($c[0]) <= 255) // error
return FALSE;
return 0;
} // function _uniord()
and
function _unichr($o) {
if (function_exists('mb_convert_encoding')) {
return mb_convert_encoding('&#'.intval($o).';', 'UTF-8', 'HTML-ENTITIES');
} else {
return chr(intval($o));
}
} // function _unichr()
Here's a more compact implementation of unichr/uniord based on pack:
// code point to UTF-8 string
function unichr($i) {
return iconv('UCS-4LE', 'UTF-8', pack('V', $i));
}
// UTF-8 string to code point
function uniord($s) {
return unpack('V', iconv('UTF-8', 'UCS-4LE', $s))[1];
}
If you're using PHP7.2 (or later), you don't need to define a new function. There are two functions for your purposes from Multibyte String extension!
To get code point of a character (i.e. Unicode value), use mb_ord(); and to get a specific character from that value, use mb_chr().
E.g.:
mb_chr(12103, "utf8"); // ⽇
mb_ord("⽇", "utf8"); // 12103
This also works, (for someone who understands bitshifting this might be more readable than Mark Bakers answer):
public function ordinal($str){
$charString = mb_substr($str, 0, 1, 'utf-8');
$size = strlen($charString);
$ordinal = ord($charString[0]) & (0xFF >> $size);
//Merge other characters into the value
for($i = 1; $i < $size; $i++){
$ordinal = $ordinal << 6 | (ord($charString[$i]) & 127);
}
return $ordinal;
}
You can use the following functions
For encoding
string utf8_encode ( string $data )
http://php.net/manual/en/function.utf8-encode.php
For decoding
string utf8_decode ( string $data )
http://php.net/manual/en/function.utf8-decode.php
Also check
http://php.net/manual/en/function.htmlspecialchars.php
<?php
echo htmlspecialchars_decode("⽇");//will print ⽇
?>