PHP Escaped special characters to html - php

I have string that looks like this "v\u00e4lkommen till mig" that I get after doing utf8_encode() on the string.
I would like that string to become
välkommen till mig
where the character
\u00e4 = ä = ä
How can I achive this in PHP?

Do not use utf8_(de|en)code. It just converts from UTF8 to ISO-8859-1 and back. ISO 8859-1 does not provide the same characters as ISO-8859-15 or Windows1252, which are the most used encodings (besides UTF-8). Better use mb_convert_encoding.
"v\u00e4lkommen till mig" > This string looks like a JSON encoded string which IS already utf8 encoded. The unicode code positiotion of "ä" is U+00E4 >> \u00e4.
Example
<?php
header('Content-Type: text/html; charset=utf-8');
$json = '"v\u00e4lkommen till mig"';
var_dump(json_decode($json)); //It will return a utf8 encoded string "välkommen till mig"
What is the source of this string?
There is no need to replace the ä with its HTML representation ä, if you print it in a utf8 encoded document and tell the browser the used encoding. If it is necessary, use htmlentities:
<?php
$json = '"v\u00e4lkommen till mig"';
$string = json_decode($json);
echo htmlentities($string, ENT_COMPAT, 'UTF-8');

Edit: Since you want to keep HTML characters, and I now think your source string isn't quite what you posted (I think it is actual unicode, rather than containing \unnnn as a string), I think your best option is this:
$html = str_replace( str_replace( str_replace( htmlentities( $whatever ), '<', '<' ), '>', '>' ), '&', '&' );
(note: no call to utf8-decode)
Original answer:
There is no direct conversion. First, decode it again:
$decoded = utf8_decode( $whatever );
then encode as HTML:
$html = htmlentities( $decoded );
and of course you can do it without a variable:
$html = htmlentities( utf8_decode( $whatever ) );
http://php.net/manual/en/function.utf8-decode.php
http://php.net/manual/en/function.htmlentities.php
To do this by regular expression (not recommended, likely slower, less reliable), you can use the fact that HTML supports &#xnnnn; constructs, where the nnnn is the same as your existing \unnnn values. So you can say:
$html = preg_replace( '/\\\\u([0-9a-f]{4})/i', '&#x$1;', $whatever )

The html_entity_decode worked for me.
$json = '"v\u00e4lkommen till mig"';
echo $decoded = html_entity_decode( json_decode($json) );

Related

How to convert ASCII to ISO-8859-1 in PHP?

I'm currently trying to figure out how to convert an ASCII encoded string to ISO-8859-1 encoding to be used for utf8_encode() to display special characters like "ñ" but I can't seem to make it work. In need of help.
I've already tried this iconv(mb_detect_encoding($text, mb_detect_order(), true), "ISO-8859-1", $text); and this mb_convert_encoding($text, "ISO-8859-1"); and also this mb_convert_encoding($text, "ASCII", "ISO-8859-1"); but it doesn't work, the string is still ASCII encoded.
I've created a temporary solution for this by creating a lookup table using the string provided by reading each character of the string. But I want to use the php built-in functions, is this possible?
Here is my code:
<?php
function convertString($text) {
$text = iconv(mb_detect_encoding($text, mb_detect_order(), true), "ISO-8859-1", $text);
echo mb_detect_encoding($text) .'<br/>'; // to check what encoding the string is in, displays ASCII
return utf8_encode($text);
}
echo convertString('\xc3\xb1');
?>

Iconv convert from UTF-8 to Windows-1250 doesn't work

I've always had problems with iconv. Now I must convert string to Windows-1250 and this doesn't seems to work:
$string = "ľaľa ho papľuha, ogrcal mi krpce!";
echo $string . ' ( ' . mb_detect_encoding($string) . ' ) <br>';
$string_encoded = iconv( mb_detect_encoding( $string ), 'Windows-1250//TRANSLIT', $string );
echo $string_encoded . ' ( ' . mb_detect_encoding($string_encoded) . ' ) <br>';
$string_encoded = mb_convert_encoding( $string, 'Windows-1250' );
echo $string_encoded . ' ( ' . mb_detect_encoding($string_encoded) . ' ) <br>';
The three echos above output exactly this:
ľaľa ho papľuha, ogrcal mi krpce! ( UTF-8 )
�a�a ho pap�uha, ogrcal mi krpce! ( )
mb_convert_encoding() Unknown encoding "Windows-1250" ( ASCII )
Since I've always seen this diamond question marks I wonder if this PHP function works at all. How can I convert UTF-8 to Windows-1250?
The file was saved in notepad+ in UTF-8
Also I've tried header('Content-Type: text/html; charset=windows-1250'); and setLocale()
I have experienced a similar issue. While reading CSV file, word "Česká republika" was read as "Èeská republika".
This solved it for me:
iconv( "Windows-1250", "UTF-8", ($string));
The � character is an indication that your text is being interpreted as UTF-8, but at this point an invalid byte sequence was encountered. Meaning, you're not serving UTF-8, yet the client is reading it as UTF-8. Which would imply that iconv is working just fine and whoever is reading the result just didn't get the message that it should be interpreting it as Windows-1250.
See What Every Programmer Absolutely, Positively Needs To Know About Encodings And Character Sets To Work With Text and Handling Unicode Front To Back In A Web App.
Is old post but you can convert UTF-8 to Windows-1252 and you will have same effect:
$str = "ľaľa ho papľuha, ogrcal mi krpce!"
$str = mb_convert_encoding( $str, "Windows-1252", "UTF-8" );
but if you realy need Windows-1250 you can use THIS SOLUTION and adapt to your need.
Correct answer is iconv( "UTF-8", "Windows-1250", $string );

PHP htmlentities and saving the data in xml format

Im trying to save some data into a xml file using the following PHP script:
<?php
$string = 'Go to google maps and some special characters ë è & ä etc.';
$string = htmlentities($string, ENT_QUOTES, 'UTF-8');
$doc = new DOMDocument('1.0', 'UTF-8');
$doc->preserveWhiteSpace = false;
$doc->formatOutput = true;
$root = $doc->createElement('top');
$root = $doc->appendChild($root);
$title = $doc->createElement('title');
$title = $root->appendChild($title);
$id = $doc->createAttribute('id');
$id->value = '1';
$text = $title->appendChild($id);
$text = $doc->createTextNode($string);
$text = $title->appendChild($text);
$doc->save('data.xml');
echo 'data saved!';
?>
I'm using htmlentities to translate all of the string into an html format, if I leave this out the special characters won't be translated to html format. this is the output:
<?xml version="1.0" encoding="UTF-8"?>
<top>
<title id="1">&lt;a href=&quot;google.com/maps&quot;&gt;Go to google maps&lt;/a&gt; and some special characters &euml; &egrave; &amp; &auml; etc.</title>
</top>
The ampersand of the html tags get a double html code: &lt; and an ampersand becomes: &amp;
Is this normal behavior? Or how can I prevent this from happening? Looks like a double encoding.
Try to remove the line:
$string = htmlentities($string, ENT_QUOTES, 'UTF-8');
Because the text passed to createTextNode() is escaped anyway.
Update:
If you want the utf-8 characters to be escaped. You could leave that line and try to add the $string directly in createElement().
For example:
$title = $doc->createElement('title', $string);
$title = $root->appendChild($title);
In PHP documentation it says that $string will not be escaped. I haven't tried it, but it should work.
It is the htmlentities that turns a & into &
When working with xml data you should not use htmlentities, as the DOMDocument will handle a & and not &.
As of php 5.3 the default encoding is UTF-8, so there is no need to convert to UTF-8.
This line:
$string = htmlentities($string, ENT_QUOTES, 'UTF-8');
… encodes a string as HTML.
This line:
$text = $doc->createTextNode($string);
… encodes your string of HTML as XML.
This gives you an XML representation of an HTML string. When the XML is parsed you get the HTML back.
how can I prevent this from happening?
If your goal is to store some text in an XML document. Remove the line that encodes it as HTML.
Looks like a double encoding.
Pretty much. It is encoded twice, it just uses different (albeit very similar) encoding methods for each of the two passes.

Migrating data, from latin1 charset to UTF-8

I'm trying to move over some fish species information profiles from a bespoke CMS using latin1 charset to a WordPress customised (custom post type, with numerous meta fields) database which uses UTF-8.
On top of that, the old CMS uses some odd bbCode bits.
Basically, I'm looking for a function which will do this:
Take information from my old database with latin1_swedish_ci collation (and latin1 charset)
Convert all of the non-standard characters (we have characters from languages including but not exclusive of Croatian, Czech, Spanish, French and German) to HTML entities such as á (numbers like &134; fine too).
Convert all of the bbCode (see below) to HTML
Convert ' and " to HTML entities
Return the information with utf-8 charset to my new database
The bbCode to and from are:
$search = array( '[i]', '[/i]', '[b]', '[/b]', '[pl]', '[/pl]' );
$replace = array( '<i>', '</i>', '<strong>', '</strong>', '', '' );
The function that I've tried so far is:
$search = array( '[i]', '[/i]', '[b]', '[/b]', '[pl]', '[/pl]' );
$replace = array( '<i>', '</i>', '<strong>', '</strong>', '', '' );
function _convert($content) {
if(!mb_check_encoding($content, 'UTF-8')
OR !($content === mb_convert_encoding(mb_convert_encoding($content, 'UTF-32', 'UTF-8' ), 'UTF-8', 'UTF-32'))) {
$content = mb_convert_encoding($content, 'UTF-8');
if (mb_check_encoding($content, 'UTF-8')) {
return $content;
} else {
echo "<p>Couldn't convert to UTF-8.</p>";
}
}
}
function _clean($content) {
$content = _convert( $content );
/* edited out because otherwise all HTML appears as <html> rather than <html>
//$content = htmlentities( $content, ENT_QUOTES, "UTF-8" );
$content = str_replace( $search, $replace, $content );
return $content;
}
However this is stopping some fields from being imported to the new database and isn't replacing the bbCode.
If I use the following code, it mostly works:
$var = str_replace( $search, $replace, htmlentities( $row["var"], ENT_QUOTES, "UTF-8" ) );
However, certain fields containing what I think are Czech/Croatian characters don't appear at all.
Does anyone have any suggestions for how I can, in the order listed above, successfully convert the information from the "old format" to the new?
I would say if you want to convert all your non-ASCII characters you won't need to do any latin1 to UTF-8 conversion what so ever. Let's say you run a function such as htmlspecialchars or htmlentities on your data, then all non-ASCII characters will be replaced with their corresponding entity code.
Basically, after this step, there shouldn't be any characters left that needs conversion to UTF-8. Also, if you wanted to convert your latin1 encoding string into UTF-8 i strongly suspect utf8_encode will du just fine.
PS. When it comes to converting bbCode into HTML I would recommend using regular expressions instead. For example you could do it all in a line like this:
$html_data = preg_replace('/\[(/?[a-z]+)\]/i', '<$1>', $bb_code_data);

How to convert some multibyte characters into its numeric html entity using PHP?

Test string:
$s = "convert this: ";
$s .= "–, —, †, ‡, •, ≤, ≥, μ, ₪, ©, ® y ™, ⅓, ⅔, ⅛, ⅜, ⅝, ⅞, ™, Ω, ℮, ∑, ⌂, ♀, ♂ ";
$s .= "but, not convert ordinary characters to entities";
$encoded = mb_convert_encoding($s, 'HTML-ENTITIES', 'UTF-8');
asssuming your input string is UTF-8, this should encode most everything into numeric entities.
Well htmlentities doesn't work correctly. Fortunately someone has posted code on the php website that seems to do the translation of multibyte characters properly
I did work on decoding ascii into html coded text (&#xxxx). https://github.com/hellonearthis/ascii2web

Categories