I'm getting troubles with data from a database containing german umlauts. Basically, whenever I receive a data containing umlauts, it is a black square with an interrogation mark. I solved this by putting
mysql_query ('SET NAMES utf8')
before the query.
The problem is, as soon as I use json_encode(...) on a result of a query, the value containing an umlaut gets null. I can see this by calling the php-file directly in the browser. Are there other solution than replacing this characters before encoding to JSON and decoding it in JS?
Check out this pretty elegant solution mentioned here:
json_encode( $json_full, JSON_UNESCAPED_UNICODE );
If the problem isn't anywhere else in your code this should fix it.
Edit: Umlaut problems can be caused by a variety of sources like the charset of your HTML document, the database format or some previous php functions your strings run through (You should definitely look into multibyte functions when having problems with umlauts).
These problems tend to be the pretty annoying because they are hard to track in most cases (altough this isn't as bad as it was a few years ago). The function above fixes – as asked – umlaut problems with json_encode … but there is a good chance that the problem is caused by a different part of your application and not this specific function.
I know this might be old but here a better solution:
Define the document type with utf-8 charset:
<?php header('Content-Type: application/json; charset=utf-8'); ?>
Make sure that all content is utf_encoded. JSON works only with utf-8!
function encode_items(&$item, $key)
{
$item = utf8_encode($item);
}
array_walk_recursive($rows, 'encode_items');
Hope this helps someone.
You probably just want to show the texts somehow in the browser, so one option would be to change the umlauts to HTML entities by using htmlentities().
The following test worked for me:
<?php
$test = array( 'bla' => 'äöü' );
$test['bla'] = htmlentities( $test['bla'] );
echo json_encode( $test );
?>
The only important point here is that json_encode() only supports UTF-8 encoding.
http://www.php.net/manual/en/function.json-encode.php
All string data must be UTF-8 encoded.
So when you have any special characters in a non utf-8 string json_encode will return a null Value.
So either you switch the whole project to utf-8 or you make sure you utf8_encode() any string before using json_encode().
make sure the translation file itself was explicitely stored as UTF-8
After that reload cache blocks and translations
Related
I have a simple `url that pass two parameters. Name and cellphone. But when I use special characters, the parameter can't be decoded. It appears the ?? instead of the character.
I already tried use urldecode($_GET['name']), rawurldecode, html_entity_decode, utf8_decode, but none of this worked.
I have the utf-8 meta tag in my HTML and I also tryed pass this as a header inside php, but it didn't work.
The code is like this
<?php echo $_GET['name']; ?>
You simply have the use the correct function, which is utf8_encode:
<?php echo utf8_encode($_GET['name']); ?>
Output:
Consultório
The function utf8_encode:
This function converts the string data from the ISO-8859-1 encoding to
UTF-8.
See the documentation here.
name=Consult%F3rio
This is the good old ISO-8859-1 encoding for Consultório of the early days of the web. If the decoded version renders incorrectly, it's very likely that your application is not using ISO-8859-1 at all, thus there's no benefit in using it there either. If your app is using UTF-8, the simplest solution would be to switch entirely to UTF-8:
Consult%C3%B3rio
This is basically what you get with any builtin PHP function when fed with UTF-8 data because they work at byte level:
var_dump(rawurlencode('Consultório')); // string(16) "Consult%C3%B3rio"
If this happens to be third-party data you can't control, please check Martin's answer.
I'm grabbing a bunch of data from a database and putting it into a PHP array. I'm then looking to json_encode that array using $output = json_encode($out).
My issue is that from time to time, something in the array is not able to be read by json_encode and the whole thing fails. If I use print_r($out) to have a look, I can clearly see where it's failing, because the character that is screwing things up always appears as a question mark inside of a black diamond �.
First - what are these characters?
Second - Is there a function I can pass the elements through prior to adding them to the array that would strip these out, or replace 'them' with blanks?
I found the answer to this. Since the data coming FROM the database was stored with the "black diamond" character, I needed to get this out POST grabbing it from the database.
$x[4] = utf8_encode(odbc_result($query, 'B'));
By passing the result through utf8_encode, the string is encoded into UTF-8 and the illegal character is removed.
Say echo json_encode($out);
This will solve your issue
Black diamonds are browser issue. Database uses plain question marks.
It seems you are getting already wrong data from databalse. But that's quite tricky to have incorrect utf with your settings. You need to check everything
if your table marked with utf8 charset
if your data indeed encoded in utf (not marked but indeed encoded)
if your server sending correct charset in Content-type header.
it is also useful to see the page choosing different charsets from your browser menu.
But first of all you have to wipe any trace of all random actions you tried, all these various encode, decode and stuff. Just plain and direct output from database. Otherwise you will never get to the problem
I have had the problem a few times now while working on projects and I would like to know if there's an elegant solution.
Problem
I am pulling tweets via XML from twitter and uploading them to my DB however when I output them to screen I get these characters:
"moved to dusseldorf.�"
OR
también
and if I have Russian characters then I get lots of ugly boxes in place.
What I would like is the correct native accents to show under one encoding. I thought was possible with UTF-8.
What I am using
PHP, MYSQL
After reading in the XML file I am doing the following to cleanse the data:
$data = trim($data);
$data = htmlentities($data);
$data = mysql_real_escape_string($data);
My Database Collation is: utf8_general_ci
Web page character set is: charset=UTF-8
I think it could have something to do with HTML entities but I really appreciate a solution that works across the board on projects.
Thanks in advance.
Replace this line:
$data = htmlentities($data);
With this:
$data = htmlentities($data, null, "UTF-8");
That way, htmlentities() will leave valid UTF-8 characters alone. For more information see the documentation for htmlentities().
You need to change your connection's encoding to UTF-8 (it's usually iso-8859-1). See here: How can I store the '€' symbol in MySQL using PHP?
Calling htmlentities() is unnecessary when you get the encodings right. I would remove it completely. You'll just have to be careful to use htmlspecialchars() when outputting the data a in HTML context.
Make sure that you set your php internal encoding ot UTF8 using iconv_set_encoding, and that you call htmlentities with the encoding information as EdoDodo said. Also make sure that you're database stores with UTF8-encoding, though you say that's already the case.
You can't use htmlentities() in it's default state for XML data, because this function produces HTML entities, not XML entities.
The difference is that the HTML DTD defines a bunch of entity codes which web browsers are programmed to interpret. But most XML DTDs don't define them (if the XML even has a DTD).
The only entitity codes that are available by default to XML are >, < and &. All other entities need to be presented using their numeric entity.
PHP doesn't have an xmlentities() function, but if you read the manual page for htmlentities(), you'll see in the comments that that plenty of people have had this same issue and have posted their solutions. After a quick browse through it, I'd suggest looking at the one named philsXMLClean().
Hope that helps.
I seem to be completely unable to get around utf-8 character encoding.
So I'm exporting content from a database as a utf-8 xml file.
The software I am importing into is quite strict about character encoding, so I can't just put everything in CDATA tags.
There's a whole bunch of weird characters, e.g. ’, — … already in the data.
These aren't working in the xml and need to be replaced out (normally with just a ' quote).
Ideally, I'd like to decode all the characters, and then use htmlspecialchars($text, ENT_COMPAT, 'UTF-8', FALSE) to encode them back again. But I can't seem to find a function that will decode them. Is there one?
I've started to manually go through each entity with a str_replace() but it's turning into a much bigger job than I anticipated.
Any help would be a lifesaver.
Thanks
html_entity_decode() perhaps?
in some cases, in character conversion issues in php, it is important to have a locale set. Doesn't matter which, e.g.
setlocale(LC_CTYPE,'en_US.utf8');
But I would advise that any time invested in getting the encoding right from the beginning, without reverting to entities, if at all possible, is worth it.
Well, I give up.
I've been messing around with all I could think of to retrieve data from a target website that has information in traditional Chinese encoding (charset=GB2312).
I've been using the simple_html_parser like always but it doesn't seem to return the Chinese characters, in fact all I get are some weird question marks embedded inside a rhomboid shape.
("�������ѯ�ؼ��֣�" Like so)
Declaring the encoding for the php file didn't do anything except of getting rid of some unwanted character showing at the start of the page.
By declaring it I mean:
header('Content-Type', 'text/html; charset=GB2312');
I can't get any data that's written in Chinese, also tried file_get_contents with the same luck. I'm probably missing something obvious since I can't find any related discussion elsewhere.
Thanks in advance.
Have you tried converting the encoding with mb_convert_encoding or iconv, e.g.
$str = mb_convert_encoding($content, 'UTF-8', 'GB2312');
or
$str = iconv("UTF-8", "GB2312//IGNORE", $content);
Get it in whatever character set the source uses, then convert it to something usable locally, such as UTF-8. Then send it to the browser.
set header('Content-Type: text/html; charset=utf-8');
It's working for me