I need to convert unicode to character in PHP. I am using MySQL database to store text The text is in unicode format with collation utf8-general-ci. When I retrive those data and display, some special characters are displayed:: like "मिनिसà¥à¤•à¤°à¥à¤Ÿà¤®à¤¾ करà¥à¤•à¥‡ नजर" for the text "मिनिस्कर्टमा कर्के नजर". This is Nepali font in unicode format. I need it in character or ascii format in PHP. I have tried utf8 encode and decode but none of them worked(displays question marks ???? in decoding and "à ¤®à ¤¿à ¤¨à ¤¿à ¤¸à ¥Âà ¤•à ¤°à ¥Âà ¤Ÿà ¤®à ¤¾ à ¤•à ¤°à ¥Âà ¤•à ¥‡ à ¤¨à ¤œà ¤°" on encoding). So, how can I get ascii value or character or unicode value of each unicode characters from mysql database in PHP???
Chnage the collation to utf_bin and in header of your pages <meta http-equiv="Content-Type" content="text/html;charset=UTF-8">. Hope it works.
Ok, I got it. I used this php library and use utf8_chr_to_unicode_style function to convert each unicode charaters to code. I converted all the codes to my required font code(preeti nepali font code). That's all :).
Try function iconv.
It's a function for conversions from different encodings. Please try some at the link provided above. If you cannot manage to figure it out then comment and I will try to research more on the subject.
i have an issue related with the utf-8 charset. I've been all around the web (well, not entirely) but for quite awhile now and the best advice was and is to set the header charset to "UTF-8".
However, I was developing my web application locally on my machine using xampp (and sometimes wamp so as to get a distinction of the two when it came to debugging my code). Everything was working great =). But as soon as i uploaded it online, the result was not all that jazzy (the kind of errors you would get if you had set the headers to a different charset like "iso-8859-1").
Every header in my code has UTF-8 as the default charset, but i still got the same "hieroglyphic thingies". Then you guys gave me the idea that the issue isn't my code but the php.ini that was running it.. Turns out my local machine was running php 5.5 and the cpanel where i had uploaded my web application was running native php 5.3.
Well, when i changed the version of php that my cpanel was set by default from Native PHP 5.3 to PHP 5.5, believe you me guys =) it worked like a charm just like as if i was right there at the localhost of machine.
NOTE: Please, if you got the same problem as i did, just make sure your PHP is 5.5 version.. I'm posting this coz i feel you guys. Cheers!
I am building a webpage in HTML with PHP and MySQL and I ran into trouble with swedish characters ÅÄÖ when running page. They show up as � instead of Å/Ä/Ö.
I have set the charset to UTF-8 in both HTML meta-tag and via PHP:
<?php
header('Content-type: text/html; charset=UTF-8');
?>
<meta charset="UTF-8">
Also, MySQL runs utf8_general_ci collation on all tables.
All files should also be encoded and saved as UTF-8 without Unicode Signature (BOM) and no normalization form.
All this have worked flawless before, but today, nomather what I try I do end up with � instead of Å/Ä/Ö. Is there a good way to debug this and find the problem?
Is any of my steps unnecessary or have I forgotten anything?
What you need from deceze's article is the part regarding the SET NAMES:
mysql_set_charset('utf8', $connection); //not mysql_query("SET NAMES 'utf8'");
Just add that at the beginning of your php code, after the database connection was started
You may try save your php files in UTF-8 encoding. I assume the files are written in something else (possibly ISO-xxxx or ANSI)
To do that with Notepad++, select all the lines and copy to clipboard, change the coding to UTF-8 without BOM in encoding menu, then paste over everything and save.
Is this for only few records or all records with swedish characters?
You can change the page encoding manually in browser settings - this is how you test it: change it to latin1/iso-8854-1 to see if it displays these correctly that are wrong as utf-8.
Chances are someone is using browser that is not supporting utf8 or fiddled with the encoding manually.
Also make sure you db connection is utf8 too. (set names utf8;)
I am trying to show नेपाल at my page, but is shows नेपाल. What is causing the unicode to render like this.
It's caused by something (likely the web browser) interpreting the characters as something else than Unicode. Browsers are quite bad at guessing the proper encoding, so it must be explicitly defined. Perhaps you should have something like this in the head section:
<meta http-equiv="content-type" content="text/html;charset=utf-8" />
It's also possible that the font being used does not cover those characters.
Write at the top of the script <?php header('Content-Type: text/html; charset=UTF-8');?>
If the data comes from a database then this
$mysqli->query('set character set utf8');
should help. Put it inside your db connection :-)
Reading from unicode.org:
If you are unable to read some Unicode
characters in your browser, it may be
because your system is not properly
configured. Here are some basic
instructions for doing that. There are
two basic steps:
Install fonts that cover the
characters you need
Configure your
browser to use them.
Well, I got a MySQL db, encoded as utf8_unicode_ci, and it runs like a charm with the current application (written in Code Igniter)
Now, I'm developing a new PHP app, and when I try to recover the data, several characters are unreadable - chars appears ok in the DB with phpMyAdmin, but when I try to put it up in a webpage, it became like "ROLA �60".
These characters are spanish letters, such as ñ, á, Ó... or ascii codes like €, Ø...
Where's the problem? I've set the page as meta http-equiv="Content-Type" content="text/html; charset=UTF-8", I've tried the mysql_set_charset() function, and still nothing.
Any experience with this kind of problems?
I have noticed this exact problem in my database driven applications, and it took me a long time to work it out!
The problem occurs because there are at least three places in your application that need their character sets defining, and they must all be the same character set (and that character set must be able to handle the characters you are handling).
The question mark symbol and it's variations occurs when the browser doesn't understand what character it is being passed.
Make sure your character sets match in the following places:
The HTML head section.
The database collation itself - this can be set in phpMyAdmin when you create a new table, or alter the schema of an existing table.
The most overlooked character set setting: The php.ini file's "default_charset" value (can be set via PHP script or the httpd.conf file).
Judging from your post, it looks like your PHP configuration might be set to be using a different default_charset. This means that your database will be storing the characters fine, and will be sendign the characters fine to your script, but the PHP script itself will not know what to do with the character, and thus outputs to the browser as the annoying question mark symbol.
Try changing the php.ini value to the same charset, and you may be surprised to see the characters displaying fine! If you don't have access to php.ini, you can change the value with the following function:
ini_set("default_charset", "UTF-8");
Hope this helps.
if mysql_set_charset() didn't work try executing
mysql_query('SET NAMES utf8');
after establishing connection.
Any kind of problem requires only one experience: understanding of what are you doing and debugging. Very rare skills nowadays, thanks to sites like stackoverflow.
there are 3 levels where encoding gets involved:
database level
server side script level.
html page level.
each should be checked respectively.
to check HTML level is easy. Just click "View" menu in your browser, then "encoding" and see which one is marked. If it's right one, you are all right. If not - wrong HTTP header being sent and you have to make it right
for the server side there is very little to do. Just to tell mysql which encoding you want. it can be done 2 ways:
mysql_set_charset() - preferred one
SET NAMES query
note that UTF-8 encoding named utf8 in mysql.
database level seems O.K. in your case.
mysql_query('SET NAMES utf8'); works when browsing via Chrome but not FF or IE
Try below may be your problem will be solved:
<?php
$text = "This is the Euro symbol '€'.";
echo 'Original : ', $text, PHP_EOL;
echo 'TRANSLIT : ', iconv("UTF-8", "ISO-8859-1//TRANSLIT", $text), PHP_EOL;
echo 'IGNORE : ', iconv("UTF-8", "ISO-8859-1//IGNORE", $text), PHP_EOL;
echo 'Plain : ', iconv("UTF-8", "ISO-8859-1", $text), PHP_EOL;
?>
http://php.net/manual/en/function.iconv.php
I'm writing a php program that pulls from a database source. Some of the varchars have quotes that are displaying as black diamonds with a question mark in them (�, REPLACEMENT CHARACTER, I assume from Microsoft Word text).
How can I use php to strip these characters out?
If you see that character (� U+FFFD "REPLACEMENT CHARACTER") it usually means that the text itself is encoded in some form of single byte encoding but interpreted in one of the unicode encodings (UTF8 or UTF16).
If it were the other way around it would (usually) look something like this: ä.
Probably the original encoding is ISO-8859-1, also known as Latin-1. You can check this without having to change your script: Browsers give you the option to re-interpret a page in a different encoding -- in Firefox use "View" -> "Character Encoding".
To make the browser use the correct encoding, add an HTTP header like this:
header("Content-Type: text/html; charset=ISO-8859-1");
or put the encoding in a meta tag:
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
Alternatively you could try to read from the database in another encoding (UTF-8, preferably) or convert the text with iconv().
I also faced this � issue. Meanwhile I ran into three cases where it happened:
substr()
I was using substr() on a UTF8 string which cut UTF8 characters, thus the cut chars could not be displayed correctly. Use mb_substr($utfstring, 0, 10, 'utf-8'); instead. Credits
htmlspecialchars()
Another problem was using htmlspecialchars() on a UTF8 string. The fix is to use: htmlspecialchars($utfstring, ENT_QUOTES, 'UTF-8');
preg_replace()
Lastly I found out that preg_replace() can lead to problems with UTF. The code $string = preg_replace('/[^A-Za-z0-9ÄäÜüÖöß]/', ' ', $string); for example transformed the UTF string "F(×)=2×-3" into "F � 2� ". The fix is to use mb_ereg_replace() instead.
I hope this additional information will help to get rid of such problems.
This is a charset issue. As such, it can have gone wrong on many different levels, but most likely, the strings in your database are utf-8 encoded, and you are presenting them as iso-8859-1. Or the other way around.
The proper way to fix this problem, is to get your character-sets straight. The simplest strategy, since you're using PHP, is to use iso-8859-1 throughout your application. To do this, you must ensure that:
All PHP source-files are saved as iso-8859-1 (Not to be confused with cp-1252).
Your web-server is configured to serve files with charset=iso-8859-1
Alternatively, you can override the webservers settings from within the PHP-document, using header.
In addition, you may insert a meta-tag in you HTML, that specifies the same thing, but this isn't strictly needed.
You may also specify the accept-charset attribute on your <form> elements.
Database tables are defined with encoding as latin1
The database connection between PHP to and database is set to latin1
If you already have data in your database, you should be aware that they are probably messed up already. If you are not already in production phase, just wipe it all and start over. Otherwise you'll have to do some data cleanup.
A note on meta-tags, since everybody misunderstands what they are:
When a web-server serves a file (A HTML-document), it sends some information, that isn't presented directly in the browser. This is known as HTTP-headers. One such header, is the Content-Type header, which specifies the mimetype of the file (Eg. text/html) as well as the encoding (aka charset).
While most webservers will send a Content-Type header with charset info, it's optional. If it isn't present, the browser will instead interpret any meta-tags with http-equiv="Content-Type". It's important to realise that the meta-tag is only interpreted if the webserver doesn't send the header. In practice this means that it's only used if the page is saved to disk and then opened from there.
This page has a very good explanation of these things.
As mentioned in earlier answers, it is happening because your text has been written to the database in iso-8859-1 encoding, or any other format.
So you just need to convert the data to utf8 before outputting it.
$text = “string from database”;
$text = utf8_encode($text);
echo $text;
To make sure your MYSQL connection is set to UTF-8 (or latin1, depending on what you're using), you can do this to:
$con = mysql_connect("localhost","username","password");
mysql_set_charset('utf8',$con);
or use this to check what charset you are using:
$con = mysql_connect("localhost","username","password");
$charset = mysql_client_encoding($con);
echo "The current character set is: $charset\n";
More info here: http://php.net/manual/en/function.mysql-set-charset.php
I chose to strip these characters out of the string by doing this -
ini_set('mbstring.substitute_character', "none");
$text= mb_convert_encoding($text, 'UTF-8', 'UTF-8');
Just Paste This Code In Starting to The Top of Page.
<?php
header("Content-Type: text/html; charset=ISO-8859-1");
?>
Based on your description of the problem, the data in your database is almost certainly encoded as Windows-1252, and your page is almost certainly being served as ISO-8859-1. These two character sets are equivalent except that Windows-1252 has 16 extra characters which are not present in ISO-8859-1, including left and right curly quotes.
Assuming my analysis is correct, the simplest solution is to serve your page as Windows-1252. This will work because all characters that are in ISO-8859-1 are also in Windows-1252. In PHP you can change the encoding as follows:
header('Content-Type: text/html; charset=Windows-1252');
However, you really should check what character encoding you are using in your HTML files and the contents of your database, and take care to be consistent, or convert properly where this is not possible.
Add this function to your variables
utf8_encode($your variable);
Try This Please
mb_substr($description, 0, 490, "UTF-8");
This will help you. Put this inside <head> tag
<meta charset="iso-8859-1">
That can be caused by unicode or other charset mismatch. Try changing charset in your browser, in of the settings the text will look OK. Then it's question of how to convert your database contents to charset you use for displaying. (Which can actually be just adding utf-8 charset statement to your output.)
what I ended up doing in the end after I fixed my tables was to back it up and change back the settings to utf-8 then I altered my dump file so that DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci are my character set entries
now I don't have characterset issues anymore because the database and browser are utf8.
I figured out what caused it. It was the web page+browser effects on the DB. On the terminals that are linux (ubuntu+firefox) it was encoding the database in latin1 which is what the tabes are set. But on the windows 10+edge terminals, the entries were force coded into utf8. Also I noticed the windows 10 has issues staying with latin1 so I decided to bend with the wind and convert all to utf8.
I figured it was a windows 10 issue because we started using win 10 terminals.
so yet again microsoft bugs causes issues. I still don't know why the encoding changes on the forms because the browser in windows 10 shows the latin1 characterset but when it goes in its utf8 encoded and I get the data anomaly. but in linux+firefox it doesn't do that.
This happened to work in my case:
$text = utf8_decode($text)
I turns the black diamond character into a question mark so you can:
$text = str_replace('?', '', utf8_decode($text));
Just add these lines before headers.
Accurate format of .doc/docx files will be retrieved:
if(ini_get('zlib.output_compression'))
ini_set('zlib.output_compression', 'Off');
ob_clean();
When you extract data from anywhere you should use functions with the prefix md_FUNC_NAME.
Had the same problem it helped me out.
Or you can find the code of this symbol and use regexp to delete these symbols.
You can also change the caracter set in your browser. Just for debug reasons.
Using the same charset (as suggested here) in both the database and the HTML has not worked for me... So remembering that the code is generated as HTML, I chose to use the "(HTML code) or the " (ISO Latin-1 code) in my database text where quotes were used. This solved the problem while providing me a quotation mark. It is odd to note that prior to this solution, only some of the quotation marks and apostrophes did not display correctly while others did, however, the special code did work in all instances.
I ran the "detect encoding" code after my collation change in phpmyadmin and now it comes up as Latin_1.
but here is something I came across looking a different data anomaly in my application and how I fixed it:
I just imported a table that has mixed encoding (with diamond question marks in some lines, and all were in the same column.) so here is my fix code. I used utf8_decode process that takes the undefined placeholder and assigns a plain question mark in the place of the "diamond question mark " then I used str_replace to replace the question mark with a space between quotes.
here is the
[code]
include 'dbconnectfile.php';
//// the variable $db comes from my db connect file
/// inx is my auto increment column
/// broke_column is the column I need to fix
$qwy = "select inx,broke_column from Table ";
$res = $db->query($qwy);
while ($data = $res->fetch_row()) {
for ($m=0; $m<$res->field_count; $m++) {
if ($m==0){
$id=0;
$id=$data[$m];
echo $id;
}else if ($m==1){
$fix=0;
$fix=$data[$m];
$fix = utf8_decode($fix);
$fixx =str_replace("?"," ",$fix);
echo $fixx;
////I echoed the data to the screen because I like to see something as I execute it :)
}
}
$insert= "UPDATE Table SET broke_column='".$fixx."' where inx='".$id."'";
$insresult= $db->query($insert);
echo"<br>";
}
?>
For global purposes.
Instead of converting, codifying, decodifying each text I prefer to let them as they are and instead change the server php settings.
So,
Let the diamonds
From the browser, on the view menu select
"text encoding" and find the one which let's you see your text
correctly.
Edit your php.ini and add:
default_charset = "ISO-8859-1"
or instead of ISO-8859 the one which fits your text encoding.
Go to your phpmyadmin and select your database and just increase the length/value of that table's field to 500 or 1000 it will solve your problem.