I'm currently sending info by webservice from Android to a function in PHP which sends that data to the database.
Although, if I have accents in my words like names, I receive this error from PHP:
Incorrect string value: '\xE1\xE1mos ...' for column 'firstname' at
row 1
The names could have accents, like:
António, João, etc.
In PHP function, before inserting into database, I did this without success:
$db->set_charset('utf8');
$query = $db->prepare("SET NAMES 'utf8';");
$query->execute();
If I send the data without any kind of accents, the service works perfectly.
Edit: Solved using utf8_encode(variable).
When sending data to your WebServices in Android, try URL encoding the strings before putting them in the URL/form data, like:
// ...
String encodedName = URLEncoder.encode(name, Http.UTF_8);
// use encodedName instead of name to pass as parameters to the webservices
In the PHP side, before using the parameter, use the urldecode function.
$name = urldecode ($_GET['my_param']);
And keep using the encoding in the database. Also, check if your tables have been created using UTF_8 colation.
It is not enough to specify the encoding of the connection. You need to change the charset of your tables and database to utf-8.
As such, do the following
ALTER DATABASE databasename CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE tbl_name CONVERT TO CHARACTER SET "utf-8" COLLATE utf8_general_ci;
Related
I am inserting some data from a sjson file into mysql database. Some of the data is formatted using unicode.
When I echo the data on the browser is showing fine however, it is inserted in the database not formatted properly.
In the database I am using as Collation utf8_unicode_ci.
JSON data:
anch\u2019io, sar\u00e0
Showing in the Browser:
anch'io, sarà
Showing in the mysql database:
anch’io, sarÃ
How can I inserted in the database the text properly formatted?
PHP
$getUrl = "https://example.com/79809000.json";
$json_level = file_get_contents($getUrl);
$data_level = json_decode($json_level);
$text = $data_level->{"text"};
mysqli_query($conn, "INSERT INTO `70_level`(`text`) VALUES ('$text')");
I have tried to use addslashes, htmlentities but it does not work.
You are probably not using utf8 as your character set connection/collation connection.
The easiest way to fix this will be to use
$conn = mysqli_connect(...);
mysqli_query($conn, "SET NAMES 'utf8'");
The SET NAMES query should be the first query you run, so make sure you put it right after your mysqli_connect function.
Note that this change will affect all the data, so if you already have data in the database - you will need to re-insert it using the new (utf8) charset.
I have form with input text, when i add text
Un sac à main de femme recèlerait une quantité importante de bactéries
it adds in database only Un sac
i have tried with addslashes, mysql_real_escape_string, htmlspecialchars etc. also using UTF-8 encoding, but still it can not insert whole string
YOu should use utf8_unicode_ci as your column's collation in orer for French strings to be added in it.
In order to store non-US strings in the database, you must ensure that each of the following 3 steps are correctly implemented:
You database table must be set to a charset compatible with French. To be future proof, I recommend creating tables with UTF-8. For more information see the MySQL documentation.
Your database connection must be set to a proper character set both when storing and when querying. To do this, use mysqli_set_charset() (or whatever your MySQL connector offers).
Your input form AND your view page must be served with the exact character set as your data. To do that, you will need to set the following header: header('Content-Type: text/html; charset=UTF-8'); (If you are using a different charset, change it accordingly.)
You can of course use a different character set for storage and representation but why would you want to do that?
Also, when working with databases and HTML, you should consider:
ALWAYS escape your data as it goes into the database. Use mysqli_real_escape_string() or whatever escape method your database connector offers. Also, do NOT set the connection charset by using SET NAMES UTF8, otherwise your connector library will not know what charset to use for escaping. For more information google "sql injection".
ALWAYS escape your data as it goes into HTML with htmlspecialchars(). Also pay attention to ALWAYS provide the correct character set. For more information google "xss".
After breaking my head for 2 days straight and reading all the possible answers here's what solved the problem and allows me to insert additional weird characters like em dash etc. and retrieve data without seeing weird characters.
Here's the complete step-by-step setup.
The collation of the db column need to be: utf8_general_ci
The type is: varchar(250)
In the PHP header set the default client character set to UTF8
mysql_set_charset("UTF8", $link);
Set the character set result so we can show french characters
$sql = "SET character_set_results=utf8";
$result = mysql_query($sql);
In the html header specify, so you can view the french characters:
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
When inserting the data do NOT use utf8_decode, just the below will work fine
$query = 'insert into tbl (col) VALUES ("'.mysql_real_escape_string($variable).'");
Use normal queries to retreive data, example query:
$query = "select * from table;";
Finally got this fixed, hope this is helpful to others.
In the php:
header ('Content-type: text/html; charset=utf-8');
After connection:
mysql_set_charset("utf8");
Just to follow up with this, I was using dbForge Studio and just pasting in French text and I had all the collations/encoding set properly. The one thing I didn't have set was the actual encoding for the connection to the db. Set it to UTF8 and all was well again. #2 in #Janoszen answer.
Had the same problem. The input text came from ANSII file, so it wasn't quite UTF8, despite all my utf8 settings. utf8_encode(input_text) solved it.
I have tried
htmlentities()
. .it saves the string as it is in the database
You should try this to insert special character in mysql :
$con = mysql_connect($server,$uname,$pass);
$res = mysql_select_db($database,$con)
mysql_set_charset("letin1", $con);
I am trying to convert a string from HTML-ENTITIES to UTF-8 and then save the encoded string in my database. The html entities are greek letters and look for example like this: νω
Now I tried thousands of different ways, starting from just using utf8_encode or html_entity_decode until now I came across the function mb_convert_encoding().
Now the really weird thing is that when converting my string and then outputting it, it is correctly encoded to utf-8, but when inserting this string into my database I end up getting something like: ξÏνω.
This is the code for the encoding:
header('Content-Type: text/html; charset=utf-8');
mb_internal_encoding('utf-8');
......
while($arr = $select->fetch_array(MYSQLI_ASSOC))
{
$text = $arr["greek"];
$result = mb_convert_encoding($text, 'UTF-8', 'HTML-ENTITIES');
$mysqli->query("UPDATE some SET greek = '".$result."'");
}
When outputting my query and then manually doing a sql query in phpmyadmin it works fine, so it doesnt seem to be a problem of my db. There must be some problem when transferring the encoded string to my database...
As you see in your script, you are instructing the browser to use UTF8. That is the first step.
However your database needs the same thing and also the encoding/collation on the tables need to be UTF8 too.
You can either recreate your tables using utf8_general_ci or utf8_unicode_ci as the collation, or convert the existing tables (see here)
You need to also make sure that your database connection i.e. php code to mysql is using UTF8. If you are using PDO there are plenty of articles that show how to do that. The simplest way is to do:
$mysqli->query('SET NAMES utf8');
NOTE The change you will make now is final. If you change the connection encoding to your database, you could affect existing data.
EDIT You can do the following to set the connection
$mysqli = new mysqli($host, $user, $pass, $db);
if (!$mysqli->set_charset("utf8")) {
die("Error loading character set utf8: %s\n", $mysqli->error);
}
$mysqli->close();
Links of interest:
Whether to use "SET NAMES"
Execute the SET NAMES 'utf8' query prior to any others.
im facing a really stressing problem here.. i have everything in UTF-8 , all my DB and tables are utf8_general_ci but when trying to insert or update from a single PHP script all i see are symbols.. but if i edit in phpmyadmin the words are shown correctly.. i found that if i run the utf8_decode() function to my strings in php i can make it work, but im not planning to do that because is a mess and it should work without doing that :S
Here is a basic code im using to test this:
<?php
$conn=mysql_connect("localhost","root","root")
or die("Error");
mysql_select_db("mydb",$conn) or
die("Error");
mysql_query("UPDATE `mydb`.`Clients` SET `name` = '".utf8_decode("Araña")."' WHERE `Clients`.`id` =25;",
$conn) or die(mysql_error());
mysql_close($conn);
echo "Success.";
?>
This is what i get if i dont decode utf8 with php utf8_decode function:
instead of Araña, i get : Araña
I've run into the same issue many times. Sometimes it's because the type of database link I'm selecting from isn't the same type that I'm using for inserting and other times, it's from file data into a database.
For the later instance, mysql_set_charset('utf8',$link); is the magic answer.
Place the call to mysql_set_charset just after you select your database via mysql_select_db.
#ref http://php.net/manual/en/function.mysql-set-charset.php
"Araña" IS UTF-8. The characters "ñ" represent the two bytes into which the Spanish ñ are encoded in UTF-8. Whatever you're reading it back with is not handling the UTF-8 and is displaying it as (it appears) ISO-8859-1.
That DDL you mentioned has to do with the collation, not the character set. The correct statement would be:
ALTER TABLE Clients CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
You still need to make sure the client library (libmysql or whatever driver PHP is using) is not transcoding the data back to ISO-8859. mysql_set_charset('utf8') will explicitly set the client encoding to UTF-8. Alternatively, you can send a SET NAMES UTF8; right after you connect to the database. To do that implicitly, you can change the my.cnf [client] block to have utf-8 as the client character encoding (and /etc/init.d/mysql reload to apply). Either way, make sure the client doesn't mangle the results it's pulling.
[client]
default-character-set=utf8
You do not need to use utf8_decode if you're using mbstrings. The following php.ini configuration should ensure UTF-8 support on the PHP side:
mbstring.internal_encoding = utf-8
mbstring.http_output = utf-8
mbstring.func_overload = 6
Finally, when you display the results in HTML, verify that the page's encoding is explicitly UTF-8.
I have a MySQL db, i've set collation = utf8_unicode_ci.
I'm trying to fetch the value through PHP but i'm getting "???" instead of the actual string.
I have read about this subject and tried using mb_convert_encoding but it didn't work, what am I missing?
Can someone please post a code snippet that actually pulls a value from a DB and echos the string to the screen?
Thanks,
I have a MySQL db, i've set collation = utf8_unicode_ci.
I'm trying to fetch the value through PHP but i'm getting "???" instead of the actual string.
Character sets are how characters are encoded.
Collations are how characters are sorted.
These are different things. Chances are that your tables or columns have the right collation, but the wrong character set. The Internationalization section of the MySQL manual has a great deal of information on how to set things up correctly.
Can someone please post a code snippet that actually pulls a value from a DB and echos the string to the screen?
Let's demonstrate how to use utf8 as a character set, and the utf8 "general case insensitive" collation. I'm using PDO in this example, but the same general idea should work with mysqli as well. I wouldn't advise using the old mysql extension.
// Let's tell MySQL we're going to be working with utf8 data.
// http://dev.mysql.com/doc/refman/5.1/en/charset-connection.html
$db->query("SET NAMES 'utf8'");
// Create a table with our proper charset and collation.
// If we needed to, we could specify the charset and collation with
// each column.
// http://dev.mysql.com/doc/refman/5.1/en/charset-column.html
// We could also set the defaults at the database level.
// http://dev.mysql.com/doc/refman/5.1/en/charset-database.html
$db->query('
CREATE TABLE foo(
bar TEXT
)
DEFAULT CHARACTER SET utf8
DEFAULT COLLATE utf8_general_ci
ENGINE=InnoDB
');
// I don't know Arabic, so I'll type this in English. It should
// work fine in Arabic, as long as the string is encoded as utf8.
$sth = $db->prepare("INSERT INTO foo(bar) VALUES(?)");
$sth->execute(array("Hello, world!"));
$sth = $db->query("SELECT bar FROM foo LIMIT 1");
$row = $sth->fetch(PDO::FETCH_NUM);
echo $row[0]; // Will echo "Hello, world!", or whatever you inserted.
#tomp's comment below is correct. Make sure to emit a proper character set with your content type header. For example:
header('Content-type: text/html; charset=utf-8'); // Note the dash!