Encoding problems when importing sql statement from file - php

I have encountered a problem, when i try to execute a sql statement I have stored in a file. The problem is that special characters like æøå becomes gibberish when they are imported to the database.
I use the code below:
$dsn = 'mysql:host='.DBHOST.';dbname='.DBBASE;
$db = new PDO($dsn, DBUSER, DBPASS);
//$db->exec("set names 'utf8'");
$sql = "TRUNCATE TABLE wtm_meta_language";
$db->exec($sql);
$sql = file_get_contents('language.sql');
$db->exec($sql);
The language.sql file is encoded in UTF-8 and so is the database.
I have tried to force the database connection to use utf-8 by adding "set names 'utf8'" to my code (row 3, that is now a comment), but when I add this row nothing gets imported at all.
I hope someone has an idea to how to solve this.

Your results are not getting retrieved because you did not add charset=utf8 .
$db = new PDO('mysql:host='.DBHOST.';dbname='.DBBASE.';charset=utf8', 'username', 'password');
and then it should allow you to get the required result.

Could be that PHP does not detect that the file is UTF-8 since, according to your comment, the BOM is missing.
I found this little function here which should force it to read UTF-8.
function file_get_contents_utf8($fn) {
$opts = array(
'http' => array(
'method'=>"GET",
'header'=>"Content-Type: text/html; charset=utf-8"
)
);
$context = stream_context_create($opts);
$result = #file_get_contents($fn,false,$context);
return $result;
}
Also, if I were you I'd use mysql_set_charset to set the character set on the mysql connection. Also I'd use SHOW VARIABLES LIKE 'character_set%'; so see what the current connection character set is.

Related

Import file to mysql database resulting in bad accents

I have a utf8 json file containing some stuff like that :
"type":"Feature","geometry":{"type":"LineString","coordinates":[[-73.6371999890417,45.451049968774],[-73.6362864025022,45.4507004803266]]},"properties":{"FIN_GCH":191,"TYP_VOIE":"avenue","LIM_GCH":"Montréal-Ouest","NOM_VOIE":"Brock","CLASSE":0,"DEB_DRT":0,"FIN_DRT":0,"LIM_DRT":"Montréal-Ouest","LIE_VOIE":"","DEB_GCH":173,"ID_TRC":1602102,"DIR_VOIE":"Sud","SENS_CIR":-1}}
,{"type":"Feature","geometry":{"type":"LineString","coordinates":[[-73.6382813452807,45.4514760942107],[-73.6371999890417,45.451049968774]]},"properties":{"FIN_GCH":171,"TYP_VOIE":"avenue","LIM_GCH":"Montréal-Ouest","NOM_VOIE":"Brock","CLASSE":0,"DEB_DRT":150,"FIN_DRT":152,"LIM_DRT":"Montréal-Ouest","LIE_VOIE":"","DEB_GCH":143,"ID_TRC":1602103,"DIR_VOIE":"Sud","SENS_CIR":-1}}
With the help of a php script, i put all the data in a mysql UTF8 database but even so, the entries have weird accents : instead of Montréal-Ouest i have Montréal-Ouest
Do you have any idea what is the problem please ?
Thank you
I had to set a utf8 connection with the server by setting the options array like this
$options = array(
PDO::MYSQL_ATTR_INIT_COMMAND => "SET NAMES utf8"
);
$connection = new PDO( $dns, $user, $password, $options );
Thanks to CBroe

Parsing UTF-8 Characters directly to UTF-8 Database [duplicate]

This question already has answers here:
UTF-8 all the way through
(13 answers)
Closed 9 years ago.
Just a simple question.. I am using UTF-8 as standard in both my MySQL database and HTML file.. One thing that I don't understand is why a letter is shown differently in SQL than when I typed it. I am sure it should be placed the exact same way as I write it, when using UTF-8.
Is there a class or something that automatically keeps all posted data in UTF-8 and then posts it as it is?
Your text editor also has an effect on your character encoding. Be sure to use a text editor or IDE that you can set the character encoding to UTF-8.
Try this 1. method if not succes 2.metod. Last method i say open HTML File with notepad2 editor and change encoding from File menu to UTF8 and save. i think it will be solution;
Method
$dsn = $config ["driver"] . ':dbname=' . $config ['dbname'] . ';host=' . $config ['host'];
. ';charset=UTF-8'; //---> this is the solution
try {
$this->link = new PDO ( $dsn, $config ['user'], $config ['pass'] );
// The mysql driver ignores the charset directive in $dsn (yeah, that sucks)
if ($config ["driver"] == 'mysql')
$this->link->exec ( "set names utf8" );
}
catch ( PDOException $e ) {
die ( "Hata var : " . $e->getMessage () );
}
Other:
$encodings = array("windows-1254","utf-8", "iso-8859-9" );
$data = file_get_contents($TempSrc);
$data=mb_convert_encoding($data, 'UTF-8', mb_detect_encoding($data, $encodings));
MY SOLUTION
In case anybody experience similar problem, mine was solved by adding this line after the PDO connection , array(PDO::MYSQL_ATTR_INIT_COMMAND => "SET NAMES 'utf8'").
Now the connection string looks like this:
$conn = new PDO('mysql:host=localhost;dbname=$dbname', $username, $password, array(PDO::MYSQL_ATTR_INIT_COMMAND => "SET NAMES 'utf8'"));

Characters not displaying correctly on a UTF-8 website

I've done everything I can think of, but special characters are not displaying correctly on this webpage.
For example, in the database it's:
But on the site it's:
Nouveaux R�alistes
Here's everything I've checked...
The database is set to UTF-8:
The page was written in NetBeans, with the document encoding set to UTF-8:
The page header declares UTF-8:
The meta charset is set to UTF-8:
I've even added the following line to my .htacess:
But there characters are still not displaying correctly, and I get the following error from the W3C Validator:
I feel like I've attempted everything, but it still won't work. (I've even tried htmlspecialchars and htmlentities in PHP, but the page doesn't even render!)
UPDATE
As requested, here is the code I'm using:
class Exhibition {
public $exhibitionDetails;
public function __construct(Database $db, $exhibitionID){
$this->_db = $db;
$params['ExhibitionID'] = $exhibitionID;
$STH = $this->_db->prepare("SELECT *
FROM Exhibition
INNER JOIN Schedule
ON Exhibition.ExhibitionID = Schedule.ExhibitionID
WHERE Schedule.Visible = 1
AND Exhibition.ExhibitionID = :ExhibitionID;");
$STH->execute($params);
$this->exhibitionDetails = $STH->fetchAll(PDO::FETCH_ASSOC);
}
}
And...
try {
$db = new Database(SITE_ROOT."/../");
$db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$exhibition = new Exhibition($db,$_GET['id']);
} catch (PDOException $e) {
echo "<p class='error'>ERROR: ".$e->getMessage()."</p>";
}
And finally...
<p><?php echo $exhibition->exhibitionDetails[0]["Desc"]; ?></p>
If you are using mysql_* functions:
mysql_query("SET NAMES 'utf8'");
If you are using PDO
$dsn = 'mysql:host=localhost;dbname=testdb';
$username = 'username';
$password = 'password';
$options = array(
PDO::MYSQL_ATTR_INIT_COMMAND => 'SET NAMES utf8',
);
$dbh = new PDO($dsn, $username, $password, $options);
It sets connection encoding.
It's been a few years since I've used PHP but back then it didn't natively support Unicode and a quick search of google tells me it still doesn't. You can still make it work though.
Here's a great link:
Encodings And Character Sets To Work With Text

Polish characters in mysql response

I have a problem with polish characters. I can't get correctly written words, like '?ukasz' instead of "Łukasz" or even "null", when it supposed to be "Kraków". I tried "mysql_set_charset('utf-8'/'iso-8859-1')" after mysql_connect or iconv(on json_encode($output)) and it's still the same (except now there is "Krak\u00f3" instead of "null"). I'll appreciate any help.
This is a php file for my Android app:
$id_client = $_REQUEST['id_klienta'];
$con=mysql_connect('localhost', 'root', '') or die(mysql_error());
mysql_select_db('courier_helper') or die(mysql_error());
$sql=mysql_query("SELECT * FROM `clients` WHERE id_klienta='$id_client'");
while($r=mysql_fetch_assoc($sql))
$output[]=$r;
print(json_encode($output));
mysql_close($con);
?>
You have to make sure, that you are using UTF-8 everywhere:
script file encoding (UTF-8 instead of ANSI) - you can set encoding it in Notepad++
html code (meta charset)
database table charset (when you are creating table or database)
database mysql_set_charset('utf8', $connection_obj);
database SET NAMES utf8 - run that SQL command after connecting
And one more thing - get familiar with PDO. This is my PDO connect function I use:
function DbConnect()
{
$db_host = "localhost";
$db_name = "database_name";
$db_user = "your_username";
$db_pass = "your_passwd";
$link = new PDO("mysql:host=$db_host;dbname=$db_name; charset=UTF-8", $db_user, $db_pass);
$link->exec("set names utf8;");
return $link;
}
You can use that function like this (this is PDO example):
$link = DbConnect();
$query = $link->prepare("SELECT id FROM wp_users");
$query->execute();
$result = $query->fetchAll(PDO::FETCH_ASSOC);
You should have your database storing data as UTF8, which means converting your existing tables.
ALTER TABLE tablename CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
You also need to make sure your connection to the database is UTF8. You can make sure of that by running a SET NAMES query right after your connect.
SET NAMES UTF8
As others mentioned, you should start using PDO.

reading utf-8 content from mysql table [duplicate]

This question already has answers here:
UTF-8 all the way through
(13 answers)
Closed 12 months ago.
I have a mysql table with contents
the structure is here:
I want to read and print the content of this table to html
This is my code:
<?php
include("config.php");
$global_dbh = mysql_connect($hostname, $username, $password)
or die("Could not connect to database");
mysql_select_db($db)
or die("Could not select database");
function display_db_query($query_string, $connection, $header_bool, $table_params) {
// perform the database query
$result_id = mysql_query($query_string, $connection)
or die("display_db_query:" . mysql_error());
// find out the number of columns in result
$column_count = mysql_num_fields($result_id)
or die("display_db_query:" . mysql_error());
// Here the table attributes from the $table_params variable are added
print("<TABLE $table_params >\n");
// optionally print a bold header at top of table
if($header_bool) {
print("<TR>");
for($column_num = 0; $column_num < $column_count; $column_num++) {
$field_name = mysql_field_name($result_id, $column_num);
print("<TH>$field_name</TH>");
}
print("</TR>\n");
}
// print the body of the table
while($row = mysql_fetch_row($result_id)) {
print("<TR ALIGN=LEFT VALIGN=TOP>");
for($column_num = 0; $column_num < $column_count; $column_num++) {
print("<TD>$row[$column_num]</TD>\n");
}
print("</TR>\n");
}
print("</TABLE>\n");
}
function display_db_table($tablename, $connection, $header_bool, $table_params) {
$query_string = "SELECT * FROM $tablename";
display_db_query($query_string, $connection,
$header_bool, $table_params);
}
?>
<HTML><HEAD><TITLE>Displaying a MySQL table</TITLE></HEAD>
<BODY>
<TABLE><TR><TD>
<?php
//In this example the table name to be displayed is static, but it could be taken from a form
$table = "submits";
display_db_table($table, $global_dbh,
TRUE, "border='2'");
?>
</TD></TR></TABLE></BODY></HTML>
but I get ???????? as the results:
Where is my mistake?
Four good steps to always get correctly encoded UTF-8 text:
1) Run this query before any other query:
mysql_query("set names 'utf8'");
2) Add this to your HTML head:
<meta http-equiv="Content-Type" content="text/html;charset=UTF-8">
3) Add this at top of your PHP code:
header("Content-Type: text/html;charset=UTF-8");
4) Save your file with UTF-8 without BOM encoding using Notepad++ or any other good text-editor / IDE.
Set the charset as utf8 as follows:
$conn = new mysqli($servername, $username, $password, $dbname);
$conn->set_charset("utf8");
You are not defining your HTML page as UTF-8. See this question on ways to do that.
You may also need to set your database connection explicitly to UTF8. Doing a
mysql_query("SET NAMES utf8;");
^
Put it right under your database connection script or include and MAKE sure you have it placed before you do any necessary queries. Also, for collocation please take the time to make sure your
setting it for your proper syntax type and general_ci seems working good for me when used. As a finale, clear your cache after banging your head, set your browser to proper encoding toolbar->view->encoding
Setting the connection to UTF8 after establishing the connection takes care of the problem. Don't do this if the first step already works.
UTF-8 content from MySQL table with PDO
To correctly get latin characters and so on from a MySQL table with PDO,
there is an hidden info coming from a "User Contributed Note" in the PHP manual website
(the crazy thing is that originally, that contribution was downvoted, now luckily turned to positive .. sometime some people need to got blamed)
my credits credits go to this article that pulled the solution and probably made that "User Contributed Note" to turn positive
If you want to have a clean database connection with correct Unicode characters
$this->dbh = new PDO(
"mysql:host=".DB_HOST.";dbname=".DB_NAME.";charset=utf8",
DB_USER,
DB_PASS);
try this :
mysql_set_charset('utf8', $yourConnection);
Old ways have been deprecated. If you are using PHP > 5.0.5 and using mysqli the new syntax is now:
$connection->set_charset("utf8")
Where $connection is a reference to your connection to the DB.
I tried several solutions but the only one that worked
is that of Hari Dass:
$conn->set_charset("utf8");

Categories