he i have a problem
i have a mysql database dump, but all the data is url encoded has + instead of spaces and other characters as %20 etc
i tried to import this directly and wrote a php script to decode the data and rewrite it back to the db,
/// make rs..........
........
do {
// decode the items in the countries table
$myCountryID = $row_countries['id_con'];
$new_country_con = urldecode($row_countries['country_con']);
$result = mysql_query("UPDATE countries_con SET country_con = '".$new_country_con."'
WHERE id_con = ".$myCountryID."")or die(mysql_error()) ;
echo("id_win: ".$myCountryID."<br>added country_con: ".$new_country_con."<br>");
} while ($row_countries = mysql_fetch_assoc($countries));
echo('----------------------------------END Countries-------------------------------------<br><br>');
the page is set to use utf8 all the tables in the db are set to use utf8
the echo statement outputs the correct characters, but it still gets stored in the db as a urlencoded string, how do i get it into the db as utf8 without the url encoded characters?????
Try to printout your query to see at which point it produces %20 (before query execution or after).
Also please see: http://recursive-design.com/blog/2008/06/23/force-mysql-encoding-to-utf8/
Related
I have one php form where i used to enter data to database(phpmyadmin), and i used SELECT query to display all values in database to view in php form.
Also i have another PHP file which i used to create JSON from the same db table.
Here when i enter foreign languages like "Experiența personală:" the value getting saved in DB is "ExperienÈ›a personală: " but when i use select query to display this in same php form it coming correctly "Experiența personală:". So the db is correct and now am using following php code to create JSON
<?php
$servername = "localhost";
$username = "root";
$password = "root";
$dbname = "aaps";
// Create connection
$con=mysqli_connect($servername,$username,$password,$dbname);
// Check connection
mysqli_set_charset($con, 'utf8');
//echo "connected";
$rslt=mysqli_query($con,"SELECT * FROM offers");
while($row=mysqli_fetch_assoc($rslt))
{
$taxi[] = array('code'=> $row["code"], 'name'=> $row["name"],'contact'=> $row["contact"], 'url'=> $row["url"], 'details'=> $row["details"]);
}
header("Content-type: application/json; charset=utf-8");
echo json_encode($taxi);
?>
and JSON looks like
[{"code":"CT1","name":"Experien\u00c8\u203aa personal\u00c4\u0192: ","contact":"4535623643","url":"images\/offers\/event-logo-8.jpg","details":"Experien\u00c8\u203aa personal\u00c4\u0192: jerhbehwgrh 234234 hjfhjerg#$%$#%#4"},{"code":"ewrw","name":"Experien\u00c8\u203aa personal\u00c4\u0192: ","contact":"ewfew","url":"","details":"eExperien\u00c8\u203aa personal\u00c4\u0192: Experien\u00c8\u203aa personal\u00c4\u0192: Experien\u00c8\u203aa personal\u00c4\u0192: "},{"code":"Experien\u00c8\u203aa personal\u00c4\u0192: ","name":"Experien\u00c8\u203aa personal\u00c4\u0192: ","contact":"","url":"","details":"Experien\u00c8\u203aa personal\u00c4\u0192: "}]
In this "\u00c8\u203aa" this is wrong it supposed to be "\u021b" (t).
So pho used to creating JSON making this issue.
But am unable to find exactly why its coming like this . please help
Avoid Unicode -- note the extra argument:
json_encode($s, JSON_UNESCAPED_UNICODE)
Don't use utf8_encode/decode.
ă turning into ă is Mojibake. It probably means that
The bytes you have in the client are correctly encoded in utf8 (good).
You connected with SET NAMES latin1 (or set_charset('latin1') or ...), probably by default. (It should have been utf8.)
The column in the tables may or may not have been CHARACTER SET utf8, but it should have been that.
If you need to fix for the data it takes a "2-step ALTER", something like
ALTER TABLE Tbl MODIFY COLUMN col VARBINARY(...) ...;
ALTER TABLE Tbl MODIFY COLUMN col VARCHAR(...) ... CHARACTER SET utf8 ...;
Before making any changes, do
SELECT col, HEX(col) FROM tbl WHERE ...
With that, ă should show hex of C483. If you see C384C692, you have "double-encoding", which is messier to fix.
Depending on the version of MySql in the database, it may not be using the full utf-8 set, as stated in the documentation:
The ucs2 and utf8 character sets do not support supplementary characters that lie outside the BMP. Characters outside the BMP compare as REPLACEMENT CHARACTER and convert to '?' when converted to a Unicode character set.
This, however, is not likely to be related to your problem. I would try a couple of different things and see if it solves your problem.
use SET NAMES utf-8
You can read more about that here
use utf8_encode() when inserting data to the database, and utf8_decode() when extracting. That way, you don't have to worry about MySql manipulating the unicode characters. Documentation
I am trying to use special danish characters (æøå) in the URL as GET parameters. So for instance I have this URL:
http://example.com?name=åge
that URL will get converted into the ASCII equivalent automatically, so in the URL it will read:
http://example.com?name=%E5ge
When i access and print out that value it works just fine, and displays that parameter as:
åge
However, i am using that to select stuff in my DB, and this won't work. If i use the ASCII version in the URL, it won't select anything form the DB and just give me an empty result. If i force the URL to not use ASCII, so it is: http://example.com?name=åge it will work fine when selecting from the DB, but when I display the parameter it shows as this:
Ã¥
I have no idea how to get around this. Any help is appreciated.
I would transform the parameter to UTF-8 and then prepare the database for a UTF-8 query, with something like this:
$name = utf8_encode($_GET['name']);
mysqli_query($mysqli_connector, "SET NAMES UTF8");
And then, prepare your query.
That should work.
I hope that helps!
I have a site developed in codeigniter where I'd like to insert a record inside my database in a table with utf8 fields.
The problem is that when I insert something inside the table I see that:
�"�h�t�t�p�:�/�/�I�m�a�g�e�1�.�u�r�l�f�o�r�i�m�a�g�e�s�.�c�o�m�/�D�e�f�a�u�l�t�/�8�8�0�4�/�2�3�6�9�2�2�6�2�-�1�8�4�3�3�8�5�2�6�6�.�j�p�g�"�
There are many more characters. The real string is a simple path. I don't know the format of the string because it is from an external server.
This is my query to insert record. I take the string from xml and if I print it inside the page I see the correct string. The problem occurs whenever I check inside the database:
foreach($img->childNodes as $node){
$data = array(
'image'=>$node->getAttribute('path'),
);
$this->db->insert('hotel_images',$data);
}
That data is not UTF-8. It is UCS-2 or UTF-16. UCS-2 is a subset of UTF-16, so treating it as UTF-16 should do the trick.
You can convert it using iconv.
$data = iconv("UTF-16", "UTF-8", $data);
I'm currently using this function to obfuscate a bit the field values in MySQL and protect it from direct dumping. It all works good and values are stored correctly, but what happens when i try to store a multibyte string?
Here's an example, let's try to encode the string álex:
<?
$v = xorencode('álex');
// step 1 - encode
echo $v."\n";
// step 2 - decode
echo xorencode($v);
?>
Works good, i see some obfuscated string first time, and then i see álex again. Now if i try to save it in a VARCHAR field in a MySQL table, and then select it - i no longer have a utf string, instead it gets returned as gllex.
Note, MySQL tables and fields collations are utf8_general_ci, files are UTF-8, and i SET NAMES utf8 after connecting. Any workaround to this?
Thanks
So I have a BINARY(255) field in MySQL, and I want to store a Base64 string in it and retrieve it as needed using PHP.
I'm storing the data using
'UPDATE table SET bindata=0x'.bin2hex(base64_decode($b64));
Which seems to be working fine, at any rate, there is a value going into the database. But I can't figure out how to get my original base64 string back. I've tried using base64_encode on the SELECT results, using HEX(bindata) and then running hex2bin and base64_encode. Neither seems to work.
If the base64 string, once converted into binary, has a bunch of zeros at the end, that's fine.
I would store the value this way:
$sql = "UPDATE table SET bindata=?";
$stmt = $pdo->prepare($sql);
Pass as your query parameter the result of base64_decode($b64). It's okay to pass a binary string this way.
$stmt->execute([base64_decode($b64)]);
When you fetch the binary string from the database, query it like this:
$sql = "SELECT bindata FROM table ...";
$stmt = $pdo->prepare($sql);
$stmt->execute();
It's okay to fetch a binary string. After you fetch that into a result row, you can re-encode it.
while ($row = $stmt->fetch(PDO::FETCH_ASSOC)) {
echo base64_encode($row['bindata']);
}