save special characters in database in core php - php

I want to save special characters like the following string in a database:-
:¦:-•:":•.-:¦:-•EXCELLENT!•-:¦:-•:•-:¦:-•:*''•
Below is the code that I am using.
$Fields ['CommentText']=$CommentText;
$Fields = prepareMySQLi($FieldsNotifications,$linkMysqli);
$insert = mysqli_query($linkMysqli,"INSERT INTO `feeds` SET $Fields");
function prepareMySQLi($MyArray,$linkMysqli) {
foreach($MyArray as $col => $val) {
if($val=='Invalid Request') $val='';
if ($val!='' && !is_array($val)) {
$col = mysqli_real_escape_string($linkMysqli,$col);
$val = mysqli_real_escape_string($linkMysqli,$val);
if(isset($fields)) {
$fields .= ", `$col` = '$val' ";
} else {
$fields = " `$col` = '$val' ";
}
}
}
return $fields;
}
But the above code saves the result like:-
•:¨¨:•.EXCELLENT.•:¨¨:••:¨¨:•.
Can anyone guide me how can I save the string same as it is in the database?

It seems like encoding issue. The encoding of the data that you are receiving should be same as the encoding of database where you are storing it.
General practice would be to use "utf-8" encoding for both.
So check in which encoding the database stores the data, and try to convert received data to that format or vice versa.
You can use utf8_encode function for encoding data to "utf-8".

please check by doing these two simple thing, i hope it helps you:-
the "collation" of the column in which you are going to save this data(special characters), made it "utf-8-bin".
either change type of the column to "blob","text" or "long text".
try in your code:- mysqli_set_charset($linkMysqli, "utf8");
encode and decode process. before saving the text encode it and if you want to show it somewhere then first decode it and then show.

simple thing is change
Collation in db to utf8 -> utf8_unicode_ci
and if you dont used field Type as text than change it to text...
May be this is simple solution.....
Other things may You can check your Mysqli Db class Where
'charset' => 'utf8',
is there or not....??
If you dont find charset than please change to utf8...
At last check your page header
and add
<meta charset="utf-8">
Thats it may u will get something...

Related

Emoji name "family_mothers_one_boy" or "woman-woman-boy"?

I have a reference emojis file used by my php code. Inside there is for example "woman-woman-boy", but the browser (chrome) replaces this name by "family_mothers_one_boy"...
Why are there two versions of emojis' names?
Is there en (some) error(s) in my file, or should I have to do something in my code to avoid the conversion?
NOTE:
The code related to this emoji is:
1F469;‍👩‍&#x1F466
Here are the two functions I'm using to manage the emojis:
1. When I display the emoji, I replace the tage :name: by the HTML rendering (using unicode)
function replaceEmojiNameByUnicode($inputText){
$emoji_unicode = getTabEmojiUnicode();
preg_match_all("/:([a-zA-Z0-9'_+-]+):/", $inputText, $emojis);
foreach ($emojis[1] as $emojiname) {
if (isset($emoji_unicode[$emojiname])) {
$inputText = str_replace(":".$emojiname.":", "&#x".$emoji_unicode[$emojiname].";", $inputText);
}
else {
$inputText = str_replace(":".$emojiname.":", "(:".$emojiname.":)", $inputText);
}
}
return $inputText;
}
2. When I want to propose the list of emoji I display an HTML SELECT in the page. Teh following function return the list of option to add inside:
/* Display the options in the HTML select */
function displayEmojisOptions(){
$emoji_unicode = getTabEmojiUnicode();
foreach ($emoji_unicode as $name => $unicode) {
echo '<option value="&#x'.$unicode.';">'.$name.' => &#x'.$unicode.';</option>';
}
}
In the array $emoji_unicode there is one entry (with 3 semi-column removed to not display emoji here):
'family_one_girl' => '1F468;&#x200D&#x1F469&#x200D&#x1F467',
For example: In order to make it works, I have to replace the line 'thinking_face' => '1F914', by 'thinking' => '1F914',
My question is: why ??
Thank you
Nop, the emoji text was changed by no code... I guess it was due to a wrong emoji file I used... I correct all the emoji manually and now I did not see the mismatch anymore...
If someone need the corrected file, I can provide it.

PHP json encode - Malformed UTF-8 characters, possibly incorrectly encoded [duplicate]

This question already has answers here:
UTF-8 all the way through
(13 answers)
Closed 7 months ago.
I'm using json_encode($data) to an data array and there's a field contains Russian characters.
I used this mb_detect_encoding() to display what encoding it is for that field and it displays UTF-8.
I think the json encode failed due to some bad characters in it like "ра▒". I tried alot of things utf8_encode on the data and it will by pass that error but then the data doesn't look correct anymore.
What can be done with this issue?
The issue happens if there are some non-utf8 characters inside even though most of them are utf8 chars. This will remove any non-utf8 characters and now it works.
$data['name'] = mb_convert_encoding($data['name'], 'UTF-8', 'UTF-8');
If you have a multidimensional array to encode in JSON format then you can use below function:
If JSON_ERROR_UTF8 occurred :
$encoded = json_encode( utf8ize( $responseForJS ) );
Below function is used to encode Array data recursively
/* Use it for json_encode some corrupt UTF-8 chars
* useful for = malformed utf-8 characters possibly incorrectly encoded by json_encode
*/
function utf8ize( $mixed ) {
if (is_array($mixed)) {
foreach ($mixed as $key => $value) {
$mixed[$key] = utf8ize($value);
}
} elseif (is_string($mixed)) {
return mb_convert_encoding($mixed, "UTF-8", "UTF-8");
}
return $mixed;
}
Please, make sure to initiate your Pdo object with the charset iso as utf8.
This should fix this problem avoiding any re-utf8izing dance.
$pdo = new PDO("mysql:host=localhost;dbname=mybase;charset=utf8", 'user', 'password');
With php 7.2, two options allow to manage invalid UTF-8 direcly in json_encode :
https://www.php.net/manual/en/function.json-encode
json_encode($text, JSON_INVALID_UTF8_IGNORE);
Or
json_encode($text, JSON_INVALID_UTF8_SUBSTITUTE);
you just add in your pdo connection charset=utf8
like below line of pdo connection:
$pdo = new PDO("mysql:host=localhost;dbname=mybase;charset=utf8", 'user', 'password');
hope this will help you
Remove HTML entities before JSON encoding. I used html_entity_decode() in PHP and the problem was solved
$json = html_entity_decode($source);
$data = json_decode($json,true);
Do you by any chance have UUIDs in your result set? In that case the following database flag will help:
PDO::DBLIB_ATTR_STRINGIFY_UNIQUEIDENTIFIER => true
If your data is well encoded in the database for example, make sure to use the mb_ * functions for string handling, before json_encode. Functions like substr or strlen do not work well with utf8mb4 and can cut your text and leave a malformed UTF8
I know this is kind of an old topic, but for me it was what I needed. I just needed to modify the answer 'jayashan perera'.
//...code
$stmt->execute();
$result = $stmt->fetchAll(PDO::FETCH_ASSOC);
for ($i=0; $i < sizeof($result) ; $i++) {
$tempCnpj = $result[$i]['CNPJ'];
$tempFornecedor = json_encode(html_entity_decode($result[$i]['Nome_fornecedor']),true) ;
$tempData = $result[$i]['efetivado_data'];
$tempNota = $result[$i]['valor_nota'];
$arrResposta[$i] = ["Status"=>"true", "Cnpj"=>"$tempCnpj", "Fornecedor"=>$tempFornecedor, "Data"=>"$tempData", "Nota"=>"$tempNota" ];
}
echo json_encode($arrResposta);
And no .js i have use
obj = JSON.parse(msg);

MySQL JSON Encoded string corrupts after INSERT..SELECT

I'm encoding array of Image URLS into json string and store them in database. (utf8_general_ci).
When I insert data into table and retrive it, json_decode() is capable of decoding it.
However, when I copy data from one table to another (INSERT INTO ... SELECT statement) data after retrieving from database cannot be decoded anymore.
Instead, i get corrupted json ENCoded string. Even empty array [] cannot be properly decoded.
It converts from http://pl.tinypic.com/r/fwoiol/8
into http://pl.tinypic.com/r/bgea05/8
(had to make images since those squares cannot be copied as text).
Edit, After checking a bit more i tried to bin2hex() both strings from database.
Both seem to be exactly same.
However, one decodes and one does not. The
5b22687474703a5c2f5c2f7777772e
changes into
0022687474703a5c2f5c2f7777772e
So, json_decode only changes 5b into 00 in string.
It's like It's losing encoding somewhere?
Edit 2
static public function jsonDecodeFieldsArray($entries, $fields = array('features','images')){
foreach($entries as $key => $entry){
$entries[$key] = self::jsonDecodeFields($entry, $fields);
}
return $entries;
}
static public function jsonDecodeFields($entry, $fields = array('features','images')){
foreach($fields as $field){
if(isset($entry[$field])){
$entry[$field] = json_decode((string) $entry[$field], true);
}
}
return $entry;
}
I'm using code above, to decode keys of array specified by $fields. However, it not only decodes wrongfully. But also affects keys that are not listed in $fields. Corrupting their encodings.
More to add. If I dont use those functions and use only json_decode on fields json_decode($array[0][images], true) it works fine.
To Clarify that I found answer/solution I write this Answer
The reason behoind this error was not SQL error and data was proper. I had an example array of:
$many_entries = array(
array(
'features' = > 'json_encoded_string'
'images' = > 'json_encoded_string'
),
array(
'features' = > 'json_encoded_string'
'images' = > 'json_encoded_string'
)
);
// And
$one_entry = array(
'features' = > 'json_encoded_string'
'images' = > 'json_encoded_string'
);
Now I had 2 functions. One to Parse $many_entries (jsonDecodeFieldsArray) array and one to Parse $one_entry array structure (jsonDecodeFields).
The problem was I used jsonDecodeFieldsArray on $one_entry which made jsonDecodeFields iterate on strings.
It is odd that the character encoding is changing through the transmission. I would say check your charset(s) in PHP but you said the issue is only occurring in a table => table SQL transfer. I would still check the charset of the column / table.
You can fix the issue by running a str_replace() upon decoding. For example:
$DB_ARRAY = $DB_QUERY->fetch_array();
$CORRECT_ENCODING = json_decode(str_replace('0x93', '[', $DB_ARRAY['urlstring']), true);
You would, of course, need to know what the wrongly encoded character is. Or its ASCII code equivalent.

XML character encoding issue with PHP

I have code which is creating an XML, my only problem is with the encoding of words like á, olá and ção.
These characters dont appear correctly and when I try reading the XML I get an error displayed relating to that character.
$dom_doc = new DOMDocument("1.0", "utf-8");
$dom_doc->preserveWhiteSpace = false;
$dom_doc->formatOutput = true;
$element = $dom->createElement("hotels");
while ($row = mysql_fetch_assoc($result)) {
$contact = $dom_doc->createElement( "m" . $row['id'] );
$nome = $dom_doc->createElement("nome", $row['nome'] );
$data1 = $dom_doc->createElement("data1", $row['data'] );
$data2 = $dom_doc->createElement("data2", $row['data2'] );
$contact->appendChild($nome);
$contact->appendChild($data1);
$contact->appendChild($data2);
$element->appendChild($contact);
$dom_doc->appendChild($element);
What can I change to fix my problem, I am using utf-8???
Please try to put directly 'á', 'olá' or 'ção' in your script.
$data1 = $dom_doc->createElement("data1", 'ção');
If you don't have problem, this is probably the data you get from mysql that are wrongly encoded.
Are you sure your mysql outputs correct UTF-8?
To know that, make your PHP dump your data in an HTML document with meta tag set to UTF-8 and see if the characters display correctly.
You can also call :
$data1 = $dom_doc->createElement("data1", mb_detect_encoding($row['data']));
and see what encoding is detected by PHP for your data.
If you can't convert the data from your database, or change its settings, you can use mb_convert to do it on-the-fly : http://www.php.net/manual/en/function.mb-convert-encoding.php
You are using utf-8, the 8-bit unicode encoding format. Even though it properly supports all 1,112,064 code points in Unicode its possible that there is an issue here.
Try UTF-16 as the standard, just an idea. See below:
$dom_doc = new DOMDocument("1.0", "utf-16");
OR
$dom_doc = new DOMDocument("1.0", "ISO-10646");

Encoding Conversion with PHP

Trying to do a Latin1 to UTF-8 conversion for WordPress, had no luck with the tutorial posted in the Codex. I came up with this to check encoding and convert.
while($row = mysql_fetch_assoc($sql)) {
if(!mb_check_encoding($row['post_content'], 'UTF-8')) {
$row = mb_convert_encoding($row['post_content'], 'ISO-8859-1', 'UTF-8');
if(!mb_check_encoding($row['post_content'], 'UTF-8')) {
echo 'Can\'t Be Converted<br/>';
}
else {
echo '<br/>'.$row.'<br/><br/>';
}
}
else {
echo 'UTF-8<br/>';
}
}
This works... sorta. I'm not getting any rows that can't converted but I did notice that Panamá becomes Panam
Am I missing a step? Or am I doing this all wrong?
UPDATE
The data stored within the database is corrupt(á characters are stored). So its looking more like a find and replace job than a conversion. I haven't found any great solutions so far for doing this automagically.
This will help you. http://php.net/manual/en/book.iconv.php
Further more you can set your mysql connection to utf8 this way:
mysql_set_charset ('utf8',$this->getConnection());
$this->getConnection in my code returns the variable which was returned by
mysql_connect(MYSQL_SERVER,DB_LOGIN,DB_PASS);
Refer to the PHP documentation for mb_convert_encoding:
string mb_convert_encoding ( string $str , string $to_encoding [, mixed $from_encoding ] )
Your code is attempting to convert to ISO-8859-1 from UTF-8!

Categories