I got a database from a customer and need to create some logic around it, like reading and inserting entries.
When I tried some basic requests, like:
$json = array ();
$query = "select * from tab1";
if ($result = $link->query ( $query )) {
while ( $row = $result->fetch_assoc () ) {
array_push ( $json, $row );
}
}
die ( json_encode ( $json ) );
I surprisingly got an empty response.
Executing the same query directly from PHPMyAdmin, I got all expected results.
When I just dumped the JSON result in the browser, I noticed (extract):
... string(30) "ColumnName(�)" ...
It took me a while to find out, that there were columns name with some special charaters, like µ and °. Apparently, they could not be displayed properly so the whole response became invalid and I got no results.
Just removing those characters from the column name solved the problem.
Is there another solution, instead of just manually looking for those characters and removing them?
Because json_encode()/json_decode() deals only with UTF-8 encoding, you must convert all non utf8 encoded string to utf-8
You can use utf8-encode() function
Related
I'm encoding array of Image URLS into json string and store them in database. (utf8_general_ci).
When I insert data into table and retrive it, json_decode() is capable of decoding it.
However, when I copy data from one table to another (INSERT INTO ... SELECT statement) data after retrieving from database cannot be decoded anymore.
Instead, i get corrupted json ENCoded string. Even empty array [] cannot be properly decoded.
It converts from http://pl.tinypic.com/r/fwoiol/8
into http://pl.tinypic.com/r/bgea05/8
(had to make images since those squares cannot be copied as text).
Edit, After checking a bit more i tried to bin2hex() both strings from database.
Both seem to be exactly same.
However, one decodes and one does not. The
5b22687474703a5c2f5c2f7777772e
changes into
0022687474703a5c2f5c2f7777772e
So, json_decode only changes 5b into 00 in string.
It's like It's losing encoding somewhere?
Edit 2
static public function jsonDecodeFieldsArray($entries, $fields = array('features','images')){
foreach($entries as $key => $entry){
$entries[$key] = self::jsonDecodeFields($entry, $fields);
}
return $entries;
}
static public function jsonDecodeFields($entry, $fields = array('features','images')){
foreach($fields as $field){
if(isset($entry[$field])){
$entry[$field] = json_decode((string) $entry[$field], true);
}
}
return $entry;
}
I'm using code above, to decode keys of array specified by $fields. However, it not only decodes wrongfully. But also affects keys that are not listed in $fields. Corrupting their encodings.
More to add. If I dont use those functions and use only json_decode on fields json_decode($array[0][images], true) it works fine.
To Clarify that I found answer/solution I write this Answer
The reason behoind this error was not SQL error and data was proper. I had an example array of:
$many_entries = array(
array(
'features' = > 'json_encoded_string'
'images' = > 'json_encoded_string'
),
array(
'features' = > 'json_encoded_string'
'images' = > 'json_encoded_string'
)
);
// And
$one_entry = array(
'features' = > 'json_encoded_string'
'images' = > 'json_encoded_string'
);
Now I had 2 functions. One to Parse $many_entries (jsonDecodeFieldsArray) array and one to Parse $one_entry array structure (jsonDecodeFields).
The problem was I used jsonDecodeFieldsArray on $one_entry which made jsonDecodeFields iterate on strings.
It is odd that the character encoding is changing through the transmission. I would say check your charset(s) in PHP but you said the issue is only occurring in a table => table SQL transfer. I would still check the charset of the column / table.
You can fix the issue by running a str_replace() upon decoding. For example:
$DB_ARRAY = $DB_QUERY->fetch_array();
$CORRECT_ENCODING = json_decode(str_replace('0x93', '[', $DB_ARRAY['urlstring']), true);
You would, of course, need to know what the wrongly encoded character is. Or its ASCII code equivalent.
i encode the user entred testarea content with json_encode
here is how it storred in mysql (there is a return to line after "aaa")
{"content":"aaa
bbb"}
now when i get the the content from database and trying to decoded it using json_decode, i get NULL for this, instead of what expected.
what wrrong? a bug in PHP?
EDIT 1: more details
$data =array('content'=>$textareaText);
$addres_insert = "INSERT INTO `rtable` (`data`) VALUES ('".json_encode($data)."');";
$rows = mysql_query($addres_insert);
then to get the content
$query = "SELECT * FROM `rtable` WHERE id = '".$id."'";
$rows = mysql_query($query);
if ( $rows ){
$row = mysql_fetch_array($rows);
$res['data'] = json_decode($row['data']);//i tryed substr($row['data'],3) but didn't work
}
JavaScript and JSON do not allow line returns to be contained within a string. You need to escape them.
json_encode() should escape them automatically for you.
Here is the output of my playing with your JSON code supplied on the PHP interactive shell:
php > $json = '{"content":"aaa
php ' bbb"}';
php > var_dump(json_decode($json, true));
NULL
As you can see when I escape your line return it works just fine:
php > $json = '{"content":"aaa\n bbb"}';
php > var_dump(json_decode($json, true));
array(1) {
["content"]=>
string(8) "aaa
bbb"
}
This is also further discussed in a previous question relating to a similar problem: Problem when retrieving text in JSON format containing line breaks with jQuery
I am using curl to get a json file which can be located here: (It's way too long to copy paste it): http://www.opap.gr/web/services/rs/betting/availableBetGames/sport/program/4100/0/sport-1.json?localeId=el_GR
After that i use json_decode to get the assosiative array.Till here everything seems ok.When i am using var_dump the characters inside the array are in Greek.After that i am using the following code:
$JsonClass = new ArrayToXML();
$mydata=$JsonClass->toXml($json);
class ArrayToXML
{
public static function toXML( $data, $rootNodeName = 'ResultSet', &$xml=null ) {
// turn off compatibility mode as simple xml throws a wobbly if you don't.
// if ( ini_get('zend.ze1_compatibility_mode') == 1 ) ini_set ( 'zend.ze1_compatibility_mode', 0 );
if ( is_null( $xml ) ) //$xml = simplexml_load_string( "" );
$xml = simplexml_load_string("<?xml version='1.0' encoding='UTF-8'?><$rootNodeName />");
// loop through the data passed in.
foreach( $data as $key => $value ) {
$numeric = false;
// no numeric keys in our xml please!
if ( is_numeric( $key ) ) {
$numeric = 1;
$key = $rootNodeName;
}
// delete any char not allowed in XML element names
`enter code here`$key = preg_replace('/[^a-z0-9\-\_\.\:]/i', '', $key);
// if there is another array found recrusively call this function
if ( is_array( $value ) ) {
$node = ArrayToXML::isAssoc( $value ) || $numeric ? $xml->addChild( $key ) : $xml;
// recrusive call.
if ( $numeric ) $key = 'anon';
ArrayToXML::toXml( $value, $key, $node );
} else {
// add single node.
$value = htmlentities( $value );
$xml->addChild( $key, $value );
}
}
// pass back as XML
return $xml->asXML();
}
public static function isAssoc( $array ) {
return (is_array($array) && 0 !== count(array_diff_key($array, array_keys(array_keys($array)))));
}
}
And here comes the problem .All the greek characters inside the result are in some strange characters Î?Î?Î¥Î?Î?ΡΩΣÎ?Î? for example.I really don't know what am i doing wrong.I am really bad with encoding /decoding things :(.
And to make this a bit more clear:
Here is how the assosiative array (on of the parts that i have the problem with) looks like:
{ ["resources"]=> array(4) { ["team-4833"]=> string(24) "ΛΕΥΚΟΡΩΣΙΑ U21" ["t-429"]=> string(72) "ΠΡΟΚΡΙΜΑΤΙΚΑ ΕΥΡΩΠΑΪΚΟΥ ΠΡΩΤΑΘΛΗΜΑΤΟΣ" ["t-429-short"]=> string(6) "ΠΕΠ" ["team-15387"]=> string(16) "ΕΛΛΑΔΑ U21" } ["locale"]=> string(5) "el_GR" } ["relatedNum"]=> NULL }
And here is what i get after the use of simplexml
<resources><team-4833>Î?Î?Î¥Î?Î?ΡΩΣÎ?Î? U21</team-4833><t-429>ΠΡÎ?Î?ΡÎ?Î?Î?ΤÎ?Î?Î? Î?ΥΡΩΠÎ?ΪÎ?Î?Î¥ ΠΡΩΤÎ?Î?Î?Î?Î?Î?ΤÎ?Σ</t-429><t-429-short>Î Î?Î </t-429-short><team-15387>Î?Î?Î?Î?Î?Î? U21</team-15387></resources><locale>el_GR</locale></lexicon><relatedNum></relatedNum></betGames>
Thanks in advance for your replies.
PS:I have also <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> in the page i display the result but it doesnt help.
I still didn't find a solution with that so i used a different approach something like Yannis suggested.I saved the XML in a file using the class i found here http://www.phpclasses.org/package/1826-PHP-Store-associative-array-data-on-file-in-XML.html .
After that i load the xml with simplexml_load_file and i used xslt to access the data in all nodes and store it in my database.It worked fine that way .If anyone still wants to try and explain me why it doesn't work with the way i tried to do it at the start feel free (Just for the learning purpose :p)Thanks for your replies :).
There is no need - The current json is given in an xml format as well here apparently:
http://www.opap.gr/web/services/rs/betting/availableBetGames/sport/program/4100/0/sport-1.xml?localeId=el_GR
Just had to play with the url parameters a bit :)
This worked for me on chrome using php version 5.3.6:
$json = file_get_contents('http://www.opap.gr/web/services/rs/betting/availableBetGames/sport/program/4100/0/sport-1.json?localeId=el_GR');
$json = json_decode($json, true);
$xml = new SimpleXMLElement('<ResultSet/>');
array_walk_recursive($json, array ($xml, 'addChild'));
print $xml->asXML();
exit();
Clearly your bug is that you are manipulating UTF‑8–encoded Unicode as though those bytes were ISO‐8859‑1.
I cannot see where this is happening; probably in your call to htmlentities, whatever that is.
It may need to use some sort of “multibyte” hack, perhaps including such things as this sort of pattern:
/([^\x00-\x7F])/u
wiht an explicit /u so it works on logical code points instead of 8‑bit code units (read: bytes). It might do this to grab one non-ASCII code point so it can replace it with a numeric entity. Without the easily forgotten /u, it would work on bytes not code points, which matches what your description shows happening.
It could be this sort of thing, or it might be that you have to swap over to some of the mb_*() functions instead of normal ones. This is to work around the fundamental underlying PHP bug that there it no real Unicode support in the language, just a few band-aides here and there that seem to like to fall off from time to time for no good reason.
If you could use a clean language with not just proper Unicode support but also a clear separation between physical bytes and abstract characters, this sort of thing would not be happening. But I bet it’s a common problem that others must be having too, so I would be really surprised if it were a library bug instead of a (perfectly understandable!) oversight somewhere in your code.
answer in your question from GREECE---------
word "? [ΛΕΥΚΟ]"? it has ASC (his code character) 203-197-213-202-207 ()----------
when however you read him [prostithete] the 206 and are doubled the letters----------
but also change code as following 206-(203-48=155)-206-(197-48=149)-206-(213-48=165)-
-206-(213-48=165)-206-(202-48=154)-206-(207-48=159)-------------
consequently the solution they is checking to a character if you find the 206 to >ignore---------
him and in the ASC of next character to add number 48 and to find the new character. >------------
Because I deal also i with the [ΑΠΟΚΟΔΙΚΟΠΟΙΗΣΗ] of [ΟΠΑΠ] every new knowledge they is >[ΕΥΠΡΟΣΔΕΚΤΟ]------
in mail -->? bluegt03#in.gr
I am retrieving json from a php file that connects to the database and then creates the json to be imported into my page via an ajax call.
Some of the database columns have file paths in them, ie images/myfolder/myfile
The json that I see when I open the page in a web browser formats the file path like this:
\/images\/myfolder\/myfile
Are the escaped characters going to break the json?
i.e, if i do var icon_image = myData.file
will icon_image hold: \/images\/myfolder\/myfile or /images/myfolder/myfile
I am hoping the second option from above, but if it doesn't how do I get it to display as /images/myfolder/myfile
I have put mb_internal_encoding( 'UTF-8' ); at the top of the php page
The script that generates the json is as follows:
mb_internal_encoding( 'UTF-8' );
mysql_select_db($database_growth_conn, $growth_conn);
$query_rs_icons = sprintf("SELECT * FROM icons_ico ORDER BY name_ico");
//echo($query_rs_icons);
$rs_icons = mysql_query($query_rs_icons, $growth_conn) or die(mysql_error());
$row_rs_icons = mysql_fetch_assoc($rs_icons);
$totalRows_rs_icons = mysql_num_rows($rs_icons)
$rows = array();
while($r = mysql_fetch_assoc($rs_icons)) {
$rows[] = $r;
}
$jsondata = json_encode($rows);
echo '{"icons":'.$jsondata.'}';
are the escaped characters going to break the json?
No. A backslash-escaped non-special punctuation is just the same as the punctuation itself. The JS string literals "a/b" and "a\/b" result in the same string value, a/b.
json_encode escapes the forward slash character for you so that if you try to include a string with the sequence </script> in it in a script element, it doesn't prematurely end the script block.
I have this code converting a mysql query to json:
$sth = mysql_query('SELECT * FROM `staff` ORDER BY `id` DESC LIMIT 20') or die(mysql_error());
$rows = array();
while($r = mysql_fetch_array($sth)) {
$rows[] = $r;
}
print json_encode($rows);
Works great, but i have in my database, a url field, here is a sample json row:
{"0":"4","id":"4","1":"Noter 2","name":"Noter 2","2":null,"desc":null,"3":"http:\/\/images.apple.com\/webapps\/productivity\/images\/noter2_20091223182720-thumb.jpg","iconurl":"http:\/\/images.apple.com\/webapps\/productivity\/images\/noter2_20091223182720-thumb.jpg","4":"1262032317","date":"1262032317","5":"dBelement, LLC","company":"dBelement, LLC","6":"http:\/\/dbelement.com\/","companyurl":"http:\/\/dbelement.com\/","7":"http:\/\/noter2.dbelement.com","appurl":"http:\/\/noter2.dbelement.com"},
How can i get around this?
I suppose the "problem" is the presence of backslashed in the URLs ?
Like :
http:\/\/images.apple.com\/webapps\/productivity\/images\/noter2_20091223182720-thumb.jpg
Well, that's not a problem for Javascript : try to assign your JSON data to a Javascript object, and display one of those URL :
var data = {"0":"4","id":"4","1":"Noter 2","name":"Noter 2","2":null,"desc":null,"3":"http:\/\/images.apple.com\/webapps\/productivity\/images\/noter2_20091223182720-thumb.jpg","iconurl":"http:\/\/images.apple.com\/webapps\/productivity\/images\/noter2_20091223182720-thumb.jpg","4":"1262032317","date":"1262032317","5":"dBelement, LLC","company":"dBelement, LLC","6":"http:\/\/dbelement.com\/","companyurl":"http:\/\/dbelement.com\/","7":"http:\/\/noter2.dbelement.com","appurl":"http:\/\/noter2.dbelement.com"};
alert(data[3]);
The URL is displayed correctly :
http://images.apple.com/webapps/productivity/images/noter2_20091223182720-thumb.jpg
So, the backslashes are not a "problem" ;-)
And, in fact, if you take a look at the "string" section of json.org, you'll see that slashes have to be backslashed for your string to be valid.
I don't see a problem here. If you're wondering why the URLs look like this:
http:\/\/www.example.com\/
Instead of like this:
http://www.example.com/
It's because the / characters are being appropriately escaped. When you unserialize the JSON object in javascript, the URLs will look as they do in your database.