MySQL JSON Encoded string corrupts after INSERT..SELECT - php

I'm encoding array of Image URLS into json string and store them in database. (utf8_general_ci).
When I insert data into table and retrive it, json_decode() is capable of decoding it.
However, when I copy data from one table to another (INSERT INTO ... SELECT statement) data after retrieving from database cannot be decoded anymore.
Instead, i get corrupted json ENCoded string. Even empty array [] cannot be properly decoded.
It converts from http://pl.tinypic.com/r/fwoiol/8
into http://pl.tinypic.com/r/bgea05/8
(had to make images since those squares cannot be copied as text).
Edit, After checking a bit more i tried to bin2hex() both strings from database.
Both seem to be exactly same.
However, one decodes and one does not. The
5b22687474703a5c2f5c2f7777772e
changes into
0022687474703a5c2f5c2f7777772e
So, json_decode only changes 5b into 00 in string.
It's like It's losing encoding somewhere?
Edit 2
static public function jsonDecodeFieldsArray($entries, $fields = array('features','images')){
foreach($entries as $key => $entry){
$entries[$key] = self::jsonDecodeFields($entry, $fields);
}
return $entries;
}
static public function jsonDecodeFields($entry, $fields = array('features','images')){
foreach($fields as $field){
if(isset($entry[$field])){
$entry[$field] = json_decode((string) $entry[$field], true);
}
}
return $entry;
}
I'm using code above, to decode keys of array specified by $fields. However, it not only decodes wrongfully. But also affects keys that are not listed in $fields. Corrupting their encodings.
More to add. If I dont use those functions and use only json_decode on fields json_decode($array[0][images], true) it works fine.

To Clarify that I found answer/solution I write this Answer
The reason behoind this error was not SQL error and data was proper. I had an example array of:
$many_entries = array(
array(
'features' = > 'json_encoded_string'
'images' = > 'json_encoded_string'
),
array(
'features' = > 'json_encoded_string'
'images' = > 'json_encoded_string'
)
);
// And
$one_entry = array(
'features' = > 'json_encoded_string'
'images' = > 'json_encoded_string'
);
Now I had 2 functions. One to Parse $many_entries (jsonDecodeFieldsArray) array and one to Parse $one_entry array structure (jsonDecodeFields).
The problem was I used jsonDecodeFieldsArray on $one_entry which made jsonDecodeFields iterate on strings.

It is odd that the character encoding is changing through the transmission. I would say check your charset(s) in PHP but you said the issue is only occurring in a table => table SQL transfer. I would still check the charset of the column / table.
You can fix the issue by running a str_replace() upon decoding. For example:
$DB_ARRAY = $DB_QUERY->fetch_array();
$CORRECT_ENCODING = json_decode(str_replace('0x93', '[', $DB_ARRAY['urlstring']), true);
You would, of course, need to know what the wrongly encoded character is. Or its ASCII code equivalent.

Related

JSON decode multiple nested JSON in PHP

I am using a page builder called King composer for wordpress, where i am trying to build some custom functions, like is intended.
My problem is, that the build-in background color picker is base64 encoding the background properties, So i need to decode it - But first i need to decode the 'my-css' json, so that i can access the different properties.
this is the return of what i get from the builder.
array (
'_id' => '69391',
'image' => '294,9,16',
'gallery-text' => 'Dette er nærmest et galleri',
'my-css' => '{
`kc-css`:{
`any`:{
`typography`{`color|`:`#ffffff`},
`background`{`background|`:`longBase64StringHere`},
`box`:{`margin|`:`100px inherit inherit inherit`}
}
}
}',
)
So far i have tried:
$decodedBackground = base64_decode($atts['my-css']);
which returns as null
then i tried :
$decodedJson = json_decode($atts['my-css']);
which returns : null
Also tried some other stuff that went horriably wrong
I don't really understand it, I can access the other properties fine, since it is just a part of an array, but the CSS part, I cannot comprehend. I think I need to go deeper in - but I can't get it to work.
Been stuck for about 1.5 hours now, so any help or pointers would be appreciated
/------ EDIT -----/
So this is how i am trying to inspect the decoded json afterwards -
might be important.
$decodedJson = json_decode($atts['my-css'], true);
echo '<pre>' . var_export($decodedJson, true) . '</pre>';
This is maybe not the best way to do because the JSON in kc-css is not well formated, but this code works for your case:
// Refomating JSON
$atts['my-css'] = str_replace('`{', '`:{', $atts['my-css']);
$atts['my-css'] = str_replace('`', '"', $atts['my-css']);
$json = json_decode($atts['my-css'], true);

Special characters in column names

I got a database from a customer and need to create some logic around it, like reading and inserting entries.
When I tried some basic requests, like:
$json = array ();
$query = "select * from tab1";
if ($result = $link->query ( $query )) {
while ( $row = $result->fetch_assoc () ) {
array_push ( $json, $row );
}
}
die ( json_encode ( $json ) );
I surprisingly got an empty response.
Executing the same query directly from PHPMyAdmin, I got all expected results.
When I just dumped the JSON result in the browser, I noticed (extract):
... string(30) "ColumnName(�)" ...
It took me a while to find out, that there were columns name with some special charaters, like µ and °. Apparently, they could not be displayed properly so the whole response became invalid and I got no results.
Just removing those characters from the column name solved the problem.
Is there another solution, instead of just manually looking for those characters and removing them?
Because json_encode()/json_decode() deals only with UTF-8 encoding, you must convert all non utf8 encoded string to utf-8
You can use utf8-encode() function

Trouble Reading json Data from hasoffer API

Trying to read json Data and I can't get it to work correctly with the following code:
$apiurl = "https://api.hasoffers.com/Apiv3/json?NetworkId=REDACTED&Target=Affiliate_Report&Method=getStats&api_key=REDACTED&fields%5B%5D=Stat.conversions&fields%5B%5D=Stat.unique_clicks&fields%5B%5D=Stat.payout&filters%5BStat.date%5D%5Bconditional%5D=LESS_THAN&filters%5BStat.date%5D%5Bvalues%5D=2016-02-21&filters%5BStat.date%5D%5Bconditional%5D=GREATER_THAN&filters%5BStat.date%5D%5Bvalues%5D=2016-02-21";
$data = json_decode(file_get_contents($apiurl), true);
foreach($data['response']['data'] as $dataline) {
echo "Conversions: {$dataline['Stat']['conversions']} Payout: {$dataline['Stat']['payout']}";
}
The query is generating the following json return, I just can't figure out how to read the stats correctly (it's also looping through 7 lines in the foreach which also makes no sense to me):
{"request":{"Target":"Affiliate_Report","Format":"json","Service":"HasOffers","Version":"2","NetworkId":"REDACTED","Method":"getStats","api_key":"REDACTED","fields":["Stat.conversions","Stat.unique_clicks","Stat.payout"],"filters":{"Stat.date":{"conditional":"GREATER_THAN","values":"2016-02-21"}},"__gaTune":"GA1.2.1289716345.1455904273","__utma":"267117079.1377304869.1455903853.1455904273.1455904273.1","__utmc":"267117079","__utmz":"267117079.1455904273.1.1.utmcsr=developers.hasoffers.com|utmccn=(referral)|utmcmd=referral|utmcct=/","_biz_uid":"1742fd1f613440a4cfbb5a510d1d7def","_biz_nA":"1","_biz_pendingA":"[]","_hp2_id_1318563364":"5257773084071598.0276720083.0714677778","_ga":"GA1.2.1377304869.1455903853"},"response":{"status":1,"httpStatus":200,"data":{"page":1,"current":50,"count":1,"pageCount":1,"data":[{"Stat":{"conversions":"1000","unique_clicks":"1000","payout":"1000.000000"}}],"dbSource":"branddb"},"errors":[],"errorMessage":null}}
If your JSON is exactly that you have posted, it is unvalid JSON, as per the comments.
The invalid part is the REDACTED words without double-quote.
To bypass this error, you can try in this way:
$data = file_get_contents( $apiurl );
$data = preg_replace( '/:REDACTED(?=\W)/', ':"REDACTED"', $data );
$json = json_decode( $data, True );
This will works on example above, but please note that is a trick, and it can not work if some JSON field has a value like "sometext:REDACTED,sometext": it is improbable, but not impossible.
Actually thank you that site helped a lot realized there was a second array also labeled "data" under the first "data" array so I needed to change it to:
$data['response']['data']['data'] as $dataline

array keys encoded when using http_build_query()

Encoding an array to a URL using http_build_query() produces strange behaviour when an array key is also a html-char code.
For example:
return http_build_query([
'id' = > ['my', 'data', 'here'], // no problem
'class' = > ['my', 'data', 'here'], // no problem
'yen' = > ['my', 'data', 'here'], // ¥ html car is ¥
'parameter' = > ['my', 'data', 'here'], // ¶ html char is ¶
]);
and the encoded result is:
id[0]=my&id[1]=data&id[2]=here&class[0]=my&class[1]=data&class[2]=here¥[0]=my¥[1]=data¥[2]=here¶meter[0]=my¶meter[1]=data¶meter[2]=here
whats going on here, it cant be possible that i cannot use the word parameter as an array key.
If you view the source of HTML output, you will see
id%5B0%5D=my&id%5B1%5D=data&id%5B2%5D=here&class%5B0%5D=my&class%5B1%5D=data&class%5B2%5D=here&yen%5B0%5D=my&yen%5B1%5D=data&yen%5B2%5D=here&parameter%5B0%5D=my&parameter%5B1%5D=data&parameter%5B2%5D=here
Which is correct. While displaying only, the browser will interpret malformed entities like &yen as ¥. There is nothing to worry about on the server side.
HTML entities reference
Demo: IDEOne

PHP json_encode() making data null

I have this code here:
case 'resource_list':
if(file_exists('content.php')){
include('../ajax/content.php');
} else {
die('does not exist');
}
$html = render_content_page($array[1],$array[2]);
$slate = 'info_slate';
$reply_array = array(
'html' => $html,
'slate' => $slate
);
echo json_encode($reply_array);
break;
i have debugged every level right up until json_encode() is called. But the data i receive back in my ajax is nul for the html key. This code is essentially a copy and paste of another case the just calls a function other than render_content_page() but that works perfectly fine.
$reply_array var_exports to:
array (
'html' => '<ol>
<li unit="quiz" identifier=""><img src="img/header/notifications.png"/>Fran�ois Vase Volute Krater</li>
</ol>',
'slate' => 'info_slate',
)
My initial thought is that special character in Fran�ois Vase Volute Krater, as json_encode only works with UTF-8 encoded data.
Try UTF-8 encoding it before JSON encoding it like so:
json_encode(utf8_encode("Fran�ois Vase Volute Krater"));
Maybe problem is with encoding?
As manual states, json_encode() works only only with utf8 encoded data:
This function only works with UTF-8 encoded data.
http://php.net/json_encode
As documented, json_encode expects its input text in UTF-8. Most likely, your input (the ç) is not in UTF-8.
Use utf8_encode (if you're currently using ISO-8859-1) or mb_convert_encoding (otherwise) to convert input strings to UTF-8.

Categories