I have a problem where I thought it was easy to Google but it seems to be not that easy. Ok, here is my problem:
I have to read a CSV file which has a bad and different encoding. I'm not able to correct the CSV file before hand, so I have to handle it in my application. So the CSV file could have the following-encodings:
'Ü5' and 'Möbelmarkt' in the same file.
If I decode (utf8_decode) the right one is correct and the left (which was correct) is wrong. When I try to find out the encoding (mb_detect_encoding) i always get the answer that this is UTF-8.
I still tried the following solutions:
public function convert( $str ) {
return iconv( "Windows-1252", "UTF-8", $str );
}
and
private function getUmlauteArray() {
return array( 'ü'=>'ü', 'ä'=>'ä', 'ö'=>'ö', 'Ö'=>'Ö', 'ß'=>'ß', 'à '=>'à', 'á'=>'á', 'â'=>'â', 'ã'=>'ã', 'ù'=>'ù', 'ú'=>'ú', 'û'=>'û', 'Ù'=>'Ù', 'Ú'=>'Ú', 'Û'=>'Û', 'Ãœ'=>'Ü', 'ò'=>'ò', 'ó'=>'ó', 'ô'=>'ô', 'è'=>'è', 'é'=>'é', 'ê'=>'ê', 'ë'=>'ë', 'À'=>'À', 'Ã'=>'Á', 'Â'=>'Â', 'Ã'=>'Ã', 'Ä'=>'Ä', 'Ã…'=>'Å', 'Ç'=>'Ç', 'È'=>'È', 'É'=>'É', 'Ê'=>'Ê', 'Ë'=>'Ë', 'ÃŒ'=>'Ì', 'Ã'=>'Í', 'ÃŽ'=>'Î', 'Ã'=>'Ï', 'Ñ'=>'Ñ', 'Ã’'=>'Ò', 'Ó'=>'Ó', 'Ô'=>'Ô', 'Õ'=>'Õ', 'Ø'=>'Ø', 'Ã¥'=>'å', 'æ'=>'æ', 'ç'=>'ç', 'ì'=>'ì', 'Ã'=>'í', 'î'=>'î', 'ï'=>'ï', 'ð'=>'ð', 'ñ'=>'ñ', 'õ'=>'õ', 'ø'=>'ø', 'ý'=>'ý', 'ÿ'=>'ÿ', '€'=>'€' );
}
public function fixeUmlaute($string) {
$umlaute = $this->getUmlauteArray();
foreach ($umlaute as $key => $value){
$value = str_replace($key, $value, $string);
}
return $string;
}
and
function valid_utf8( $string ){
return !((bool)preg_match('~[\xF5\xF6\xF7\xF8\xF9\xFA\xFB\xFC\xFD\xFE\xFF\xC0\xC1]~ms',$string));
}
That are all solutions I found with a Google search to change the encoding...(perhaps this "collection" helps anybody else...) So, how can I really detect the wrong characters or where is it my mistake?
Can anybody give me a hint?
Greetz
V
There is a nice PHP class that can help you with that: https://github.com/neitanod/forceutf8
It will convert any charset to UTF8, and handle the detection for you. Hope it helps.
Related
I'm working on Laravel (v5.7) app that converts uploaded CSV (with contacts) into array that is then passed as argument when job class is being dispatched.
Here is the example of CSV file (format that is supported):
123456,Richard,Smith
654321,John,Doe
Uploaded (CSV) file is handled like this:
$file_path = $request->file_name->store('contacts');
$file = storage_path('app/' . $file_path);
$contactsIterator = $this->getContacts($file);
$contacts = iterator_to_array($contactsIterator); // Array of contacts from uploaded CSV file
protected function getContacts($file)
{
$f = fopen($file, 'r');
while ($line = fgets($f))
{
$row = explode(",", $line);
yield [
'phone' => !empty($row[0]) ? trim($row[0]) : '',
'firstname' => !empty($row[1]) ? trim($row[1]) : '',
'lastname' => !empty($row[2]) ? trim($row[2]) : '',
];
}
}
Finally, $contacts array is passed to a job that is dispatched:
ImportContacts::dispatch($contacts);
This job class looks like this:
public function __construct($contacts)
{
Log::info('ImportContacts#__construct START');
$this->contacts = $contacts;
Log::info('ImportContacts#__construct END');
}
public function handle()
{
Log::info('ImportContacts#handle');
}
... and everything worked fine (no errors) until I've tried with this CSV:
123456,Richardÿ,Smith
654321,John,Doe
Please notice ÿ. So, when I try with this CSV - I get this error exception:
/code_smsto/vendor/laravel/framework/src/Illuminate/Queue/Queue.php | 91 | Unable to JSON encode payload. Error code: 5
... and my log file looks like this:
error local 2019-11-11 17:17:18 /code_smsto/vendor/laravel/framework/src/Illuminate/Queue/Queue.php | 91 | Unable to JSON encode payload. Error code: 5
info local 2019-11-11 17:17:18 ImportContacts#__construct END
info local 2019-11-11 17:17:18 ImportContacts#__construct START
As you can see - handle method was never executed. If I remove ÿ - no errors and handle is executed.
I've tried to solve this, but without success:
Apply utf8_encode:
protected function getContacts($file, $listId)
{
$f = fopen($file, 'r');
while ($line = fgets($f))
{
$row = explode(",", $line);
yield [
'phone' => !empty($row[0]) ? utf8_encode($row[0]) : '',
'firstname' => !empty($row[1]) ? utf8_encode($row[1]) : '',
'lastname' => !empty($row[2]) ? utf8_encode($row[2]) : '',
];
}
}
... and it works (no errors, no matter if there's that ÿ), but then Greek and Cyrillic letters are turned into question marks. For example, this: Εθνικής will become ???????.
I also tried with mb_convert_encoding($row[1], 'utf-8') - and it doesn't turn Greek or Cyrillic letter into question marks, but this ÿ character will become ?.
Move "handling" (converting to array) of uploaded CSV file into #handle method of a Job class worked, but then I was not able to store the data from that array into DB (MongoDB). Please see the update below.
DEBUGGING:
This is what I get from dd($contacts);:
So, it has that "b" where ÿ is. And, after some "googling" I found that this "b" means "binary string", that is, a non unicode string, on which functions operate at the byte level (What does the b in front of string literals do?).
What I understand is this: When dispatching Job class, Laravel tries to "JSON encode" it (passed arguments/data) but it fails because there are binary data (non-unicode strings).
Anyway, I was not able to find a solution (to be able to handle such CSV file with ÿ).
I am using:
Laravel 5.7
PHP 7.1.31-1+ubuntu16.04.1+deb.sury.org+1 (cli) (built: Aug 7 2019 10:22:48) ( NTS )
Redis powered queues
UPDATE
When I move "handling" (converting to array) of uploaded CSV file into #handle method of a Job class - I don't get this error (Unable to JSON encode payload. Error code: 5), but when I try to store that problematic binary data with ÿ (b"Richardÿ") into MongoDB - it fails. The weird thing is that I don't get any error-exception message in log file, so I put all in try-catch like this:
try {
// Insert data into MongoDB
} catch (Exception $e) {
Log::info($e->getFile());
Log::info($e->getLine());
Log::info($e->getMessage());
}
... and this is the result:
Anyway, I believe that it failed because of b"Richardÿ", and I guess that the solution is in encoding string, but as I've mentioned - I was not able to find a solution that works:
utf8_encode works (no errors, no matter if there's that ÿ), but then Greek and Cyrillic letters are turned into question marks. For example, this: Εθνικής will become ???????
mb_convert_encoding($row[1], 'utf-8') - it doesn't turn Greek or Cyrillic letter into question marks, but this ÿ character will become ?.
iconv('windows-1252', 'UTF-8', $row[1]) - works (no errors, no matter if there's that ÿ), but when there are Greek or Cyrillic letters - it fails (I get this error exception: iconv(): Detected an illegal character in input string)
You have several ways to deal with it but I'd recommend the following two. In both cases, the idea is that you store a UTF-8 string.
A simpler approach, figure out what encoding it is out of the (your) predefined list and convert it to UTF8.
$encoding = mb_detect_encoding($content, 'UTF-8, ISO-8859-1, WINDOWS-1252, WINDOWS-1251', true);
if ($encoding != 'UTF-8') {
$string = iconv($encoding, 'UTF-8//IGNORE', $row[1]);
}
The second approach is to use a third party library outlined in this answer
I am using imap_mail_move() to move emails from one folder to another. This works pretty well, but not if it comes to special characters in the folder name. I am sure I need to encode the name, but all test where not succesful.
Anybody that has a nice idea? Thanks in advance.
class EmailReader {
[...]
function doMoveEmail($uid, $targetFolder) {
$targetFolder = imap_utf8_to_mutf7($targetFolder);
$return = imap_mail_move($this->conn, $uid, $targetFolder, CP_UID);
if (!$return) {
$this->printValue(imap_errors());
die("stop");
}
return $return;
}
[...]
}
Calling the function in the script
[...]
$uid = 1234;
$folderTarget1 = "INBOX.00_Korrespondenz";
$this->doMoveEmail($uid, $folderTarget1);
$folderTarget2 = "INBOX.01_Anmeldevorgang.011_Bestätigungslink";
$this->doMoveEmail($uid, $folderTarget2);
[...]
The execution of the first call (folderTarget1) is working pretty well.
The execution of the secound call (folderTarget2) is creating an error:
[TRYCREATE] Mailbox doesn't exist: INBOX.01_Anmeldevorgang.011_Bestätigungslink (0.001 + 0.000 secs).
Remark 1:
if I call imap_list(), the name of the folder is shown as
"INBOX.01_Anmeldevorgang.011_Besta&Awg-tigungslink" (=$val)
using:
$new = mb_convert_encoding($val,'UTF-8','UTF7-IMAP')
echo $new; // gives --> "INBOX.01_Anmeldevorgang.011_Bestätigungslink"
but:
$new2 = mb_convert_encoding($new,'UTF7-IMAP', 'UTF-8')
echo $new2; // gives --> "INBOX.01_Anmeldevorgang.011_Best&AOQ-tigungslink"
Remark 2
I checked each possible encoding, with the following script, but none of them matchs the value that is returned by imap_list().
// looking for "INBOX.01_Anmeldevorgang.011_Besta&Awg-tigungslink" given by imap_list().
$targetFolder = "INBOX.01_Anmeldevorgang.011_Bestätigungslink";
foreach(mb_list_encodings() as $chr){
echo mb_convert_encoding($targetFolder, $chr, 'UTF-8')." : ".$chr."<br>";
}
Your folder name, as on the server, Besta&Awg-tigungslink is not canonically encoded:
&Awg- decodes as the combining diaereses character. Using some convenient python to look it up:
import base64
import unicode data
x = base64.b64decode('Awg=').decode('utf-16be'); # equals added to satisfy base64 padding requirements
unicodedata.name(x)
# Returns 'COMBINING DIAERESIS'
This combines with the a in front of it to show ä.
Your encoder is returning the more common precomposed form:
x = base64.b64decode('AOQ=').decode('utf-16be')
unicodedata.name(x)
# Returns: 'LATIN SMALL LETTER A WITH DIAERESIS'
This is a representation of ä directly.
Normally, when you work with IMAP folders, you pass around the raw name, and only convert the folder name for display. As you can see, there is not necessarily a one-way mapping from glyphs to encodings in unicode.
It does surprise me that PHP does seem to be doing a canonicalization step when encoding; I would expect round tripping the same data to return the same thing.
I created a workaround, which helps me to work with UTF8-values and to translate it to the original (raw) IMAP folder name.
function getFolderList() {
$folders = imap_list($this->conn, "{".$this->server."}", "*");
if (is_array($folders)) {
// Remove Server details of each element of array
$folders = array_map(function($val) { return str_replace("{".$this->server."}","",$val); }, $folders);
// Sort array
asort($folders);
// Renumber the list
$folders = array_values($folders);
// add UTF-8 encoded value to array
// this is needed as the original value is so wiered, that it is not possible to encode it
// with a function on the fly. This additional utf-8 value is needed to map the utf-8 value
// to the original value. The original value is still needed to do some operations like e.g.:
// - imap_mail_move()
// - imap_reopen()
// ==> the trick is to use normalizer_normalize()
$return = array();
foreach ($folders as $key => $folder) {
$return[$key]['original'] = $folder;
$return[$key]['utf8'] = normalizer_normalize(mb_convert_encoding($folder,'UTF-8','UTF7-IMAP'));
}
return $return;
} else {
die("IMAP_Folder-List failed: " . imap_last_error() . "\n");
}
}
When I dump the property of my model and it has an accent it returns with this "b" prefix
dump($venda_item->produto->nomeproduto); // b"teste téste"
My database is setted to utf8 and utf8_general_ci collation
This causes me the following error Malformed UTF-8 characters, possibly incorrectly encoded when I'm returning the response in json, at this line
$json_response = Response::json($response, $this->getStatusCode(), $headers);
Update
I discovered that if I die and dump the record on the web route it shows normal teste téste
Route::get('/', function () {
dd(App\Vendasitem::where('codigovi', 112685)->first()->produto->nomeproduto);
}
otherwise if I do the same in the controller or request and other files that I tryied it keeps returning me with the "b" prefix
Update 2
If I save my record like this PROMO - VIRICAPS (GUARANá + POLIVIT) 60 CáPS - CAIXA 18 UND and dump($venda_item->produto->nomeproduto); it returns me the right result with the accents.
All my database, including the column is set up to utf8mb4 and utf8mb4_unicode_ci
If you already confirmed the encoding on your database. Have a look on config/database.php, on charset and collation proprieties.
'charset' => 'utf8',
'collation' => 'utf8_unicode_ci'
If the issue is on the "print" then you might use the utf8_encode function.
{{utf8_encode($yourVar)}}
I had this problem when importing data from a different database. Solved with this method, maybe it works for you to.
public function fixEncoding($str) {
return mb_check_encoding($str, 'UTF-8')
? html_entity_decode($str)
: utf8_encode(html_entity_decode($str));
}
So, it looks like your best bet is to write a accessor for the object using iconv:
public function getNomeprodutoAttribute($value) {
return iconv("utf-8", "utf-8//ignore", $value);
}
Unless you want to recode the whole database, which probably isn't an option.
The value passed in will the value out of the database (which is the unfortunate binary data). Then you are telling it the input is utf (which it is), and that the output is utf8 (which we want) but to ignore chars that aren't supported in utf-8. //ignore may work in some versions of the library, so you'd just have to do the following, and hope for the best:
public function getNomeprodutoAttribute($value) {
return iconv("utf-8", "utf-8", $value);
}
Use this function
public function utf8ize($value) {
if (is_array($value)) {
foreach ($value as $k => $v) {
$value[$k] = utf8ize($v);
}
} else if (is_string ($value)) {
return utf8_encode($value);
}
return $value;
}
I am getting Windows-1256 encoded text from the web and nee to convert it to utf-8.
I tried using mb_convert_encoding and iconv but they don't seem to work.
none of them seem to be capable of handling windows-1256.
How to do it?
Edit: More details about the errors.
When trying
mb_convert_encoding($text,"utf-8", "windows-1256");
I get
Message: mb_convert_encoding() [function.mb-convert-encoding]: Illegal character encoding specified
And when i try
iconv("windows-1256", "utf-8", $text);
I get no errors but it returns an empty string
Trying
echo iconv('WINDOWS-1256', 'UTF-8', 'testÍÊ');
...on http://writecodeonline.com/php/ seems to work correctly (produces testأچأٹ)
Try this, should work:
iconv("windows-1256", "utf-8//TRANSLIT//IGNORE", $text)
Check this:
http://rayed.com/wordpress/wp-content/upload/lib.utf2win.php.txt
Apparently he also had some problems, because he wrote this script, if you can reverse that, it might work.
I reversed it for you, try that:
$f[]="\xc2\xac"; $t[]="\x80";
$f[]="\xd9\xbe"; $t[]="\x81";
$f[]="\xc0\x9a"; $t[]="\x82";
$f[]="\xc6\x92"; $t[]="\x83";
$f[]="\xc0\x9e"; $t[]="\x84";
$f[]="\xc0\xa6"; $t[]="\x85";
$f[]="\xc0\xa0"; $t[]="\x86";
$f[]="\xc0\xa1"; $t[]="\x87";
$f[]="\xcb\x86"; $t[]="\x88";
$f[]="\xc0\xb0"; $t[]="\x89";
$f[]="\xd9\xb9"; $t[]="\x8a";
$f[]="\xc0\xb9"; $t[]="\x8b";
$f[]="\xc5\x92"; $t[]="\x8c";
$f[]="\xda\x86"; $t[]="\x8d";
$f[]="\xda\x98"; $t[]="\x8e";
$f[]="\xda\x88"; $t[]="\x8f";
$f[]="\xda\xaf"; $t[]="\x90";
$f[]="\xc0\x98"; $t[]="\x91";
$f[]="\xc0\x99"; $t[]="\x92";
$f[]="\xc0\x9c"; $t[]="\x93";
$f[]="\xc0\x9d"; $t[]="\x94";
$f[]="\xc0\xa2"; $t[]="\x95";
$f[]="\xc0\x93"; $t[]="\x96";
$f[]="\xc0\x94"; $t[]="\x97";
$f[]="\xda\xa9"; $t[]="\x98";
$f[]="\xc4\xa2"; $t[]="\x99";
$f[]="\xda\x91"; $t[]="\x9a";
$f[]="\xc0\xba"; $t[]="\x9b";
$f[]="\xc5\x93"; $t[]="\x9c";
$f[]="\xc0\x8c"; $t[]="\x9d";
$f[]="\xc0\x8d"; $t[]="\x9e";
$f[]="\xda\xba"; $t[]="\x9f";
$f[]="\xd8\x8c"; $t[]="\xa1";
$f[]="\xda\xbe"; $t[]="\xaa";
$f[]="\xd8\x9b"; $t[]="\xba";
$f[]="\xd8\x9f"; $t[]="\xbf";
$f[]="\xdb\x81"; $t[]="\xc0";
$f[]="\xd8\xa1"; $t[]="\xc1";
$f[]="\xd8\xa2"; $t[]="\xc2";
$f[]="\xd8\xa3"; $t[]="\xc3";
$f[]="\xd8\xa4"; $t[]="\xc4";
$f[]="\xd8\xa5"; $t[]="\xc5";
$f[]="\xd8\xa6"; $t[]="\xc6";
$f[]="\xd8\xa7"; $t[]="\xc7";
$f[]="\xd8\xa8"; $t[]="\xc8";
$f[]="\xd8\xa9"; $t[]="\xc9";
$f[]="\xd8\xaa"; $t[]="\xca";
$f[]="\xd8\xab"; $t[]="\xcb";
$f[]="\xd8\xac"; $t[]="\xcc";
$f[]="\xd8\xad"; $t[]="\xcd";
$f[]="\xd8\xae"; $t[]="\xce";
$f[]="\xd8\xaf"; $t[]="\xcf";
$f[]="\xd8\xb0"; $t[]="\xd0";
$f[]="\xd8\xb1"; $t[]="\xd1";
$f[]="\xd8\xb2"; $t[]="\xd2";
$f[]="\xd8\xb3"; $t[]="\xd3";
$f[]="\xd8\xb4"; $t[]="\xd4";
$f[]="\xd8\xb5"; $t[]="\xd5";
$f[]="\xd8\xb6"; $t[]="\xd6";
$f[]="\xd8\xb7"; $t[]="\xd8";
$f[]="\xd8\xb8"; $t[]="\xd9";
$f[]="\xd8\xb9"; $t[]="\xda";
$f[]="\xd8\xba"; $t[]="\xdb";
$f[]="\xd9\x80"; $t[]="\xdc";
$f[]="\xd9\x81"; $t[]="\xdd";
$f[]="\xd9\x82"; $t[]="\xde";
$f[]="\xd9\x83"; $t[]="\xdf";
$f[]="\xd9\x84"; $t[]="\xe1";
$f[]="\xd9\x85"; $t[]="\xe3";
$f[]="\xd9\x86"; $t[]="\xe4";
$f[]="\xd9\x87"; $t[]="\xe5";
$f[]="\xd9\x88"; $t[]="\xe6";
$f[]="\xd9\x89"; $t[]="\xec";
$f[]="\xd9\x8a"; $t[]="\xed";
$f[]="\xd9\x8b"; $t[]="\xf0";
$f[]="\xd9\x8c"; $t[]="\xf1";
$f[]="\xd9\x8d"; $t[]="\xf2";
$f[]="\xd9\x8e"; $t[]="\xf3";
$f[]="\xd9\x8f"; $t[]="\xf5";
$f[]="\xd9\x90"; $t[]="\xf6";
$f[]="\xd9\x91"; $t[]="\xf8";
$f[]="\xd9\x92"; $t[]="\xfa";
$f[]="\xc0\x8e"; $t[]="\xfd";
$f[]="\xc0\x8f"; $t[]="\xfe";
$f[]="\xdb\x92"; $t[]="\xff";
function win_to_utf8($str) {
global $f, $t;
return str_replace($t, $f, $str);
}
This question already has answers here:
PHP json_decode() returns NULL with seemingly valid JSON?
(29 answers)
Closed 4 months ago.
There is a strange behaviour with json_encode and json_decode and I can't find a solution:
My php application calls a php web service. The webservice returns json that looks like this:
var_dump($foo):
string(62) "{"action":"set","user":"123123123123","status":"OK"}"
now I like to decode the json in my application:
$data = json_decode($foo, true)
but it returns NULL:
var_dump($data):
NULL
I use php5.
The Content-Type of the response from the webservice: "text/html; charset=utf-8" (also tried to use "application/json; charset=utf-8")
What could be the reason?
Well, i had a similar issue and the problems was the PHP magic quotes in the server... here is my solution:
if(get_magic_quotes_gpc()){
$param = stripslashes($_POST['param']);
}else{
$param = $_POST['param'];
}
$param = json_decode($param,true);
EDIT:
Just did some quick inspection of the string provided by the OP. The small "character" in front of the curly brace is a UTF-8 B(yte) O(rder) M(ark) 0xEF 0xBB 0xBF. I don't know why this byte sequence is displayed as here.
Essentially the system you aquire the data from sends it encoded in UTF-8 with a BOM preceding the data. You should remove the first three bytes from the string before you throw it into json_decode() (a substr($string, 3) will do).
string(62) "{"action":"set","user":"123123123123","status":"OK"}"
^
|
This is the UTF-8 BOM
As Kuroki Kaze discovered, this character surely is the reason why json_decode fails. The string in its given form is not correctly a JSON formated structure (see RFC 4627)
Print the last json error when debugging.
json_decode( $so, true, 9 );
$json_errors = array(
JSON_ERROR_NONE => 'No error has occurred',
JSON_ERROR_DEPTH => 'The maximum stack depth has been exceeded',
JSON_ERROR_CTRL_CHAR => 'Control character error, possibly incorrectly encoded',
JSON_ERROR_SYNTAX => 'Syntax error',
);
echo 'Last error : ', $json_errors[json_last_error()], PHP_EOL, PHP_EOL;
Also use the json.stringify() function to double check your JSON syntax.
None of the solutions above worked for me, but html_entity_decode($json_string) did the trick
Try this
$foo = utf8_encode($foo);
$data = json_decode($foo, true);
make sure that if you sent the data by POST / GET, the server has not escape the quotes
$my_array = json_decode(str_replace ('\"','"', $json_string), true);
"{"action":"set","user":"123123123123","status":"OK"}"
This little apostrophe in the beginning - what is it? First symbol after the doublequote.
I had the similar problem in a live site. In my local site it was working fine. For fixing the same I Just have added the below code
json_decode(stripslashes($_GET['arr']));
I just put this
$result = mb_convert_encoding($result,'UTF-8','UTF-8');
$result = json_decode($result);
and it's working
Yesterday I spent 2 hours on checking and fixing that error finally I found that in JSON string that I wanted to decode were '\' slashes. So the logical thing to do is to use stripslashes function or something similiar to different PL.
Of course the best way is sill to print this var out and see what it becomes after json_decode, if it is null you can also use json_last_error() function to determine the error it will return integer but here are those int described:
0 = JSON_ERROR_NONE
1 = JSON_ERROR_DEPTH
2 = JSON_ERROR_STATE_MISMATCH
3 = JSON_ERROR_CTRL_CHAR
4 = JSON_ERROR_SYNTAX
5 = JSON_ERROR_UTF8
In my case I got output of json_last_error() as number 4 so it is JSON_ERROR_SYNTAX. Then I went and take a look into the string it self which I wanted to convert and it had in last line:
'\'title\' error ...'
After that is really just an easy fix.
$json = json_decode(stripslashes($response));
if (json_last_error() == 0) { // you've got an object in $json}
I had such problem with storage json-string in MySQL.
Don't really know why, but using htmlspecialchars_decode berofe json_decode resolved problem.
Non of these solutions worked for me.
What DID eventually work was checking the string encoding by saving it to a local file and opening with Notepad++.
I found out it was 'UTF-16', so I was able to convert it this way:
$str = mb_convert_encoding($str,'UTF-8','UTF-16');
Maybe you use thing as $ ${: these chars should be quoted.
I was having this problem, when I was calling a soap method to obtain my data, and then return a json string, when I tried to do json_decode I just keep getting null.
Since I was using nusoap to do the soap call I tried to just return json string and now I could do a json_decode, since I really neaded to get my data with a SOAP call, what I did was add ob_start() before include nusoap, id did my call genereate json string, and then before returning my json string I did ob_end_clean(), and GOT MY PROBLEM FIXED :)
EXAMPLE
//HRT - SIGNED
//20130116
//verifica se um num assoc deco é valido
ob_start();
require('/nusoap.php');
$aResponse['SimpleIsMemberResult']['IsMember'] = FALSE;
if(!empty($iNumAssociadoTmp))
{
try
{
$client = new soapclientNusoap(PartnerService.svc?wsdl',
array(
// OPTS
'trace' => 0,
'exceptions' => false,
'cache_wsdl' => WSDL_CACHE_NONE
)
);
//MENSAGEM A ENVIAR
$sMensagem1 = '
<SimpleIsMember>
<request>
<CheckDigit>'.$iCheckDigitAssociado.'</CheckDigit>
<Country>Portugal</Country>
<MemberNumber">'.$iNumAssociadoDeco.'</MemberNumber>
</request>
</SimpleIsMember>';
$aResponse = $client->call('SimpleIsMember',$sMensagem1);
$aData = array('dados'=>$aResponse->xpto, 'success'=>$aResponse->example);
}
}
ob_end_clean();
return json_encode($aData);
I don't know Why?
But this work:
$out = curl_exec($curl);
$out = utf8_encode($out);
$out = str_replace("?", "", $out);
if (substr($out,1,1)!='{'){
$out = substr($out,3);
}
$arResult["questions"] = json_decode($out,true);
without utf8_encode() - Don't work
Check the encoding of your file. I was using netbeans and had to use iso windows 1252 encoding for an old project and netbeans was using this encoding since then for every new file. json_decode will then return NULL. Saving the file again with UTF-8 encoding solved the problem for me.
In Notepad++, select Encoding (from the top menu) and then ensure that "Encode in UTF-8" is selected.
This will display any characters that shouldn't be in your json that would cause json_decode to fail.
Try using json_encode on the string prior to using json_decode... idk if will work for you but it did for me... I'm using laravel 4 ajaxing through a route param.
$username = "{username: john}";
public function getAjaxSearchName($username)
{
$username = json_encode($username);
die(var_dump(json_decode($username, true)));
}
You should try out json_last_error_msg(). It will give you the error message and tell you what is wrong. It was introduced in PHP 5.5.
$foo = "{"action":"set","user":"123123123123","status":"OK"}";
$data = json_decode($foo, true);
if($data == null) {
throw new Exception('Decoding JSON failed with the following message: '
. json_last_error_msg());
}
// ... JSON decode was good => Let's use the data
Before applying PHP related solutions, validate your JSON format. That may be the problem. Try below online JSON format validator. If your JSON format is invalid, correct it first, because PHP doesn't decode invalid JSON strings.
https://jsonformatter.org/
Laravel specific answer:
I got the same issue in Laravel. And this did the trick for me
$result = json_decode($result->getContent(), true);
In my case, when I was printing to the screen, json was fine and I copied and decode with json_deocode() function. It was working fine. But, when I was trying to put jsonString directly in the function, it was returning null because quotes were coming like these ". So I used htmlspecialchars_decode() function and now it is working fine.
I am new here, so if I am making any mistakes in writing answer then sorry for that. I hope it'll help somebody.
Sometimes the problem is generated when the content is compressed, so adding the Accept-Encoding: identity header can solve the problem without having to wrangle with the response.
$opts = array(
'http' =>
array(
'header' =>
array(
'Accept-Encoding: identity',
),
),
);
$context = stream_context_create($opts);
$contents = file_get_contents('URL', false, $context);
i had a similar problem, got it to work after adding '' (single quotes) around the json_encode string. Following from my js file:
var myJsVar = <?php echo json_encode($var); ?> ; -------> NOT WORKING
var myJsVar = '<?php echo json_encode($var); ?>' ; -------> WORKING
just thought of posting it in case someone stumbles upon this post like me :)