Correctly decoding double-encoded UTF-8 on PHP - php

So im trying to send data from an HTML page through Ajax, to a PHP page.
Thats the piece of jQuery code that im using:
$.ajax({
url: "test.php",
type: "POST",
data: {
name: "João"
}
}).done(function (data) {
alert(data);
})
As you can see, the parameter im sending is "João". Before making the Ajax request jQuery encodes it on the background, "João" becomes "Jo%C3%A3o" which is double encoded UTF-8.
My problem arises when the request is sent and PHP tries to decode it on the background. PHP decodes automatically it only once when I use $_POST, so instead of getting "João" I get "João". That happens because PHP is decoding every % individually, so %C3 becomes à and %A3 becomes £.
If I try to decode it manually through utf8_decode() it will work, but im here to know if there's a better solution. What I really need is a way for PHP to decode my data correctly, even if it's double-encoded, or even triple-encoded.

That's not double-encoded, it's correct UTF-8. It looks like the PHP is expecting latin-1 encoding instead, and is showing you what the same bytes would mean if they were not
UTF-8.
In this case, since your characters seem to be below 0xFF, you could also URL-encode them first as Jo%E3o in latin-1 if you can't work out how to have PHP recognize UTF-8.

Related

ajax returns json with added character

I have a Jquery code getting data from ajax call.
$.ajax({
type: 'GET',
url: 'php/upd_dashboard.php',
data: {'name': 'GET_ANUAL_TOTAL_OBJECTIF'},
success: function(data){
list_total_objectif=data;
},
dataType: 'JSON',
async: false
});
the file upd_dashboard.php, get data from MySql database, and return json result.
echo json_encode($result);
I have no problem when executing this code in localhost, but when I deploy the site I can't get the result of json_encode.
by inspecting that in Chrome, I get a character '\ufeff' at the beginning of the json result !!
Chrome inspection result
Is it a problem of encoding?
Regards,
Hamza.
You can try doing ob_clean(); before sending out json output
ob_clean();//clears the output buffer
echo json_encode($result);
But it seems your $result is just malformed. Is it a string or an array? No character encoding issues there anyway
From what I've found on Google, \ufeff is the Byte Order Mark.
So you probably have your PHP source files in UTF-8 With BOM.
Some PHP versions don't like that and send the BOM output.
Try converting all your PHP source files to UTF-8 Without BOM and it should fix the issue.

The & is not allowed for my new password

I have a window on my site where the visitors can change them password.
The problem is that the & character is never taken :
If I put those two new passwords :
stack&
stack& (the second is the confirmation)
The insertion in the BD is stack (without the &).
This is the js code :
data: 'nouveau_mdp=' + $('input#champ_nouveau_mdp').val(),
the alert shwos me "stack&"
In PHP, a var_dump of $_POST gives me :
stack (without the &).
Is & a reserved word for jquery ?
This is my js code :
$.ajax({
type: 'POST',
url: 'modification_mdp.php',
data: 'nouveau_mdp=' + $('input#champ_nouveau_mdp').val(),
dataType: 'text',
success: function(reponse) {
reponse = $.trim(reponse);
Have you an idea to reselove this problem please ?
Thanks in advance.
Encode the field value
data: 'nouveau_mdp=' + encodeURI($('input#champ_nouveau_mdp').val()),
by this way & is encoded with the ascii number.
Try passing an associative array, which jQuery will encode for you:
data: { nouveau_mdp: $('input#champ_nouveau_mdp').val() }
You'll need to change the PHP code that receives this value to match (it gets an array rather than a string).
The ampersand character is not allowed to pass by Get method or by Post method in PHP. As the usual syntax to access the PHP by GET method is ines.php?user=username&password=mypassword,
the '&' character separates the two Variables.
That is why it is does not take '&' from 'Stack&' You may use Javascript to Validate if & is not entered in the text Box. Use JavaScript to encode the varaible... No decoding is required at the PHP end
encodeURIComponent("stack&")
This line is the problem:
data: 'nouveau_mdp=' + $('input#champ_nouveau_mdp').val(),
The & character has a special meaning in HTTP. If you had multiple parameters they would be separated by & e.g. nouveau_mdp=mynewpassword&old_mdp=myoldpassword
What you need to do is encode characters such as &, = etc as their hex equivalents i.e. %26 and %3D respectively.
In JavaScript you can do this with the encodeURI() function.
Though, as mentioned in the other answer, jQuery also allows you to pass a JSON dictionary of parameters, and it will do the encoding for you.

How to properly encode UTF-8 for JavaScript and JSON?

I have a problem creating an input validation hash. JavaScript submits data to API and API validates the sent data with json_encode. Basically it works like this:
$input=array('name'='John Doe','city'=>'New York');
$validationHash=sha1(json_encode($input).$key); // Key is known to both servers
If PHP connects to another server, then everything works. It also works through JavaScript (I have a custom sha1() function there as well):
var validationHash=sha1(JSON.stringify({'name':'John Doe','city'=>'New York'})+key);
My problem comes when the string contains UTF-8 characters. For example, if one of the values is:
Ränisipelgasöösel
Then PHP server that receives the command converts it to this after JSON encoding:
R\u00e4nisipelgas\u00f6\u00f6sel
I need to do this in JavaScript as well, but I haven't been able to work out how. I need to make sure that I send proper validation hash to the server or the command fails. I found from Google that unescape(encodeURIComponent(string)) and decodeURIComponent() could be used, but neither gives me the same string that PHP has and validates with.
UTF-8 is used on both client and server.
Any ideas?
It does not seem to be possible. The only working solution I have found is to encode all data with encodeURIComponent() on browser side and with rawurlencode() on PHP side and then calculate the JSON from these values in arrays.
My fix was to raw url encode my json data like so.
rawurlencode( json_encode( $data ) );
And then from within javascript decode the raw url encoded json and then parse the json string like so.
JSON.parse( decodeURIComponent( data ) );
Hope this helps.
Why not base64 encode the data for safe transport? It encodes UTF-8 characters in a safe string that can be sent across different mediums - php, javascript etc. That way you can base64 decode the string at the receiving end. Voila!
By base64 encoding the data, i mean base64 encoding the values and not the whole json string
is you html page encoding utf-8?
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">

jQuery ajax + responseText + Character encoding

I perform an AJAX request to a PHP script in JavaScript and by using the jQuery library.
Here is my command which performs the AJAX request :
$.ajax({
async: "false",
cache: "false",
data: {login: strLogin, password: strPassword},
datatype: "text",
error: connexionAjaxError,
success: connexionAjaxSuccess,
type: "POST",
url: "./Connexion"
});
".Connexion" is a URL redirection to my PHP script.
connexionAjaxError is a JavaScript function inevitably executed as soon as the HTTP response is received.
The reason : My PHP script results to a text/plain page with HTTP status code equal to 500.
Here is the most relevant part of my PHP script :
header("HTTP/1.0 500 Internal Server Error;
Content-type: text/plain;
charset=ISO-8859-1");
echo "été";
Here is the signature of the connexionAjaxError callback function :
function connexionAjaxError(jqXHR, textStatus, errorThrown)
My PHP results to the string "été".
My PHP internal encoding is ISO-8859-1 so the result string is encoded [e9 74 e9] in hexadecimal notation.
But jqXHR.responseText in the connexionAjaxError callback function contains a 1-character string.
I have used the charCodeAt method of the JavaScript String object to get the unicode (UTF-8 ?) code for the character.
The character is encoded 65535 in decimal notation equivalent to [ff ff] in hexadecimal notation.
I do not understand why jqXHR.responseText does not contain the 3-character string "été".
I am not an expert in JavaScript but I suppose jqXHR.responseText can only contains a string encoded in UTF-8.
So the string "été" in jqXHR.responseText would be encoded [c3a9 74 c3a9] in hexadecimal notation.
I have retrieve the UTF-8 code for the character "é" from the following web site : http://www.utf8-chartable.de/
I make a lot of development in Spanish and I've noticed a few things that might help you, since I've gone through this already:
The charset in the header function should be UTF-8.
Also, the FILE itself (the PHP script) must be saved in UTF-8, you can easily achieve this if you edit the file with Notepad++ and select the encoding to be UTF-8 without BOM.
Same goes for your javascript and HTML files, they should all be encoded with UTF-8 without BOM rather than ANSI.
Let me know!
Try to set the Contenttype like this in your call, as far as I know jQuery Ajax is always utf-8 if not specified.
$.ajax({
...
datatype: "text",
contentType: "application/x-www-form-urlencoded;charset=ISO-8859-1",
error: connexionAjaxError,
success: connexionAjaxSuccess,
type: "POST",
...
});
Edit:
If my answer isn't helpfull, it might help if you check the servers charset configuration, and also post it.

Special characters break json returned to jQuery

The following jQuery ajax function runs a PHP script that queries a MySQL database containing entries that are encoded as UTF-8:
function searchLocations() {
var stateSelected = $("#stateSelect").val();
$.ajax({
url: 'ajax/json.php',
dataType: 'json',
data: 'state='+stateSelected,
success: function(data) {
placeMarkers(data.markerdata.markers);
}
});
}
The JSON object returned to the function contains the longitudes and latitudes of map marker objects as well as a name to display in an info window when each marker is clicked.
Every name loads fine, and is displayed without a problem except for a single name which contains the character "ñ". This name is returned in the JSON object as "null". How can I make this name display properly?
I used to have this problem and I found there was two options:
In PHP convert all characters into HTML friendly characters using the PHP function htmlentities (http://php.net/manual/en/function.htmlentities.php) before returning it to the jQuery function.
If that fails or the characters are not converted you could try using base64 encode and decode. In PHP you can encode the data using base64_encode function (http://php.net/manual/en/function.base64-encode.php). Then in jQuery you could use one of many base64 encode and decode plugins to decode the data. I have used this plugin successfully in the past: http://plugins.jquery.com/project/base64-encode-and-decode
Failing both those options it may be worth looking at the character set used by your page. Try and use UTF-8 to encode your HTML and see if that helps.
Without using jQuery and AJAX if you did a basic PHP page that queried the data in question and prints the name out does it still not display correctly. If it doesn't it could also be a character set issue with MySQL.
Hope any of these pointers help. Let me know how you get on.

Categories