Does ajax post data need to be URI encoded?

Does ajax post data need to be URI encoded? - php

Regarding this line:
var data = encodeURIComponent(JSON.stringify(object_literal));
I don't understand why this is being URI encoded.
Later data will be sent via ajax POST.
I understand that URLs, particularly the one you can see in the browser address bar require special characters as described here:
http://www.blooberry.com/indexdot/html/topics/urlencoding.htm
But what exactly does this have to do with Ajax posting?
Do both the url address bar and the internal ajax post utilize the same mechanism?

It all depends on the content type.
Normally when a <form> uses the HTTP method POST then the form values are URL Encoded and placed in the body of the request. The content type header looks like this:
content-type: application/x-www-form-urlencoded
Most AJAX libraries will do this by default since it is universally accepted among web servers. However, there is nothing preventing you from simply serializing the data as JSON or XML and then sending it with a different content type.
content-type: application/json
or
content-type: text/xml
It's worth noting that the payload has nothing to do with AJAX! Under the hood they are all using the XmlHttpRequest object to send an HTTP request asynchronously to the server. It doesn't matter if you send Plain Text, JSON, XML, or even raw binary data so long as you tell the server how to interpret those bits.
The url encoding is purely a historical artifact of how <form> elements posted their data to the server before AJAX was around.

Josh's answer is good, but I think it's missing something. Traditionally form data was posted using the same format as querystrings. eg: param1=value1&param2=value2 and the only difference between a GET and POST is that POST puts these parameters in the message body whereas a GET puts them in the URL. However, the obvious problem is that if your parameter names or values include unescaped characters like & and =, then you'll screw up the server's ability to automatically parse the parameter collection, resulting in corruption of the data. Using Javascript's encodeURIComponent() function on each parameter value will escape all these characters for you.
So bottom line: if you are not using the old standard parameter collection on the server side--for example, if you are parsing JSON instead--then there's no need to URL encode the text. Also, there's no reason to URL encode if you aren't parsing out multiple parameters on the server side so running encodeURIComponent once on the entire message body makes no sense.
Side note: if you are using asp.net and trying to let users pass html to the server, encodeURIComponent will let you do this without disabling the request validation that normally prohibits this. I don't think sending it as JSON, alone, would accomplish this.

encodeURIComponent is to pass variables over links using GET
JSON.stringify() encodes automatically into utf8
only in some rare cases for example when you want to convert strange charachters to base64 the encodeURIComponent is used outside from GET.
here is an example
base64_encode=function(a){
return window.btoa(unescape(encodeURIComponent(a)));
};
https://developer.mozilla.org/en-US/docs/Web/API/window.btoa
said that ..
if you use a REST API service every ajax GET request which is a string parameter needs to be encoded with encodeURIComponent.
here is an example using yql
https://stackoverflow.com/a/18302867/2450730

No they do not. There is no need for it when it is send as data through Ajax POST.
You can send pure JSON unencoded using AJAX
var ajaxObject = $.ajax({
url: 'url',
type: 'POST',
data: JSON.stringify(object_literal);
});

Related

Submitted POST not read by php://input [duplicate]

I have been directed to use the method php://input instead of $_POST when interacting with Ajax requests from JQuery. What I do not understand is the benefits of using this vs the global method of $_POST or $_GET.

The reason is that php://input returns all the raw data after the HTTP-headers of the request, regardless of the content type.
The PHP superglobal $_POST, only is supposed to wrap data that is either
application/x-www-form-urlencoded (standard content type for simple form-posts) or
multipart/form-data (mostly used for file uploads)
This is because these are the only content types that must be supported by user agents. So the server and PHP traditionally don't expect to receive any other content type (which doesn't mean they couldn't).
So, if you simply POST a good old HTML form, the request looks something like this:
POST /page.php HTTP/1.1
key1=value1&key2=value2&key3=value3
But if you are working with Ajax a lot, this probaby also includes exchanging more complex data with types (string, int, bool) and structures (arrays, objects), so in most cases JSON is the best choice. But a request with a JSON-payload would look something like this:
POST /page.php HTTP/1.1
{"key1":"value1","key2":"value2","key3":"value3"}
The content would now be application/json (or at least none of the above mentioned), so PHP's $_POST-wrapper doesn't know how to handle that (yet).
The data is still there, you just can't access it through the wrapper. So you need to fetch it yourself in raw format with file_get_contents('php://input') (as long as it's not multipart/form-data-encoded).
This is also how you would access XML-data or any other non-standard content type.

First, a basic truth about PHP.
PHP was not designed to explicitly give you a pure REST (GET, POST, PUT, PATCH, DELETE) like interface for handling HTTP requests.
However, the $_SERVER, $_COOKIE, $_POST, $_GET, and $_FILES superglobals, and the function filter_input_array() are very useful for the average person's / layman's needs.
The number one hidden advantage of $_POST (and $_GET) is that your input data is url-decoded automatically by PHP. You never even think about having to do it, especially for query string parameters within a standard GET request, or HTTP body data submitted with a POST request.
Other HTTP Request Methods
Those studying the underlying HTTP protocol and its various request methods come to understand that there are many HTTP request methods, including the often referenced PUT, PATCH (not used in Google's Apigee), and DELETE.
In PHP, there are no superglobals or input filter functions for getting HTTP request body data when POST is not used. What are disciples of Roy Fielding to do? ;-)
However, then you learn more ...
That being said, as you advance in your PHP programming knowledge and want to use JavaScript's XmlHttpRequest object (jQuery for some), you come to see the limitation of this scheme.
$_POST limits you to the use of two media types in the HTTP Content-Type header:
application/x-www-form-urlencoded, and
multipart/form-data
Thus, if you want to send data values to PHP on the server, and have it show up in the $_POST superglobal, then you must urlencode it on the client-side and send said data as key/value pairs--an inconvenient step for novices (especially when trying to figure out if different parts of the URL require different forms of urlencoding: normal, raw, etc..).
For all you jQuery users, the $.ajax() method is converting your JSON to URL encoded key/value pairs before transmitting them to the server. You can override this behavior by setting processData: false. Just read the $.ajax() documentation, and don't forget to send the correct media type in the Content-Type header.
php://input, but ...
Even if you use php://input instead of $_POST for your HTTP POST request body data, it will not work with an HTTP Content-Type of multipart/form-data This is the content type that you use on an HTML form when you want to allow file uploads!
<form enctype="multipart/form-data" accept-charset="utf-8" action="post">
<input type="file" name="resume">
</form>
Therefore, in traditional PHP, to deal with a diversity of content types from an HTTP POST request, you will learn to use $_POST or filter_input_array(POST), $_FILES, and php://input. There is no way to just use one, universal input source for HTTP POST requests in PHP.
You cannot get files through $_POST, filter_input_array(POST), or php://input, and you cannot get JSON/XML/YAML in either filter_input_array(POST) or $_POST.
PHP Manual: php://input
php://input is a read-only stream that allows you to read raw data
from the request body...php://input is not available with
enctype="multipart/form-data".
PHP Frameworks to the rescue?
PHP frameworks like Codeigniter 4 and Laravel use a facade to provide a cleaner interface (IncomingRequest or Request objects) to the above. This is why professional PHP developers use frameworks instead of raw PHP.
Of course, if you like to program, you can devise your own facade object to provide what frameworks do. It is because I have taken time to investigate this issue that I am able to write this answer.
URL encoding? What the heck!!!???
Typically, if you are doing a normal, synchronous (when the entire page redraws) HTTP requests with an HTML form, the user-agent (web browser) will urlencode your form data for you. If you want to do an asynchronous HTTP requests using the XmlHttpRequest object, then you must fashion a urlencoded string and send it, if you want that data to show up in the $_POST superglobal.
How in touch are you with JavaScript? :-)
Converting from a JavaScript array or object to a urlencoded string bothers many developers (even with new APIs like Form Data). They would much rather just be able to send JSON, and it would be more efficient for the client code to do so.
Remember (wink, wink), the average web developer does not learn to use the XmlHttpRequest object directly, global functions, string functions, array functions, and regular expressions like you and I ;-). Urlencoding for them is a nightmare. ;-)
PHP, what gives?
PHP's lack of intuitive XML and JSON handling turns many people off. You would think it would be part of PHP by now (sigh).
So many media types (MIME types in the past)
XML, JSON, and YAML all have media types that can be put into an HTTP Content-Type header.
application/xml
applicaiton/json
application/yaml (although IANA has no official designation listed)
Look how many media-types (formerly, MIME types) are defined by IANA.
Look how many HTTP headers there are.
php://input or bust
Using the php://input stream allows you to circumvent the baby-sitting / hand holding level of abstraction that PHP has forced on the world. :-) With great power comes great responsibility!
Now, before you deal with data values streamed through php://input, you should / must do a few things.
Determine if the correct HTTP method has been indicated (GET, POST,
PUT, PATCH, DELETE, ...)
Determine if the HTTP Content-Type header has been transmitted.
Determine if the value for the Content-Type is the desired media
type.
Determine if the data sent is well formed XML / JSON / YAML /
etc.
If necessary, convert the data to a PHP datatype: array or
object.
If any of these basic checks or conversions fails, throw an exception!
What about the character encoding?
AH, HA! Yes, you might want the data stream being sent into your application to be UTF-8 encoded, but how can you know if it is or not?
Two critical problems.
You do not know how much data is coming through php://input.
You do not know for certain the current encoding of the data stream.
Are you going to attempt to handle stream data without knowing how much is there first? That is a terrible idea. You cannot rely exclusively on the HTTP Content-Length header for guidance on the size of streamed input because it can be spoofed.
You are going to need a:
Stream size detection algorithm.
Application defined stream size limits (Apache / Nginx / PHP limits may be too broad).
Are you going to attempt to convert stream data to UTF-8 without knowing the current encoding of the stream? How? The iconv stream filter (iconv stream filter example) seems to want a starting and ending encoding, like this.
'convert.iconv.ISO-8859-1/UTF-8'
Thus, if you are conscientious, you will need:
Stream encoding detection algorithm.
Dynamic / runtime stream filter definition algorithm (because you cannot know the starting encoding a priori).
(Update: 'convert.iconv.UTF-8/UTF-8' will force everything to UTF-8, but you still have to account for characters that the iconv library might not know how to translate. In other words, you have to some how define what action to take when a character cannot be translated: 1) Insert a dummy character, 2) Fail / throw and exception).
You cannot rely exclusively on the HTTP Content-Encoding header, as this might indicate something like compression as in the following. This is not what you want to make a decision off of in regards to iconv.
Content-Encoding: gzip
Therefore, the general steps might be ...
Part I: HTTP Request Related
Determine if the correct HTTP method has been indicated (GET, POST,
PUT, PATCH, DELETE, ...)
Determine if the HTTP Content-Type header has been transmitted.
Determine if the value for the Content-Type is the desired media
type.
Part II: Stream Data Related
Determine the size of the input stream (optional, but recommended).
Determine the encoding of the input stream.
If necessary, convert the input stream to the desired character
encoding (UTF-8).
If necessary, reverse any application level compression or encryption, and then repeat steps 4, 5, and 6.
Part III: Data Type Related
Determine if the data sent is well formed XML / JSON / YMAL /
etc.
(Remember, the data can still be a URL encoded string which you must then parse and URL decode).
If necessary, convert the data to a PHP datatype: array or
object.
Part IV: Data Value Related
Filter input data.
Validate input data.
Now do you see?
The $_POST superglobal, along with php.ini settings for limits on input, are simpler for the layman. However, dealing with character encoding is much more intuitive and efficient when using streams because there is no need to loop through superglobals (or arrays, generally) to check input values for the proper encoding.

php://input can give you the raw bytes of the data. This is useful if the POSTed data is a JSON encoded structure, which is often the case for an AJAX POST request.
Here's a function to do just that:
/**
* Returns the JSON encoded POST data, if any, as an object.
*
* #return Object|null
*/
private function retrieveJsonPostData()
{
// get the raw POST data
$rawData = file_get_contents("php://input");
// this returns null if not valid json
return json_decode($rawData);
}
The $_POST array is more useful when you're handling key-value data from a form, submitted by a traditional POST. This only works if the POSTed data is in a recognised format, usually application/x-www-form-urlencoded (see http://www.w3.org/TR/html4/interact/forms.html#h-17.13.4 for details).

If post data is malformed, $_POST will not contain anything. Yet, php://input will have the malformed string.
For example there is some ajax applications, that do not form correct post key-value sequence for uploading a file, and just dump all the file as post data, without variable names or anything.
$_POST will be empty, $_FILES empty also, and php://input will contain exact file, written as a string.

if (strtoupper($_SERVER['REQUEST_METHOD']) != 'POST') {
throw new Exception('Only POST requests are allowed');
}
// Make sure Content-Type is application/json
$content_type = isset($_SERVER['CONTENT_TYPE']) ? $_SERVER['CONTENT_TYPE'] : '';
if (stripos($content_type, 'application/json') === false) {
throw new Exception('Content-Type must be application/json');
}
// Read the input stream
$body = file_get_contents("php://input");
// Decode the JSON object
$object = json_decode($body, true);

Simple example of how to use it
<?php
if(!isset($_POST) || empty($_POST)) {
?>
<form name="form1" method="post" action="">
<input type="text" name="textfield"><br />
<input type="submit" name="Submit" value="submit">
</form>
<?php
} else {
$example = file_get_contents("php://input");
echo $example; }
?>

Are PHP “php://input” and $_POST mutually exclusive? [duplicate]

I have been directed to use the method php://input instead of $_POST when interacting with Ajax requests from JQuery. What I do not understand is the benefits of using this vs the global method of $_POST or $_GET.

The reason is that php://input returns all the raw data after the HTTP-headers of the request, regardless of the content type.
The PHP superglobal $_POST, only is supposed to wrap data that is either
application/x-www-form-urlencoded (standard content type for simple form-posts) or
multipart/form-data (mostly used for file uploads)
This is because these are the only content types that must be supported by user agents. So the server and PHP traditionally don't expect to receive any other content type (which doesn't mean they couldn't).
So, if you simply POST a good old HTML form, the request looks something like this:
POST /page.php HTTP/1.1
key1=value1&key2=value2&key3=value3
But if you are working with Ajax a lot, this probaby also includes exchanging more complex data with types (string, int, bool) and structures (arrays, objects), so in most cases JSON is the best choice. But a request with a JSON-payload would look something like this:
POST /page.php HTTP/1.1
{"key1":"value1","key2":"value2","key3":"value3"}
The content would now be application/json (or at least none of the above mentioned), so PHP's $_POST-wrapper doesn't know how to handle that (yet).
The data is still there, you just can't access it through the wrapper. So you need to fetch it yourself in raw format with file_get_contents('php://input') (as long as it's not multipart/form-data-encoded).
This is also how you would access XML-data or any other non-standard content type.

php://input can give you the raw bytes of the data. This is useful if the POSTed data is a JSON encoded structure, which is often the case for an AJAX POST request.
Here's a function to do just that:
/**
* Returns the JSON encoded POST data, if any, as an object.
*
* #return Object|null
*/
private function retrieveJsonPostData()
{
// get the raw POST data
$rawData = file_get_contents("php://input");
// this returns null if not valid json
return json_decode($rawData);
}
The $_POST array is more useful when you're handling key-value data from a form, submitted by a traditional POST. This only works if the POSTed data is in a recognised format, usually application/x-www-form-urlencoded (see http://www.w3.org/TR/html4/interact/forms.html#h-17.13.4 for details).

If post data is malformed, $_POST will not contain anything. Yet, php://input will have the malformed string.
For example there is some ajax applications, that do not form correct post key-value sequence for uploading a file, and just dump all the file as post data, without variable names or anything.
$_POST will be empty, $_FILES empty also, and php://input will contain exact file, written as a string.

if (strtoupper($_SERVER['REQUEST_METHOD']) != 'POST') {
throw new Exception('Only POST requests are allowed');
}
// Make sure Content-Type is application/json
$content_type = isset($_SERVER['CONTENT_TYPE']) ? $_SERVER['CONTENT_TYPE'] : '';
if (stripos($content_type, 'application/json') === false) {
throw new Exception('Content-Type must be application/json');
}
// Read the input stream
$body = file_get_contents("php://input");
// Decode the JSON object
$object = json_decode($body, true);

Simple example of how to use it
<?php
if(!isset($_POST) || empty($_POST)) {
?>
<form name="form1" method="post" action="">
<input type="text" name="textfield"><br />
<input type="submit" name="Submit" value="submit">
</form>
<?php
} else {
$example = file_get_contents("php://input");
echo $example; }
?>

Angular http.get/post with URL in data being passed

I have the following http.get request...
$http.get("api/checkSave/"+ JSON.stringify($scope.programDetails)).then(function(data) {
....
});
In my programDetails object, there is at least one field that can have a URL in it. I thought the stringify would've properly encoded the object so it would get passed correctly to my PHP function. What seems to happen though is that the URL isn't sent properly. I get an error that the server responded with a 404 not found error. I manually tried removing the URL in the request (copied it to the browser and sent an empty string for the URL field) and the request was processed correctly - that's why I am thinking the stringify can't do what I need.
I also need to send this data via a http.post later to save the data in the object. Hopefully, whatever I need to do in the get will also apply when I call the post.
Should the stringify pass along the data correctly or do I need to encode it differently? I've been using stringify in other apps, but haven't had to pass a URL in the data.
If I need to do it differently, I would need to know what to do on the PHP side to decode it for processing. The examples I find for get/post all seem to pass individual variables and not an object and not with an URL in one of the variables. I've seen encodeURIComponent mentioned but wasn't sure if that's what I need to use instead and if so, then what to use on the PHP side to decode it properly.

This doesn't really have much to do with the data consisting of a URL (only that a URL has a much better chance of having characters in it that have special meaning within a URL than other data).
The problem is that you are putting the data into a URL and URLs are not JSON texts.
To encode a string to put it in a URL use encodeURIComponent.

So it ended up that I had to replace the '/' in the fields that contained links and then it worked. Otherwise, the '/' was making it seem as though I was passing additional parameters in my get call. Thanks for the help!

PHP "php://input" vs $_POST

I have been directed to use the method php://input instead of $_POST when interacting with Ajax requests from JQuery. What I do not understand is the benefits of using this vs the global method of $_POST or $_GET.

The reason is that php://input returns all the raw data after the HTTP-headers of the request, regardless of the content type.
The PHP superglobal $_POST, only is supposed to wrap data that is either
application/x-www-form-urlencoded (standard content type for simple form-posts) or
multipart/form-data (mostly used for file uploads)
This is because these are the only content types that must be supported by user agents. So the server and PHP traditionally don't expect to receive any other content type (which doesn't mean they couldn't).
So, if you simply POST a good old HTML form, the request looks something like this:
POST /page.php HTTP/1.1
key1=value1&key2=value2&key3=value3
But if you are working with Ajax a lot, this probaby also includes exchanging more complex data with types (string, int, bool) and structures (arrays, objects), so in most cases JSON is the best choice. But a request with a JSON-payload would look something like this:
POST /page.php HTTP/1.1
{"key1":"value1","key2":"value2","key3":"value3"}
The content would now be application/json (or at least none of the above mentioned), so PHP's $_POST-wrapper doesn't know how to handle that (yet).
The data is still there, you just can't access it through the wrapper. So you need to fetch it yourself in raw format with file_get_contents('php://input') (as long as it's not multipart/form-data-encoded).
This is also how you would access XML-data or any other non-standard content type.

php://input can give you the raw bytes of the data. This is useful if the POSTed data is a JSON encoded structure, which is often the case for an AJAX POST request.
Here's a function to do just that:
/**
* Returns the JSON encoded POST data, if any, as an object.
*
* #return Object|null
*/
private function retrieveJsonPostData()
{
// get the raw POST data
$rawData = file_get_contents("php://input");
// this returns null if not valid json
return json_decode($rawData);
}
The $_POST array is more useful when you're handling key-value data from a form, submitted by a traditional POST. This only works if the POSTed data is in a recognised format, usually application/x-www-form-urlencoded (see http://www.w3.org/TR/html4/interact/forms.html#h-17.13.4 for details).

If post data is malformed, $_POST will not contain anything. Yet, php://input will have the malformed string.
For example there is some ajax applications, that do not form correct post key-value sequence for uploading a file, and just dump all the file as post data, without variable names or anything.
$_POST will be empty, $_FILES empty also, and php://input will contain exact file, written as a string.

if (strtoupper($_SERVER['REQUEST_METHOD']) != 'POST') {
throw new Exception('Only POST requests are allowed');
}
// Make sure Content-Type is application/json
$content_type = isset($_SERVER['CONTENT_TYPE']) ? $_SERVER['CONTENT_TYPE'] : '';
if (stripos($content_type, 'application/json') === false) {
throw new Exception('Content-Type must be application/json');
}
// Read the input stream
$body = file_get_contents("php://input");
// Decode the JSON object
$object = json_decode($body, true);

Simple example of how to use it
<?php
if(!isset($_POST) || empty($_POST)) {
?>
<form name="form1" method="post" action="">
<input type="text" name="textfield"><br />
<input type="submit" name="Submit" value="submit">
</form>
<?php
} else {
$example = file_get_contents("php://input");
echo $example; }
?>

file_get_contents("php://input") or $HTTP_RAW_POST_DATA, which one is better to get the body of JSON request?

file_get_contents("php://input") or $HTTP_RAW_POST_DATA - which one is better to get the body of JSON request?
And which request type (GET or POST) should I use to send JSON data when using client side XmlHTTPRequest?
My question was inspired from this answer:
How to post JSON to PHP with curl
Quote from that answer:
From a protocol perspective file_get_contents("php://input") is actually more correct, since you're not really processing http multipart form data anyway.

Actually php://input allows you to read raw request body.
It is a less memory intensive alternative to $HTTP_RAW_POST_DATA and does not need any special php.ini directives.
From Reference
php://input is not available with enctype="multipart/form-data".

php://input is a read-only stream that allows you to read raw data
from the request body. In the case of POST requests, it is preferable
to use php://input instead of $HTTP_RAW_POST_DATA as it does not
depend on special php.ini directives. Moreover, for those cases
where $HTTP_RAW_POST_DATA is not populated by default, it is a
potentially less memory intensive alternative to activating
always_populate_raw_post_data.
Source: http://php.net/manual/en/wrappers.php.php.

file_get_contents(php://input) - gets the raw POST data and you need to use this when you write APIs and need XML/JSON/... input that cannot be decoded to $_POST by PHP
some example :
send by post JSON string
<input type="button" value= "click" onclick="fn()">
<script>
function fn(){
var js_obj = {plugin: 'jquery-json', version: 2.3};
var encoded = JSON.stringify( js_obj );
var data= encoded
$.ajax({
type: "POST",
url: '1.php',
data: data,
success: function(data){
console.log(data);
}
});
}
</script>
1.php
//print_r($_POST); //empty!!! don't work ...
var_dump( file_get_contents('php://input'));

For JSON data, it's much easier to POST it as "application/json" content-type. If you use GET, you have to URL-encode the JSON in a parameter and it's kind of messy. Also, there is no size limit when you do POST. GET's size if very limited (4K at most).

The usual rules should apply for how you send the request. If the request is to retrieve information (e.g. a partial search 'hint' result, or a new page to be displayed, etc...) you can use GET. If the data being sent is part of a request to change something (update a database, delete a record, etc..) then use POST.
Server-side, there's no reason to use the raw input, unless you want to grab the entire post/get data block in a single go. You can retrieve the specific information you want via the _GET/_POST arrays as usual. AJAX libraries such as MooTools/jQuery will handle the hard part of doing the actual AJAX calls and encoding form data into appropriate formats for you.

Your second question is easy, GET has a size limitation of 1-2 kilobytes on both the server and browser side, so any kind of larger amounts of data you'd have to send through POST.

We Keep Coding

PHP, A popular general-purpose scripting language that is especially suited to web development.