When I use
<form method="post" enctype="text/plain" action="proc.php">
form data can not be sent to proc.php file properly. Why? What is the problem? Why I can't use text/plain encoding with post but I can use it with get method?
[Revised]
The answer is, because PHP doesn't handle it (and it is not a bug):
https://bugs.php.net/bug.php?id=33741
Valid values for enctype in <form> tag are:
application/x-www-form-urlencoded
multipart/form-data
The first is the default, the second one you need only when you upload files.
#Alohci provided explanation why PHP doesn't populate $_POST array, but store the value inside a variable $HTTP_RAW_POST_DATA.
Example of what can go wrong with text/plain enctype:
file1.php:
<form method="post" enctype="text/plain" action="file2.php">
<textarea name="input1">abc
input2=def</textarea>
<input name="input2" value="ghi" />
<input type="submit">
</form>
file2.php:
<?php
print($HTTP_RAW_POST_DATA);
?>
Result:
input1=abc
input2=def
input2=ghi
No way to distinguish what is the value of input1 and input2 variables. It can be
input1=abc\r\ninput2=def, input2=ghi, as well as
input1=abc, input2=def\r\ninput2=ghi
No such problem when using the other two encodings mentioned before.
The difference between GET and POST:
in GET, the variables are part of URL and are present in URL as query string, therefore they must be URL-encoded (and they are, even if you write enctype="text/plain" - it just gets ignored by the browser; you can test it using Wireshark to sniff the request packets),
when sending POST, the variables are not part of URL, but are sent as the last header in HTTP request (POSTDATA), and you can choose whether you want to send them as text/plain or application/x-www-form-urlencoded, but the second one is the only non-ambiguous solution.
HTML5 does define how to format form data submitted as text/plain here: https://w3c.github.io/html/sec-forms.html#plain-text-form-data.
At the bottom of that section, it says:
Payloads using the text/plain format are intended to be human
readable. They are not reliably interpretable by computer, as the
format is ambiguous (for example, there is no way to distinguish a
literal newline in a value from the newline at the end of the value).
So it not unreasonable that PHP does not attempt to interpret it and only makes it available in raw form. To me, that seems the correct approach.
Related
What does enctype='multipart/form-data' mean in an HTML form and when should we use it?
When you make a POST request, you have to encode the data that forms the body of the request in some way.
HTML forms provide three methods of encoding.
application/x-www-form-urlencoded (the default)
multipart/form-data
text/plain
Work was being done on adding application/json, but that has been abandoned.
(Other encodings are possible with HTTP requests generated using other means than an HTML form submission. JSON is a common format for use with web services and some still use SOAP.)
The specifics of the formats don't matter to most developers. The important points are:
Never use text/plain.
When you are writing client-side code:
use multipart/form-data when your form includes any <input type="file"> elements
otherwise you can use multipart/form-data or application/x-www-form-urlencoded but application/x-www-form-urlencoded will be more efficient
When you are writing server-side code:
Use a prewritten form handling library
Most (such as Perl's CGI->param or the one exposed by PHP's $_POST superglobal) will take care of the differences for you. Don't bother trying to parse the raw input received by the server.
Sometimes you will find a library that can't handle both formats. Node.js's most popular library for handling form data is body-parser which cannot handle multipart requests (but has documentation that recommends some alternatives which can).
If you are writing (or debugging) a library for parsing or generating the raw data, then you need to start worrying about the format. You might also want to know about it for interest's sake.
application/x-www-form-urlencoded is more or less the same as a query string on the end of the URL.
multipart/form-data is significantly more complicated but it allows entire files to be included in the data. An example of the result can be found in the HTML 4 specification.
text/plain is introduced by HTML 5 and is useful only for debugging — from the spec: They are not reliably interpretable by computer — and I'd argue that the others combined with tools (like the Network Panel in the developer tools of most browsers) are better for that).
when should we use it?
Quentin's answer is right: use multipart/form-data if the form contains a file upload, and application/x-www-form-urlencoded otherwise, which is the default if you omit enctype.
I'm going to:
add some more HTML5 references
explain why he is right with a form submit example
HTML5 references
There are three possibilities for enctype:
application/x-www-form-urlencoded
multipart/form-data (spec points to RFC7578)
text/plain. This is "not reliably interpretable by computer", so it should never be used in production, and we will not look further into it.
How to generate the examples
Once you see an example of each method, it becomes obvious how they work, and when you should use each one.
You can produce examples using:
nc -l or an ECHO server: HTTP test server accepting GET/POST requests
a user agent like a browser or cURL
Save the form to a minimal .html file:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8"/>
<title>upload</title>
</head>
<body>
<form action="http://localhost:8000" method="post" enctype="multipart/form-data">
<p><input type="text" name="text1" value="text default">
<p><input type="text" name="text2" value="aωb">
<p><input type="file" name="file1">
<p><input type="file" name="file2">
<p><input type="file" name="file3">
<p><button type="submit">Submit</button>
</form>
</body>
</html>
We set the default text value to aωb, which means aωb because ω is U+03C9, which are the bytes 61 CF 89 62 in UTF-8.
Create files to upload:
echo 'Content of a.txt.' > a.txt
echo '<!DOCTYPE html><title>Content of a.html.</title>' > a.html
# Binary file containing 4 bytes: 'a', 1, 2 and 'b'.
printf 'a\xCF\x89b' > binary
Run our little echo server:
while true; do printf '' | nc -l localhost 8000; done
Open the HTML on your browser, select the files and click on submit and check the terminal.
nc prints the request received.
Tested on: Ubuntu 14.04.3, nc BSD 1.105, Firefox 40.
multipart/form-data
Firefox sent:
POST / HTTP/1.1
[[ Less interesting headers ... ]]
Content-Type: multipart/form-data; boundary=---------------------------735323031399963166993862150
Content-Length: 834
-----------------------------735323031399963166993862150
Content-Disposition: form-data; name="text1"
text default
-----------------------------735323031399963166993862150
Content-Disposition: form-data; name="text2"
aωb
-----------------------------735323031399963166993862150
Content-Disposition: form-data; name="file1"; filename="a.txt"
Content-Type: text/plain
Content of a.txt.
-----------------------------735323031399963166993862150
Content-Disposition: form-data; name="file2"; filename="a.html"
Content-Type: text/html
<!DOCTYPE html><title>Content of a.html.</title>
-----------------------------735323031399963166993862150
Content-Disposition: form-data; name="file3"; filename="binary"
Content-Type: application/octet-stream
aωb
-----------------------------735323031399963166993862150--
For the binary file and text field, the bytes 61 CF 89 62 (aωb in UTF-8) are sent literally. You could verify that with nc -l localhost 8000 | hd, which says that the bytes:
61 CF 89 62
were sent (61 == 'a' and 62 == 'b').
Therefore it is clear that:
Content-Type: multipart/form-data; boundary=---------------------------735323031399963166993862150 sets the content type to multipart/form-data and says that the fields are separated by the given boundary string.
But note that the:
boundary=---------------------------735323031399963166993862150
has two less dashes -- than the actual barrier
-----------------------------735323031399963166993862150
This is because the standard requires the boundary to start with two dashes --. The other dashes appear to be just how Firefox chose to implement the arbitrary boundary. RFC 7578 clearly mentions that those two leading dashes -- are required:
4.1. "Boundary" Parameter of multipart/form-data
As with other multipart types, the parts are delimited with a
boundary delimiter, constructed using CRLF, "--", and the value of
the "boundary" parameter.
every field gets some sub headers before its data: Content-Disposition: form-data;, the field name, the filename, followed by the data.
The server reads the data until the next boundary string. The browser must choose a boundary that will not appear in any of the fields, so this is why the boundary may vary between requests.
Because we have the unique boundary, no encoding of the data is necessary: binary data is sent as is.
TODO: what is the optimal boundary size (log(N) I bet), and name / running time of the algorithm that finds it? Asked at: https://cs.stackexchange.com/questions/39687/find-the-shortest-sequence-that-is-not-a-sub-sequence-of-a-set-of-sequences
Content-Type is automatically determined by the browser.
How it is determined exactly was asked at: How is mime type of an uploaded file determined by browser?
application/x-www-form-urlencoded
Now change the enctype to application/x-www-form-urlencoded, reload the browser, and resubmit.
Firefox sent:
POST / HTTP/1.1
[[ Less interesting headers ... ]]
Content-Type: application/x-www-form-urlencoded
Content-Length: 51
text1=text+default&text2=a%CF%89b&file1=a.txt&file2=a.html&file3=binary
Clearly the file data was not sent, only the basenames. So this cannot be used for files.
As for the text field, we see that usual printable characters like a and b were sent in one byte, while non-printable ones like 0xCF and 0x89 took up 3 bytes each: %CF%89!
Comparison
File uploads often contain lots of non-printable characters (e.g. images), while text forms almost never do.
From the examples we have seen that:
multipart/form-data: adds a few bytes of boundary overhead to the message, and must spend some time calculating it, but sends each byte in one byte.
application/x-www-form-urlencoded: has a single byte boundary per field (&), but adds a linear overhead factor of 3x for every non-printable character.
Therefore, even if we could send files with application/x-www-form-urlencoded, we wouldn't want to, because it is so inefficient.
But for printable characters found in text fields, it does not matter and generates less overhead, so we just use it.
enctype='multipart/form-data is an encoding type that allows files to be sent through a POST. Quite simply, without this encoding the files cannot be sent through POST.
If you want to allow a user to upload a file via a form, you must use this enctype.
When submitting a form, you tell your browser to send, via the HTTP protocol, a message on the network, properly enveloped in a TCP/IP protocol message structure. An HTML page has a way to send data to the server: by using <form>s.
When a form is submitted, an HTTP Request is created and sent to the server, the message will contain the field names in the form and the values filled in by the user. This transmission can happen with POST or GET HTTP methods.
POST tells your browser to build an HTTP message and put all content in the body of the message (a very useful way of doing things, more safe and also flexible).
GET will submit the form data in the querystring. It has some constraints about data representation and length.
Stating how to send your form to the server
Attribute enctype has sense only when using POST method. When specified, it instructs the browser to send the form by encoding its content in a specific way. From MDN - Form enctype:
When the value of the method attribute is post, enctype is the MIME
type of content that is used to submit the form to the server.
application/x-www-form-urlencoded: This is the default. When the form is sent, all names and values are collected and URL Encoding is performed on the final string.
multipart/form-data: Characters are NOT encoded. This is important when the form has a file upload control. You want to send the file binary and this ensures that bitstream is not altered.
text/plain: Spaces get converted, but no more encoding is performed.
Security
When submitting forms, some security concerns can arise as stated in RFC 7578 Section 7: Multipart form data - Security considerations:
All form-processing software should treat user supplied form-data
with sensitivity, as it often contains confidential or personally
identifying information. There is widespread use of form "auto-fill"
features in web browsers; these might be used to trick users to
unknowingly send confidential information when completing otherwise
innocuous tasks. multipart/form-data does not supply any features
for checking integrity, ensuring confidentiality, avoiding user
confusion, or other security features; those concerns must be
addressed by the form-filling and form-data-interpreting applications.
Applications that receive forms and process them must be careful
not to supply data back to the requesting form-processing site that
was not intended to be sent.
It is important when interpreting the filename of the Content-
Disposition header field to not inadvertently overwrite files in the
recipient's file space.
This concerns you if you are a developer and your server will process forms submitted by users which might end up containing sensitive information.
enctype='multipart/form-data' means that no characters will be encoded. that is why this type is used while uploading files to server.
So multipart/form-data is used when a form requires binary data, like the contents of a file, to be uploaded
Set the method attribute to POST because file content can't be put inside a URL parameter using a form.
Set the value of enctype to multipart/form-data because the data will be split into multiple parts, one for each file plus one for the text of the form body that may be sent with them.
enctype(ENCode TYPE) attribute specifies how the form-data should be encoded when submitting it to the server.
multipart/form-data is one of the value of enctype attribute, which is used in form element that have a file upload. multi-part means form data divides into multiple parts and send to server.
Usually this is when you have a POST form which needs to take a file upload as data... this will tell the server how it will encode the data transferred, in such case it won't get encoded because it will just transfer and upload the files to the server, Like for example when uploading an image or a pdf
The enctype attribute specifies how the form-data should be encoded when submitting it to the server.
The enctype attribute can be used only if method="post".
No characters are encoded. This value is required when you are using forms that have a file upload control
From W3Schools
I generate this in a view:
<form method="post">
<input type="hidden"
name="test"
value="<?=htmlentities('<>"&ščé', ENT_QUOTES, 'UTF-8')?>">
<input type="submit>
</form>
Now, should I do this when processing data from the form?
$decodedTest = html_entity_decode($_POST['test'], ENT_QUOTES, 'UTF-8');
I think that this should be allright:
$decodedTest = $_POST['test'];
But I have not found a reference to this.
EDIT: I had printed the posted value of test and I had seen that the value is not encoded. What I don't know is If I can rely on this behaviour and why. I am asking about theory of operation. If I look into the raw post request, I can see that the post data is urlencoded (which is I guess a different type of encoding than htmlentities does). Does that mean that client must perform some recoding before sending the request. Does (client) browser store input values in encoded form or decoded form in memory before sending? (I already know that php automatically decodes urlencoded data in requests so that part is fairly clear to me).
You don't really need a reference because printing htmlspecialchars($_POST['test']) (or just setting Content-Type: text/plain) will immediately reveal that the data inside $_POST is not entity-encoded.
You also don't need to call htmlentities to encode the data in the view -- htmlspecialchars will suffice if your aim is to generate valid markup.
Sending the form, you can do it. Better check
I'm trying to understand the meaning of raw POST data. The PHP manual page for $HTTP_RAW_POST_DATA just state that this variable contains Raw POST data.
When will this variable be set and what's the meaning of raw POST data?
I understand the $_POST, but I am totally confused with $HTTP_RAW_POST_DATA.
An HTTP request consists of two parts. A set of headers and a body.
The headers include things like the URL being requested and caching control helpers (such as "I have a version of this from yesterday, only give me a new one if there are changes, OK?").
The body may or may not appear depending on the type of request. POST requests have bodies.
The body can be in any format the client likes. One of the headers will tell the server what the format is.
There are a couple of formats used by HTML forms, and PHP knows how to parse these and put the data into $_POST.
If the data is in another format, such as JSON, or if the data doesn't conform to PHP's quirks (such as the rules for having [] on the end of keys with the same name) then you might want to access the data directly so you can parse it yourself.
That is the raw POST data.
$_POST can be said to be an outcome after splitting the $HTTP_RAW_POST_DATA variable. PHP splits the raw post data and formats it in the way we see it in the $_POST array. For example:
$HTTP_RAW_POST_DATA looks something like this
key1=value1&key2=value2
then $_POST would look like this:
$_POST = array(
"key1" => "value1",
"key2" => "value2",);
HTTP is a text-based protocol, so all the data is passed as a strings. When you work with $_POST - you already have the passed data processed for you to be in an array form. This is done by PHP automatically right before the control is passed to your script.
So in the raw POST data there is data as it was passed through the network.
Likely you see a=1&b=2 data, as you see it in URLs.
One of things you can do with the HTTP protocol is do a POST request which sends some text back to the server.
$HTTP_RAW_POST_DATA will contain that text, no matter what it says.
Most of the time when we do a POST request, we will be adding a "content-type" to the text that is sent. This tells the server what sort of content it is. Most of the time on the web we are sending content type as 'application/x-www-form-urlencoded'.
When a server receives a POST request with this content type marker, the server will know to try to turn the data into a $_POST array so that "test=hello" becomes:
$_POST['test']='hello'
$HTTP_RAW_POST_DATA contains the raw POST data like in the following formats:
text
JSON
XML
HTML
In general, php://input should be used instead of $HTTP_RAW_POST_DATA. because this feature has been DEPRECATED as of PHP 5.6.0. Relying on this feature is highly discouraged.
Source: php.net - $HTTP_RAW_POST_DATA
When I use
<form method="post" enctype="text/plain" action="proc.php">
form data can not be sent to proc.php file properly. Why? What is the problem? Why I can't use text/plain encoding with post but I can use it with get method?
[Revised]
The answer is, because PHP doesn't handle it (and it is not a bug):
https://bugs.php.net/bug.php?id=33741
Valid values for enctype in <form> tag are:
application/x-www-form-urlencoded
multipart/form-data
The first is the default, the second one you need only when you upload files.
#Alohci provided explanation why PHP doesn't populate $_POST array, but store the value inside a variable $HTTP_RAW_POST_DATA.
Example of what can go wrong with text/plain enctype:
file1.php:
<form method="post" enctype="text/plain" action="file2.php">
<textarea name="input1">abc
input2=def</textarea>
<input name="input2" value="ghi" />
<input type="submit">
</form>
file2.php:
<?php
print($HTTP_RAW_POST_DATA);
?>
Result:
input1=abc
input2=def
input2=ghi
No way to distinguish what is the value of input1 and input2 variables. It can be
input1=abc\r\ninput2=def, input2=ghi, as well as
input1=abc, input2=def\r\ninput2=ghi
No such problem when using the other two encodings mentioned before.
The difference between GET and POST:
in GET, the variables are part of URL and are present in URL as query string, therefore they must be URL-encoded (and they are, even if you write enctype="text/plain" - it just gets ignored by the browser; you can test it using Wireshark to sniff the request packets),
when sending POST, the variables are not part of URL, but are sent as the last header in HTTP request (POSTDATA), and you can choose whether you want to send them as text/plain or application/x-www-form-urlencoded, but the second one is the only non-ambiguous solution.
HTML5 does define how to format form data submitted as text/plain here: https://w3c.github.io/html/sec-forms.html#plain-text-form-data.
At the bottom of that section, it says:
Payloads using the text/plain format are intended to be human
readable. They are not reliably interpretable by computer, as the
format is ambiguous (for example, there is no way to distinguish a
literal newline in a value from the newline at the end of the value).
So it not unreasonable that PHP does not attempt to interpret it and only makes it available in raw form. To me, that seems the correct approach.
I was building a simple web based calculator which takes equations from a HTML form, evaluates it on the server using PHP and sends the result back.
I am using Mootools to send the data via the req.send AJAX operation.
But, each time I have a '+' in an equation, it is not seen on the POST data the server gets.
Any ideas why this is happening and how I can work around it?
eg:
10 + 12 in HTML form is seen as 10 12 in the $_POST data.
The Mootools send command I am using is something like this with
<textarea name="equationTextArea">10+12</textarea>
req.send("eqn="+$('equationTextArea').value);
Upon submit, I see $_REQUEST['eqn'] as 10 12.
Try using the function encodeURIComponent over your text value. It, well.. uri encodes your text.
Set your form's encoding to multipart/form-data - this is an alternative to the default application/x-www-form-urlencoded and doesn't encode a space into a plus sign +.
Example from the w3.org reference:
<form action="http://example.com/cgi/handle"
enctype="multipart/form-data"
method="post">
Your text most likely either need to be URLEncoded.