What are this Weird Characters on my Ajax result 138d on the start and 0 on the end of my ajax result. how to disable this?
138d
{"feeds":[{"pubdate":"Sun, 28 Nov 2021 23:00:00 EST"]}
0
Your HTTP client is buggy. It improperly extracted the body from the HTTP response.
There are three ways to signaling the end of an HTTP response:
Using a Content-Length header.
Using the chunked transfer coding.
Closing the socket.
The first two methods allow more than one request-response exchange to occur over the same connection, which is far more efficient than creating a new connection for each request. (Especially for HTTPS.)
The Content-Length header is quite simple to use, but it requires knowing the size of the message body before starting to send it. The chunked transfer coding doesn't have that limitation. It allows servers to start sending a response before its size becomes known.
An HTTP response that uses the chunked transfer coding might look like this:
HTTP/1.1 200 OK␍␊
Transfer-Encoding: chunked␍␊
Content-Type: application/json␍␊
␍␊
28␍␊
..............................␍␊
28␍␊
..............................␊␍␊
0␍␊
The message body of that response is
............................................................␊
The response that returned your JSON apparently used the chunked transfer coding. And whatever you used to parse the HTTP response incorrectly returned the still-chunked message body. This is a bug. Quote RFC2616,
All HTTP/1.1 applications MUST be able to receive and decode the
"chunked" transfer-coding
Related
I have a symfony2 application where I am using the Guzzle http client to send a GET request to a server in order to retrieve the contents of a json file. The Guzzle response gets transformed into a Symfony2 response to the browser.
The Guzzle response comes back with the following headers:
Content-Encoding: gzip
Content-Length: 2255
Content-Type: application/json
When outputting the data to the UI/browser I notice that it gets cut off because the Content-Length is incorrect. The size of the file is closer to 4905 bytes, not 2255. 2255 is the exact length of the data up to the cut-off point. I suspect that the 2255 is the size of the gzipped data and it gets uncompressed at some point without updating the content-length. Now I did verify that I get all of the data back, however the content-length header is honored which is why the data gets cut off when I forward it to the browser. Interestingly, hitting the url to the json file directly yields the full contents even though the content-length is 2255 which means it gets ignored by Chrome when hitting the file directly. Same if I use the POSTman REST client to make the GET request - full contents get displayed.
By default, Guzzle has a request option decode_content = true for how the responses should be handled. I set it to false when submitting the request but that didn't seem to resolve the issue.
Before converting the Guzzle response to a Symfony response I removed the content-length header and that seems to solve the problem however I am not sure that's the best approach since RFC protocol states that a content-length header should be present unless a transfer-encoding header is present, which it isn't. https://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html
Another alternative is, since this is a streamed response, to get the size of the stream and correct the content-length, however the Guzzle implementation uses strlen() for this which has the undesirable affect of reading the whole stream.
What possible issues might I run into if I choose to omit the content-length header? And alternatively, is there a way to get the TRUE length of the contents without reading the whole stream and simply update the content-length header with the correct amount?
I have this HTTP response content :
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Transfer-Encoding: chunked
Date: Mon, 12 Aug 2013 15:08:10 GMT
PK�Ctemps_attente.json���n� �߅9�Bw���VU��Uߠs���^��#�CGç��ͷ�r7G�3Hnp����^pYSu\#Qo%~x��FGa�Y�ا����S���-ua���t��j-���s�%э��+,g�xq.��������t�fb� �0:)�:�K�}^�N�L����>�щ%#�̲x`C#��m݃ :^��$~�i8���WzCh�a�ă���7t�O|��AX˂��UO$���<��y"�;�'F��]��{֘Ha}F��<��l6��o벰V���66t�&��f�Ť��x�H��툗���/PKA�Y�1�PK�CA�Y�1�temps_attente.jsonPK#q
I would like to know what format is the response and how to decompile to have the final response.
I tried to use this function: http_chunked_decode but I did not succeed.
The body (or at least what appears to be the body) of the response is not chunked.
It does appear to be compressed - with HTTP this should be expressly stated in the headers,
There is no blank line between the what appears to be the headers and what appears to be the body.
If this is really the response you are getting it's not HTTP - an off-the-shelf function is not going to make sense of it.
A centain web client that I need to support, is sending back the Cookies header to my application twice in the HTTP headers, this in turn is making PHP unable to read the correct value for the cookie thus ignoring the session.
Here is the relevant part of the request I am seeing:
GET / HTTP/1.1
Cache-Control: max-age=0
Accept-Language: en-US
Cookie: PHPSESSID=49af82ddf12740e6a35b15985e93d91a
Connection: Keep-Alive
Cookie: PHPSESSID=49af82ddf12740e6a35b15985e93d91a
[...] Other irrelevant headers
I have two questions:
Is that a PHP bug? or is the behavior undefined when the client sends that same header twice?
Is there a quick workaround to make things work without having to manually parse the HTTP headers so I can read the right value of the cookie (and session) in my application? Or should I manually parse the HTTP header to set the session to its correct value?
According to the HTTP spec, a double header simply concatenates the values together with a comma, making it:
Cookie: PHPSESSID=49af82ddf12740e6a35b15985e93d91a, PHPSESSID=49af82ddf12740e6a35b15985e93d91a
PHP should be able to parse the cookies, but the behavior of sessions is undefined when there are two session IDs.
I strongly recommend fixing the client. If that's not an option, you'll have to parse the headers manually.
I'm working on carddav client. As server i use davical v. 0.9.9.6. I don't understand why i'm getting invalid content-type error when http headers contains correct value. I look into source code and found this condition:
if ( isset($_SERVER['CONTENT_LENGTH']) && $_SERVER['CONTENT_LENGTH'] > 7) {...
After little research I found php set $_SERVER['CONTENT_LENGTH'] only with POST method and uploading file. Is there any way to configure php to always set $_SERVER['CONTENT_LENGTH']?
I'm asking generally, not only for this case...
//EDIT
I'm doing HTTP PUT request to davical server (using php curl).
PUT /caldav.php/testuser/contacts/newc.vcf HTTP/1.1
Host: davical
Content-Type: text/vcard;
BEGIN:VCARD
VERSION:3.0
FN:ME
...
On davical side is condition testing CONTENT_LENGTH which is not set. So it's a davical bug?
//EDIT 2
Finally I figure it out!
PUT request with calback readfunc requires set INFILE_SIZE via curl_setopt(...)
There is none auto value and put Content-Length field manualy into header is also wrong.
Example (incorrect):
// PUT REQUEST
curl_setopt($ch,CURLOPT_HTTPHEADER,"Content-Length: $length"); //mistake
curl_setopt($ch,CURLOPT_PUT,true);
curl_setopt($ch,CURLOPT_READFUNCTION,array($this,'readfunc'));
....
--------------------------------------------------------------
// WIRESHARK TCP STREAM DUMP
PUT /caldav.php/testuser/contacts/novy.vcf HTTP/1.1
Authorization: Basic xxxxxxxxxxxxxxx
Host: davical
Accept: */*
Content-Type: text/vcard
Content-Length: xxx
Expect: 100-continue
HTTP/1.1 100 Continue
155
BEGIN:VCARD
VERSION:3.0
...
END:VCARD
0
HTTP/1.1 200 OK
----------------------------------------------------------------
// On server side
isset($_SERVER['CONTENT_LENGTH'])==false
Second (correct) example
// PUT REQUEST
curl_setopt($ch,CURLOPT_INFILESIZE,$length);
curl_setopt($ch,CURLOPT_PUT,true);
curl_setopt($ch,CURLOPT_READFUNCTION,array($this,'readfunc'));
....
--------------------------------------------------------------
// WIRESHARK TCP STREAM DUMP
PUT /caldav.php/testuser/contacts/novy.vcf HTTP/1.1
Authorization: Basic xxxxxxxxxxxxxxx
Host: davical
Accept: */*
Content-Type: text/vcard
Content-Length: xxx
Expect: 100-continue
HTTP/1.1 100 Continue
BEGIN:VCARD
VERSION:3.0
...
END:VCARD
HTTP/1.1 200 OK
----------------------------------------------------------------
// On server side
isset($_SERVER['CONTENT_LENGTH'])==true
Although i have never used CONTENT_LENGHT i can tell you why this is probably happening:
In a request, you don't have to set the Content-Lenght header... IT IS NOT MANDATORY. Except for specific situations. If your POSTed content is of type "multipart/form-data" it becomes necessary to use content-lenght for each part because each part is seperated by a boundary and each part will have its own headers...
For example:
Content-Type: MultiPart/Form-Data
Boundary: #FGJ4823024562DGGRT3455
MyData=1&Username=Blabla&Password=Blue
#FGJ4823024562DGGRT3455==
Content-Type: image/jpef:base64
Content-Lenght: 256
HNSIFRTGNOHVDFNSIAH$5346twSADVni56hntgsIGHFNR$Iasdf==
So here this is a crude example of what a multi part request works, you see that the second part has a content-lenght. This is why sometimes the content-lenght is set and sometimes not, because you need to read X bytes before finding another boundary and extract the correct data.
It doesn't mean your server will never send it in in other cases, but my 2 cents are this is the case right now. Its because you are not in POST, but in some other modes.
Only requests that have a request body have a content length request header (or at least only then it makes sense) and so therefore the $_SERVER variable is set.
If you need it to be always set (which I think is bogus), you can do this yourself on the very beginning of your script:
isset($_SERVER['CONTENT_LENGTH']) && $_SERVER['CONTENT_LENGTH'] = 0;
Assuming that if it is not set, it's of zero length. See as well Improved handling of HTTP requests in PHP.
You could probably set them by yourself. Why do you need that this values are set? And what should they set to?
Maybe you're missing information on $_SERVER['CONTENT_TYPE'] or
$_SERVER['CONTENT_LENGTH'] as I did. On POST-requests these are
available in addition to those listed above.
-> http://www.php.net/manual/en/reserved.variables.server.php#86495
Attempting to submit a form with CURL, both via PHP and the command line. The response from the server consists of null content (the headers posted below).
When the same URL is submitted via a browser, the response consists of a proper webapge.
Have tried submitting the CURL request parameters via POST and GET via each of the following command line curl flags "-d" "-F" and "-G".
If the query string parameters are posted with "-d" flag, resulting header is:
HTTP/1.1 302 Moved Temporarily
Date: Thu, 02 Jun 2011 21:41:54 GMT
Server: Apache
Set-Cookie: JSESSIONID=DC5F435A96A353289F58593D54B89570; Path=/XXXXXXX
P3P: CP="CAO PSA OUR"
Location: http://www.XXXXXXXX.com/
Content-Length: 0
Connection: close
Content-Type: text/html;charset=UTF-8
Set-Cookie: XXXXXXXXXXXXXXXX=1318103232.20480.0000; path=/
If the query string parameters are posted with "-F" flag, the resulting header is:
HTTP/1.1 100 Continue
HTTP/1.1 500 Internal Server Error
Date: Thu, 02 Jun 2011 21:52:54 GMT
Server: Apache
Content-Length: 1677
Connection: close
Content-Type: text/html;charset=utf-8
Set-Cookie: XXXXXXXXXXXXXX=1318103232.20480.0000; path=/
Vary: Accept-Encoding
<html><head><title>Apache Tomcat/5.5.26 - Error report</title><style><!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}HR {color : #525D76;}--></style> </head><body><h1>HTTP Status 500 - </h1><HR size="1" noshade="noshade"><p><b>type</b> Exception report</p><p><b>message</b> <u></u></p><p><b>description</b> <u>The server encountered an internal error () that prevented it from fulfilling this request.</u></p><p><b>exception</b> <pre>javax.servlet.ServletException: Servlet execution threw an exception<br>
</pre></p><p><b>root cause</b> <pre>java.lang.NoClassDefFoundError: com/oreilly/servlet/multipart/MultipartParser<br>
com.corsis.tuesday.servlet.mp.MPRequest.<init>(MPRequest.java:27)<br>
com.corsis.tuesday.servlet.mp.MPRequest.<init>(MPRequest.java:21)<br>
com.corsis.tuesday.servlet.TuesdayServlet.doPost(TuesdayServlet.java:494)<br>
javax.servlet.http.HttpServlet.service(HttpServlet.java:710)<br>
javax.servlet.http.HttpServlet.service(HttpServlet.java:803)<br>
</pre></p><p><b>note</b> <u>The full stack trace of the root cause is available in the Apache Tomcat/5.5.26 logs.</u></p><HR size="1" noshade="noshade"><h3>Apache Tomcat/5.5.26</h3></body></html>
Questions:
What might cause a server to respond different depending on the nature of the CURL request.
How to successfully submit request via CURL?
HTTP/1.1 100 Continue
I had problems associated with this header before. Some servers simply do not understand it. Try this option to override Expect header.
curl_setopt( $curl_handle, CURLOPT_HTTPHEADER, array( 'Expect:' ) );
To add to what Richard said, I have seen cases where servers check the User-Agent string and behave differently based on its value.
I have just had an experience with this and what fixed it was surprising. In my situation I was logging into a server so I could upload a file, have the server do work on it, and then download the new file. I did this in Chrome first and used the dev tools to capture over 100 HTTP requests in this simple transaction. Most are simply grabbing resources I don't need if I am trying to do all of this from the command line, so I filtered out only the ones I knew at a minimum I should need.
Initially this boiled down to a GET to set the cookie and log in with a username and password, a POST to upload the file, a POST to execute the work on the file, and a GET to retrieve the new file. I could not get the first POST to actually work though. The response from that POST is supposed to be information containing the upload ID, time uploaded, etc, but instead I was getting empty JSON lists even though the status was 200 OK.
I used CURL to spoof the requests from the browser exactly (copying the User-Agent, overriding Expect, etc) and was still getting nothing. Then I started arbitrarily adding in some of the requests that I captured from Chrome between the first GET and POST, and low and behold after adding in a GET request for the JSON history before the POST the POST actually returned what it was supposed to.
TL;DR Some websites require more requests after the initial log in before you can POST. I would try to capture a successful exchange between the server and browser and look at all of the requests. Some requests might not be as superfluous as the seem.