How if at all can I strip all the header from a PHP response through apache in order to just stream the text response. I've tried adding a custom htaccess but to no avail. I've got limited control of the hosting server. The stream is read by an embedded device which doesn't need any headers.
It get's to a point where certain headers are NEEDED to be interpreted by the browser so it can render the output. If the reason why you want to remove the header is for a chat-like feature, think about using a persitant keep-alive connection
Tips in reducing bandwidth
Use ajax: keep the response from PHP in JSON format and update DOM elements
Gzip.
Just don't worry about headers -- typically a HTTP OK response will only take up < 200 bytes, hardly anything in comparison to the actual page content. Focus on where it really matters.
Edit:
To suit your case look into using sockets (UDP would be a good option if wanting to cut back on a lot of bandwidth) socket_listen() (non UDP) or socket_bind() capabable of UDP
That's impossible.
You are using HTTP protocol and HTTP protocol response always contains headers.
Either do not use HTTP or teach your device to strip headers. It's not that hard.
Anyway, php has very little to do with removing headers. There is also a web-server that actually interacts with your device and taught to send proper headers.
There is a PHP function called header_remove(). I never used it before but you can try if this works for you. Note that this function is available since PHP 5.3.0.
Related
Sometimes when using multiple concurrent connections and scraping with cURL in my PHP script, incomplete webpages are returned. Is there some value in curl_getinfo() that will let me know if a webpage was 100% fetched vs. only 90% fetched?
Would the content-size header of a returned page be the actual size of what was returned or would it be the entire page? If so, I could check the content-size against the actual size of the response..
Thanks!
Assuming your question is whether you can check if the content size header comes from the other side or is calculated on your side, yes, you can use that header to check if you've received the full response because it is generated on the other side from the actually intended content. A few things, though:
It's Content-Length, not Content-Size;
you can use it as long as you trust the other party to set it correctly;
it may not be available because while it SHOULD exist, it is not strictly necessary.
This is typical scenario: a page is evaluated, and there's a buffer - once the buffer is full, the part of the page that is evaluated is sent to the browser. This uses the HTTP 1.1 chunked encoding.
However, an error can occur in one of the chunks (after the first one is already sent). In that case:
you can't redirect (send a Location header), because the headers and the response status were already sent
you can't do server-side redirect (forward), because the new page will have to be rendered after the part that is already sent - it will look ugly for sure.
So what should you do in this case? I asked a question whether you can send a Location header in the chunked trailer, but this is low-level http and the abstraction of languages may not allow it, even if it is possible (and it is likely not to be supported across browsers)
Another option is to send a <script>window.href.location="errorPage"</script> and thus force the client to redirect, but that's ugly. Plus you have to put </script> to close any potential unclosed <script> tag in the partial page.
(I'm tagging major web languages/frameworks, because this is an universal issue)
You cannot redirect from the server in a chunked encoding because the headers have already been sent. The only way to perform a redirect is to send a <script> tag from the server and perform a client side redirect. Just out of curiosity are you trying to implement a COMET server? If this is the case HTML5 WebSockets seem better way (if the browsers you are targeting support them of course) compared to the hidden iframe technique.
I know want to know what happens behind the scene of a HTTP post method.
i.e browser sends a HTTP post request to a server side script in PHP (eg).
How does PHP's $_POST variable get the values from the client.
Could someone explain in details or point to a guide.
The HTTP protocol(*) specifies how the browser should send the request.
HTTP basically consists of a set of headers in plain text, separated by line feeds, followed by the data being transmitted. Inside the HTTP request, POST data is actually formatted pretty much the same as GET data; it's just in a different part of the HTTP headers.
You can use tools like Firebug or Fiddler to see exactly how the headers and data are formatted for incoming and outgoing HTTP requests. It's actually all quite simple to read, so you should be able to work it out just by looking.
Once it gets to the server, the PHP interpreter is responsible for translating the raw HTTP request data into its standard $_GET, $_POST, etc variables. This is something that PHP does for you.
Other languages (eg Perl) do not have this functionality built in, so a Perl programmer would have to have code in their program to parse the incoming request data into useful variables. Fortunately, even Perl has a standard library which can be included that does the job, so even Perl programmers don't generally have to write the code themselves any more.
The way PHP, and any other language, does it is simply string manipulation. As I said, the HTTP data is plain text and is received in simple string format, so it's just a case of breaking it down by splitting it on question mark and equal sign characters.
As PHP does it all behind the scenes, you probably don't need to worry about the exact mechanisms it uses, but the PHP source code is available if you really want to find out.
I said it's all in plain text. HTTPS, of course, is encrypted. However by the time PHP gets hold of it, the Apache server has already done the decryption, so as far as PHP is concerned it's still plain text.
(*) Before anyone pulls me up on it, yes, I know that saying "HTTP protocol" is a redundancy, like "ATM machine" or "PIN number".
The browser encodes the data according to the content-type of the form, then transmits it as the body of a POST request. PHP then picks it up and populates $_POST with the names and values (performing special handling when the name includes the characters [ and ] or .).
I'd suggest to get a capturing proxy (e.g. Fiddler) or a network capture tool (e.g. Wireshark) and watch your own browsing traffic for a while; it will give you a nice view of the issue.
Other than that, POST is rather similar to GET, except that the data is sent in the body of the request instead of the URL, and there are two ways to encode them (multipart-form-data in addition to the urlencode that's shared with GET)
Well, let's ilustrate step by step, starting with a page containing a [form action="foo.php" method="post"]
Once you click submit (or hit enter), browser will trigger an event named "submit". This event can be catched internally for processing with javascript/dom, and this is what most sites do for validation or Ajax routines.
If routines does not stop the flow with a return false, browser continues to process the post request (this process is the same as making a post with XMLHttpRequest Object).
Browser will check first method, action and content encoding, then parse inputs values to know the size of data it will send, and encode it.
Finally it send something like this (raw values):
POST /foo.php HTTP/1.1
Host: example.org
Content-Type: application/x-www-form-urlencoded
Content-Length: 7
foo=bar
This is a POST request. But note that it can send content-length and send variables in chunks. Browser and server know this can happen (this is the POST method purpose). When a server receives a POST request, it keeps listening to the browser until the content received match the informed content length.
Now the other side. Server receives the request, listen the content, parse it (foo = bar; xxx = baz), and make it available on its environment for that specific request, thus you can catch it with PHP or Python, or Java...
That's it. Ah note you can pass both GET and POST variables in the same request!
Using a [form action="foo.php?someVar=123&anotherVar=TRUE" method="post"]
Will make the browser send the request as
POST /foo.php?someVar=123&anotherVar=TRUE HTTP/1.1
Host: example.org
Content-Type: application/x-www-form-urlencoded
Content-Length: 7
foo=bar
And server when parsing this request will make the following variables available:
GET[someVar] = 123
GET[anotherVar] = TRUE
POST[foo] = bar
I have to ensure that curl is sending cookies propertly, so I should view full text of request (not response!) http headers. Is there any way to do it?
Thank you
Obviously, you need to intercept the request headers at some point. Considering you want to see headers from curl, I would suggest setting up a transparent proxy. For a relatively simpler solution, I would suggest a tool such as WireShark.
I know it's possible, but I can't seem to figure it out.
I have a mySQL query that has a couple hundred thousand results. I want to be able to send the results, but it seems the response requires content-length header to start downloading.
In phpMyAdmin, if you go to export a database, it starts the download right away, FF just says unknown file size, but it works. I looked at that code, but couldn't figure it out.
Help?
Thanks.
this document in section 14.14 states that In HTTP, it SHOULD be sent whenever the message's length can be determined prior to being transferred, unless this is prohibited by the rules in section 4.4. This means it DOESN'T have to be sent if you can't say the size of it.
Just don't send it.
If you want to send parts of data to the browser before all data is available, do you flush your output buffer? Maybe that's what is the problem, not lack of a header?
The way you use flush is like that:
generate some output, which should add it to the buffer
flush() it, which should send current buffer to the client
goto 1
So, if your query returns a lot of results, you could just generate the output for, lets say, 100 or 1000 of them, then flush, and so on.
Also, to tell the client browser to attempt to save a file instead of displaying it in window, you can try using the Content-disposition: attachment header as well. See the specification here, 19.5.1 section.
You can use chunked transfer-encoding. Basically, you send a "Transfer-encoding: chunked" header, and then the data is sent in chunked mode, meaning that you send the length of a chunk followed by the chunk. You keep repeating this until the end of data, at which point you send a zero-length chunk.
Details are in RFC 2616.
It’s possible that you are using gzip which waits for all content to be generated. Check your .htaccess for directives regarding this.
You don’t need to set a Content-Length header field. This header field is just to tell the client amount of data it has to expected. In fact, a “wrong” value can cause that the client discards some data.
If you use the LiveHTTPHeaders plugin in FireFox, you can see the difference between the headers being sent by phpMyAdmin and the headers being sent by your application. I don't know specifically what you're missing, but this should give you a hint. One header that I see running a quick test is "Transfer-Encoding: chunked", and I also see that they're not sending content-length. Here's a link to the FireFox plugin if you don't already have it:
LiveHTTPHeaders
http content length is a SHOULD field, so you can drop it...
but you have to set transfer encoding then
have a look at Why "Content-Length: 0" in POST requests?