I just discovered an oddity in PHP's header() method silently converting some of my statuses to 500. Since I had no luck in finding mention of this behavior in various web searches, I'm adding this here in the hope of saving others some aggravation, but also to ask if anyone has discovered a better workaround (either with PHP or Zend1) than I've come up with.
Given a simple PHP script like:
<?php
header('HTTP/1.1 429');
echo "Too Many Requests\n";
I would expect to receive something like:
HTTP/1.1 429
Date: Thu, 18 Jul 2013 22:19:45 GMT
Content-Length: 11
Content-Type: text/html; charset=UTF-8
Too Many Requests
Instead, it actually returns:
HTTP/1.1 500 Internal Server Error
Date: Thu, 18 Jul 2013 22:19:45 GMT
Content-Length: 11
Content-Type: text/html; charset=UTF-8
Too Many Requests
Adding to the mystery, there are no events in my apache error log, and the access log shows the correct status code (thus different from what got sent to the browser):
$IP - - [18/Jul/2013:16:31:34 -0700] "GET /test/429.php HTTP/1.1" 429 11 "-" "curl/7.30.0"
Everything works fine when testing with many other status codes like 401, 420, 426.
Everything also works fine if I am explicit and send header('HTTP/1.1 429 Too Many Requests'); This would be a useful workaround except that I'm using Zend Framework and its setHttpResponseCode method expects an integer, which it uses as the third parameter to php's header() function.
I've since discovered that it seems to apply specifically to statuses added in RFC 6585 (see https://github.com/php/php-src/pull/274), though I'm a little confused why statuses like 426 work when they're clearly not present in the source code for 5.4.14 and 5.4.16 (the two versions I've tested against) but non-functional ones like 429 are.
Update:
As answers have indicated, this is mostly an Apache issue, not PHP, I've updated the title accordingly. Most interesting seems to be that this is fixed only in certain versions of Apache (with no apparent consistency between old and new). I believe the upstream issue in question is here: https://issues.apache.org/bugzilla/show_bug.cgi?id=44995
It's Apache, 99% sure, I can't find it direcly in it's docs, but I can infer it from the test below (Apache version 2.2.22)
Add this in your config:
ErrorDocument 429 Aaargh to heavy
Restart:
$ sudo /etc/init.d/apache2 restart
Syntax error on line 6 of /etc/apache2/conf.d/localized-error-pages:
Unsupported HTTP response code 429
Action 'configtest' failed.
The Apache error log may have more information.
...fail!
429 also seems a recent addition in rfc6585, status: proposed, date: April 2012. One year old for HTTP RFCs is... just a baby in my experience. Add to that the process of getting it in Apache, and then in your package repositories... Well, you could try Apache 2.4...
It's perhaps your SAPI configuraiton. Last time I tested something similar, the conclusion looked like this:
<?php
header('HTTP/ 429 Too Many Requests', false, 429);
echo "Too Many Requests\n";
Which in your case works still well for me (Apache 2.2 / FCGI / Windows):
>curl -i "http://local.example.com/header-test.php"
HTTP/1.1 429 Too Many Requests
Date: Thu, 18 Jul 2013 23:49:09 GMT
Server: Apache/2.2.22 (Win32) mod_fcgid/2.3.6
X-Powered-By: PHP/5.4.13
Transfer-Encoding: chunked
Content-Type: text/html
Too Many Requests
Related
We have upgraded our hosting platform with latest tech stack which includes PHP updates from Version 7.0 to 7.3 and enabled SSL certification.
After the upgrade, one of our user authentication method has failed though, it was working till the hosting platform upgrade.
Here is copy of PHP code - codecheck.php,
<html>
<body>
<?php
$header = "Content-Type: application/json";
header($header);
$code = $_GET["code"];
$codelistFile = "./codelist.txt";
$codeList = file( $codelistFile, FILE_SKIP_EMPTY_LINES);
$codelistOutput = sprintf('%s%s', $code, "\r\n" );
file_put_contents( $codelistFile, $codelistOutput, FILE_APPEND);
?>
</body>
</html>
Here is result of codelist.txt before the platform upgrade (with PHP version 7.0)
65cafead50f6d205d66f90c74f1683344ca86c8cc60fc0370c278ecb880da5c8
6e85e436538335da64f6e9172bd4191686e591aa390cca69acb9346668a48bd5
Here is result of codelist.txt after the platform upgrade (with PHP version 7.3)
774cad9dd07761fe79db8baa9370a3dd84abca558c73c1f46b39e7c996a26d70?code=774cad9dd07761fe79db8baa9370a3dd84abca558c73c1f46b39e7c996a26d70
f10bb27fb82b0d539d3607012655012764c60794cc656aa6912eccc16d927a82?code=f10bb27fb82b0d539d3607012655012764c60794cc656aa6912eccc16d927a82
Here is value of code repeated along with 'code' text itself hence the value of 'code' does not match when it compared.
Here is what I can see in ssl_access log files, ssl_access.log-20190629:79.1.200.79 - - [29/Jun/2019:07:46:24 +0100] "GET /codelist.php?code=ae21250db8b20cac3b7016e6d36a63de5846d537f032ed841a3e5c9121202cf4?code=ae21250db8b20cac3b7016e6d36a63de5846d537f032ed841a3e5c9121202cf4 HTTP/1.1" 200 19 "-" "Registration"
From this log file, I can see all GET requests to server appending the data twice.
I would expect it would be something like,
example.com/?code=123456789
but not as
example.com/?code=123456789?code=123456789
I am very new to PHP and HTTPS stuff, please help to figure out the issue. Thank you.
Here is an update:
As suggested, the issue seems to be more with SSL re-writing,
Here is code from desktop app where the app will connect and check the code with the server,
C++:
CString RegistrationServer::Uri( CString page, CString code )
{
CString sServer;
sServer.Format("http://www.mywebsite,com/%s?code=%s", page, code);
//Here page=codecheck.php and code = 10;
return sServer;
}
Here is log when submitted through desktop app,
27.62.66.34 - - [30/Jun/2019:21:55:51 +0100] "GET /codecheck.php?code=10?code=10 HTTP/1.1" 200 - "-" "Hack-o-Matic ver 0.01"
I can simulate the same request through web browser as below,
https://www.mywebsite/codecheck.php?code=10
Here is log when submitted through web browser,
27.62.66.34 - - [30/Jun/2019:21:46:28 +0100] "GET /codecheck.php?code=10 HTTP/1.1" 200 - "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36"
You can see the difference in both the request is http vs https.
When the request is coming from desktop app, the code data is appended twice which uses http.
It appears that changing desktop app to have https will help fix the issue but that's something that we can't do anything with desktop app.
So we have to relay on fix from Server side but our hosting company doesn't seem to understand the problem exactly.
They keep analysing the issue since last 3 days and coming up some fixes like googleapi call fixes but that's not helping to fixing up our real issue.
I'm not sure if I'm missing some better phrases/terms to explain this issue to them better. Please let me know if there is better way to explain the issue to our hosting company.
If nothing working out, Can I ask them to remove SSL certification?
Another Update:
Here is response from our hosting company,
We have this referred to our engineers and they confirmed that this only happens when calling http and not https. You need to use https now since you have enabled SSL.
Latitude-E6540:~$ curl -I http://www.mywebsite.com/codecheck.php?code=10
HTTP/1.1 301 Moved Permanently
Server: nginx/1.15.8
Date: Mon, 01 Jul 2019 11:03:47 GMT
Content-Type: text/html
Content-Length: 169
Connection: keep-alive
Location: https://www.mywebsite.com/codecheck.php?code=10?code=10
Strict-Transport-Security: max-age=15768000
Our engineers made some tests and they were not able to replicate when they set to https.
Latitude-E6540:~$ curl -I https://www.mywebsite.com/codecheck.php?code=10
HTTP/1.1 200 OK
Server: nginx/1.15.8
Date: Mon, 01 Jul 2019 11:03:35 GMT
Content-Type: application/json
Connection: keep-alive
Strict-Transport-Security: max-age=15768000
Here is log from server,
213.171.217.184 - - [01/Jul/2019:12:03:35 +0100] "HEAD /usage7.php?code=10 HTTP/1.1" 200 - "-" "curl/7.58.0"
They confirmed that this looks to be something with your local software settings as this only seems to get in the case of "after submitting the requests through browser, HTTP GET data is not appended twice but when the same is submitted through their desktop software, the HTTP GET data is appended twice"
What I wanted to ask you is, from below curl output itself where I can see the code is appended twice when request is made with http, Does this having any clue to spot where the issue resides?
Location: https://www.mywebsite/codecheck.php?code=10?code=10
How to solve PHP upgrade errors:
Post-event, how to find, diagnose and fix errors apparently caused by PHP updates?
1) Check your scripts for PHP Errors.
2) Check changes to your php.ini file caused by updates.Depending on your system and upgrade method, the php.ini file may be adjusted or even a new default one. Read the Migration Notes to see if this may apply to you. You will need to review and explore what's changed. Also manually compare your reserved/backup php.ini with the current/new live one.
3) Read the PHP Migration notes for each version you have upgraded into and then out of(These are best done from oldest to newest).
4) Read the corresponding PHP Changelog(s) and search this text (it's loooong) for the functions you've found be failing in step (1).
For your specific instances; your code is of a very low quality (you are sending HTTP heders after you are sending HTML code) so the issue may well be caused by PHP upgrading an already existing error from E_WARNING to E_ERROR, or similar.
Low quality code is most easily fixed by turing on error_reporting(E_ALL); either in the scripts or in the php.ini and reading the resulting error logs.
Good Luck.
Update
Even with this SSL log, I can see the value for code twice and the same written to the file. I would expect it would be something like example.com/?code=123456789 but not as example.com/?code=123456789?code=123456789.
The sign you have two ? means you should be exploring the code that sets the code= value, please update your question with this information, how is code set?
Your issue may be with your HTTP Host routing, Apache, Nginx, etc., your HTTP Host is possibly double loading, first the HTTP_ page and then secondly redirecting on to the HTTPS page with the original query string appended, thus appending twice.
I think one or both of the above is where your problem lies.
Update 2:
Comment by Thi:
Here is what my hosting company responded, "as per our engineers the cause of the logs is de to the website making http (not https) calls to the google api for css and other things. They have advised that you need to ensure that any code that relates to http is switched to https." - There is below line in all of our html pages and have changed it to https but it didn't help <link href="fonts.googleapis.com/…" rel="stylesheet" type="text/css">
This relates to what I reference above about checking your server routing for HTTP and HTTPS protocols.
Solutons:
1) Update all your outgoing links to https:// (or simply //) so:
<link href="//fonts.googleapis.com/..." rel="stylesheet" type="text/css">
will always connect securely, if loaded securely.
2) Use Content Security Policy (CSP) Upgrade Insecure Requests flag to do just that; to force all http:// links within your website to be turned into https:// links by the client browser.
In your .htaccess, or equivilant file:
Content-Security-Policy: upgrade-insecure-requests;
However, insecure calls to 3rd party resources will NOT be the cause of your code block being appended to your URL twice.
I've been playing with output_buffering in php (confirmed by phpinfo()) and I just discovered that upon turning it on, I will start getting random 404 errors on my pages, but the page content loads fine and everything looks ok.
This only happens on my production site. PHP 5.5.35, Apache/2.4.18 (Unix). I have no idea where this 404 is being sent from. Does anyone have any ideas as to what might be happening, or what I can search for on my server to fix it?
These are my response headers in case that's at all relevant:
Cache-Control:s-maxage=10
Cache-Control:no-cache, must-revalidate, max-age=0
Connection:Keep-Alive
Content-Encoding:gzip
Content-Length:13021
Content-Type:text/html; charset=UTF-8
Date:Fri, 02 Feb 2018 18:31:59 GMT
Expires:Wed, 11 Jan 1984 05:00:00 GMT
Keep-Alive:timeout=5, max=96
Server:Apache
Strict-Transport-Security:max-age=31536000; includeSubDomains; preload
Vary:Accept-Encoding
X-Powered-By:PHP/5.5.35
EDIT:
I discovered that this issue only happens on pages on which I am loading 'recent posts' from a Wordpress instance. It's not really an option for me to remove this little widget but I assume there's a reason behind setting a 404 header. Any thoughts?
I think what is happening is that your code is issuing 404s in some cases regardless of whether output buffering is on or off; but when output buffering is off the header fails to be set because it has already been written generating a warning like this one:
Warning: Cannot modify header information - headers already sent by
(output started at /file.php:20) in /file.php on line 100
So you can only see the 404 when output buffering is on which enables you to set headers at any point before the buffers are written to output.
The solution doesn't work for me from other links.
I am consuming web services from android using Ksoap2.
Here is my problem I am getting this error.
Here are my server details
HTTP/1.1 500 Internal Server Error
Server: nginx
Content-Type: text/html
Content-Length: 3
Accept-Ranges: bytes
Date: Wed, 25 Nov 2015 16:01:16 GMT
X-Varnish: 1781493534
Age: 0
Via: 1.1 varnish
Connection: keep-alive
From PHP client
[message:protected] => looks like we got no XML document
From android application using Ksoap2
org.xmlpull.v1.XmlPullParserException: expected: '>' actual: '' (position:END_TAG </SOAP-ENV:Envelop>#8:19 in java.io.InputStreamReader#41f06f28)
My code and web service work fine for all the server. Except the one from Iran. Is there is any encoding issue?
Looks like your webservice is busted and the server (nginx) is throwing an error. All the other errors are just a consequence. You need to fix the webservice by figuring out what is going wrong there. Check the nginx logs and details about whatever is implementing the webservice.
The problem is not with the web service. It's working on other servers. I installed a new version of the framework (Magento for my case) on Nginx, and now my web service works fine. It may be due to some module interfacing.
Edited:
I have got the same error when I was using different store id to access the data. I was getting the same error looks like we got no XML document.
So I think there is no proper answer to the question. The answer must be how to trace the source of this error.
EDITED [26 June 2016]
This might be the possible answer if you are using Magento. This happens to me once where API throwing a warning before outputting the XML response which was causing the error.
https://stackoverflow.com/a/10679915/5028508
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
404 header - HTTP 1.0 or 1.1?
I have a simple PHP code which send 404 using the header
header($_SERVER["SERVER_PROTOCOL"]." 404 Not Found");
echo $_SERVER["SERVER_PROTOCOL"];
But when using the curl command and force 1.0, it return..
curl -0 -v 'http://www.example.com/test.php'
> GET /test.php HTTP/1.0
< HTTP/1.1 404 Not Found
< Server: nginx
< Date: Sat, 27 Oct 2012 08:51:27 GMT
< Content-Type: text/html
< Connection: close
<
* Closing connection #0
HTTP/1.0
As you can see $_SERVER["SERVER_PROTOCOL"] is 1.0 but header give me 1.1, what is the reason?
Great question.
You have to understand that this is not something specific to PHP so don't think there is a bug or something. It's specific to:
A web server's configuration
A client's (think browser) configuration
Usually clients, when sending a request to a web server, indicate which protocol they support. The web server then responds with the protocol it supports or it is configured to.
In your PHP code, when you flush that 404 header you are hinting to the server which protocol to use. You can't force it - at least I am not aware of a way to force it from PHP. The web server will ultimately settle for the highest protocol supported by both itself and the requesting client.
Unless the client forces it to, nginx will always respond over HTTP/1.1. I am not aware of a method of configuring it like Apache for example. But that is the reason for the behavior you are seeing.
Maybe you should edit your question, or open a new question, to be more specific to nginx since, like I explained this is not a problem with PHP. That way more people might be able to find the question and help out.
If you're using FastCGI (which you probably are) between nginx and PHP, you should instead use
header("Status: 404 Not Found");
as advised by PHP's documentation for header.
When i am requesting a page which is already loaded, and checking in firebug inspector
i get
Status Code:304 Not Modified
Response Headers
Cache-Control:public
Connection:Keep-Alive
Date:Tue, 23 Oct 2012 09:28:57 GMT
ETag:"200000000296d-12ca-4cca274ac8a98"
Expires:Tue, 30 Oct 2012 09:28:57 GMT
Keep-Alive:timeout=5, max=99
Server:Apache/2.2.22 (Win32) PHP/5.3.13
Vary:Accept-Encoding
My doubt is .. is there any way in which i can reduce the response headers passed over from the server to browser when a page is not modified.
like in one reference website, on inspecting with firebug i could find:
Response Headers
Date:Tue, 23 Oct 2012 09:28:56 GMT
Etag:"79ee2bbda49fcd1:0"
with 2 values. Please provide me with some helpful resource.
Correct me if i am wrong some where? Thanks for help.
This reference website is hosted on iis. and the longer response iam geting on apache.
UPDATE:
Upon googling i got to this url.
https://github.com/apache/httpd/blob/2.2.x/modules/http/http_filters.c#L1281
reading the code, tells me it is filtering header fields if its a HTTP_NOT_MODIFIED type od response. Is this module default installed in apache? or how to get this installed on my apache and get this working?
Depending on those headers your Browser "Knows what to do" with that page...
You may hide your server information "Server:Apache/2.2.22 (Win32) PHP/5.3.13".
But users browser need other lines to proccess correctly that page.