"no acceptable variant" from MultiViews in Apache - php

Question

In one deployment of a PHP-based application, Apache's MultiViews option is being used to hide the .php extension of a request dispatcher script. E.g. a request to
/page/about
...would be handled by
/page.php
...with the trailing part of the request URI available in PATH_INFO.
Most of the time this works fine, but occasionally results in errors like
[error] [client 86.x.x.x] no acceptable variant: /path/to/document/root/page
My question is: What triggers this error occasionally, and how can I fix the problem?

Answer 1

Short Answer
This error can occur when all the following are simultaneously true:
Your webserver has Multiviews enabled
You are allowing Multiviews to serve PHP files by assigning them an arbitrary type with the AddType directive, most likely with a line like this:
AddType application/x-httpd-php .php
Your client's browser sends with requests an Accept header that does not include */* as an acceptable MIME type (this is highly unusual, which is why you see the error only rarely).
You have your MultiviewsMatch directive set to its default of NegotiatedOnly.
You can resolve the error by adding the following incantation to your Apache config:
<Files "*.php">
MultiviewsMatch Any
</Files>
Explanation
Understanding what is going on here requires getting at least a superficial overview of the workings of Apache's mod_negotiation and HTTP's Accept and Accept-Foo headers. Prior to hitting the bug described by the OP, I knew nothing about either of these; I had mod_negotiation enabled not by deliberate choice but because that's how apt-get set up Apache for me, and I had enabled MultiViews without much understanding of the implications of that besides that it would let me leave .php off the end of my URLs. Your circumstances may be similar or identical.
So here are some important fundamentals that I didn't know:
request headers like Accept and Accept-Language let the client specify what MIME types or languages it is acceptable for them to receive the response in, as well as specifying weighted preferences for the acceptable types or languages. (Naturally, these are only useful if the server has, or is capable of generating, different responses based upon these headers.) For example, Chromium sends off the following headers for me whenever I load a page:
Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-GB,en-US;q=0.8,en;q=0.6
Apache's mod_negotiation lets you store multiple files like myresource.html.en, myresource.html.fr, myresource.pdf.en and myresource.pdf.fr in the same folder and then automatically use the request's Accept-* headers to decide which to serve when the client sends a request to myresource. There are two ways of doing this. The first is to create a Type Map file in the same folder that explicitly declares the MIME Type and language for each of the available documents. The other is Multiviews.
When Multiviews are enabled...
Multiviews
... If the server receives a request for /some/dir/foo and /some/dir/foo does not exist, then the server reads the directory looking for all files named foo.*, and effectively fakes up a type map which names all those files, assigning them the same media types and content-encodings it would have if the client had asked for one of them by name. It then chooses the best match to the client's requirements, and returns that document.
The important thing to note here is that the Accept header is still being respected by Apache even with Multiviews enabled; the only difference from the type map approach is that Apache is inferring the MIME types of files from their file extensions rather than through you explicitly declaring it in a type map.
The no acceptable variant error is thrown (and a 406 response sent) by Apache when there exist files for the URL it has received, but it's not allowed to serve any of them because their MIME types don't match any of the possibilities provided in the request's Accept header. (The same thing can happen if there is, for example, no variant in an acceptable language.) This is compliant with the HTTP spec, which states:
If an Accept header field is present, and if the server cannot send a response which is acceptable according to the combined Accept field value, then the server SHOULD send a 406 (not acceptable) response.
You can test this behaviour easily enough. Just create a file called test.html containing the string "Hello World" in the webroot of an Apache server with Multiviews enabled and then try to request it with an Accept header that permits HTML responses versus one that doesn't. I demonstrate this here on my local (Ubuntu) machine with curl:
$ curl --header "Accept: text/html" localhost/test
Hello World
$ curl --header "Accept: image/png" localhost/test
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>406 Not Acceptable</title>
</head><body>
<h1>Not Acceptable</h1>
<p>An appropriate representation of the requested resource /test could not be found on this server.</p>
Available variants:
<ul>
<li>test.html , type text/html</li>
</ul>
<hr>
<address>Apache/2.4.6 (Ubuntu) Server at localhost Port 80</address>
</body></html>
This brings us to a question that we haven't yet addressed: how does mod_negotiate determine the MIME type of a PHP file when deciding whether it can serve it? Since the file is going to be executed, and could spit out any Content-Type header it likes, the type isn't known prior to execution.
Well, by default, the answer is that MultiViews simply won't serve .php files. But chances are that you followed the advice of one of the many, many posts on the internet (I get 4 on the first page if I Google 'php apache multiviews', the top one clearly being the one the OP of this question followed, since he actually commented upon it) advocating getting around this using an AddType directive, probably looking something like this:
AddType application/x-httpd-php .php
Huh? Why does this magically cause Apache to be happy to serve .php files? Surely browsers aren't including application/x-httpd-php as one of the types they'll accept in their Accept headers?
Well, not exactly. But all the major ones do include */* (thus permitting a response of any MIME type - they're using the Accept header only for expressing preference weighting, not for restricting the types they'll accept.) This causes mod_negotiation to be willing to select and serve .php files as long as some MIME type - any at all! - is associated with them.
For example, if I just type a URL into the address bar in Chromium or Firefox, the Accept header the browser sends is, in the case of Chromium...
Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
... and in the case of Firefox:
Accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Both of these headers contain */* as an acceptable content type, and thus permit the server to serve a file of any content type it likes. But some less popular browsers don't accept */* - or perhaps only include it for page requests, not when loading the content of a <script> or <img> tag that you might also be serving through PHP - and that's where our problem comes from.
If you check the user agents of the requests that result in 406 errors, you'll likely see that they're from relatively unusual user agents. When I experienced this error, it was when I had the src of an <img> element pointing to a PHP script that dynamically served images (with the .php extension omitted from the URL), and I first witnessed it failing for BlackBerry users:
Mozilla/5.0 (BlackBerry; U; BlackBerry 9320; fr) AppleWebKit/534.11+ (KHTML, like Gecko) Version/7.1.0.714 Mobile Safari/534.11+
To get around this, we need to let mod_negotiate serve PHP scripts via some means other than giving them an arbitrary type and then relying upon the browser to send an Accept: */* header. To do this, we use the MultiviewsMatch directive to specify that multiviews can serve PHP files regardless of whether they match the request's Accept header. The default option is NegotiatedOnly:
The NegotiatedOnly option provides that every extension following the base name must correlate to a recognized mod_mime extension for content negotiation, e.g. Charset, Content-Type, Language, or Encoding. This is the strictest implementation with the fewest unexpected side effects, and is the default behavior.
But we can get what we want with the Any option:
You may finally allow Any extensions to match, even if mod_mime doesn't recognize the extension.
To restrict this rule change only to .php files, we use a <Files> directive, like this:
<Files "*.php">
MultiviewsMatch Any
</Files>
And with that tiny (but difficult-to-figure-out) change, we're done!

Answer 2

The answer given by Mark Amery is almost complete, however it is missing the sweet spot and does not address the 'no extension is given in the request thus negotiation fails with alternatives.
You can resolve this error by adding the follwing config-snippets:
Your PHP config should be something like this:
<FilesMatch "\.ph(p3?|tml)$">
SetHandler application/x-httpd-php
</FilesMatch>
Do NOT use AddType application/x-httpd-php .php or any other AddType
And your additional config should be like this:
RemoveType .php
<Files "*.php">
MultiviewsMatch Any
</Files>
If you do use AddType you will get errors like this:
GET /index/123/434 HTTP/1.1
Host: test.net
Accept: image/*
HTTP/1.1 406 Not Acceptable
Date: Tue, 15 Jul 2014 13:08:27 GMT
Server: Apache
Alternates: {"index.php" 1 {type application/x-httpd-php}}
Vary: Accept-Encoding
Content-Length: 427
Connection: close
Content-Type: text/html; charset=iso-8859-1
As you can see, it does find index.php, however it does not use this alternative as it cannot match the Accept: image/* to application/x-httpd-php. If you request /index.php/1/2/3/4 it works fine.
The reason for this I found in the source code of the mod_negotiation module. I was trying to find out why Apache would work if the .php type was 'cgi' but not otherwise (hint: application/x-httpd-cgi is hardcoded..). While in the source i noticed that apache would only see the file as a match if the Content-Type of that file matched the Accept header, or if the Content-Type of that file was empty.
If you use the SetHandler than apache won't see the .php files as application/x-httpd-php, but unfortunatly, many distro's also define this in the /etc/mime.types file. So to be sure, just add the RemoveType .php to your config if this bug is bothering you.