php curl with digest returns two responses - php

Question

I have spotted a "weird" php CURL behavior that is sending me nuts. Basically what I am doing is making a digest authenticated call with curl. Here's an extract of my code:
curl_setopt($this->c, CURLOPT_HTTPAUTH, CURLAUTH_DIGEST);
curl_setopt($this->c, CURLOPT_USERPWD, $username . ":" . $password);
It works fine and the server actually comes back with a "YES, YOU PROVIDED THE RIGHT CREDENTIALS" kind of message. Only trouble is, the raw http response is a bit odd as it includes, as a matter of fact, 2 responses instead of one. Here's what curl_exec($this->c) spits out:
HTTP/1.0 401 Unauthorized
Date: Tue, 23 Oct 2012 08:41:18 GMT
Server: Apache/2.2.20 (Ubuntu)
X-Powered-By: PHP/5.3.6-13ubuntu3.9
WWW-Authenticate: Digest realm="dynamikrest-testing",qop="auth",nonce="5086582e95104",opaque="4b24e95490812b28b3bf139f9fbc9a66"
Vary: Accept-Encoding
Content-Length: 9
Connection: close
Content-Type: text/html
HTTP/1.1 200 OK
Date: Tue, 23 Oct 2012 08:41:18 GMT
Server: Apache/2.2.20 (Ubuntu)
X-Powered-By: PHP/5.3.6-13ubuntu3.9
Vary: Accept-Encoding
Content-Length: 9
Connection: close
Content-Type: text/html
"success"
I don't get why it includes the first response from the server (the one in which it states it requires authentication).
Can anyone throw some light on the issue? How do I avoid the responses' cumulation?
Cheers

Answer 1

It looks like curl has the same behavior if you use the -I option for headers:
curl -I --digest -u root:somepassword http://localhost/digest-test/
returns:
HTTP/1.1 401 Authorization Required
Date: Fri, 31 May 2013 13:48:35 GMT
Server: Apache/2.2.22 (Ubuntu)
WWW-Authenticate: Digest realm="Test Page", nonce="9RUL3wPeBAA=52ef6531dcdd1de61f239ed6dd234a3288d81701", algorithm=MD5, domain="/digest-test/ http://localhost", qop="auth"
Vary: Accept-Encoding
Content-Type: text/html; charset=iso-8859-1
HTTP/1.1 200 OK
Date: Fri, 31 May 2013 13:48:35 GMT
Server: Apache/2.2.22 (Ubuntu)
Authentication-Info: rspauth="4f5f8237e9760f777255f6618c21df4c", cnonce="MTQ3NDk1", nc=00000001, qop=auth
Vary: Accept-Encoding
Content-Type: text/html;charset=UTF-8
X-Pad: avoid browser bug
To only get the second header you could try this (not very optimal solution):
<?php
$ch = curl_init();
// set url
curl_setopt($ch, CURLOPT_URL, "http://localhost/digest-test/");
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_DIGEST);
curl_setopt($ch, CURLOPT_USERPWD, "root:test");
// first authentication with a head request
curl_setopt($ch, CURLOPT_NOBODY, 1);
curl_exec($ch);
// the get the real output
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_HTTPGET, 1);
$output = curl_exec($ch);
echo $output;

Answer 2

I hit the same problem, and I think it was caused by PHP being compiled against an ancient version of libcurl (7.11.0 in my case, which is now nearly 10 years old). On a different machine with a more recent version of libcurl (7.29.0) the same code was fine, and my problems ended after getting my host to recompile their PHP to use the latest they had available (7.30.0).
This fix was suggested by a thread on the curl-library mailing list from 2008, where a user discovered the problem affected version 7.10.6 but not 7.12.1. I've searched the libcurl changelog around 7.12.0 and failed to find any clear entry about fixing this problem, though it might be covered by "general HTTP authentication improvements". Still, I'm now pretty confident that an old libcurl is the problem.
You can check which version of libcurl is used by your PHP from the 'cURL Information' entry in the output of phpinfo();