cURL freeze Apache and time out on a 404 error - php

Question

I have a really strange issue with a cURL request. Basically I have some code that I use to wrap a legacy PHP app on Symfony2 (http://symfonybricks.com/it/brick/wrap-legacy-php-code-by-a-symfony2-application). They are basic cURL commands that should pull a webpage and display it. However, I have noticed the following :
If I make a query to an existing file, it's working fine
If I make a query to a file that doesn't exist, cURL time out (Because I have set up a time out of 8sec), and during that time, it freeze the whole server, I can't access my website from any other devices until it actually time out.
Here's my code :
/**
* #Route("/", defaults={"controller" = "index.php", "controller2" = "", "controller3" = ""})
* #Route("/{controller}", defaults={"controller2" = "index.php", "controller3" = ""})
* #Route("/{controller}/", defaults={"controller2" = "index.php", "controller3" = ""})
* #Route("/{controller}/{controller2}", defaults={"controller3" = ""})
* #Route("/{controller}/{controller2}/", defaults={"controller3" = ""})
* #Route("/{controller}/{controller2}/{controller3}", defaults={"controller3" = "index.php"})
*/
public function getLegacyResourceAction($controller, $controller2, $controller3, Request $request)
{
$path_to_legacy_code = "http://XXX/";
$originalController = $request->getPathInfo();
$originalQueryString = $request->getQueryString();
$url = "{$path_to_legacy_code}{$originalController}?{$originalQueryString}";
//open connection
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
//Timeout
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 8);
curl_setopt($ch, CURLOPT_TIMEOUT, 8);
curl_setopt($ch, CURLOPT_MAXREDIRS, 5);
$stderr = fopen("{$this->container->getParameter('kernel.root_dir')}/logs/curl.txt", "a");
curl_setopt($ch, CURLOPT_STDERR, $stderr);
//echo "Login stuff in ".$this->container->getParameter('kernel.root_dir')."/logs/curl.txt";
curl_setopt($ch, CURLOPT_COOKIESESSION, 0);
curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookies.txt');
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookies.txt');
$result = curl_exec($ch);
if (false === $result) {
echo curl_error($ch);
exit;
}
curl_close($ch);
fclose($stderr);
$Response = new Response($result);
return $Response;
Here's the log I'm getting from a valid response :
* About to connect() to www.xxx.com port 80 (#0)
* Trying 178.32.223.113...
* connected
* Connected to www.xxx.com (178.32.223.113) port 80 (#0)
> GET /web/legacy/index.php HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:40.0) Gecko/20100101 Firefo$
Host: www.xxx.com
Accept: */*
Cookie: idto=116; PHPSESSID=a159rcjvh0fk6otukqqq9bkrd5
* additional stuff not fine transfer.c:1037: 0 0
* HTTP 1.1 or later with persistent connection, pipelining supported
< HTTP/1.1 200 OK
< Date: Sat, 05 Sep 2015 04:26:32 GMT
< Server: Apache/2.2.22 (Debian)
< X-Powered-By: PHP/5.5.28-1~dotdeb+7.1
< Expires: Thu, 19 Nov 1981 08:52:00 GMT
< Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
< Pragma: no-cache
< Vary: Accept-Encoding
< Transfer-Encoding: chunked
< Content-Type: text/html
<
* Connection #0 to host www.xxx.com left intact
* Closing connection #0
And here's the code from an invalid response :
* About to connect() to www.xxx.com port 80 (#0)
* Trying 178.32.223.113...
* connected
* Connected to www.xxx.com (178.32.223.113) port 80 (#0)
> GET /web/legacy/whatever.php HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:40.0) Gecko/20100101 Firefo$
Host: www.xxx.com
Accept: */*
Cookie: idto=116; PHPSESSID=a159rcjvh0fk6otukqqq9bkrd5
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* Operation timed out after 8001 milliseconds with 0 bytes received
* Closing connection #0
* About to connect() to www.xxx.com port 80 (#0)
* Trying 178.32.223.113...
* connected
* Connected to www.xxx.com (178.32.223.113) port 80 (#0)
> GET /web/legacy/legacy/whatever.php HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:40.0) Gecko/20100101 Firefo$
Host: www.xxx.com
Accept: */*
Cookie: idto=116; PHPSESSID=a159rcjvh0fk6otukqqq9bkrd5
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* Operation timed out after 8001 milliseconds with 0 bytes received
* Closing connection #0
* About to connect() to www.xxx.com port 80 (#0)
* Trying 178.32.223.113...
* connected
* Connected to www.xxx.com (178.32.223.113) port 80 (#0)
> GET /web/legacy/legacy/legacy/whatever.php HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:40.0) Gecko/20100101 Firefo$
Host: www.xxx.com
Accept: */*
Everything that's bellow curl_exec($ch); is not executed until it time out, not giving me a chance to check for a 404 error or something. I've been searching for days, no luck so far.
Thanks a lot !
EDIT :
Okay, so I solved the issue by removing this line :
curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookies.txt');
Now it give a proper 404 error. I have no idea why it's working without this line. The problem is, without this line I'm breaking some session stuff from my legacy code !
EDIT 2 :
I've changed :
curl_setopt($ch, CURLOPT_COOKIESESSION, 0);
to
curl_setopt($ch, CURLOPT_COOKIESESSION, 1);
And now it's working fine with 404 error and my session. I have NO IDEA why.
EDIT 3 :
Okay, I thing I finally understand what's going on :
On every cURL request, it write on the cookie.txt file
If someone query a 404 url, before the time out, it file lock the cookie.txt file, and that's why the whole thing seems to be frozen.
As of my understanding, the best solution would be to generate a different cookie.txt file for each user, preventing a file lock.

Answer 1

I finally got to the bottom of this, turns out I had two issues :
No timeout configured, so if I'd try to query a non-existing webpage, it would loop indefinitely.
I was using the same cookies file for everyone, and if someone was stuck in such loop as mentionned before, it would file lock the cookie file, preventing other users to access the site.