Making cURL cookies work in successives curl_exec connections - php

Question

I'm crawling a web with cURL and DOM PHP. The web has a products sections where you can go page by page viewing all the products and also you have subsections for more concise searching, in each page 9 products are listed.
I need to store the information of the subsection to witch the product belongs. I start with all the subsections URL's and the program above shows how I try to get the next 9 products page of a subsection.
The problem it's that the web makes redirects with some information that I suppose it's on a cookie because there is not post traces in the network.
For example: In the ALL PRODUCTS section the URL of the second page is like:
www.example.com/product/?n=2
The first page of any subsection has a unique URL like:
www.example.com/product/subsection
The problem is that the link to the next subsection page (next 9 products) is
www.example.com/product/?n=2
The URL it's THE SAME as the all product section but it shows the subsection products.
The problem it's that I get the ALL PRODUCTS page instead of the SUBSECTION page.
I have tried with cookies but I don't get distinct results. Any suggestion?
<?php
private ckfile;
public function main()
{
$this->ckfile = tempnam ("C:/Web/", "CURLCOOKIE");
$copy = $this->get_page();
$next_visit = $this->link_next($copy);
while($next_visit != false){//it's not last page
$copy = $this->get_page($next_visit,$get_name($next_visit));
$next_visit = $this->link_next($copy);
}
}
public function get_page($URL = "http://www.example.com" , $nombre = "example" )
{
$ch = curl_init();
$options = array(
CURLOPT_HTTPHEADER => array("Accept-Language: es-es,en"),
CURLOPT_USERAGENT => "Googlebot/2.1 (+http://www.google.com/bot.html)",
CURLOPT_AUTOREFERER => true, // set referer on redirect ,
CURLOPT_ENCODING => "", //allow all encodings
CURLOPT_FOLLOWLOCATION => true, // follow redirects
CURLOPT_HEADER => false,
CURLOPT_CONNECTTIMEOUT => 120, // timeout on connect
CURLOPT_TIMEOUT => 120, // timeout on response
CURLOPT_MAXREDIRS => 10, // stop after 10 redirects
CURLOPT_COOKIEFILE => $this->ckfile,
CURLOPT_COOKIEJAR => $this->ckfile,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_URL => $URL
);
curl_setopt_array($ch, $options);
$g = 'C:/Web/'.$nombre.'.html';
if(!is_file($g)){
$fp=fopen ($g, "w");
curl_setopt ($ch,CURLOPT_FILE, $fp);
$trash = curl_exec ($ch); // don't browse them
fclose($fp);
}
curl_close ($ch);
return $g;
}
public function link_next($value)
{
# function that searches the DOM for a link and returns a well formed URL
# or returns false if doesn't find one( last page)
}
?>

Answer 1

To make multiple calls you want to use curl multi:
$ch = curl_multi_init();
Not
$ch = curl_init();
See this post for an example Multiple PHP cUrl posts to same page