How to reduce load time of curl and parse html - php

Question

i need to reduce load time of my script. It is curl and simple parse dom.
This is my script, i need help :(
It lasts about 2 minutes, i need to parse many different pages!
require_once ("simple_html_dom.php");
function curl ($page){
ob_start();
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "URL");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, "POSTFIELDS");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$result = curl_exec($ch);
curl_close($ch);
ob_end_clean();
return $result;
}
$start = microtime(true);
set_time_limit(0);
$text = "here text of last page";
$i = 0;
while(strpos(str_get_html(curl($i)), $text) == null){
$html = str_get_html(curl($i));
foreach($html->find('div#box-container-inner div.box') as $e){
PRINT etc... only for test
}
echo "parsata la pagina ".($i+1)."<br>";
$i++;
}
$time_elapsed_secs = (microtime(true) - $start)/60;
echo $time_elapsed_secs;

Answer 1

You appear to be running CURL twice in each loop (once to evaluate the while-loop condition, once to set $html) and converting the resulting string into an object for each loop. That's four potentially-intensive processes each time you loop that you can knock down to two per loop.
Instead, you can assign the $html variable to the result of str_get_html(curl($i)) while within the while-loop evaluation:
while(strpos(($html = str_get_html(curl($i))), $text) === false) {
// $html = str_get_html(curl($i));

Answer 2

Just add curl timeout as parameter set to 60 seconds or your choice.
curl_setopt($ch, CURLOPT_TIMEOUT, 60);