simple_http_dom - how to send request to each url - php

Question

I have got 16 links in my get-listing.php output and I need to send the request to each URL to get the responses, for which I need to receive the list of elements when send the request to each URL.
$base1 = "http://testbox.elementfx.com/get-listing.php";
$html = file_get_html($base1);
$links = $html->find('p[id=links] a');
foreach ($links as $element)
{
//open each url in each array
$urls[] = $url = $element->href;
$data = file_get_html($url);
}
When I use the code as above, it will only send the request to each url to get the response which I have got 9 responses. I should have more than 9 responses.
Can you please tell me how I can send request to every url to get the responses using with simple_http_dom?

Answer 1

If your question is to send a simple request to each of the urls you've already parsed and get a response back, try file_get_contents:
foreach ($links as $element)
{
// This array stack is only necessary if you plan on using it later
$urls[] = $url = $element->href;
// $opts and $context are optional for specifying options like method
$opts = array(
'http'=>array(
'method'=>"GET", // "GET" or "POST"
)
);
$context = stream_context_create($opts);
// Remove context argument if not using options array
$data = file_get_contents($url, false, $context);
// ... Do something with $data
}
Your other option is more complex but has more flexibility (depending on the application) is Curl:
foreach ($links as $element)
{
$urls[] = $url = $element->href;
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
// Used for GET request
curl_setopt($ch, CURLOPT_POSTFIELDS, null);
curl_setopt($ch, CURLOPT_POST, FALSE);
curl_setopt($ch, CURLOPT_HTTPGET, TRUE);
// Necessary to return data
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$data = curl_exec($ch);
curl_close($ch);
// ... Do something with $data
}
This is scratching the surface of Curl, the documentation on PHP's website (linked above) has more information.
If the data being returned from the urls is HTML, you can pass it through PHP's DomDocument to parse after it has been pulled. Documentation and examples are readily available on PHP's site (I can't post more links right now, sorry).