Why is curl links redirecting through localhost? - php

Question

Right now, I have the current php code:
<?php
include('simple_html_dom.php');
# set up the request parameters
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, 'https://www.google.com/search?q=sport+news');
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curl, CURLOPT_MAXREDIRS, 0);
$result = curl_exec($curl);
curl_close($curl);
echo $result;
?>
when this code is run, it returns a google page with the search results corresponding to the search sport news. Although, when you try to click on any one of theese links, it redirects you to 'localhost:/--url--'. How do I prevent curl from redirecting to localhost and rather redirect to the actual site?
I am currently using wampserver for testing.

Answer 1

This happens because Google's result page is using relative URLs in the links.
<a href="/url?q=https://www.bbc.co.uk/sport/43634915&sa=U&ved=2ahUKEwjX (...)
Notice that the href starts with: / not with a domain such as href="https://foobar.com/url?q=.
Therefore, the links will use the hostname of the page serving the results.
Te reason you get localhost in the results when clicking them, is that you are serving this code from localhost.
One solution could be to use the DOMDocument PHP extension to parse links, and add a hostname, so that the result links are absolute, rather than relative.
For example:
// Ignore HTML errors
libxml_use_internal_errors(true);
// Instantiate parser
$dom = new DOMDocument;
// Load HTML into DOM document parser
$dom->loadXML($result);
// Select anchor tags
$books = $dom->getElementsByTagName('a');
// Iterate through all links
foreach ($links as $link) {
// Get relative link value
$relativePath = $link->getAttribute('href');
// Check if this is a relative link
if (substr($relativePath, 0, 1) === '/') {
// Prepend Google domain
$link->setAttribute('href', "https://google.com/" . $relativePath);
}
}
echo $dom->saveHTML();