Wwriting links to database in a foreach loop fails - php

Question

I am using simple html dom to write records to a database, however it doesn't seem to write the records.
The problem is with the foreach loop. It outputs all the urls and the following error:
Notice: Undefined variable: url in C:\xampp\htdocs\meh\crawler.php on line 28
<?php
// Create connection
$con=mysqli_connect("localhost","root","spidermankillssuperman","expatriates");
// Check connection
if (mysqli_connect_errno()) {
echo "Failed to connect to MySQL: " . mysqli_connect_error();
}
?>
<?php
include_once('../simple_html_dom.php');
$links = array (
'http://www.expatriates.com/classifieds/bhr/hs/index100.html'
);
foreach ($links as $link) {
$html = file_get_html($link);
foreach($html->find('a') as $element) {
if(strpos($element->href, "cls"))
$url = "http://expatriates.com".$element->href . '<br>';
echo $url;
}
$sql="INSERT INTO urlstocrawl (url)
VALUES ('$url')";
if (!mysqli_query($con,$sql)) {
die('Error: ' . mysqli_error($con));
}
echo '<br>'.'<p>'."1 record added";
}
mysqli_close($con);
?>

Answer 1

you want your mysql query to insert foreach link in you html file, not run once after the loop. move the code inside the loop:
foreach ( $html->find('a') as $element ) {
if ( strpos($element->href, "cls") !== false ) {
$url = "http://expatriates.com" . $element->href . '<br>';
$sql = "INSERT INTO urlstocrawl (url) VALUES ('$url')";
if ( !mysqli_query($con,$sql) ) {
die('Error: ' . mysqli_error($con));
}
echo '<br>'.'<p>'."1 record added" . $url;
}
}
Note the modification to the if condition. strpos() returns either false when the string is absent or an integer indicating the position, starting at 0. This means the condition could fail if cls is at the start of the string. With this strict type checking you can be sure of the desired behavior.