Not able to parse HTML content using PHP because of special characters - php

Question

I'm trying to scrape a website by using CURL. So far I have written the following:
Curl class:
<?php
class Curl
{
public $cookieJar = "";
public function __construct($cookieJarFile = 'cookies.txt') {
$this->cookieJar = $cookieJarFile;
}
function setup()
{
$header = array();
$header[0] = "Accept: text/xml,application/xml,application/xhtml+xml,";
$header[0] .= "text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
$header[] = "Cache-Control: max-age=0";
$header[] = "Connection: keep-alive";
$header[] = "Keep-Alive: 300";
$header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
$header[] = "Accept-Language: en-us,en;q=0.5";
$header[] = "Pragma: "; // browsers keep this blank.
curl_setopt($this->curl, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7');
curl_setopt($this->curl, CURLOPT_HTTPHEADER, $header);
curl_setopt($this->curl,CURLOPT_COOKIEJAR, $cookieJar);
curl_setopt($this->curl,CURLOPT_COOKIEFILE, $cookieJar);
curl_setopt($this->curl,CURLOPT_AUTOREFERER, true);
curl_setopt($this->curl,CURLOPT_FOLLOWLOCATION, true);
curl_setopt($this->curl,CURLOPT_RETURNTRANSFER, true);
}
function get($url)
{
$this->curl = curl_init($url);
$this->setup();
return $this->request();
}
function getAll($reg,$str)
{
preg_match_all($reg,$str,$matches);
return $matches[1];
}
function postForm($url, $fields, $referer='')
{
$this->curl = curl_init($url);
$this->setup();
curl_setopt($this->curl, CURLOPT_URL, $url);
curl_setopt($this->curl, CURLOPT_POST, 1);
curl_setopt($this->curl, CURLOPT_REFERER, $referer);
curl_setopt($this->curl, CURLOPT_POSTFIELDS, $fields);
return $this->request();
}
function getInfo($info)
{
$info = ($info == 'lasturl') ? curl_getinfo($this->curl, CURLINFO_EFFECTIVE_URL) : curl_getinfo($this->curl, $info);
return $info;
}
function request()
{
return curl_exec($this->curl);
}
}
?>
And then I'm calling this curl class in my php file:
include_once("curl.php");
$curl = new Curl();
$html = $curl->get("www.somewebsite.com");
$html = htmlentities($html);
//echo $html;
$pattern = htmlentities("<span class=\"review-text\">");
function get_string_between($string, $start, $end)
{
$string = " ".$string;
$ini = strpos($string,$start);
if ($ini == 0)
return "";
$ini += strlen($start);
$len = strpos($string,$end,$ini) - $ini;
return substr($string,$ini,$len);
}
echo get_string_between($html, '<span class=\"review-text\">', '<\/span>');
Now that I'm trying to get the string between the two strings, I'm getting a blank page. However when I see the html content, I am clearly able to spot the string.
The HTML content is very big and I'm trying search and get the content between the huge file.
I even tried replacing the "<" symbol with "<" sign but it does not seem to find the string.

Answer 1

'There is a better way to get the value of a html tag, by using the dom.
$dom = new DomDocument();
#$dom -> loadHTML($html);
$dom -> preserveWhiteSpace = false;
$spans = getElementsByTagName('span');
foreach($spans as $span){
if($span -> getAttribute('class') == 'review-text'){ print $span-> nodeValue }
}
Or there is another way:
$dompath = new DOMXPath($dom);
$review_div = $dompath -> query('//*[#class="review-text"]')->item(0)
$string = $review_div -> nodeValue;
Hope this helps you.