I'm dealing with a full html document, and I need to extract the urls but only if matches the required domain
<html>
<div id="" class="">junk
example.com
morejunk
notexample.com
</div>
</html>
from that junky part I would need to get the full url of example.com, but not the rest (notexample.com). that would be "http://example.com/foo/bar" or even better, only the last part of that url (bar) witch of course would be different each time.
Hope I've been clear enough, thanks a lot!
Edit: using php
Regex is something you must avoid for parsing HTML like this. Here is a DOM parser based code that will get what you need:
$html = <<< EOF
<html>
<div id="" class="">junk
example.com
morejunk
notexample.com
</div>
</html>
EOF;
$doc = new DOMDocument();
libxml_use_internal_errors(true);
$doc->loadHTML($html); // loads your html
$xpath = new DOMXPath($doc);
$nodelist = $xpath->query("//a"); // gets all the links
for($i=0; $i < $nodelist->length; $i++) {
$node = $nodelist->item($i);
$val = $node->attributes->getNamedItem('href')->nodeValue;
if (preg_match('#^https?://example\.com/foo/(.*)$#', $val, $m))
echo "$m[1]\n"; // prints bar
}
Related
I have the following code to retrieve all hyper links in an HTML document
and my question is how to retrieve the text nodes inside every anchor tag
(even if the text node is a child of a child like if the anchor node has a span node which has a text node)?
<?PHP
$content = "
<html>
<head>
<title>bar , this is an example</title>
</head>
<body>
<a href='aaa'><span>bbb</span></a>
</body>
</html>
";
$dom = new DOMDocument();
#$dom->loadHTML($content);
$xpath = new DOMXPath($dom);
$row = $xpath->evaluate("/html/body//a");
for ($i = 0; $i < $row->length; $i++) {
$anchor = $row->item($i);
$href = $anchor->getAttribute('href');
// I want the grab the text value which is inside the anchor
$text = //should have the value "bbb"
}
?>
Thanks
$anchor->textContent
A slightly more info here DOMNode->textContent
:D
Heres what you can do:
(string)$anchor->nodeValue;
As referenced in the DomDocument::DomNode page
The page on another of my domains which I'd like to scrape one div from contains:
<div id="thisone">
<p>Stuff</p>
</div>
<div id="notthisone">
<p>More stuff</p>
</div>
Using this php...
<?php
$page = file_get_contents('http://thisite.org/source.html');
$doc = new DOMDocument();
$doc->loadHTML($page);
foreach ($doc->getElementsByTagName('div') as $node) {
echo $doc->saveHtml($node), PHP_EOL;
}
?>
...gives me all divs on http://thisite.org/source.html, with html. However, I only want to pull through the div with an id of "thisone" but using:
foreach ($doc->getElementById('thisone') as $node) {
doesn't bring up anything.
$doc->getElementById('thisone');// returns a single element with id this one
Try $node=$doc->getElementById('thisone'); and then print $node
On a side note, you can use phpQuery for a jquery like syntext: pq("#thisone")
$doc->getElementById('thisone') returns a single DOMElement, not an array, so you can't iterate through it
just do:
$node = $doc->getElementById('thisone');
echo $doc->saveHtml($node), PHP_EOL;
Look at PHP manual http://php.net/manual/en/domdocument.getelementbyid.php
getElementByID returns an element or NULL. Not an array and therefore you can't iterate over it.
Instead do this
<?php
$page = file_get_contents('example.html');
$doc = new DOMDocument();
$doc->loadHTML($page);
$node = $doc->getElementById('thisone');
echo $doc->saveHtml($node), PHP_EOL;
?>
On running
php edit.php you get something like this
<div id="thisone">
<p>Stuff</p>
</div>
How would I get content from HTML between h3 tags inside an element that has class pricebox? For example, the following string fragment
<!-- snip a lot of other html content -->
<div class="pricebox">
<div class="misc_info">Some misc info</div>
<h3>599.99</h3>
</div>
<!-- snip a lot of other html content -->
The catch is 599.99 has to be the first match returned, that is if the function call is
preg_match_all($regex,$string,$matches)
the 599.99 has to be in $matches[0][1] (because I use the same script to get numbers from dissimilar looking strings with different $regex - the script looks for the first match).
Try using XPath; definitely NOT RegEx.
Code :
$html = new DOMDocument();
#$html->loadHtmlFile('http://www.path.to/your_html_file_html');
$xpath = new DOMXPath( $html );
$nodes = $xpath->query("//div[#class='pricebox']/h3");
foreach ($nodes as $node)
{
echo $node->nodeValue."";
}
I have this html:
<a href=" URL TO KEEP" class="class_to_check">
<strong> TEXT TO KEEP</strong>
</a>
I have a long html code with many link as above, I have to keep the links that have the <strong> inside, I have to keep the HREF of the link and the text inside the <strong>, how can i do using DOMDocument?
Thank you!
$html = "...";
$dom = new DOMDOcument();
$dom->loadHTML($html);
$xp = new XPath($dom);
$a = $xp->query('//a')->item(0);
$href = $a->getAttribute('href');
$strong = $a->nodeValue;
Of course, this XPath stuff works for just this particular html snippet. You'll have to adjust it to work with a more fully populated HTML tree.
Until the website give me an access to his API, i need to display only 2 things from this website :
What i want to grab
// Example on a live page
Those 2 things are contained in a div :
<div style="float: right; margin: 10px;">
here what i want to display on my website
</div>
The problem is that i found an example on stackoverflow, but i never wrote preg_match before. How to do this with the data i want to grabb ? Thank you
<?php $html = file_get_contents($st_player_cv->getUrlEsl());
preg_match_all(
'What do i need to write here ?',
$html,
$posts, // will contain the data
PREG_SET_ORDER // formats data into an array of posts
);
foreach ($posts as $post) {
$premium = $post[1];
$level = $post[2];
// do something with data
}
The DOM way to do it would be
libxml_use_internal_errors(TRUE);
$dom = new DOMDocument;
$dom->loadHTMLFile('http://www.esl.eu/fr/player/5178309/');
libxml_clear_errors();
$xPath = new DOMXPath($dom);
$nodes = $xPath->query('//div[#style="float: right; margin: 10px;"]');
foreach($nodes as $node) {
echo $node->nodeValue, PHP_EOL;
}
but there is a whole slew of JavaScript in the page that modifies the DOM heavily after the page was loaded. Since any PHP script based fetching will not execute any JavaScript, the style we search for in the XPath does not exist yet and we won't get any results (the Regex suggesed by Hannes doesn't work for the same reason). Neither do the level numbers on the badge exist yet.
As Wrikken pointed out in the comments, there also seems to be some mechanism to block certain requests. I had the message once, but I am not sure what triggers it, because I could also fetch page on several occasions.
To cut a long story short: you cannot achieve what you are trying to do with this page.
If you want something more generic
preg_match('/<div[^>]+?>(.*?)<\/div>/', $myhtml, $result);
echo $result[1] . "\n";
$myhtml contains the code html you have to analyze. $result is the array that contains the regexp and () content after the regular expression was applied. $result[1] will give you what is between the <div ... > and </div>.
This way, even if the <div differs (class name change or different attributes), it'll still work.
this regex '#<div style="float: right; margin: 10px;">(.*)</div>#' should do the trick (yeah) but i would advice you to use DOM & XPath.
edit:
Here is an Xpath / DOM Example:
$html = <<<HTML
<html>
<body>
<em>nonsense</em>
<div style="float: right; margin: 10px;"> here what i want to display on my website </div>
<div> even more nonsense </div>
</body>
</html>
HTML;
$doc = new DOMDocument();
$doc->loadHTML($html);
$xpath = new DOMXpath($doc);
$elements = $xpath->query('//div[#style="float: right; margin: 10px;"]');
echo $elements->item(0)->nodeValue;