How to use DOMDocument to get child elements? - php

I am trying to get the text of child elements using the PHP DOM.
Specifically, I am trying to get only the first <a> tag within every <tr>.
The HTML is like this...
<table>
<tbody>
<tr>
<td>
1st Link
</td>
<td>
2nd Link
</td>
<td>
3rd Link
</td>
</tr>
<tr>
<td>
1st Link
</td>
<td>
2nd Link
</td>
<td>
3rd Link
</td>
</tr>
</tbody>
</table>
My sad attempt at it involved using foreach() loops, but would only return Array() when doing a print_r() on the $aVal.
$dom = new DOMDocument();
libxml_use_internal_errors(true);
$dom->loadHTML(returnURLData($url));
libxml_use_internal_errors(false);
$tables = $dom->getElementsByTagName('table');
$aVal = array();
foreach ($tables as $table) {
foreach ($table as $tr){
$trVal = $tr->getElementsByTagName('tr');
foreach ($trVal as $td){
$tdVal = $td->getElementsByTagName('td');
foreach($tdVal as $a){
$aVal[] = $a->getElementsByTagName('a')->nodeValue;
}
}
}
}
Am I on the right track or am I completely off?

Put this code in test.php
require 'simple_html_dom.php';
$html = file_get_html('test1.php');
foreach($html->find('table tr') as $element)
{
foreach($element->find('a',0) as $element)
{
echo $element->plaintext;
}
}
and put your html code in test1.php
<table>
<tbody>
<tr>
<td>
1st Link
</td>
<td>
2nd Link
</td>
<td>
3rd Link
</td>
</tr>
<tr>
<td>
1st Link
</td>
<td>
2nd Link
</td>
<td>
3rd Link
</td>
</tr>
</tbody>
</table>

I am pretty sure I am late, but better way should be to iterate through all "tr" with getElementByTagName and then while iterating through each node in nodelist recieved use getElementByTagName"a". Now no need to iterate through nodeList point out the first element recieved by item(0). That's it! Another way can be to use xPath.
I personally don't like SimpleHtmlDom because of the loads of extra added features it uses where a small functionality is required. In case of heavy scraping also memory management issue can hold you back, its better if you yourself do DOM Analysis rather than depending thrid party application.
Just My opinion. Even I used SHD initially but later realized this.

You're not setting $trVal and $tdVal yet you're looping them ?

Related

How do you web scrape nested table using PHP?

Backstory
First off, I have used PHP to read email and extract out the HTML attachment which later on I stored in a PHP variable.
Now the problem I am facing is that I am trying to extract out information from the HTML in a nested table and hoping to convert to some sort of array so that I can store in SQL.
May I know if there are solutions for this? As I have tried finding but to no avail.
Example
<html>
<table>
<tr>
<td>
<table>
<tr>
<td></td>
</td>
</tr>
</table>
</td>
</tr>
<tr>
<td>
<table>
<tr>
<td>
<p>hi</p>
</td>
</tr>
</table>
</td>
</tr>
</table>
</html>
I want to locate the nearest table tag where "hi" is located so that I can get all the information from that table
Stuff I have tried
I have tried using simple HTML DOM but I guess the HTML file that I tried scraping was too large that it causes memory issues.
include('./simple_html_dom.php');
/* Parse the HTML, stored as a string in $woString */ <br>
$html = str_get_html($worksOrder);
/* Locate the ultimate grandfather table id that wraps all his generation */<br>
$messytables = $html->find('table');
print_r($messytables);
Rather than using simple HTML DOM, this uses DOMDocument and XPath to find the elements.
This draws on the answer XPath to find nearest ancestor element that contains an element that has an attribute with a certain value to locate the <table> tags that enclose the <p> tags which have hi in it. As there are a few levels of enclosing <table> tags, it then uses last() (from XSLT getting last element) to find the innermost enclosing <table>...
libxml_use_internal_errors(true);
$doc = new DOMDOcument();
$doc->loadHTML( $worksOrder );
$xp = new DOMXPath($doc);
$table = $xp->query('(//ancestor::table[descendant::p="hi"])[last()]');
echo $doc->saveHTML($table[0]);
The last line is just to display the data, you can just start with $table[0] and fetch the data as needed.
This outputs with your test data...
<table><tr>
<td>
<p>hi</p>
</td>
</tr></table>

PHP DOM grabbing a specific subset of information

The webpage in question is http://assignments.uspto.gov/assignments/q?db=pat&pub=20060030630
Now, let's just say I want to capture the Assignees in the first assignment. The relevant code there looks like
<div class="t3">Assignee:</div>
</td>
</tr>
</table>
</td><td>
<table width="100%" cellpadding="0" cellspacing="0" border="0">
<tbody valign="top">
<tr>
<td>
<table>
<tr>
<td>
<div class="p1">
LEAR CORPORATION
</div>
</td>
</tr>
<tr>
<td><span class="p1">21557 TELEGRAPH ROAD</span></td>
</tr>
<tr>
<td><span class="p1">SOUTHFIELD, MICHIGAN 48034</span></td>
</tr>
</table>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
I could I suppose use xpath and grab everything out of spans with class p1, except that thing is used all throughout the page for basically everything, same for the div class that lear corporation is in.
So is there a way for me to just read "Assignees" and then grab just the information relevant to it?
I figure if I can understand how to do that, then I can extrapolate from that and figure out how to grab any specific data on the page that I want, i.e. grabbing the conveyance data on any particular assignment.
But if say, I were just to grab all the data on the page (reel/frame, conveyance, assignors, assignee, correspondent for every assignment, and the header information about the patent itself), might that be easier to do than trying to grab each individual piece of information?
There is no clear way to do it since we have no designation in the DOM where this information is.. It's very arbitrary.
I would recommend using some math to figure out the pattern of where in the DOM the Assignee resides.
For example, we know that for every class of p1, the assignee value is position 16, and a new Assignment occurs every 23rd position. Using a loop you could figure it out.
This should get you started at the very least.
$Site = file_get_contents('http://assignments.uspto.gov/assignments/q?db=pat&pub=20060030630');
$Dom = new DomDocument();
$Dom->loadHTML($Site);
$Finder = new DomXPath($Dom);
$Nodes = $Finder->query("//*[contains(concat(' ', normalize-space(#class), ' '), ' p1 ')]");
$position = 0;
foreach($Nodes as $node) {
if(($position % 16) == 0 && $position > 0) {
var_dump($node->nodeValue);
break;
}
$position++;
}

PHP ganon dom parser get next element when match is found

I am parsing and html dom string from ganon dom parser and want to get the next element plain text when a match is found on previous element e.g my html is like
<tr class="last even">
<th class="label">SKU</th>
<td class="data last">some sku here i want to get </td>
</tr>
I have used the following code for now
$html = str_get_dom('html string here');
foreach ($html('th.label') as $elem){
if($elem->getPlainText()=='SKU'){ //this is right
echo $elem->getSibling(1)->getPlainText(); // this is not working
}
}
If the th with class lable and innerhtml SKU is found then get the innerhtml from next sibling that is SKU value
Please help to sort this out.
It's probably a bug in "ganon" of the html - if you take your example of html:
$html = '<table>
<tr class="last even">
<th class="label">SKU</th>
<td class="data last">some sku here i want to get </td>
</tr>
</table>';
$html = str_get_dom($html);
for some reason because of the new line in the html "ganon" thinks that the next element is a text element and only then there is the desire td - so you have to do this:
foreach ($html('th.label') as $elem){
if($elem->getPlainText()=='SKU'){
//elem -> text node -> td node
echo($elem->getSibling(1)->getSibling(1)->getPlainText());
}
}
If you organize your html like this (without new line):
$html = '<table>
<tr class="last even">
<th class="label">SKU</th><td class="data last">some sku here i want to get </td>
</tr>
</table>';
Then your original code will work $elem->getSibling(1)->getPlainText()
Maybe consider using the php simple html dom class - it's much more intuitive, using full oop methods, jquery dom parser like and don't uses this awful var-function method :):
require('simple_html_dom.php');
$html = '<table>
<tr class="last even">
<th class="label">SKU</th>
<td class="data last">some sku here i want to get </td>
</tr>
</table>';
$dom = str_get_html($html);
foreach($dom->find('th.label') as $el){
if($el->plaintext == 'SKU'){
echo($el->next_sibling()->plaintext);
}
}

Simple HTML DOM: Notice->Trying to get property of non-object

I am getting an php notice when using simple html dom to scrape a website. There are 2 notices displayed and everything rendered underneath looks perfect when using the print_r function to display it.
The website table structure is as follows:
<table class=data schedTbl>
<thead>
<tr>
<th>DATA</th>
<th>DATA</th>
<th>DATA</th>
etc....
</tr>
</thead>
<tbody>
<tr>
<td>
<div class="class1">DATA</div>
<div class="class2">SAME DATA AS PREVIOUS DIV</div>
</td>
<td>DATA</td>
<td>DATA</td>
etc....
</tr>
<tr>
<td>
<div class="class1">DATA</div>
<div class="class2">SAME DATA AS PREVIOUS DIV</div>
</td>
<td>DATA</td>
<td>DATA</td>
etc....
</tr>
<tr>
<td>
<div class="class1">DATA</div>
<div class="class2">SAME DATA AS PREVIOUS DIV</div>
</td>
<td>DATA</td>
<td>DATA</td>
etc....
</tr>
etc....
</tbody>
</table>
The code below is used to find all tr in table[class=data schedTbl]. I have a tbody selector in there, but it seems to pay no attention to this selector as it still selects the tr in the thead.
include('simple_html_dom.php');
$articles = array();
getArticles('www.somesite.com');
function getArticles($page) {
global $articles;
$html = new simple_html_dom();
$html->load_file($page);
$items = $html->find('table[class=data schedTbl] tbody tr');
foreach($items as $post) {
$articles[] = array($post->children(0)->first_child(0)->plaintext,//0 -- GAME DATE
$post->children(1)->plaintext,//1 -- AWAY TEAM
$post->children(2)->plaintext);//2 -- HOME TEAM
}
}
So, I believe notices come from the tr in the thead because I am calling on the first child of the first td which only has one record. The reason for two is there is actually two tables with the same data structure in the body.
Again, I believe there are 2 ways of solving this:
1) PROBABLY THE EASIEST (fix the find selector so the TBODY works and only selects the tds within the tbodies)
2) Figure out a way to not do the first_child filter when it is not needed?
Please let me know if you would like a snapshot of the print_r($articles) output I am receiving.
Thanks in advance for any help provided!
Sincerely,
Bill C.
Just comment out line #695 in the simple_html_dom.php
if ($m[1]==='tbody') continue;
Then it should read the tbody.

Missing SimpleXML Parsing

<?php
// load SimpleXML
$entry = new SimpleXMLElement('http://bit.ly/c3IqMF', null, true);
echo <<<EOF
<table>
<tr>
<th>Title</th>
<th>Image</th>
</tr>
EOF;
foreach($entry as $item) //
{
echo <<<EOF
<tr>
<td>{$item->title}</td>
<td><img src="{$item->children('im', true)->image}"></td>
</tr>
EOF;
}
echo '</table>';
?>
The above php works but somehow, I got 8 empty table entities above the result
<tr>
<td></td>
<td><img src=""></td>
</tr>
What's wrong with the code? How do I get rid of the empty table entities?
The way you have it now it gets the <id>, <title>, <updated> from the the start of the xml. Actually you needed all the entry entries in the xml. So it should be $entry->entry
foreach($entry->entry as $item) //
{
echo <<<EOF
<tr>
<td>{$item->title}</td>
<td><img src="{$item->children('im', true)->image}"></td>
</tr>
EOF;
}
Honestly, I think you are approaching this the wrong way. Since it seems that you are trying to parse an Atom feed, try using something designed for that, like Magpie RSS. It will probably save you a lot of time.

Categories