Backstory
First off, I have used PHP to read email and extract out the HTML attachment which later on I stored in a PHP variable.
Now the problem I am facing is that I am trying to extract out information from the HTML in a nested table and hoping to convert to some sort of array so that I can store in SQL.
May I know if there are solutions for this? As I have tried finding but to no avail.
Example
<html>
<table>
<tr>
<td>
<table>
<tr>
<td></td>
</td>
</tr>
</table>
</td>
</tr>
<tr>
<td>
<table>
<tr>
<td>
<p>hi</p>
</td>
</tr>
</table>
</td>
</tr>
</table>
</html>
I want to locate the nearest table tag where "hi" is located so that I can get all the information from that table
Stuff I have tried
I have tried using simple HTML DOM but I guess the HTML file that I tried scraping was too large that it causes memory issues.
include('./simple_html_dom.php');
/* Parse the HTML, stored as a string in $woString */ <br>
$html = str_get_html($worksOrder);
/* Locate the ultimate grandfather table id that wraps all his generation */<br>
$messytables = $html->find('table');
print_r($messytables);
Rather than using simple HTML DOM, this uses DOMDocument and XPath to find the elements.
This draws on the answer XPath to find nearest ancestor element that contains an element that has an attribute with a certain value to locate the <table> tags that enclose the <p> tags which have hi in it. As there are a few levels of enclosing <table> tags, it then uses last() (from XSLT getting last element) to find the innermost enclosing <table>...
libxml_use_internal_errors(true);
$doc = new DOMDOcument();
$doc->loadHTML( $worksOrder );
$xp = new DOMXPath($doc);
$table = $xp->query('(//ancestor::table[descendant::p="hi"])[last()]');
echo $doc->saveHTML($table[0]);
The last line is just to display the data, you can just start with $table[0] and fetch the data as needed.
This outputs with your test data...
<table><tr>
<td>
<p>hi</p>
</td>
</tr></table>
Related
I'm trying to extract only the second cell of the second column of a html table using php.
This is an example of the table:
<table border="1" bordercolor="#FFCC00" style="background-color:#FFFFCC" width="100%" cellpadding="3" cellspacing="3">
<tr>
<td>Name</td>
<td>Marcos</td>
</tr>
<tr>
<td>Address</td>
<td>1234 west 34 st</td>
</tr>
<tr>
<td>Phone</td>
<td>2013336666</td>
</tr>
<tr>
<td>fax</td>
<td>201456789</td>
</tr>
I just want to pull the cell with the address.
First, I would recommend you use the PHP DOMDocument class which is much more full-featured and well-supported. I also use DOMXPath for easy traversal.
$dom = DOMDocument::loadHTML($your_html_string);
$dom_xpath = new DOMXPath($dom);
$value_you_want = $dom_xpath->evaluate('string(/table/tr[2]/td[2])');
Well, since you didn't state how you plan to do this, I'll just throw something out there. Try the PHP DOM found here:
http://us2.php.net/manual/en/book.dom.php
It's a great OOP approach to reading and manipulating an XML or HTML DOM in PHP.
I am trying to get the text of child elements using the PHP DOM.
Specifically, I am trying to get only the first <a> tag within every <tr>.
The HTML is like this...
<table>
<tbody>
<tr>
<td>
1st Link
</td>
<td>
2nd Link
</td>
<td>
3rd Link
</td>
</tr>
<tr>
<td>
1st Link
</td>
<td>
2nd Link
</td>
<td>
3rd Link
</td>
</tr>
</tbody>
</table>
My sad attempt at it involved using foreach() loops, but would only return Array() when doing a print_r() on the $aVal.
$dom = new DOMDocument();
libxml_use_internal_errors(true);
$dom->loadHTML(returnURLData($url));
libxml_use_internal_errors(false);
$tables = $dom->getElementsByTagName('table');
$aVal = array();
foreach ($tables as $table) {
foreach ($table as $tr){
$trVal = $tr->getElementsByTagName('tr');
foreach ($trVal as $td){
$tdVal = $td->getElementsByTagName('td');
foreach($tdVal as $a){
$aVal[] = $a->getElementsByTagName('a')->nodeValue;
}
}
}
}
Am I on the right track or am I completely off?
Put this code in test.php
require 'simple_html_dom.php';
$html = file_get_html('test1.php');
foreach($html->find('table tr') as $element)
{
foreach($element->find('a',0) as $element)
{
echo $element->plaintext;
}
}
and put your html code in test1.php
<table>
<tbody>
<tr>
<td>
1st Link
</td>
<td>
2nd Link
</td>
<td>
3rd Link
</td>
</tr>
<tr>
<td>
1st Link
</td>
<td>
2nd Link
</td>
<td>
3rd Link
</td>
</tr>
</tbody>
</table>
I am pretty sure I am late, but better way should be to iterate through all "tr" with getElementByTagName and then while iterating through each node in nodelist recieved use getElementByTagName"a". Now no need to iterate through nodeList point out the first element recieved by item(0). That's it! Another way can be to use xPath.
I personally don't like SimpleHtmlDom because of the loads of extra added features it uses where a small functionality is required. In case of heavy scraping also memory management issue can hold you back, its better if you yourself do DOM Analysis rather than depending thrid party application.
Just My opinion. Even I used SHD initially but later realized this.
You're not setting $trVal and $tdVal yet you're looping them ?
The webpage in question is http://assignments.uspto.gov/assignments/q?db=pat&pub=20060030630
Now, let's just say I want to capture the Assignees in the first assignment. The relevant code there looks like
<div class="t3">Assignee:</div>
</td>
</tr>
</table>
</td><td>
<table width="100%" cellpadding="0" cellspacing="0" border="0">
<tbody valign="top">
<tr>
<td>
<table>
<tr>
<td>
<div class="p1">
LEAR CORPORATION
</div>
</td>
</tr>
<tr>
<td><span class="p1">21557 TELEGRAPH ROAD</span></td>
</tr>
<tr>
<td><span class="p1">SOUTHFIELD, MICHIGAN 48034</span></td>
</tr>
</table>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
I could I suppose use xpath and grab everything out of spans with class p1, except that thing is used all throughout the page for basically everything, same for the div class that lear corporation is in.
So is there a way for me to just read "Assignees" and then grab just the information relevant to it?
I figure if I can understand how to do that, then I can extrapolate from that and figure out how to grab any specific data on the page that I want, i.e. grabbing the conveyance data on any particular assignment.
But if say, I were just to grab all the data on the page (reel/frame, conveyance, assignors, assignee, correspondent for every assignment, and the header information about the patent itself), might that be easier to do than trying to grab each individual piece of information?
There is no clear way to do it since we have no designation in the DOM where this information is.. It's very arbitrary.
I would recommend using some math to figure out the pattern of where in the DOM the Assignee resides.
For example, we know that for every class of p1, the assignee value is position 16, and a new Assignment occurs every 23rd position. Using a loop you could figure it out.
This should get you started at the very least.
$Site = file_get_contents('http://assignments.uspto.gov/assignments/q?db=pat&pub=20060030630');
$Dom = new DomDocument();
$Dom->loadHTML($Site);
$Finder = new DomXPath($Dom);
$Nodes = $Finder->query("//*[contains(concat(' ', normalize-space(#class), ' '), ' p1 ')]");
$position = 0;
foreach($Nodes as $node) {
if(($position % 16) == 0 && $position > 0) {
var_dump($node->nodeValue);
break;
}
$position++;
}
I am parsing and html dom string from ganon dom parser and want to get the next element plain text when a match is found on previous element e.g my html is like
<tr class="last even">
<th class="label">SKU</th>
<td class="data last">some sku here i want to get </td>
</tr>
I have used the following code for now
$html = str_get_dom('html string here');
foreach ($html('th.label') as $elem){
if($elem->getPlainText()=='SKU'){ //this is right
echo $elem->getSibling(1)->getPlainText(); // this is not working
}
}
If the th with class lable and innerhtml SKU is found then get the innerhtml from next sibling that is SKU value
Please help to sort this out.
It's probably a bug in "ganon" of the html - if you take your example of html:
$html = '<table>
<tr class="last even">
<th class="label">SKU</th>
<td class="data last">some sku here i want to get </td>
</tr>
</table>';
$html = str_get_dom($html);
for some reason because of the new line in the html "ganon" thinks that the next element is a text element and only then there is the desire td - so you have to do this:
foreach ($html('th.label') as $elem){
if($elem->getPlainText()=='SKU'){
//elem -> text node -> td node
echo($elem->getSibling(1)->getSibling(1)->getPlainText());
}
}
If you organize your html like this (without new line):
$html = '<table>
<tr class="last even">
<th class="label">SKU</th><td class="data last">some sku here i want to get </td>
</tr>
</table>';
Then your original code will work $elem->getSibling(1)->getPlainText()
Maybe consider using the php simple html dom class - it's much more intuitive, using full oop methods, jquery dom parser like and don't uses this awful var-function method :):
require('simple_html_dom.php');
$html = '<table>
<tr class="last even">
<th class="label">SKU</th>
<td class="data last">some sku here i want to get </td>
</tr>
</table>';
$dom = str_get_html($html);
foreach($dom->find('th.label') as $el){
if($el->plaintext == 'SKU'){
echo($el->next_sibling()->plaintext);
}
}
I have just started reading documentation and examples about DOM, in order to crawl and parse the document.
For example I have part of document shown below:
<div id="showContent">
<table>
<tr>
<td>
Crap
</td>
</tr>
<tr>
<td width="172" valign="top"><img height="91" border="0" width="172" class="" src="img"></td>
<td width="10"> </td>
<td valign="top"><table cellspacing="0" cellpadding="0" border="0">
<tbody><tr>
<td height="30"><a class="px11" href="link">title</a><a><br>
<span class="px10"></span>
</a></td>
</tr>
<tr>
<td><img height="1" width="580" src="crap"></td>
</tr>
<tr>
<td align="right">
<img height="16" border="0" width="65" src="/buy">
</td>
</tr>
<tr>
<td valign="top" class="px10">
<p style="width: 500px;">description.</p>
</td>
</tr>
</tbody></table></td>
</tr>
<tr>
<td>
Crap
</td>
</tr>
<tr>
<td>
Crap
</td>
</tr>
</table>
</div>
I'm trying to use the following code to get all the tr tags and analyze whether there is crap or information inside them:
$dom = new DOMDocument();
#$dom->loadHTML($html);
$xpath = new DOMXPath($dom);
$tags = $xpath->query('.//div[#id="showContent"]');
foreach ($tags as $tag) {
$string="";
$string=trim($tag->nodeValue);
if(strlen($string)>3) {
echo $string;
echo '<br>';
}
}
However I'm getting just stripped string without the tags, for example:
Crap
Crap
Title
Description
But I would like to get:
<tr>
<td>Crap</td>
</tr>
<tr>
title
</tr>
How to keep html nodes (tags)?
If you want to work with DOM you have to understand the concept. Everything in a DOM Document, including the DOMDocument, is a Node.
The DOMDocument is a hierarchical tree structure of nodes. It starts with a root node. That root node can have child nodes and all these child nodes can have child nodes on their own. Basically everything in a DOMDocument is a node type of some sort, be it elements, attributes or text content.
HTML Legend:
/ \ UPPERCASE = DOMElement
HEAD BODY lowercase = DOMAttr
/ \ "Quoted" = DOMText
TITLE DIV - class - "header"
| \
"The Title" H1
|
"Welcome to Nodeville"
The diagram above shows a DOMDocument with some nodes. There is a root element (HTML) with two children (HEAD and BODY). The connecting lines are called axes. If you follow down the axis to the TITLE element, you will see that it has one DOMText leaf. This is important because it illustrates an often overlooked thing:
<title>The Title</title>
is not one, but two nodes. A DOMElement with a DOMText child. Likewise, this
<div class="header">
is really three nodes: the DOMElement with a DOMAttr holding a DOMText. Because all these inherit their properties and methods from DOMNode, it is essential to familiarize yourself with the DOMNode class.
In practise, this means the DIV you fetched is linked to all the other nodes in the document. You could go all the way to the root element or down to the leaves at any time. It's all there. You just have to query or traverse the document for the wanted information.
Whether you do that by iterating the childNodes of the DIV or use getElementByTagName() or XPath is up to you. You just have to understand that you are not working with raw HTML, but with nodes representing that entire HTML document.
If you need help with extracting specific information from the document, you need to clarify what information you want to fetch from it. For instance, you could ask how to fetch all the links from the table and then we could answer something like:
$div = $dom->getElementById('showContent');
foreach ($div->getElementsByTagName('a') as $link)
{
echo $dom->saveXML($link);
}
But unless you are more specific, we can only guess which nodes might be relevant.
If you need more examples and code snippets on how to work with DOM browse through my previous answers to related questions:
https://stackoverflow.com/search?q=user%3A208809+DOM
By now, there should be a snippet for every basic to medium UseCase you might have with DOM.
To create a parser you can use htmlDOM.
It is very simple easy to use DOM parser written in php. By using it you can easily fetch the contents of div tag.
For example, find all div tags which have attribute id with a value of text.
$ret = $html->find('div[id=text]');