I have just started reading documentation and examples about DOM, in order to crawl and parse the document.
For example I have part of document shown below:
<div id="showContent">
<table>
<tr>
<td>
Crap
</td>
</tr>
<tr>
<td width="172" valign="top"><img height="91" border="0" width="172" class="" src="img"></td>
<td width="10"> </td>
<td valign="top"><table cellspacing="0" cellpadding="0" border="0">
<tbody><tr>
<td height="30"><a class="px11" href="link">title</a><a><br>
<span class="px10"></span>
</a></td>
</tr>
<tr>
<td><img height="1" width="580" src="crap"></td>
</tr>
<tr>
<td align="right">
<img height="16" border="0" width="65" src="/buy">
</td>
</tr>
<tr>
<td valign="top" class="px10">
<p style="width: 500px;">description.</p>
</td>
</tr>
</tbody></table></td>
</tr>
<tr>
<td>
Crap
</td>
</tr>
<tr>
<td>
Crap
</td>
</tr>
</table>
</div>
I'm trying to use the following code to get all the tr tags and analyze whether there is crap or information inside them:
$dom = new DOMDocument();
#$dom->loadHTML($html);
$xpath = new DOMXPath($dom);
$tags = $xpath->query('.//div[#id="showContent"]');
foreach ($tags as $tag) {
$string="";
$string=trim($tag->nodeValue);
if(strlen($string)>3) {
echo $string;
echo '<br>';
}
}
However I'm getting just stripped string without the tags, for example:
Crap
Crap
Title
Description
But I would like to get:
<tr>
<td>Crap</td>
</tr>
<tr>
title
</tr>
How to keep html nodes (tags)?
If you want to work with DOM you have to understand the concept. Everything in a DOM Document, including the DOMDocument, is a Node.
The DOMDocument is a hierarchical tree structure of nodes. It starts with a root node. That root node can have child nodes and all these child nodes can have child nodes on their own. Basically everything in a DOMDocument is a node type of some sort, be it elements, attributes or text content.
HTML Legend:
/ \ UPPERCASE = DOMElement
HEAD BODY lowercase = DOMAttr
/ \ "Quoted" = DOMText
TITLE DIV - class - "header"
| \
"The Title" H1
|
"Welcome to Nodeville"
The diagram above shows a DOMDocument with some nodes. There is a root element (HTML) with two children (HEAD and BODY). The connecting lines are called axes. If you follow down the axis to the TITLE element, you will see that it has one DOMText leaf. This is important because it illustrates an often overlooked thing:
<title>The Title</title>
is not one, but two nodes. A DOMElement with a DOMText child. Likewise, this
<div class="header">
is really three nodes: the DOMElement with a DOMAttr holding a DOMText. Because all these inherit their properties and methods from DOMNode, it is essential to familiarize yourself with the DOMNode class.
In practise, this means the DIV you fetched is linked to all the other nodes in the document. You could go all the way to the root element or down to the leaves at any time. It's all there. You just have to query or traverse the document for the wanted information.
Whether you do that by iterating the childNodes of the DIV or use getElementByTagName() or XPath is up to you. You just have to understand that you are not working with raw HTML, but with nodes representing that entire HTML document.
If you need help with extracting specific information from the document, you need to clarify what information you want to fetch from it. For instance, you could ask how to fetch all the links from the table and then we could answer something like:
$div = $dom->getElementById('showContent');
foreach ($div->getElementsByTagName('a') as $link)
{
echo $dom->saveXML($link);
}
But unless you are more specific, we can only guess which nodes might be relevant.
If you need more examples and code snippets on how to work with DOM browse through my previous answers to related questions:
https://stackoverflow.com/search?q=user%3A208809+DOM
By now, there should be a snippet for every basic to medium UseCase you might have with DOM.
To create a parser you can use htmlDOM.
It is very simple easy to use DOM parser written in php. By using it you can easily fetch the contents of div tag.
For example, find all div tags which have attribute id with a value of text.
$ret = $html->find('div[id=text]');
Related
Backstory
First off, I have used PHP to read email and extract out the HTML attachment which later on I stored in a PHP variable.
Now the problem I am facing is that I am trying to extract out information from the HTML in a nested table and hoping to convert to some sort of array so that I can store in SQL.
May I know if there are solutions for this? As I have tried finding but to no avail.
Example
<html>
<table>
<tr>
<td>
<table>
<tr>
<td></td>
</td>
</tr>
</table>
</td>
</tr>
<tr>
<td>
<table>
<tr>
<td>
<p>hi</p>
</td>
</tr>
</table>
</td>
</tr>
</table>
</html>
I want to locate the nearest table tag where "hi" is located so that I can get all the information from that table
Stuff I have tried
I have tried using simple HTML DOM but I guess the HTML file that I tried scraping was too large that it causes memory issues.
include('./simple_html_dom.php');
/* Parse the HTML, stored as a string in $woString */ <br>
$html = str_get_html($worksOrder);
/* Locate the ultimate grandfather table id that wraps all his generation */<br>
$messytables = $html->find('table');
print_r($messytables);
Rather than using simple HTML DOM, this uses DOMDocument and XPath to find the elements.
This draws on the answer XPath to find nearest ancestor element that contains an element that has an attribute with a certain value to locate the <table> tags that enclose the <p> tags which have hi in it. As there are a few levels of enclosing <table> tags, it then uses last() (from XSLT getting last element) to find the innermost enclosing <table>...
libxml_use_internal_errors(true);
$doc = new DOMDOcument();
$doc->loadHTML( $worksOrder );
$xp = new DOMXPath($doc);
$table = $xp->query('(//ancestor::table[descendant::p="hi"])[last()]');
echo $doc->saveHTML($table[0]);
The last line is just to display the data, you can just start with $table[0] and fetch the data as needed.
This outputs with your test data...
<table><tr>
<td>
<p>hi</p>
</td>
</tr></table>
I'm trying to extract only the second cell of the second column of a html table using php.
This is an example of the table:
<table border="1" bordercolor="#FFCC00" style="background-color:#FFFFCC" width="100%" cellpadding="3" cellspacing="3">
<tr>
<td>Name</td>
<td>Marcos</td>
</tr>
<tr>
<td>Address</td>
<td>1234 west 34 st</td>
</tr>
<tr>
<td>Phone</td>
<td>2013336666</td>
</tr>
<tr>
<td>fax</td>
<td>201456789</td>
</tr>
I just want to pull the cell with the address.
First, I would recommend you use the PHP DOMDocument class which is much more full-featured and well-supported. I also use DOMXPath for easy traversal.
$dom = DOMDocument::loadHTML($your_html_string);
$dom_xpath = new DOMXPath($dom);
$value_you_want = $dom_xpath->evaluate('string(/table/tr[2]/td[2])');
Well, since you didn't state how you plan to do this, I'll just throw something out there. Try the PHP DOM found here:
http://us2.php.net/manual/en/book.dom.php
It's a great OOP approach to reading and manipulating an XML or HTML DOM in PHP.
The webpage in question is http://assignments.uspto.gov/assignments/q?db=pat&pub=20060030630
Now, let's just say I want to capture the Assignees in the first assignment. The relevant code there looks like
<div class="t3">Assignee:</div>
</td>
</tr>
</table>
</td><td>
<table width="100%" cellpadding="0" cellspacing="0" border="0">
<tbody valign="top">
<tr>
<td>
<table>
<tr>
<td>
<div class="p1">
LEAR CORPORATION
</div>
</td>
</tr>
<tr>
<td><span class="p1">21557 TELEGRAPH ROAD</span></td>
</tr>
<tr>
<td><span class="p1">SOUTHFIELD, MICHIGAN 48034</span></td>
</tr>
</table>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
I could I suppose use xpath and grab everything out of spans with class p1, except that thing is used all throughout the page for basically everything, same for the div class that lear corporation is in.
So is there a way for me to just read "Assignees" and then grab just the information relevant to it?
I figure if I can understand how to do that, then I can extrapolate from that and figure out how to grab any specific data on the page that I want, i.e. grabbing the conveyance data on any particular assignment.
But if say, I were just to grab all the data on the page (reel/frame, conveyance, assignors, assignee, correspondent for every assignment, and the header information about the patent itself), might that be easier to do than trying to grab each individual piece of information?
There is no clear way to do it since we have no designation in the DOM where this information is.. It's very arbitrary.
I would recommend using some math to figure out the pattern of where in the DOM the Assignee resides.
For example, we know that for every class of p1, the assignee value is position 16, and a new Assignment occurs every 23rd position. Using a loop you could figure it out.
This should get you started at the very least.
$Site = file_get_contents('http://assignments.uspto.gov/assignments/q?db=pat&pub=20060030630');
$Dom = new DomDocument();
$Dom->loadHTML($Site);
$Finder = new DomXPath($Dom);
$Nodes = $Finder->query("//*[contains(concat(' ', normalize-space(#class), ' '), ' p1 ')]");
$position = 0;
foreach($Nodes as $node) {
if(($position % 16) == 0 && $position > 0) {
var_dump($node->nodeValue);
break;
}
$position++;
}
I am parsing and html dom string from ganon dom parser and want to get the next element plain text when a match is found on previous element e.g my html is like
<tr class="last even">
<th class="label">SKU</th>
<td class="data last">some sku here i want to get </td>
</tr>
I have used the following code for now
$html = str_get_dom('html string here');
foreach ($html('th.label') as $elem){
if($elem->getPlainText()=='SKU'){ //this is right
echo $elem->getSibling(1)->getPlainText(); // this is not working
}
}
If the th with class lable and innerhtml SKU is found then get the innerhtml from next sibling that is SKU value
Please help to sort this out.
It's probably a bug in "ganon" of the html - if you take your example of html:
$html = '<table>
<tr class="last even">
<th class="label">SKU</th>
<td class="data last">some sku here i want to get </td>
</tr>
</table>';
$html = str_get_dom($html);
for some reason because of the new line in the html "ganon" thinks that the next element is a text element and only then there is the desire td - so you have to do this:
foreach ($html('th.label') as $elem){
if($elem->getPlainText()=='SKU'){
//elem -> text node -> td node
echo($elem->getSibling(1)->getSibling(1)->getPlainText());
}
}
If you organize your html like this (without new line):
$html = '<table>
<tr class="last even">
<th class="label">SKU</th><td class="data last">some sku here i want to get </td>
</tr>
</table>';
Then your original code will work $elem->getSibling(1)->getPlainText()
Maybe consider using the php simple html dom class - it's much more intuitive, using full oop methods, jquery dom parser like and don't uses this awful var-function method :):
require('simple_html_dom.php');
$html = '<table>
<tr class="last even">
<th class="label">SKU</th>
<td class="data last">some sku here i want to get </td>
</tr>
</table>';
$dom = str_get_html($html);
foreach($dom->find('th.label') as $el){
if($el->plaintext == 'SKU'){
echo($el->next_sibling()->plaintext);
}
}
I have got HTML source code, and i must get some information text in the HTML. I can not use DOM, because the document isn't well-formed.
Maybe, the source could change later, I can not be aware of this situation. So, the solution of this problem must be advisible for most situation.
Im getting source with curl, and i will edit it with preg_match_all function and regular expressions.
Source :
...
<TR Class="Head1">
<TD width="15%"><font size="12">Name</font></TD>
<TD>: </TD>
<TD align="center"><font color="red">Alex</font></TD>
<TD width="25%"><b>Job</b></TD>
<TD>: </B></TD>
<TD align="center" width="25%"><font color="red">Doctor</font></TD>
</TR>
...
...
<TR Class="Head2">
<TD width="15%" align="left">Age</B></TD>
<TD>: </TD>
<TD align="center"><font color="red">32</font></TD>
<TD width="15%"><font size="10">data</TD></font>
<TD> </B></TD>
<TD width="40%"> </TD>
</TR>
...
As we have seen, the source is not well-formed. In fact, terrible! But there is nothing I can do.
The source is longer than this.
How can I get the data from the source? I can delete all of HTML codes, but how can i know sequence of data? What can I do with preg_match_all and regex? What else can I do?
Im waiting for your help.
If you can use the DOM this is far better than regexes. Take a look a PHP Tidy - it's designed to manage badly formed HTML.
You can use DOMDocument to load badly formed HTML:
$doc = new DOMDocument();
#$doc->loadHTML('<TR Class="Head2">
<TD width="15%" align="left">Age</B></TD>
<TD>: </TD>
<TD align="center"><font color="red">32</font></TD>
<TD width="15%"><font size="10">data</TD></font>
<TD> </B></TD>
<TD width="40%"> </TD>
</TR>');
$tds = #$doc->getElementsByTagName('td');
foreach ($tds as $td) {
echo $td->textContent, "\n";
}
I'm suppressing warnings in the above code for brevity.
Output:
Age
:
32
data
<!-- space -->
<!-- space -->
Using regex to parse HTML can be a futile effort as HTML is not a regular language.
Don't use RegEx. The link is funny but not informative, so the long and short of it is that HTML markup is not a regular language, hence cannot be parsed simply using regular expressions.
You could use RegEx to parse individual 'tokens' ( a single open tag; a single attribute name or value...) as part of a recursive parsing algorithm, but you cannot use a magic RegEx to parse HTML all on its own.
Or you could use a parser.
Since the markup isn't valid, maybe you could use TagSoup or PHP:Tidy.
$regex = <<<EOF
<TR Class="Head2">\s+<TD width="15%" align="left">Age</B></TD>\s+<TD>: </TD>\s+<TD align="center"><font color="red">(\d+)</font></TD>\s+<TD width="15%"><font size="10">(\w+)</TD></font>\s+<TD> </B></TD>\s+<TD width="40%"> </TD>\s+</TR>
EOF;
preg_match_all($regex, $text, $result);
var_dump($result)