I'm trying to parse an HTML snippet, using the PHP DOM functions. I have stripped out everything apart from paragraph, span and line break tags, and now I want to retrieve all the text, along with its accompanying styles.
So, I'd like to get each piece of text, one by one, and for each one I can then go back up the tree to get the values of particular attributes (I'm only interested in some specific ones, like color etc.).
How can I do this? Or am I thinking about it the wrong way?
Suppose you have a DOMDocument here:
$doc = new DOMDocument();
$doc->loadHTMLFile('http://stackoverflow.com/');
You can find all text nodes using a simple Xpath.
$xpath = new DOMXpath($doc);
$textNodes = $xpath->query('//text()');
Just foreach over it to iterate over all textnodes:
foreach ($textNodes as $textNode) {
echo $textNode->data . "\n";
}
From that, you can go up the DOM tree by using ->parentNode.
Hope that this can give you a good start.
For those who are more comfortable with CSS3 selectors, and are willing to include a single extra PHP class into their project, I would suggest the use of Simple PHP DOM parser. The solution would look something like the following:
$html = file_get_html('http://www.example.com/');
$ret = $html->find('p, span');
$store = array();
foreach($ret as $element) {
$store[] = array($element->tag => array('text' => $element->innertext,
'color' => $element->color,
'style' => $element->style));
}
print_r($store);
Related
I need to remove some tags (e.g. <div></div>) in HTML document and keep inner tags and text.
I managed to do that with Simple HTML Dom Parser. But it can't process big files due to huge memory requirements.
I would prefer to use native PHP tools like DOMDocument cause I read that it's more optimized and quicker in processing HTML documents.
But I struggle at the first stage - how to remove some tags while keeping inner text and tags.
Source HTML sample is:
<html><body><div>00000</div>aaaaa<div>bbbbbb<div>ccc<a>link</a>ccc</div>dddddd</div>eeeee<div>1111</div></body></html>
I try this code:
$htmltext='<html><body><div>00000</div>aaaaa<div>bbbbbb<div>ccc<a>link</a>ccc</div>dddddd</div>eeeee<div>1111</div></body></html>';
libxml_use_internal_errors(true);
$doc = new DOMDocument();
$doc->loadHTML($htmltext);
$oldnodes = $doc->getElementsByTagName('div');
foreach ($oldnodes as $node) {
$fragment = $doc->createDocumentFragment();
while($node->childNodes->length > 0) {
$fragment->appendChild($node->childNodes->item(0));
}
$node->parentNode->replaceChild($fragment, $node);
}
echo $doc->saveHTML();
It produces the output:
<html><body>00000aaaaa<div>bbbbbbccc<a>link</a>cccdddddd</div>eeeee<div>1111</div></body></html>
I need the following:
<html><body>00000aaaaabbbbbbccc<a>link</a>cccddddddeeeee1111</body></html>
Could someone please help me with proper code for the task?
You can use strip_tags function in PHP.
$thmltext = '<html><body><div>00000</div>aaaaa<div>bbbbbb<div>ccc<a>link</a>ccc</div>dddddd</div>eeeee<div>1111</div></body></html>';
strip_tags($htmltext, '<html>,<body>,<a>');
This remove all tags except html,body,a
And output is:
<html><body>00000aaaaabbbbbbccc<a>link</a>cccddddddeeeee1111</body></html>
EDIT:
If it is input from user, it's better for security reason to use whitelist tags and not blacklist.
If your code only contains simple HTML tags without any attributes you can keep it simple like:
$value = '<html><body><div>00000</div>aaaaa<div>bbbbbb<div>ccc<a>link</a>ccc</div>dddddd</div>eeeee<div>1111</div></body></html>';
$pattern = '/<[\/]*(div|h1)>/';
$removedTags = preg_replace($pattern, '', $value);
Since you wrote in your comment that there are more than just div tags you want to remove, I added a h1 tag to the pattern in case you also want to remove h1 tags.
This code snippet is only for simple code, but fits to your HTML input and output example.
Try this..
Just replace the for loop with the below code.
foreach ($oldnodes as $node) {
$children = $node->childNodes;
$string = "";
foreach($children as $child) {
$childString = $doc->saveXML($child);
$string = $string."".$childString;
}
$fragment = $doc->createDocumentFragment();
$fragment->appendXML($string);
$node->parentNode->insertBefore($fragment,$node);
$node->parentNode->removeChild($node);
}
I found a way to make it work.
The reason code in question not working is the manipulation with nodes in nodelist ruin nodelist. So "foreach" function wents through only 2 out of 4 items in nodelist - the rest 2 become distorted.
So I had to deal with only the 1st element of the list and then rebuild list until there are some items in the list left.
The code is:
$htmltext='<html><body><div>00000</div>aaaaa<div>bbbbbb<div>ccc<a>link</a>ccc</div>dddddd</div>eeeee<div>1111</div></body></html>';
echo "<!--
".$htmltext."
-->
";
libxml_use_internal_errors(true);
$doc = new DOMDocument();
$doc->loadHTML($htmltext);
$oldnodes = $doc->getElementsByTagName('div');
while ($oldnodes->length>0){
$node=$oldnodes->item(0);
$fragment = $doc->createDocumentFragment();
while($node->childNodes->length > 0) {
$fragment->appendChild($node->childNodes->item(0));
}
$node->parentNode->replaceChild($fragment, $node);
$oldnodes = $doc->getElementsByTagName('div');
}
echo $doc->saveHTML();
I hope that will be helpful for someone who finds same difficulties.
Ok, I want to create a "website mobilizer" by using PHP Simple HTML DOM Parser. In the present phase, I want to-
change all 'ul' and 'li' tag to 'div' tag and
change all 'table' elements (e.g. table,tr,td,th) to div. I tried an workaround for the first problem in following way:
.
$html=new new simple_html_dom();
$html>load_file($sourceurl);
$div="div";
foreach ($html->find('ul') as $element) {
$element=$div;
}
It does seem dull, but I'm not being able to find any other solution. I am discouraged for using preg_match, though I don't know if it can give me the desired output. Any help will be appreciated.
It is possible:
$html=new new simple_html_dom();
$html>load_file($sourceurl);
foreach ($html->find('ul') as $element) {
$element->innertext = "<div>".$element->innertext."</div>";
}
Of course you can do the same with table.
More in doucmetation: Simple HTML DOM Parser Manual
$html=new new simple_html_dom();
$html>load_file($sourceurl);
$replace="ul,li,table,tr,td,th";
foreach($html->find($replace) as $key=>$element){
$html->find($replace,$key)->outertext="<div>".$element->innertext."</div>"
}
this replaces all the elements from $replace array to <div> in $html DOM without changing the contents of those tags. Everything is stored in $html DOM.
As you can see you cant use $element to change anything in $html, even using $element as a reference, so you have to access $html directly.
i want get all link in page by class "page1" in php.
the same code in jquery
$("a#page1").echo(function()
{
});
can do that in php?
$pattern = '`.*?((http|ftp)://[\w#$&+,\/:;=?#%.-]+)[^\w#$&+,\/:;=?#%.-]*?`i';
preg_match_all($pattern,$page_g,$matches);
this code get all href in the $page_g but its not work for class="page1".
i want only all href in $page_g by class="page1"
can help me for optimize reqular ex or other way?
for example
$page_g="the <strong>office</strong> us s01 05 xvid mu asd";
i want return only /?s=cache:16001429:office+s01e02
tnx
You lack the expertise to use a regular expression for that. Hencewhy using DOMdocument is the advisable solution here. If you want to have a simpler API then use the jQuery-lookalikes phpQuery or QueryPath:
$link = qp($html)->find("a#page1")->attr("href");
print $link;
Edit Edited since you clarified the question.
To get all <a> links with the class .page1:
// Load the HTML from a file
$your_HTML_string = file_get_contents("html_filename.html");
$doc = new DOMDocument();
$doc->loadHTML($your_HTML_string);
// Then select all <a> tags under #page1
$a_links = $doc->getElementsByTagName("a");
foreach ($a_links as $link) {
// If they have more than one class,
// you'll need to use (strpos($link->getAttribute("class"), "page1") >=0)
// instead of == "page1"
if ($link->getAttribute("class") == "page1") {
// do something
}
}
Use DomDocument to parse HTML page, here's a tutorial:
Tutorial
DOM is preferred to be used here, as regex is difficult to maintain if underlying HTML changes, besides, DOM can deal with invalid HTML and provides you access to other HTML parsing related tools.
So, assuming that have a file that contains HTML, and you are searching for classes, this could be the way to go:
$doc = new DOMDocument;
$doc->load(PATH_TO_YOUR_FILE);
//we will use Xpath to find all a containing your class, as a tag can have more than one class and it's just easier to do it with Xpath.
$xpath = new DOMXpath($doc);
$list = $xpath->query("//a[contains(#class, 'page1')]");
foreach ($list as $a_tag) {
$href = $a_tag->getAttribute('href');
//do something
}
I'm just getting started with using php DOMDocument and am having a little trouble.
How would I select all link nodes under a specific node lets say
in jquery i could simply do.. $('h5 > a')
and this would give me all the links under h5.
how would i do this in php using DOMDocument methods?
I tried using phpquery but for some reason it can't read the html page i'm trying to parse.
As far as I know, jQuery rewrites the selector queries to XPath. Any node jQuery can select, XPath also can.
h5 > a means select any a node for which the direct parent node is h5. This can easily be translated to a XPath query: //h5/a.
So, using DOMDocument:
$dom = new DOMDocument;
$dom->loadHTML($html);
$xpath = new DOMXPath($dom);
$nodes = $xpath->query('//h5/a');
foreach ($nodes as $node) {
// do stuff
}
Retrieve the DOMElement whose children you are interested in and call DOMElement::getElementsByTagName on it.
Get all h5 tags from it, and loop through each one, checking if it's parent is an a tag.
// ...
$h5s = $document->getElementsByTagName('h5');
$correct_tags = array();
foreach ($h5s as $h5) {
if ($h5->parentNode->tagName == 'a') {
$correct_tags[] = $h5;
}
}
// do something with $correct_tags
I have the contents of a web page assigned to a variable $html
Here's an example of the contents of $html:
<div class="content">something here</div>
<span>something random thrown in <strong>here</strong></span>
<div class="content">more stuff</div>
How, using PHP can I create an array from that that finds the contents of <div class="content"></div> regions like this (for the example above) so:
echo $array[0] . "\n" . $array[1]; //etc
outputs
something here
more stuff
Assuming this is just a simplified case in the OP and the real situation is more complicated, you'll want to use XPath.
If it's really complex, then you may want to use DOMDocument (with DOMXPath), but here's a simple example using SimpleXML
$xml = new SimpleXMLElement($html);
$result = $xml->xpath('//div[#class="content"]');
while(list( , $node) = each($result)) {
echo $node,"\n";
}
Since you explicitly asked about creating an array for this, you could use:
$res_Arr = array();
while(list( , $node) = each($result)) {
$res_Arr[] = $node;
}
and $res_Arr would be an array with the contents you're looking for.
See http://php.net/manual/en/simplexmlelement.xpath.php for php SimpleXML Xpath info and http://www.w3.org/TR/xpath for the XPath specifications
PHP has several means of processing HTML, including DomDocument and SimpleXML. See Parse HTML With PHP And DOM. Here is an example:
$dom = new DomDocument;
$dom->loadHTML($html);
$dom->preserveWhiteSpace = false;
$divs = $dom->getElementsByTagName('div');
foreach ($divs as $div) {
$class = $div->getAttribute('class');
if ($class == 'content') {
echo $div->nodeValue . "\n";
}
}
Technically the class attribute could be multiple classes so you might want to use:
$classes = explode(' ', $class);
if (in_array('content', $classes)) {
...
}
The SimpleXML/XPath approach is more concise but if you don't want to go the XPath route (and learning another technology, at least enough to do these sorts of tasks) then the above is a programmatic alternative.
There not much you can do short of using string manipulations function or regular expressions. you can load your HTML as XML using the DOM library and use that to traverse to your div, but that can become cumbersome if your not careful or if the structure is complex.
http://ca3.php.net/manual/en/book.dom.php
It looks like Kalem13 beat me to it, but I agree. You could use the DOMDocument class. I haven't used it personally, but I think it would work for you. First you instantiate a DOMDocument object, then you load your $html variable using the loadHTML() function. Then you can use the getElementsByTagName() function.
You probaly need to use preg_match_all()
$matches = array();
preg_match_all('`\<div(.*?)class\=\"content\"(.*?)\>(.*?)\<\/div\>`iUsm',$html,$matches,PREG_SET_ORDER);
foreach($matches as $m){
// $m[3] represents the content in <div class="content">
}