I'm scraping a website that's mostly table based. I have <tr> tags that each represent a category and <td> tags inside these that represent properties of the category.
Using Xpath I get the <tr> fine but with all the <td> info inside it bunched as one string:
$html_string = file_get_contents('testpage.html');
$dom = new DOMDocument();
$dom->loadHTML($html_string);
$xpath = new DOMXpath($dom);
$context_nodes = $xpath->query('//table[#id="category"]/tr[not(starts-with(#id, "category"))]');
And can each get <td> fine but with no retrospective reference to the category with:
$context_nodes = $xpath->query('//table[#id="category"]/tr[not(starts-with(#id, "category"))]/td');
What I would like to do later is be able to reference the properties of each category. I presumed I could do so with $context_nodes[2] etc., thinking that the array it created was a multidimensional string array. This doesn't seem to be the case.
How would I go about creating an array from the xpath info where I can grab a property of a category based on identifying what category I specifically want. E.g. train[1][2]?
Your second attempt is on the right lines. PHP (or, rather, libxml) retains a reference to the context the nodes you selected were returned from, allowing you to do precisely what you need in your case.
XML
<root>
<cat name="category 1">
<prop>prop 1.1</prop>
<prop>prop 1.2</prop>
</cat>
<cat name="category 2">
<prop>prop 2.1</prop>
<prop>prop 2.2</prop>
</cat>
</root>
PHP
$xml = new SimpleXMLElement($xml);
$props = $xml->xpath('cat/prop');
foreach($props as $prop) {
//let's go back up...
$parent_cat = $prop->xpath('parent::*/#name');
echo '<p>'.$prop.' (property of '.$parent_cat[0].')</p>';
}
Notice how we navigate back up the tree, from the point of the prop node, to reference the parent category. Not sure if this is what you meant but hope it helps.
Related
I have xml like:
<root xmlns="urn:test:apis:baseComponents">
<books>
<book>
<name>50 shades of grey</name>
</book>
</books>
<disks>
<disk>
<name>Britney Spears</name>
</disk>
</disks>
</root>
And such php code:
$xml = new SimpleXMLElement($xml);
$books = $xml->books;
$disks = $xml->disks;
$disks->registerXPathNamespace('x', 'urn:test:apis:baseComponents');
$books->registerXPathNamespace('x', 'urn:test:apis:baseComponents');
$b_names = $books->xpath('//x:name');
b_names contains array with 2 values instead of 1. First holds books->book->name, second holds disks->disk->name.
Can you please explain what am I doing wrong and how could I find children of only one element?
The reason that I am using xpath instead of taking manually values using SimpleXMLElement, is that I don't know what value, which I want to search in advance.
Use $books->xpath('.//x:name') to search descendants of your $books variable and not descendants of the root node/document node (which the path //x:name does).
I have an XML file with following tree structure.
<rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:media="http://search.yahoo.com/mrss/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0">
<channel>
<title>Videos</title>
<link>https://www.example.com/r/videos/</link>
<description>A long description of the video.</description>
<image>...</image>
<atom:link rel="self" href="http://www.example.com/videos/.xml" type="application/rss+xml"/>
<item>
<title>The most used Jazz lick in history.</title>
<link>
http://www.example.com/
</link>
<guid isPermaLink="true">
http://www.example.com/
</guid>
<pubDate>Mon, 07 Sep 2015 14:43:34 +0000</pubDate>
<description>
<table>
<tr>
<td>
<a href="http://www.example.com/">
<img src="http://www.example.com/.jpg" alt="The most used Jazz lick in history." title="The most used Jazz lick in history." />
</a>
</td>
<td> submitted by
jcepiano
<br/>
[link]
<a href="http://www.example.com/">
[508 comments]
</a>
</td>
</tr>
</table>
</description>
<media:title>The most used Jazz lick in history.</media:title>
<media:thumbnail url="http://example.jpg"/>
</item>
</channel>
</rss>
Here, the html table element is embedded inside XML and that's confusing me.
Now I want to pick the text node values for //channel/item/title and href value for //channel/item/description/table/tr/td[1]/a[1] (with a text node value = "[link]")
Above in 2nd case, I am looking for the value of 2nd a (with a text node value = "[link]"), inside 2nd td inside tr, table, description, item, channel.
I am using PHP DOMDocument();
I have been looking for a perfect solution for this for 2 days now, can you please let me know how would this happen?
Also I need to count the total number of items in the feed, right now I am doing like this:
...
$queryResult = $xpathvar->query('//item/title');
$total = 1;
foreach($queryResult as $result){
$total++;
}
echo $title;
And I also need a reference link for XPath query selectors' rules.
Thanks in advance! :)
You wrote that you wanted the length of the result set of the following query:
$queryResult = $xpathvar->query('//item/title');
I assume that $xpathvar here is of type DOMXPath. If so, it has a length property as described here. Instead of using foreach, simply use:
$length = $xpathvar->query('//item/title')->length;
Now I want to pick the text node values for //channel/item/title
Which you can get with the expression //channel/item/title/text().
and href value for //channel/item/description/table/tr/td[1]/a[1] (with a text node value = "[link]")
Your expression here selects any tr, the first td under that, then the first a. But the first a does not have a value of "[link]" in your source. If you want that, though, you can use:
//channel/item/description/table/tr/td[1]/a[1]/#href
but it looks like you rather want:
//channel/item/description/table/tr/td/a[. = "[link]"][1]/#href
which finds the first a element in the tree that has the value (text node) that is "[link]".
Above in 2nd case, I am looking for the value of 2nd a (with a text node value = "[link]"), inside 2nd td inside tr, table, description, item, channel.
Not sure if this was a separate question or meant to explain the previous one. Regardless, the answer the same as in the previous one, unless you explicitly want to search for 2nd a etc (i.e., search by position), in which case you can use numeric predicates.
Note: you start most of your expressions with //expr, which essentially means: search the whole tree at any depth for the expression expr. This is potentially expensive and if all you need is a (relative) root node for which you know the starting point or expression, it is better, and far more performant, to use a direct path. In your case, you can replace //channel for /*/channel (because it is the first under the root element).
I finally could make it work with the code below
$url = "https://www.example.com/r/videos/.xml";
$feed_dom = new domDocument;
$feed_dom->load($url);
$feed_dom->preserveWhiteSpace = false;
$items = $feed_dom->getElementsByTagName('item');
foreach($items as $item){
$title = $item->getElementsByTagName('title')->item(0)->nodeValue;
$desc_table = $item->getElementsByTagName('description')->item(0)->nodeValue;
echo $title . "<br>";
$table_dom = new domDocument;
$table_dom->loadHTML($desc_table);
$xpath = new DOMXpath($table_dom);
$table_dom->preserveWhiteSpace = false;
$yt_link_node = $xpath->query("//table/tr/td[2]/a[2]");
foreach($yt_link_node as $yt_link){
$yt = $yt_link->getAttribute('href');
echo $yt . "<br>";
echo "<br>";
}
}
I thank Abel, your help was greatly useful to achieve the tasks. :)
Based on the following XML I want to retrieve the product name and image of a certain node (ID)
<?xml version="1.0" encoding="utf-8"?>
<clients>
<client id="A">
<product>Name of product A</product>
<image>Product image name A</image>
</client>
<client id="B">
<product>Name of product B</product>
<image>Product image name B</image>
</client>
</clients>
This is the PHP Code:
$doc = DOMDocument::load('clients.xml');
$xpath = new DOMXPath($doc);
$query = '//client[#id="A"]';
$info = $xpath->query($query);
If I do $info->item(0)->nodeValue I get both information together and not individually:
Name of product A
Product image name A
But I want to get ->product->nodeValue and image->nodeValue based on the client ID.
How can I do that? Doing $info->item(0)->product->nodeValue for example doesn't work.
->item(0)->nodeValue gives everything inside that particular item (in this case 0)
You use XPath to fetch the nodes. The result of an XPath location path is a list of node. ->item(0) returns the first node in this list.This is the client element node.
Calling DOMElement:$nodeValue always returns all descendent text nodes as a string. In you case the text nodes inside the product and the image element nodes.
You will have the to fetch the child nodes if you want the values separately.
$dom = new DOMDocument();
$dom->loadXML($xml);
$xpath = new DOMXpath($dom);
foreach ($xpath->evaluate('//client[#id="A"]/*[self::product or self::image]') as $child) {
echo $child->localName, ': ', $child->nodeValue, "\n";
}
Output:
product: Name of product A
image: Product image name A
Your result returns a single node (client). The nodevalue of that node is all of the text underneath it.
Try:
"//client [#id='A']/*[name()='product' or name()='image']" would return two nodes. Remember to check the name if you want to use them by position.
I want to completely remove the size="id" attribute from every <door> element.
<?xml version="1.0" encoding="UTF-8"?>
<doors>
<door id="1" entry="3249" size="30"/>
<door id="1041" entry="6523" size="3094"/>
-- and 1000 more....
</doors>
The PHP code:
$xml = new SimpleXMLElement('http://mysite/doors.xml', NULL, TRUE);
$ids_to_delete = array( 1, 1506 );
foreach ($ids_to_delete as $id) {
$result = $xml->xpath( "//door[#size='$id']" );
foreach ( $result as $node ) {
$dom = dom_import_simplexml($node);
$dom->parentNode->removeChild($dom);
}
}
$xml->saveXml();
I get no errors but it does not delete the size attribute. Why?
I get no errors but it does not delete the size attribute. Why?
There are mulitple reasons why it does not delete the size attribute. The one that popped first into my mind was that attributes are no child nodes. Using a method to remove a child does just not fit to remove an attribute.
Each element node has an associated set of attribute nodes; the element is the parent of each of these attribute nodes; however, an attribute node is not a child of its parent element.
From: Attribute Nodes - XML Path Language (XPath), bold by me.
However, you don't see an error here, because the $result you have is an empty array. You just don't select any nodes to remove - neither elements nor attributes - with your xpath. That is because there is no such element you look for:
//door[#size='1']
You're searching for the id in the size attribute: No match.
These are the reasons why you get no errors and it does not delete any size attribute: 1.) you don't delete attributes here, 2.) you don't query any elements to delete attributes from.
How to delete attributes in SimpleXML queried by Xpath?
You can remove the attribute nodes by selecting them with an Xpath query and then unset the SimpleXMLElement self-reference:
// all size attributes of all doors
$result = $xml->xpath("//door/#size");
foreach ($result as $node) {
unset($node[0]);
}
In this example, all attribute nodes are queried by the Xpath expressions that are size attributes of door elements (which is what you ask for in your question) and then those are removed from the XML.
//door/#size
(see Abbreviated Syntax)
Now here the full example:
<?php
/**
* #link https://eval.in/215817
*/
$buffer = <<<XML
<?xml version="1.0" encoding="UTF-8"?>
<doors>
<door id="1" entry="3249" size="30"/>
<door id="1041" entry="6523" size="3094"/>
-- and 1000 more....
</doors>
XML;
$xml = new SimpleXMLElement($buffer);
// all size attributes of all doors
$result = $xml->xpath("//door/#size");
foreach ($result as $node) {
unset($node[0]);
}
$xml->saveXml("php://output");
Output (Online Demo):
<?xml version="1.0" encoding="UTF-8"?>
<doors>
<door id="1" entry="3249"/>
<door id="1041" entry="6523"/>
-- and 1000 more....
</doors>
You can do your whole query in DOMDocument using DOMXPath, rather than switching between SimpleXML and DOM:
$dom = new DOMDocument;
$dom->load('my_xml_file.xml');
# initialise an XPath object to act on the $dom object
$xp = new DOMXPath( $dom );
# run the query
foreach ($xp->query( "//door[#size]" ) as $door) {
# remove the attribute
$door->removeAttribute('size');
}
print $dom->saveXML();
Output for the input you supplied:
<?xml version="1.0" encoding="UTF-8"?>
<doors>
<door id="1" entry="3249"/>
<door id="1041" entry="6523"/>
</doors>
If you do want only to remove the size attribute for the IDs in your list, you should use the code:
foreach ($ids_to_delete as $id) {
# searches for elements with a matching ID and a size attribute
foreach ($xp->query("//door[#id='$id' and #size]") as $door) {
$door->removeAttribute('size');
}
}
Your code wasn't working for several reasons:
it looks like your XPath was wrong, since your array is called $ids_to_delete and your XPATH is looking for door elements with the size attribute equal to the value from $ids_to_delete;
you're converting the nodes to DOMDocument objects ($dom = dom_import_simplexml($node);) to do the deletion, but $xml->saveXml();, which I presume you printed somehow, is a SimpleXML object;
you need to remove the element attribute; removeChild removes the whole element.
Hello I know there is many questions here about those three topics combined together to update XML entries, but it seems everyone is very specific to a given problem.
I have been spending some time trying to understand XPath and its way, but I still can't get what I need to do.
Here we go
I have this XML file
<?xml version="1.0" encoding="UTF-8"?>
<storagehouse xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="schema.xsd">
<item id="c7278e33ef0f4aff88da10dfeeaaae7a">
<name>HDMI Cable 3m</name>
<weight>0.5</weight>
<category>Cables</category>
<location>B3</location>
</item>
<item id="df799fb47bc1e13f3e1c8b04ebd16a96">
<name>Dell U2410</name>
<weight>2.5</weight>
<category>Monitors</category>
<location>C2</location>
</item>
</storagehouse>
What I would like to do is to update/edit any of the nodes above when I need to. I will do a Html form for that.
But my biggest conserne is how do I find and update a the desired node and update it?
Here I have some of what I am trying to do
<?php
function fnDOMEditElementCond()
{
$dom = new DOMDocument();
$dom->load('storage.xml');
$library = $dom->documentElement;
$xpath = new DOMXPath($dom);
// I kind of understand this one here
$result = $xpath->query('/storagehouse/item[1]/name');
//This one not so much
$result->item(0)->nodeValue .= ' Series';
// This will remove the CDATA property of the element.
//To retain it, delete this element (see delete eg) & recreate it with CDATA (see create xml eg).
//2nd Way
//$result = $xpath->query('/library/book[author="J.R.R.Tolkein"]');
// $result->item(0)->getElementsByTagName('title')->item(0)->nodeValue .= ' Series';
header("Content-type: text/xml");
echo $dom->saveXML();
}
?>
Could someone maybe give me an examples with attributes and so on, so one a user decides to update a desired node, I could find that node with XPath and then update it?
The following example is making use of simplexml which is a close friend of DOMDocument. The xpath shown is the same regardless which method you use, and I use simplexml here to keep the code low. I'll show a more advanced DOMDocument example later on.
So about the xpath: How to find the node and update it. First of all how to find the node:
The node has the element/tagname item. You are looking for it inside the storagehouse element, which is the root element of your XML document. All item elements in your document are expressed like this in xpath:
/storagehouse/item
From the root, first storagehouse, then item. Divided with /. You already know that, so the interesting part is how to only take those item elements that have the specific ID. For that the predicate is used and added at the end:
/storagehouse/item[#id="id"]
This will return all item elements again, but this time only those which have the attribute id with the value id (string). For example in your case with the following XML:
$xml = <<<XML
<?xml version="1.0" encoding="UTF-8"?>
<storagehouse xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="schema.xsd">
<item id="c7278e33ef0f4aff88da10dfeeaaae7a">
<name>HDMI Cable 3m</name>
<weight>0.5</weight>
<category>Cables</category>
<location>B3</location>
</item>
<item id="df799fb47bc1e13f3e1c8b04ebd16a96">
<name>Dell U2410</name>
<weight>2.5</weight>
<category>Monitors</category>
<location>C2</location>
</item>
</storagehouse>
XML;
that xpath:
/storagehouse/item[#id="df799fb47bc1e13f3e1c8b04ebd16a96"]
will return the computer monitor (because such an item with that id exists). If there would be multiple items with the same id value, multiple would be returned. If there were none, none would be returned. So let's wrap that into a code-example:
$simplexml = simplexml_load_string($xml);
$result = $simplexml->xpath(sprintf('/storagehouse/item[#id="%s"]', $id));
if (!$result || count($result) !== 1) {
throw new Exception(sprintf('Item with id "%s" does not exists or is not unique.', $id));
}
list($item) = $result;
In this example, $titem is the SimpleXMLElement object of that computer monitor xml element name item.
So now for the changes, which are extremely easy with SimpleXML in your case:
$item->category = 'LCD Monitor';
And to finally see the result:
echo $simplexml->asXML();
Yes that's all with SimpleXML in your case.
If you want to do this with DOMDocument, it works quite similar. However, for updating an element's value, you need to access the child element of that item as well. Let's see the following example which first of all fetches the item as well. If you compare with the SimpleXML example above, you can see that things not really differ:
$doc = new DOMDocument();
$doc->loadXML($xml);
$xpath = new DOMXPath($doc);
$result = $xpath->query(sprintf('/storagehouse/item[#id="%s"]', $id));
if (!$result || $result->length !== 1) {
throw new Exception(sprintf('Item with id "%s" does not exists or is not unique.', $id));
}
$item = $result->item(0);
Again, $item contains the item XML element of the computer monitor. But this time as a DOMElement. To modify the category element in there (or more precisely it's nodeValue), that children needs to be obtained first. You can do this again with xpath, but this time with an expression relative to the $item element:
./category
Assuming that there always is a category child-element in the item element, this could be written as such:
$category = $xpath->query('./category', $item)->item(0);
$category does now contain the first category child element of $item. What's left is updating the value of it:
$category->nodeValue = "LCD Monitor";
And to finally see the result:
echo $doc->saveXML();
And that's it. Whether you choose SimpleXML or DOMDocument, that depends on your needs. You can even switch between both. You probably might want to map and check for changes:
$repository = new Repository($xml);
$item = $repository->getItemByID($id);
$item->category = 'LCD Monitor';
$repository->saveChanges();
echo $repository->getXML();
Naturally this requires more code, which is too much for this answer.