HTML DOM Merge paged XML file into single file with a loop - php

Question

Hey I'm working on an import for a list of elements. The code works for now, but it is not futureproof if there are more items added. The XML uses an unique key and pagination (every 100 items a new key).
Below is my PHP code for the function I've build.
<?php
$feedUrl = '[url of the feed]';
$doc1 = new DOMDocument();
$doc1->load($feedUrl);
$doc1_token = $doc1->getElementsByTagName('resumptionToken')[0]->nodeValue;
$doc2 = new DOMDocument();
$doc2->load($feedUrl . '&resumptionToken=' . $doc1_token);
$doc2_token = $doc2->getElementsByTagName('resumptionToken')[0]->nodeValue;
$doc3 = new DOMDocument();
$doc3->load($feedUrl . '&resumptionToken=' . $doc2_token);
$doc3_token = $doc3->getElementsByTagName('resumptionToken')[0]->nodeValue;
$doc4 = new DOMDocument();
$doc4->load($feedUrl . '&resumptionToken=' . $doc3_token);
$doc4_token = $doc4->getElementsByTagName('resumptionToken')[0]->nodeValue;
$doc5 = new DOMDocument();
$doc5->load($feedUrl . '&resumptionToken=' . $doc4_token);
$doc5_token = $doc5->getElementsByTagName('resumptionToken')[0]->nodeValue;
// get 'ListRecordes' element of document 1
$list_records = $doc1->getElementsByTagName('ListRecords')->item(0); //edited res - items
// iterate over 'item' elements of document 2
$items2 = $doc2->getElementsByTagName('record');
for ($i = 0; $i < $items2->length; $i ++) {
$item2 = $items2->item($i);
// import/copy item from document 2 to document 1
$item1 = $doc1->importNode($item2, true);
// append imported item to document 1 'res' element
$list_records->appendChild($item1);
}
// iterate over 'item' elements of document 3
$items3 = $doc3->getElementsByTagName('record');
for ($i = 0; $i < $items3->length; $i ++) {
$item3 = $items3->item($i);
// import/copy item from document 3 to document 1
$item1 = $doc1->importNode($item3, true);
// append imported item to document 1 'res' element
$list_records->appendChild($item1);
}
// iterate over 'item' elements of document 4
$items4 = $doc4->getElementsByTagName('record');
for ($i = 0; $i < $items4->length; $i ++) {
$item4 = $items4->item($i);
// import/copy item from document 4 to document 1
$item1 = $doc1->importNode($item4, true);
// append imported item to document 1 'res' element
$list_records->appendChild($item1);
}
// iterate over 'item' elements of document 5
$items5 = $doc5->getElementsByTagName('record');
for ($i = 0; $i < $items5->length; $i ++) {
$item5 = $$items5->item($i);
// import/copy item from document 5 to document 1
$item1 = $doc1->importNode($item5, true);
// append imported item to document 1 'res' element
$list_records->appendChild($item1);
}
$doc1->save('merged.xml'); //edited -added saving into xml file
I think the code is not perfect, because if the we add more records than 600, the latest one's are not imported in the merged xml.
Besides this there is also an other issue. We have nested "" nodes. We need to merge the "" direct childs only.
<ListRecords>
<record>
<header>
...
</header>
<metadata>
<record xmlns="http://www.openarchives.org/OAI/2.0/" priref="100000002">
...
</record>
</metadata>
</record>
</ListRecords>

Answer 1

You can use Xpath expressions to address specific elements. On your snippet that would be /ListRecords/record. However I think it misses the document element node with the namespace declaration for the Open Archives Initiative Protocol for Metadata Harvesting. It should be something like:
<OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/">
<ListRecords>
<record>
<header>
<identifier>oai:arXiv.org:hep-th/9901001</identifier>
</header>
</record>
</ListRecords>
</OAI-PMH>
To address the namespace with Xpath you need to register a prefix for it. Then put the feed urls in an array and iterate them:
$mergeDocument = new DOMDocument();
$mergeDocument->loadXML(
'<OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/"><ListRecords/></OAI-PMH>'
);
$mergeTarget = $mergeDocument->documentElement->firstChild;
foreach ($feedUrls as $feedUrl) {
$document = new DOMDocument();
$document->load($feedUrl);
$xpath = new DOMXpath($document);
$xpath->registerNamespace('oai', 'http://www.openarchives.org/OAI/2.0/');
foreach ($xpath->evaluate('/oai:OAI-PMH/oai:ListRecords/oai:record') as $record) {
$mergeTarget->appendChild($mergeDocument->importNode($record, TRUE));
}
}
$mergeDocument->formatOutput = TRUE;
echo $mergeDocument->saveXML();