Puphpeteer - Get text and href-attribute from link - php

Question

I am using "#nesk/puphpeteer": "^2.0.0" Link to Github-Repo and want get the text and the href-attribute from a link.
I tried the following:
<?php
require_once '../vendor/autoload.php';
use Nesk\Puphpeteer\Puppeteer;
use Nesk\Rialto\Data\JsFunction;
$debug = true;
$puppeteer = new Puppeteer([
'read_timeout' => 100,
'debug' => $debug,
]);
$browser = $puppeteer->launch([
'headless' => !$debug,
'ignoreHTTPSErrors' => true,
]);
$page = $browser->newPage();
$page->goto('http://example.python-scraping.com/');
//get text and link
$links = $page->querySelectorXPath('//*[#id="results"]/table/tbody/tr/td/div/a', JsFunction::createWithParameters(['node'])
->body('return node.textContent;'));
// iterate over links and print each link and its text
// get single text
$singleText = $page->querySelectorXPath('//*[#id="pagination"]/a', JsFunction::createWithParameters(['node'])
->body('return node.textContent;'));
$browser->close();
When I run the above script I get the nodes from the page, BUT I cannot access the attributes or the text?
Any suggestions how to do this?
I appreciate your replies!

Answer 1

querySelectorXPath return array of ElementHandle. one more thing querySelectorXPath does not support callback function.
first get all node ElementHandle
$links = $page->querySelectorXPath('//*[#id="results"]/table/tbody/tr/td/div/a');
then loop over links to access attributes or text of node
foreach($links as $link){
// for text
$text = $link->evaluate(JsFunction::createWithParameters(['node'])
->body('return node.innerText;'));
// for link
$link = $link->evaluate(JsFunction::createWithParameters(['node'])
->body('return node.href;'));
}

Answer 2

Disclaimer: This is just an intermediate answer - I would update once I've got more specific requests on HTML attrs or other expectations to be retrieved.
tl;dr: Mentioned composer package nesk/puphpeteer really is just a wrapper to underlying NodeJS based implementation of puppeteer. Thus, accessing data and structures has to be "similar" to their JavaScript counterparts...
Maybe Codeception (headless) or symfony/dom-crawler (raw markup) might be better and more mature alternatives.
Anyway, let's pick the example from above and go through it step by step:
$links = $page->querySelectorXPath(
'//*[#id="results"]/table/tbody/tr/td/div/a',
JsFunction::createWithParameters(['node'])->body('return node.textContent;')
);
XPath query $x() would result in an array of ElementHandle items
to access exported node.textContent (from JsFunction), corresponding data gets fetched via ElementHandle.getProperty(prop)
exporting a scalar value (to PHP) is then done via ElementHandle.jsonValue()
Thus, after that we would have something like this:
$links = $page->querySelectorXPath(
'//*[#id="results"]/table/tbody/tr/td/div/a',
JsFunction::createWithParameters(['node'])->body('return node.textContent;')
);
/** #var \Nesk\Puphpeteer\Resources\ElementHandle $link */
foreach ($links as $link) {
var_dump($link->getProperty('textContent')->jsonValue());
}
Which outputs the following raw data (as retrieved from http://example.python-scraping.com/):
string(12) " Afghanistan"
string(14) " Aland Islands"
string(8) " Albania"
string(8) " Algeria"
string(15) " American Samoa"
string(8) " Andorra"
string(7) " Angola"
string(9) " Anguilla"
string(11) " Antarctica"
string(20) " Antigua and Barbuda"