I am iterating through the results of a service call to yahoo news thus:
//Send service request
if (!$yahooResults = file_get_contents($yahooRequest)) {
echo 'Error processing service request';
}
//Read result into xml document
$yahooResultXml = new DOMDocument('1.0', 'UTF-8');
$yahooResultXml->loadXML($yahooResults);
//Build page
include_once('components/pageHeader.php');
echo 'Search Results';
//echo $yahooResultXml->saveHTML();
//Iterate over each Result node
$stories = $yahooResultXml->getElementsByTagName('Result');
foreach ($stories as $story) {
//Title
//Summary
//Url
//Source
//Language
//Publish Date
//Modification Date
}
include_once('components/pageFooter.php');
Each Title is in a Title node within a Result Node. I cannot figure out how to simply echo the content of the Title node!
Check the first user comment at PHP: DOMDocument::getElementsByTagName
$items =
$doc->getElementsByTagName('item');
$headlines = array();
foreach($items as $item) {
$headline = array();
if($item->childNodes->length) {
foreach($item->childNodes as $i) {
$headline[$i->nodeName] =
$i->nodeValue;
}
}
$headlines[] = $headline;
}
Related
I am trying to scrape this webpage. In this webpage I have to get the job title and its location. Which I am able to get from my code. But the problem is coming that when I am sending it in XML, then only one detail is going from the array list.
I am using goutte CSS selector library and also please tell me how to scrap pagination in goutte CSS selector library.
here is my code:
$httpClient = new \Goutte\Client();
$response = $httpClient->request('GET', 'https://www.simplyhired.com/search?q=pharmacy+technician&l=American+Canyon%2C+CA&job=X5clbvspTaqzIHlgOPNXJARu8o4ejpaOtgTprLm2CpPuoeOFjioGdQ');
$job_posting_location = [];
$response->filter('.LeftPane article .SerpJob-jobCard.card .jobposting-subtitle span.JobPosting-labelWithIcon.jobposting-location span.jobposting-location')
->each(function ($node) use (&$job_posting_location) {
$job_posting_location[] = $node->text() . PHP_EOL;
});
$joblocation = 0;
$response->filter('.LeftPane article .SerpJob-jobCard.card .jobposting-title-container h3 a')
->each( function ($node) use ($job_posting_location, &$joblocation, $httpClient) {
$job_title = $node->text() . PHP_EOL; //job title
$job_posting_location = $job_posting_location[$joblocation]; //job posting location
// display the result
$items = "{$job_title} # {$job_posting_location}\n\n";
global $results;
$result = explode('#', $items);
$results['job_title'] = $result[0];
$results['job_posting_location'] = $result[1];
$joblocation++;
});
function convertToXML($results, &$xml_user_info){
foreach($results as $key => $value){
if(is_array($value)){
$subnode = $xml_user_info->addChild($key);
foreach ($value as $k=>$v) {
$xml_user_info->addChild("$k",htmlspecialchars("$v"));
}
}else{
$xml_user_info->addChild("$key",htmlspecialchars("$value"));
}
}
return $xml_user_info->asXML();
}
$xml_user_info = new SimpleXMLElement('<root/>');
$xml_content = convertToXML($results,$xml_user_info);
$xmlFile = 'details.xml';
$handle = fopen($xmlFile, 'w') or die('Unable to open the file: '.$xmlFile);
if(fwrite($handle, $xml_content)) {
echo 'Successfully written to an XML file.';
}
else{
echo 'Error in file generating';
}
what i got in xml file --
<?xml version="1.0"?>
<root><job_title>Pharmacy Technician
</job_title><job_posting_location> Vallejo, CA
</job_posting_location></root>
what i want in xml file --
<?xml version="1.0"?>
<root>
<job_title>Pharmacy Technician</job_title>
<job_posting_location> Vallejo, CA</job_posting_location>
<job_title>Pharmacy Technician 1</job_title>
<job_posting_location> Vallejo, CA</job_posting_location>
<job_title>Pharmacy Technician New</job_title>
<job_posting_location> Vallejo, CA</job_posting_location>
and so on...
</root>
You overwrite the values in the $results variable. You're would need to do something like this to append:
$results[] = [
'job_title' => $result[0];
'job_posting_location' => $result[1]
];
However here is no need to put the data into an array at all, just create the
XML directly with DOM.
Both your selectors share the same start. Iterate the card and then fetch
related data.
$httpClient = new \Goutte\Client();
$response = $httpClient->request('GET', $url);
$document = new DOMDocument();
// append document element node
$postings = $document->appendChild($document->createElement('jobs'));
// iterate job posting cards
$response->filter('.LeftPane article .SerpJob-jobCard.card')->each(
function($jobCard) use ($document, $postings) {
// fetch data
$location = $jobCard
->filter(
'.jobposting-subtitle span.JobPosting-labelWithIcon.jobposting-location span.jobposting-location'
)
->text();
$title = $jobCard->filter('.jobposting-title-container h3 a')->text();
// append 'job' node to group data in result
$job = $postings->appendChild($document->createElement('job'));
// append data nodes
$job->appendChild($document->createElement('job_title'))->textContent = $title;
$job->appendChild($document->createElement('job_posting_location'))->textContent = $location;
}
);
echo $document->saveXML();
I'm stuck on something extremely simple.
Here is my xml feed:
http://xml.betfred.com/Horse-Racing-Daily.xml
Here is my code
<?php
function HRList5($viewbets) {
$xmlData = 'http://xml.betfred.com/Horse-Racing-Daily.xml';
$xml = simplexml_load_file($xmlData);
$curdate = date('d/m/Y');
$new_array = array();
foreach ($xml->event as $event) {
if($event->bettype->attributes()->bettypeid == $viewbets){//$_GET['evid']){
// $eventid = $_GET['eventid'];
// if ($limit == $c) {
// break;
// }
// $c++;
$eventd = substr($event->attributes()->{'date'},6,2);
$eventm = substr($event->attributes()->{'date'},4,2);
$eventy = substr($event->attributes()->{'date'},0,4);
$eventt = $event->attributes()->{'time'};
$eventid = $event->attributes()->{'eventid'};
$betname = $event->bettype->bet->attributes()->{'name'};
$bettypeid = $event->bettype->attributes()->{'bettypeid'};
$betprice = $event->bettype->bet->attributes()->{'price'};
$betid = $event->bettype->bet->attributes()->{'id'};
$new_array[$betname.$betid] = array(
'betname' => $betname,
'viewbets' => $viewbets,
'betid' => $betid,
'betname' => $betname,
'betprice' => $betprice,
'betpriceid' => $event->bettype->attributes()->{'betid'},
);
}
ksort($new_array);
$limit = 10;
$c = 0;
foreach ($new_array as $event_time => $event_data) {
// $racedate = $event_data['eventy'].$event_data['eventm'].$event_data['eventd'];
$today = date('Ymd');
//if($today == $racedate){
// if ($limit == $c) {
// break;
//}
//$c++;
$replace = array("/"," ");
// $eventname = str_replace($replace,'-', $event_data['eventname']);
//$venue = str_replace($replace,'-', $event_data['venue']);
echo "<div class=\"units-row unit-100\">
<div class=\"unit-20\" style=\"margin-left:0px;\">
".$event_data['betprice']."
</div>
<div class=\"unit-50\">
".$event_data['betname'].' - '.$event_data['betprice']."
</div>
<div class=\"unit-20\">
<img src=\"betnow.gif\" ><br />
</div>
</div>";
}
}//echo "<strong>View ALL Horse Races</strong> <strong>>></strong>";
//var_dump($event_data);
}
?>
Now basically the XML file contains a list of horse races that are happening today.
The page I call the function on also declares
<?php $viewbets = $_GET['EVID'];?>
Then where the function is called I have
<?php HRList5($viewbets);?>
I've just had a play around and now it displays the data in the first <bet> node
but the issue is it's not displaying them ALL, its just repeating the 1st one down the page.
I basically need the xml feed queried & if the event->bettype->attributes()->{'bettypeid'} == $viewbets I want the bet nodes repeated down the page.
I don't use simplexml so can offer no guidance with that - I would say however that to find the elements and attributes you need within the xml feed that you ought to use an XPath query. The following code will hopefully be of use in that respect, it probably has an easy translation into simplexml methods.
Edit: Rather than targeting each bet as the original xpath did which then caused issues, the following should be more useful. It targets the bettype and then processes the childnodes.
/* The `eid` to search for in the DOM document */
$eid=25573360.20;
/* create the DOM object & load the xml */
$dom=new DOMDocument;
$dom->load( 'http://xml.betfred.com/Horse-Racing-Daily.xml' );
/* Create a new XPath object */
$xp=new DOMXPath( $dom );
/* Search the DOM for nodes with particular attribute - bettypeid - use number function from XSLT to test */
$oCol=$xp->query('//event/bettype[ number( #bettypeid )="'.$eid.'" ]');
/* If the query was successful there should be a nodelist object to work with */
if( $oCol ){
foreach( $oCol as $node ) {
echo '
<h1>'.$node->parentNode->getAttribute('name').'</h1>
<h2>'.date('D, j F, Y',strtotime($node->getAttribute('bet-start-date'))).'</h2>';
foreach( $node->childNodes as $bet ){
echo "<div>Name: {$bet->getAttribute('name')} ID: {$bet->getAttribute('id')} Price: {$bet->getAttribute('price')}</div>";
}
}
} else {
echo 'XPath query failed';
}
$dom = $xp = $col = null;
I am trying to get the data in <div id listing-page-cart-inner> and <div id="description text"> and <div id="tags">, but i am finding it difficult to mine data.
Can anyone guide me? I am not able to fetch data though first div that I mentioned I am able to scrape, but other div I am not able to. When I loop through the second foreach it takes longer time.
<?php
include_once('simple_html_dom.php');
$html = file_get_html('https://etsy.com/listing/107492702/');
//$val = $html->find('div[id=listing-page-cart-inner]');
function scraping_etsy() {
// create HTML DOM
$html = file_get_html('https://etsy.com/listing/107492702/');
foreach($html->find('div[id=listing-page-cart-inner]') as $article)
{
// get title
//$item['title'] = trim($article->find('h3', 0)->plaintext);
// get details
$item['details'] = trim($article->find('span', 0)->plaintext);
// get intro
//$lists = $articles->find('div[id=item-overview]');
$item['list1'] = trim($article->find('li',0)->plaintext);
$item['list2'] = trim($article->find('li',1)->plaintext);
$item['list3'] = trim($article->find('li',2)->plaintext);
$item['list4'] = trim($article->find('li',3)->plaintext);
$item['list5'] = trim($article->find('li',4)->plaintext);
/*foreach($article->find('li') as $al){
$item['lists'] =trim($al->find('li')->plaintext);
}*/
$ret[] = $item;
}
foreach($html->find('div[id=description]') as $content){
var_dump($content->find('text'));
// $item['content'] = trim($content->find('div[id=description]')->plaintext);
// $ret[] = $item;
}
// clean up memory
$html->clear();
unset($html);
return $ret ;
}
$ret = scraping_etsy();
var_dump($ret);
/*foreach($ret as $v) {
echo $v['title'].'<br>';
echo '<ul>';
echo '<li>'.$v['details'].'</li>';
echo '<li>Diggs: '.$v['diggs'].'</li>';
echo '</ul>';
}*/
?>
As for getting children of those divs, just remember that if found the parent element, always use ->find('<the selector here>', 0) always use the index to actually point to that element.
$html = file_get_html('https://etsy.com/listing/107492702/');
// listings with description
$div = $html->find('div#listing-page-cart-inner', 0); // here index zero
$main_description = $div->find('h1', 0)->innertext;
echo $main_description . '<br/><br/>';
$div_item_overview = $div->find('div#item-overview ul.properties li');
foreach ($div_item_overview as $overview) {
echo $overview->innertext . '<br/>';
}
// tags
$div_tag = $html->find('div#tags', 0); // here index zero pointing to that element
$tags = array();
foreach($div_tag->find('ul li') as $li) {
$tags[] = $li->find('a', 0)->innertext;
}
echo '<pre>', print_r($tags, 1), '</pre>';
// description
$div_description = $html->find('div#description', 0)->plaintext; // here pointing to index zero
echo $div_description;
The easiest way to start always is to use 3d-party library, i.e. Symfony DomCrawler
It usage as easy as
use Symfony\Component\DomCrawler\Crawler;
$html = <<<'HTML'
<!DOCTYPE html>
<html>
<body>
<p class="message">Hello World!</p>
<p>Hello Crawler!</p>
</body>
</html>
HTML;
$crawler = new Crawler($html);
foreach ($crawler as $domElement) {
print $domElement->nodeName;
}
And you can use filters like
$crawler = $crawler->filter('body > p');
I have code trying to extract the Event SKU from the Robot Events Page, here is an example. The code that I am using dosn't find any of the SKU on the page. The SKU is on line 411, with a div of the class "product-sku". My code doesn't event find the Div on the page and just downloads all the events. Here is my code:
<?php
require('simple_html_dom.php');
$html = new simple_html_dom();
if(!$events)
{
echo mysqli_error($con);
}
while($event = mysqli_fetch_row($events))
{
$htmldown = file_get_html($event[4]);
$html->load($htmldown);
echo "Downloaded";
foreach ($html->find('div[class=product-sku]') as $row) {
$sku = $row->plaintext;
echo $sku;
}
}
?>
Can anyone help me fix my code?
This code is used DOMDocument php class. It works successfully for below sample HTML. Please try this code.
// new dom object
$dom = new DOMDocument();
// HTML string
$html_string = '<html>
<body>
<div class="product-sku1" name="div_name">The this the div content product-sku</div>
<div class="product-sku2" name="div_name">The this the div content product-sku</div>
<div class="product-sku" name="div_name">The this the div content product-sku</div>
</body>
</html>';
//load the html
$html = $dom->loadHTML($html_string);
//discard white space
$dom->preserveWhiteSpace = TRUE;
//the table by its tag name
$divs = $dom->getElementsByTagName('div');
// loop over the all DIVs
foreach ($divs as $div) {
if ($div->hasAttributes()) {
foreach ($div->attributes as $attribute){
if($attribute->name === 'class' && $attribute->value == 'product-sku'){
// Peri DIV class name and content
echo 'DIV Class Name: '.$attribute->value.PHP_EOL;
echo 'DIV Content: '.$div->nodeValue.PHP_EOL;
}
}
}
}
I would use a regex (regular expression) to accomplish pulling skus out.
The regex:
preg_match('~<div class="product-sku"><b>Event Code:</b>(.*?)</div>~',$html,$matches);
See php regex docs.
New code:
<?php
if(!$events)
{
echo mysqli_error($con);
}
while($event = mysqli_fetch_row($events))
{
$htmldown = curl_init($event[4]);
curl_setopt($htmldown, CURLOPT_RETURNTRANSFER, true);
$html=curl_exec($htmldown);
curl_close($htmldown)
echo "Downloaded";
preg_match('~<div class="product-sku"><b>Event Code:</b>(.*?)</div>~',$html,$matches);
foreach ($matches as $row) {
echo $row;
}
}
?>
And actually in this case (using that webpage) being that there is only one sku...
instead of:
foreach ($matches as $row) {
echo $row;
}
You could just use: echo $matches[1]; (The reason for array index 1 is because the whole regex pattern plus the sku will be in $matches[0] but just the subgroup containing the sku is in $matches[1].)
try to use
require('simple_html_dom.php');
$html = new simple_html_dom();
if(!$events)
{
echo mysqli_error($con);
}
while($event = mysqli_fetch_row($events))
{
$htmldown = str_get_html($event[4]);
echo "Downloaded";
foreach ($htmldown->find('div[class=product-sku]') as $row) {
$sku = $row->plaintext;
echo $sku;
}
}
and if class "product-sku" is only for div's then you can use
$htmldown->find('.product-sku')
I'm building a photo gallery, building an object based on an xml file.
How can I grab the next and previous nodes? Here's what my base code looks like:
$xmlData = new SimpleXMLElement(file_get_contents("data.xml"));
foreach($xmlData->row as $item) {
if ($item->url == $_GET['id']) {
// show photo
$title = $item->title;
}
}
Only usable if the next/prev nodes are of the same type. If you want more complex processing use DOM.
$xmlData = new SimpleXMLElement(file_get_contents("data.xml"));
$index = 0;
foreach($xmlData->row as $item) {
if ($item->url == $_GET['id']) {
// show photo
$title = $item->title;
$prev = $xmlData->row[$index-1];
$next = $xmlData->row[$index+1];
}
$index++;
}