I have script which scrap/fetch html table data from another website. The website date format is 26/8/2011, How I can change it to this format 2011-12-13??
function createRSSFile($tag,$value,$data)
{
# this will return the each element with tag.
$tag=strtolower(str_replace(" ","_",$tag));
$tag=strtolower(str_replace(":","",$tag));
$tag=strtolower(str_replace("&","and",$tag));
$returnITEM = "<".$tag.">".htmlspecialchars(str_replace(" 00:00:00","",$value))."</".$tag.">";
return $returnITEM;
}
function fetchData($jobid) {
$html=file_get_contents('http://acbar.org/JobDetail.aspx?id='.$jobid);
$html=str_replace("<td></td>", "",$html);
$html=str_replace("<td style=\"font-size:8pt;font-weight:bold;\"></td>","<td style=\"font-size:8pt;font-weight:bold;\">Null</td>",$html);
$html=str_replace("<td style=\"font-size:8pt;font-weight:bold;\" colspan=\"2\" ></td>","<td style=\"font-size:8pt;font-weight:bold;\" colspan=\"2\" >Null</td>",$html);
$html=str_replace(" ", " ",$html);
$html=str_replace("?", "<br>",$html);
$html=str_replace("<br>", "_br_",$html);
$dom = new DOMDocument;
$dom->loadHTML( $html );
//echo $dom->saveHTML();
//exit;
$rows = array();
foreach( $dom->getElementsByTagName( 'tr' ) as $tr ) {
$cells = array();
foreach( $tr->getElementsByTagName( 'td' ) as $td ) {
if(trim($td->nodeValue)!='')
$cells[] = str_replace("_br_","<br>",trim($td->nodeValue));
}
if(sizeof($cells)>0)
$rows[] = $cells;
}
Could always strtotime the date passed, then format it however you wish with date, like so...
$timeToModify = strtotime($passedTime);
$formattedTime = date("Y-m-D", $timeToModify);
Related
Hey I've been trying to scrape data from an html table and I'm not having much luck.
Website: https://www.dnr.state.mn.us/hunting/seasons.html
What I'm trying to do: I want to grab the contents of the table and encode it into json like
['event_title' 'Waterfowl'] and ['event_date' '09/25/21']
but I don't know how to do this, I've tried a couple different things but in the end I can't get it to work.
Code Example (Closest I got):
<?php
$dom = new DOMDocument;
$page = file_get_contents('https://www.dnr.state.mn.us/hunting/seasons.html');
$dom->loadHTML($page);
$xpath = new DOMXPath($dom);
foreach ($xpath->query('//tbody/tr') as $tr) {
$tmp = []; // reset the temporary array so previous entries are removed
foreach ($xpath->query("td[#class]", $tr) as $td) {
$key = preg_match('~[a-z]+$~', $td->getAttribute('class'), $out) ? $out[0] : 'no_class';
if ($key === "event-title") {
$tmp['event_title'] = $xpath->query("a", $td);
}
$tmp[$key] = trim($td->textContent);
}
//$tmp['event_date'] = date("M. dS 'y", strtotime(preg_replace('~\.|\d+[ap]m *~', '', $tmp['date'])));
//$result[] = $tmp;
$marray[] = array_unique($tmp);
print_r($marray);
}
//$array2 = var_export($result);
//print_r($array2[1]);
//var_export($result);
//echo "\n----\n";
//echo json_encode($result);
?>
I am trying to generate a sitemap but somehow an extra DIV tag at the initial line of xml. I need to remove this wrong tag DIV from the xml output.
I've tried to gather the logic at first and segregate the generation of the xml side at the bottom.
set header 'text/xml'.
I tried to strip_tags the whole xml string before output, but then, it shows document empty
private function removeImageAndEmbeds ( $content )
{
// remove img tags
$re1='(<img).*?\\/.*?\\/.*?\\/.*?\\/.*?\\/.*?\\/.*?(\\/>)';
if ( $c=preg_replace("/".$re1."/is", "", $content) ) $content = $c;
// remove embedded tags
$re2='(<div).*?(data-oembed-url=)(".*?").*?<\\/div>.*?(<\\/div>)';
if ( $c=preg_replace("/".$re2."/is", "", $content) ) $content = $c;
return $content;
}
public function sitemaps ($tenantName="") {
if ( !empty($tenantName) ) {
$this->db->like( 't.name', str_replace('-', ' ', rawurldecode($tenantName)), 'none' );
$results = $this->db->get($this->TBL . ' t')->result_array();
foreach ( $results as $result ) {
$tenantId = $result['id'];
$tenantNameinURL = formatTenantNameinURL( $result['name'] );
$AllItems = $this->db->get_where($this->DIVIEW . ' di', 'di.account_id = '. $tenantId)->result_array();
$topics = [];
$itemIds = [];
$ddIds = [];
$urls = [];
foreach ( $AllItems as $k => $item ) {
$pieces = explode('_', $item['id']);
if ( $pieces[1] === $this->ITEMTBL ) {
if( !in_array($item['record_id'], $itemIds) ){
$itemIds[] = $item['record_id'];
$content = $this->removeImageAndEmbeds( $item['content'] );
$AllItems[$k]['content'] = $content;
$topics[$k][] = $AllItems[$k];
$urls[$k]['url'] = formatFrontEndURL( $this->current_class_name, $tenantName, 'show', $pieces[0] );
}
} else if ( $pieces[1] === 'dataDefinitions' ) {
if( !in_array($item['record_id'], $ddIds) ){
$ddIds[] = $item['record_id'];
$content = $this->removeImageAndEmbeds( $item['content'] );
$AllItems[$k]['content'] = $content;
$topics[$k][] = $AllItems[$k];
$urls[$k]['url'] = formatFrontEndURL( $this->current_class_name, $tenantName, 'data_definition', $pieces[0] );
}
}
}
$urlset = new SimpleXMLElement('<?xml version="1.0" encoding="UTF-8"?><urlset />');
$urlset->addAttribute('xmlns', 'http://www.sitemaps.org/schemas/sitemap/0.9');
foreach ($topics as $i => $itemsInTopic) {
$url = $urlset->addChild('url');
$url->loc = $urls[$i]['url'];
$pageMap = $url->addChild('PageMap');
$pageMap->addAttribute('xmlns', 'http://www.google.com/schemas/sitemap-pagemap/1.0');
foreach ( $itemsInTopic as $item ) {
$content = $item['content'];
$content = trim( str_replace([" ","\r","\n","\t", "
", "
"], ' ', strip_tags( utf8_decode( $content ) )) );
$dataObject = $pageMap->addChild('DataObject');
$dataObject->addAttribute('type', 'document');
$dataObject->addAttribute('id', $item['record_id']);
$dataObject->Attribute[0]['name'] = 'title';
$dataObject->Attribute[0] = $item['title'];
$dataObject->Attribute[1]['name'] = 'content';
$dataObject->Attribute[1] = $content;
}
}
$xmlContent = $urlset->asXML();
$this->output->set_content_type('text/xml')->set_output( $xmlContent );
}
}
}
here are two errors generated from seochat validator
https://drive.google.com/file/d/1vacmuJL6hnMErzqZ5zZWkkObT74rKOmT/view?usp=sharing
https://drive.google.com/file/d/1y3z85D1WtJIT9GvOC-DeYwS-DtQCAxK5/view?usp=sharing
here is google console error
https://drive.google.com/file/d/1qMvifyjGILqAjJzdWdc90jyymvdUFV5A/view?usp=sharing
I'm stuck on something extremely simple.
Here is my xml feed:
http://xml.betfred.com/Horse-Racing-Daily.xml
Here is my code
<?php
function HRList5($viewbets) {
$xmlData = 'http://xml.betfred.com/Horse-Racing-Daily.xml';
$xml = simplexml_load_file($xmlData);
$curdate = date('d/m/Y');
$new_array = array();
foreach ($xml->event as $event) {
if($event->bettype->attributes()->bettypeid == $viewbets){//$_GET['evid']){
// $eventid = $_GET['eventid'];
// if ($limit == $c) {
// break;
// }
// $c++;
$eventd = substr($event->attributes()->{'date'},6,2);
$eventm = substr($event->attributes()->{'date'},4,2);
$eventy = substr($event->attributes()->{'date'},0,4);
$eventt = $event->attributes()->{'time'};
$eventid = $event->attributes()->{'eventid'};
$betname = $event->bettype->bet->attributes()->{'name'};
$bettypeid = $event->bettype->attributes()->{'bettypeid'};
$betprice = $event->bettype->bet->attributes()->{'price'};
$betid = $event->bettype->bet->attributes()->{'id'};
$new_array[$betname.$betid] = array(
'betname' => $betname,
'viewbets' => $viewbets,
'betid' => $betid,
'betname' => $betname,
'betprice' => $betprice,
'betpriceid' => $event->bettype->attributes()->{'betid'},
);
}
ksort($new_array);
$limit = 10;
$c = 0;
foreach ($new_array as $event_time => $event_data) {
// $racedate = $event_data['eventy'].$event_data['eventm'].$event_data['eventd'];
$today = date('Ymd');
//if($today == $racedate){
// if ($limit == $c) {
// break;
//}
//$c++;
$replace = array("/"," ");
// $eventname = str_replace($replace,'-', $event_data['eventname']);
//$venue = str_replace($replace,'-', $event_data['venue']);
echo "<div class=\"units-row unit-100\">
<div class=\"unit-20\" style=\"margin-left:0px;\">
".$event_data['betprice']."
</div>
<div class=\"unit-50\">
".$event_data['betname'].' - '.$event_data['betprice']."
</div>
<div class=\"unit-20\">
<img src=\"betnow.gif\" ><br />
</div>
</div>";
}
}//echo "<strong>View ALL Horse Races</strong> <strong>>></strong>";
//var_dump($event_data);
}
?>
Now basically the XML file contains a list of horse races that are happening today.
The page I call the function on also declares
<?php $viewbets = $_GET['EVID'];?>
Then where the function is called I have
<?php HRList5($viewbets);?>
I've just had a play around and now it displays the data in the first <bet> node
but the issue is it's not displaying them ALL, its just repeating the 1st one down the page.
I basically need the xml feed queried & if the event->bettype->attributes()->{'bettypeid'} == $viewbets I want the bet nodes repeated down the page.
I don't use simplexml so can offer no guidance with that - I would say however that to find the elements and attributes you need within the xml feed that you ought to use an XPath query. The following code will hopefully be of use in that respect, it probably has an easy translation into simplexml methods.
Edit: Rather than targeting each bet as the original xpath did which then caused issues, the following should be more useful. It targets the bettype and then processes the childnodes.
/* The `eid` to search for in the DOM document */
$eid=25573360.20;
/* create the DOM object & load the xml */
$dom=new DOMDocument;
$dom->load( 'http://xml.betfred.com/Horse-Racing-Daily.xml' );
/* Create a new XPath object */
$xp=new DOMXPath( $dom );
/* Search the DOM for nodes with particular attribute - bettypeid - use number function from XSLT to test */
$oCol=$xp->query('//event/bettype[ number( #bettypeid )="'.$eid.'" ]');
/* If the query was successful there should be a nodelist object to work with */
if( $oCol ){
foreach( $oCol as $node ) {
echo '
<h1>'.$node->parentNode->getAttribute('name').'</h1>
<h2>'.date('D, j F, Y',strtotime($node->getAttribute('bet-start-date'))).'</h2>';
foreach( $node->childNodes as $bet ){
echo "<div>Name: {$bet->getAttribute('name')} ID: {$bet->getAttribute('id')} Price: {$bet->getAttribute('price')}</div>";
}
}
} else {
echo 'XPath query failed';
}
$dom = $xp = $col = null;
$some_link = 'http://www.example.com';
$abc = 'killer';
$bcd = 'awsome';
$cde = 'qwerty';
$dom = new DOMDocument;
$dom->preserveWhiteSpace = false;
#$dom->loadHTMLFile($some_link);
$html = getTags( $dom, $abc, $bcd, $cde );
echo $html;
function getTags( $dom, $abc, $bcd, $cde ){
$html = '';
$domxpath = new DOMXPath($dom);
$newDom = new DOMDocument;
$newDom->formatOutput = true;
$defffff = $domxpath->query("//$abc" . '[#' . $bcd . "='$cde']");
// since above returns DomNodeList Object
// converting to string(html)
$i = 0;
while( $myItem = $defffff->item($i++) ){
$node = $newDom->importNode( $myItem, true ); // import node
$newDom->appendChild($node); // append node
}
$html = $newDom->saveHTML();
return $html;
}
?>
this is the whole code. it is returning multiple results in a row, now what I want is to have ONLY the result no.1 and no.5. How can I do it?
I am new to DOM, tried several things but no success. Thanks in Advance
Change this
$i = 0;
while( $myItem = $defffff->item($i++) ){
$node = $newDom->importNode( $myItem, true ); // import node
$newDom->appendChild($node); // append node
}
into this, in order to append only selected nodes
$i = 0;
while( $myItem = $defffff->item($i++) ){
if ($i==0 or $i==4){
$node = $newDom->importNode( $myItem, true ); // import node
$newDom->appendChild($node); // append node
}
}
or you if you know the indexes you want already, you can do this
$myIndexes = array (0,4);
foreach ($myIndexes as $i){
$myItem = $defffff->item($i++);
$node = $newDom->importNode( $myItem, true ); // import node
$newDom->appendChild($node); // append node
}
Basically, I'm taking an SQL query and converting it into a dynamic XML. I'm trying to create custom XML tags for elements that I obtain from my query, and I'd like to pick and choose which results from my query are used as XML elements.
In a perfect world, I'd like to be able to take each row that I get from the query and determine the XML properties. I'd like for this to be a loop, but I just can't seem to get it to work.
$sql = "SELECT
COUNT( l.log_id ) AS id,
l.status AS 'requestStatus',
d.firmname AS 'name',
DAY (FROM_UNIXTIME( l.time ) ) AS DAY,
WEEK( FROM_UNIXTIME( l.time ) ) AS week,
YEAR( FROM_UNIXTIME( l.time ) ) AS year
FROM $table_id1, $table_id2
WHERE
l.client = 'XXXXX' AND
l.time > 1 AND
l.work_id = d.subid AND
d.deleted = 0 AND
d.user_id = l.user_id
GROUP BY
YEAR( FROM_UNIXTIME( l.time ) ),
WEEK( FROM_UNIXTIME( l.time ) )
ORDER BY
week DESC";
$dbresult = mysql_query($sql);
// create a new XML document
$doc = new DomDocument('1.0', 'UTF-8');
$doc->formatOutput = true;
// create root node
$root = $doc->createElement('workResponse');
$root = $doc->appendChild($root);
$occ2 = $doc->createElement('contentResponses');
$occ2 = $root->appendChild($occ2);
// process one row at a time
while($row = mysql_fetch_assoc($dbresult)) {
// add node for each row
$occ = $doc->createElement("contentResponse");
$occ = $root->appendChild($occ);
// add a child node for each field
foreach ($row as $fieldname => $fieldvalue) {
$child = $doc->createElement($fieldname);
$child = $occ->appendChild($child);
$value = $doc->createTextNode($fieldvalue);
$value = $child->appendChild($value);
} // foreach
}// while
// get completed xml document
$xml_string = $doc->saveXML();
echo $xml_string;
?>
First, I suggest not using mysql_* functions. They are deprecated and going away.
Dom extension seems to be overkill in this situation. I prefer to use SimpleXML when I can.
<?php
$dbh = new PDO('mysql:host=localhost;dbname=test','username','password');
$sxe = new SimpleXMLElement('<workResponse></workResponse>');
$sxe_crs = $sxe->addChild('contentResponses');
function array_walk_simplexml(&$value, $key, &$sx) {
$sx->addChild($key, $value);
}
$stmt = $dbh->query('SELECT * FROM sometable');
while ($row = $stmt->fetch(PDO::FETCH_ASSOC)) {
$sx_cr = $sxe_crs->addChild('contentResponse');
array_walk($row, 'array_walk_simplexml', $sx_cr);
}
echo $sxe->asXML();
If you want to "pretty print" the XML (admire your work?), then you'll need to use the Dom extension.
$dom_sxe = dom_import_simplexml($sxe);
$dom = new DOMDocument('1.0');
$dom->formatOutput = true;
$dom_sxe = $dom->importNode($dom_sxe, true);
$dom_sxe = $dom->appendChild($dom_sxe);
echo $dom->saveXML();
XML_Serializer is very useful for php pear users