How to parse the data obtained by using CURL to get DL? - php

Question

I want to show list of journals and their abbreviation like:
Journal Name, Abbreviation
I am getting the data I need from :
http://images.webofknowledge.com/WOK46/help/WOS/D_abrvjt.html
So I am running the following:
$ch = curl_init();
//Set options
$curl = curl_init();
curl_setopt_array($curl, array(
CURLOPT_URL => 'http://images.webofknowledge.com/WOK46/help/WOS/A_abrvjt.html'
));
$result = curl_exec($curl);
curl_close($curl);
$data=json_decode($result, true);
//!End function, make_call
But now what it shows me is the whole page, but as I said I only need the name of the journals(dt) and the abbreviation (dd). So How can I pars the result?

Answer 1

HTML DOM parsing via Simple HTML DOM
Scraping method ...
<?php
Function Scraper($file, $cnt = NULL) {
/*
#param $file, url or path/file
#param $cnt, (number of results to list) empty for all, or number
*/
require_once('PATH/TO/simple_html_dom.php');
//set_time_limit(0); // uncomment for large files
$result = array();
// Create DOM from URL
$html = file_get_html($file);
IF ($html) {
IF (empty($cnt)) { $cnt = count($html->find('DT')); }
foreach($html->find('DL') as $dl) {
for ($i = 0; $i < $cnt; $i++) {
$dt = $dl->find('DT', $i)->plaintext;
$dd = $dl->find('DD', $i)->plaintext;
$result[] = array(trim($dt) => trim($dd));
}
}
}
return $result;
}
$array = Scraper('http://somesite.com/page.html');
print_r($array);
?>
Example output ...
Array
(
[0] => Array
(
[D H LAWRENCE REVIEW] => D H LAWRENCE REV
)
[1] => Array
(
[D-D EXCITATIONS IN TRANSITION-METAL OXIDES] => SPRINGER TR MOD PHYS
)
[2] => Array
(
[DADOS-REVISTA DE CIENCIAS SOCIAIS] => DADOS-REV CIENC SOC
)
[3] => Array
(
[DAEDALUS] => DAEDALUS
)
[4] => Array
(
[DAEDALUS] => DAEDALUS-US
)
[5] => Array
(
[DAGHESTAN AND THE WORLD OF ISLAM] => SUOMAL TIED TOIM SAR
)
)
Updated example specific to user350082's issue ...
The definition lists DT and DD tags were not closed resulting in the dd being included in the find('dt') result.
<DT>D H LAWRENCE REVIEW<B><DD> D H LAWRENCE REV</B>
<DT>D-D EXCITATIONS IN TRANSITION-METAL OXIDES<B><DD> SPRINGER TR MOD PHYS</B>
etc. etc. etc.
Updated Function ...
Function Scraper($file, $cnt = NULL) {
/*
#param $file, url or path/file
#param $cnt, (number of results to list) empty for all, or number
*/
require_once('PATH/TO/simple_html_dom.php');
//set_time_limit(0); // uncomment for large files
$result = array();
// Create DOM from URL
$html = file_get_html($file);
IF ($html) {
foreach($html->find('DL') as $dl) {
IF (empty($cnt)) { $cnt = count($html->find('DT')); } // set count if null
for ($i = 0; $i < $cnt; $i++) {
$dd = $dl->find('DD', $i)->plaintext;
$dt = $dl->find('DT', $i)->innertext; // dt with html tags, easier for removing dd duplication
$dt = preg_replace('/\s+/', ' ',$dt); // remove extra whitespace, tabs etc.
// strip DD text duplication from DT
IF (($pos = strrpos($dt ,$dd)) !== false) {
$strlen = strlen($dd);
$dt = substr_replace($dt, "", $pos, $strlen);
}
$dt = strip_tags($dt); // remove html tags
IF (empty($dt)) { $dt = $dd; } // make sure dt is not empty
$result[] = array(trim($dt) => trim($dd));
}
}
}
return $result;
}