I need to get the resulting data from a website query. for example
http://www.uniprot.org/uniprot/?query=organism:9606+AND+gene:AEBP1+AND+reviewed:yes&sort=score&format=tab&columns=entry%20name
resulting page shows
Entry name
AEBP1_HUMAN
I need the result, in this case "AEBP1_HUMAN" to be display on my website. Confused how to get it. Thanks
The goal is, that you can read the content of any url like a file, because php supports wrappers for variety of protocols.
First example uses function file that read entire content and split it by lines into an array.
<?php
$content = file($url);
echo $content[1];
?>
In the second example you get the whole content as a string, so you have to split it with explode function by line endings.
<?php
$content = file_get_contents($url);
$lines = explode("\n", $content);
echo $lines[1];
?>
Third example uses standard file open in combination with function fgets that read the content line by line.
<?php
$fp = fopen($url);
$fp = fopen($url, 'r');
$line = fgets($fp);
$line = fgets($fp);
echo $line;
?>
The last example shows usage of curl. Don't forget to use right options.
<?php
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$content = curl_exec($ch);
$lines = explode("\n", $content);
echo $lines[1];
?>
Sometimes you may experience problems on public hosting servers, where reading remote content is blocked.
Related
I have a temperature monitor set up, and I would like to use some of the data for other things (cron jobs, etc). The data from the sensor can be accessed from our local network (192.168.123.123). The element in question is:
<td id="1E5410ECC9D90FC3-entity-0-measurement-0" class="">69.08</td>
<!-- I NEED THE 69.08 -->
I can't do it via ajax since I get the Allow-Access-Origin error (CORS).
I tried this:
$url = 'http://192.168.123.123';
$content = file_get_contents($url);
$first = explode( '<div id="1E5410ECC9D90FC3-entity-0-measurement-0">' , $content );
$second = explode("</div>" , $first[0] );
echo $second[0];
but I got this:
��UMS�0��+��$���94С�2����؋-�%#Ʉ�뻲���Bۓ%����ݷr��m4�yyF*_+ry���ӈP������S��|��&�ȵ�2���}��V�7ǜO��dz�[�� (�!�_2��$�/�p/ g�=B� D����<��1�#�=h���J�˨�'��I^ ��g7��=�=��^�0��ϔ����p�Q��L��I�%TF�Q�) ������;c��o$��a����g��mWr�ܹ��;�(��bE��O�i� ��y�҉)f=�6=�,2� �#I��s����>����kNƕt/W2^��# Xp�3^݅$ѵ��T U�ʲ�#f��db�ԁ%��b�`G|��D�{sι1�� ]#2ZH�(1;&�h8��^0er��3���D�Q�5B�u� ^!5X:�{a U\:߰0�~Ɍ�3+S�^1��qB:�g����C>�.�P~n��$\֢D����%J+�b�ELc�Gq���K �]��xV��j�[���Ԧ��nAɍ��<�ZT#���zc�Q(f܁�(~�^�ZKwk:8�·n>��(=�"aB)�Fl5�b]/�_�$���_��ɴ��9�H}��B [#�V�ԅp��r�g�A�j���2����Ju*������{�bY�,O4�����M��B�#�e���,� ��_֔���o����
How can I properly get the 'td' text within the specific div id?
You are trying to retrieve data from <td id="1E5410ECC9D90FC3-entity-0-measurement-0" class=""> not <div id="1E5410ECC9D90FC3-entity-0-measurement-0">, so not from a <div>, so just change it into:
$url = 'http://192.168.123.123';
$content = file_get_contents($url);
$first = explode( '<td id="1E5410ECC9D90FC3-entity-0-measurement-0">' , $content );
$second = explode("</td>" , $first[0] );
echo $second[0];
Or am I crazy?
Step 1:
I suggest using php's curl library to manage and configure your web request/response.
Using this mechanism allows you to better manage/control encoding, compression and encryption.
http://php.net/manual/en/book.curl.php
// create curl resource
$ch = curl_init();
// set url
curl_setopt($ch, CURLOPT_URL, "http://192.168.123.123");
//return the transfer as a string
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
// $output contains the output string
$output = curl_exec($ch);
// close curl resource to free up system resources
curl_close($ch);
Step 2:
Let's extract the details out of the returned response string from the web server. I suggest PHP's PCRE function preg_match to extract the needed data.
http://php.net/manual/en/ref.pcre.php
// Looking for <td id="1E5410ECC9D90FC3-entity-0-measurement-0" class="">69.08</td>
$pattern = '/id="1E5410ECC9D90FC3-entity-0-measurement-0".*>([\d]{1,2}?\.[\d]{1,2})<\//';
// run the regex match and collect the hit
preg_match($pattern, $output, $matches);
// print_r of the array
/*
Array
(
[0] => id="1E5410ECC9D90FC3-entity-0-measurement-0" class="">69.08</
[1] => 69.08
)
*/
// Print out the result to check
echo $matches[1];
I want to get the whole element <article> which represents 1 listing but it doesn't work. Can someone help me please?
containing the image + title + it's link + description
<?php
$url = 'http://www.polkmugshot.com/';
$content = file_get_contents($url);
$first_step = explode( '<article>' , $content );
$second_step = explode("</article>" , $first_step[3] );
echo $second_step[0];
?>
You should definitely be using curl for this type of requests.
function curl_download($url){
// is cURL installed?
if (!function_exists('curl_init')){
die('cURL is not installed!');
}
$ch = curl_init();
// URL to download
curl_setopt($ch, CURLOPT_URL, $url);
// User agent
curl_setopt($ch, CURLOPT_USERAGENT, "Set your user agent here...");
// Include header in result? (0 = yes, 1 = no)
curl_setopt($ch, CURLOPT_HEADER, 0);
// Should cURL return or print out the data? (true = retu rn, false = print)
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
// Timeout in seconds
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
// Download the given URL, and return output
$output = curl_exec($ch);
// Close the cURL resource, and free system resources
curl_close($ch);
return $output;
}
for best results for your question. Combine it with HTML Dom Parser
use it like:
// Find all images
foreach($output->find('img') as $element)
echo $element->src . '<br>';
// Find all links
foreach($output->find('a') as $element)
echo $element->href . '<br>';
Good Luck!
I'm not sure I get you right, But I guess you need a PHP DOM Parser. I suggest this one (This is a great PHP library to parser HTML codes)
Also you can get whole HTML code like this:
$url = 'http://www.polkmugshot.com/';
$html = file_get_html($url);
echo $html;
Probably a better way would be to parse the document and run some xpath queries over it afterwards, like so:
$url = 'http://www.polkmugshot.com/';
$xml = simplexml_load_file($url);
$articles = $xml->xpath("//articles");
foreach ($articles as $article) {
// do sth. useful here
}
Read about SimpleXML here.
extract the articles with DOMDocument. working example:
<?php
$url = 'http://www.polkmugshot.com/';
$content = file_get_contents($url);
$domd=#DOMDocument::loadHTML($content);
foreach($domd->getElementsByTagName("article") as $article){
var_dump($domd->saveHTML($article));
}
and as pointed out by #Guns , you'd better use curl, for several reasons:
1: file_get_contents will fail if allow_url_fopen is not set to true in php.ini
2: until php 5.5.0 (somewhere around there), file_get_contents kept reading from the connection until the connection was actually closed, which for many servers can be many seconds after all content is sent, while curl will only read until it reaches content-length HTTP header, which makes for much faster transfers (luckily this was fixed)
3: curl supports gzip and deflate compressed transfers, which again, makes for much faster transfer (when content is compressible, such as html), while file_get_contents will always transfer plain
I am reading a CSV file which contain URLs. I am trying to output the result of those URLs but facing strange issue.
I can't seem to understand why this code doesn't print variable $output when you try to print item which is on first line.
This is my CSV file containing two records:
www.serverfault.com
www.stackoverflow.com
This is my code
<?php
$myfile = fopen("products.csv", "r") or die("Unable to open file!");
while(!feof($myfile))
{
$myline = fgets($myfile);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $myline);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$output = curl_exec($ch);
curl_close($ch);
if($myline == "www.serverfault.com")
{
echo $output;
}
}
?>
Notice in CSV file the first record is www.serverfault.com and it never prints the $output. If I move this record to second line then it prints $output but then it doesn't print $output for www.stackoverflow.com which is on first line now.
What's going on?
You're just assuming success. curl_exec returns boolean false on failure, which prints as a zero-length string.
Add this:
if($output === false) {
die(curl_error($ch));
}
And don't forget to check for whitespace (e.g. linebreaks) on your string. Your $myline might actually be www....com\n or similar.
Need a code example and/or guidance about fetching multiple urls stored in a .txt file using curl. Do I need to use a spider, or can I modify the code below which works well for one url?
<?php
$c = curl_init('http://www.example.com/robots.txt');
curl_setopt($c, CURLOPT_RETURNTRANSFER, true);
$page = curl_exec($c);
curl_close($c);
?>
Your question is vague but I will try to answer it with the information you provided.
I would use explode() PHP function.
$lines = explode(PHP_EOL, $page);
foreach($lines as $line) {
$val = explode(':', $line);
echo $val[1];
}
Something like this should do the job.
Im currently working with an API which requires we send our collection details in xml to their server using a post request.
Nothing major there but its not working, so I want to output the sent xml to a txt file so I can look at actually whats being sent!!
Instead of posting to the API im posting to a document called target, but the xml its outputting its recording seems to be really wrong. Here is my target script, note that the posting script posts 3 items, so the file being written should have details of each post request one after the other.
<?php
error_reporting(E_ALL);
ini_set('display_errors', 1);
// get the request data...
$payload = '';
$fp = fopen('php://input','r');
$output_file = fopen('output.txt', 'w');
while (!feof($fp)) {
$payload .= fgets($fp);
fwrite($output_file, $payload);
}
fclose($fp);
fclose($output_file);
?>
I also tried the following, but this just recorded the last post request so only 1 collection item was recorded in the txt file, instead of all 3
output_file = fopen('output.txt', 'w');
while (!feof($fp)) {
$payload .= fgets($fp);
}
fwrite($output_file, $payload);
fclose($fp);
fclose($output_file);
I know im missing something really obvious, but ive been looking at this all morning!
Change
$output_file = fopen('output.txt', 'w');
to
$output_file = fopen('output.txt', 'a');
Also, change
$payload .= fgets($fp);
to
$payload = fgets($fp);
you should probably use curl instead to fetch the content and then write it via fwrite
change
$payload .= fgets($fp);
to
$payload = fgets($fp);