I am trying to read a link from one page, print the URL, go to that page, and read the link on the next page in the same location, print the url, go to that page (and so on...).
All I'm doing is reading the URL and passing it as an argument to the get_links() function until there are no more links.
This is my code but it throws:
Fatal error: Call to a member function find() on a non-object.
Anyone know how to fix this?
<?php
$mainPage = 'https://www.bu.edu/link/bin/uiscgi_studentlink.pl/1346752597?ModuleName=univschr.pl&SearchOptionDesc=Class+Subject&SearchOptionCd=C&KeySem=20133&ViewSem=Fall+2012&Subject=&MtgDay=&MtgTime=';
get_links($mainPage);
function get_links($url) {
$data = new simple_html_dom();
$data = file_get_html($url);
$nodes = $data->find("input[type=hidden]");
$fURL = $data->find("/html/body/form");
$firstPart = $fURL[0]->action . '<br>';
foreach ($nodes as $node) {
$val = $node->value;
$name = $node->name;
$name . '<br />';
$val . "<br />";
$str1 = $str1 . "&" . $name . "=" . $val;
}
$fixStr1 = str_replace('&College', '?College', $str1);
$fixStr2 = str_replace('Fall 2012', 'Fall+2012', $fixStr1);
$fixStr3 = str_replace('Class Subject', 'Class+Subject', $fixStr2);
$fixStr4 = $firstPart . $fixStr3;
echo $nextPageURL = chop($fixStr4);
get_links($nextPageURL);
}
?>
Alright so I was using the load->file() function somewhere in my code and did not see it until I really scraped through it. Finally have a running script :) The key is to use file_get_html instead of loading the webpage as an object using the load->file() function.
Related
I have a Laravel app, which the following PHP code:
public function handle()
{
$post_item->category_id = $source->category_id;
$post_item->featured = 0;
$post_item->type = Posts::TYPE_SOURCE;
$post_item->render_type = $item['render_type'];
$post_item->source_id = $source->id;
$post_item->description = is_array($item['description'])?'':$item['description'];
$post_item->featured_image = $item['featured_image'];
$post_item->video_embed_code = $item['video_embed_code'];
$post_item->dont_show_author_publisher = $source->dont_show_author_publisher;
$post_item->show_post_source = $source->show_post_source;
$post_item->show_author_box = $source->dont_show_author_publisher == 1 ? 0 : 1;
$post_item->show_author_socials = $source->dont_show_author_publisher == 1 ? 0 : 1;
$post_item->rating_box = 0;
$post_item->created_at = $item['pubDate'];
$post_item->views = 1;
$post_item->save();
$this->createTags($item['categories'], $post_item->id);
// This is where I want to add my echo
}
public function createTags($tags, $post_id)
{
$post_tags = PostTags::where('post_id', $post_id)->get();
foreach ($post_tags as $post_tag) {
Tags::where('id', $post_tag->tag_id)->delete();
}
PostTags::where('post_id', $post_id)->delete();
foreach ($tags as $tag) {
$old_tag=Tags::where('title',$tag)->first();
if(isset($old_tag))
{
$pt = new PostTags();
$pt->post_id = $post_id;
$pt->tag_id = $old_tag->id;
$pt->save();
}
else {
$new_tag = new Tags();
$new_tag->title = $tag;
$new_tag->slug = Str::slug($tag);
$new_tag->save();
$pt = new PostTags();
$pt->post_id = $post_id;
$pt->tag_id = $new_tag->id;
$pt->save();
}
}
}
Im trying to echo the the title along with the tags, right after the commented place, but it fails to provide the correct output. I was wondering if Im using the correct way or my workaround is completely wrong.
Im using this code in the commented part:
$tweet = $post_item->title . ' tags: ' . $post_item->tags;
After doing some tests, I realized that if I use
var_dump($tag);
right after
foreach ($tags as $tag)
at my createTags function, it seems that all tags are output correctly.
Im wondering if I can store all $tags inside the createTag function's foreach, under a globally accessed variable that would be used in the initial handle function echoed.
Guessing the post item model has a relation "tags" you could try this:
$tweet = $post_item->title . ' tags: ' . implode(', ', $post_item->tags->toArray());
Also if you would just like to echo the tags on the commented place, try this:
echo implode(',', $item['categories']);
try if $post_item->tags - is array then
$tweet = $post_item->title . ' tags: ' . implode(', ', $post_item->tags);
if - collection then
$tweet = $post_item->title . ' tags: ' . $post_item->tags->join(', ');
I've written some code in php to scrape some preferable links out of the main page of wikipedia. When I execute my script, the links are coming through accordingly.
However, at this point I've defined two functions within my script in order to learn how to pass links from one function to another. Now, my goal is to print the links in the latter function but it only prints the first link and nothing else.
If I use only this function fetch_wiki_links(), I can get several links but when i try to print the same within get_links_in_ano_func() then it prints the first link only.
How can I get them all even when I use the second function?
This is what I've written so far:
include("simple_html_dom.php");
$prefix = "https://en.wikipedia.org";
function fetch_wiki_links($prefix)
{
$weblink = "https://en.wikipedia.org/wiki/Main_Page";
$htmldoc = file_get_html($weblink);
foreach ($htmldoc->find("a[href^='/wiki/']") as $a) {
$links = $a->href . '<br>';
$absolute_links = $prefix . $links;
return $absolute_links;
}
}
function get_links_in_ano_func($absolute_links)
{
echo $absolute_links;
}
$items = fetch_wiki_links($prefix);
get_links_in_ano_func($items);
Your function returned the value at the very first iteration. You will need something like this:
function fetch_wiki_links($prefix)
{
$weblink = "https://en.wikipedia.org/wiki/Main_Page";
$htmldoc = file_get_html($weblink);
$absolute_links = array();
foreach ($htmldoc->find("a[href^='/wiki/']") as $a) {
$links = $a->href . '<br>';
$absolute_links []= $prefix . $links;
}
return implode("\n", $absolute_links);
}
I am trying to scrape this product table,
http://www.dropforyou.com/search2.php?mode=search&posted_data%5Bcategoryid%5D=2&posted_data%5Bsearch_in_subcategories%5D=on
I need the product id, the quantity and the price.
Since the site uses cookies and a post form I am grabbing the site with CURL. Which works fine. I am then loading that into simple html dom with $html = str_get_html($content);
I have been able to load all the table values into an array, however I can't label them. They just come in as 0,1,2 and I can't tell what's what.
I tried using a different method posted here on stackoverflow, however it gives me Fatal error: Call to a member function find() on a non-object in
My working code that isn't labeled
$content = curlscraper($urltoscrape);
$html = str_get_html($content);
$tds = $html->find('table',2)->find('td');
$num = NULL;
foreach($tds as $td)
{
$num[] = $td->plaintext;
}
echo '<pre>';
var_dump ($num);
echo '</pre>';
The code I found on Stackoverflow that just gives me Fatal error: Call to a member function find() on a non-object in
$content = curlscraper($urltoscrape);
$html = str_get_html($content);
foreach($html->find('tr',2) as $page)
{
$item['sku'] = $page->find('td',0)->plaintext;
$item['product'] = $page->find('td',1)->plaintext;
$item['Qty'] = $page->find('td',2)->plaintext;
$item['description'] = $page->find('td',3)->plaintext;
$item['price'] = $page->find('td',4)->plaintext;
$table[] = $item;
}
print_r($table);
Try initialize variable for your foreach function and then use your code. But you don't say which line created that error?
$line = $html->find('tr',2);
foreach($line as $page)
{
//var_dump($page) //You can check array
$item['sku'] = $page->find('td',0)->plaintext;
$item['product'] = $page->find('td',1)->plaintext;
$item['Qty'] = $page->find('td',2)->plaintext;
$item['description'] = $page->find('td',3)->plaintext;
$item['price'] = $page->find('td',4)->plaintext;
$table[] = $item;
}
I am trying to pull text from a single element in an xml file, i was able to pull the entire file but i really only need a couple of lines from the file...
On this page i was able to pull the entire xml file ( http://smyrnainlet.com/testing.php )
But when i try to just single out one line from that file.
http://smyrnainlet.com/current_data.php
This is the error i am receiving:
Fatal error: Call to a member function getbyElementID() on a non-object in /home/content/74/8620474/html/current_data.php on line 9
If anyone could help me that would be amazing, i have been struggling with this for hours.
This is my code:
<?php
function startElemHandler($parser, $name, $attribs)
{
if (strcasecmp($name, "current_observation") ==0 ) {
echo "<div id='waves'>\n";
}
if (strcasecmp($name, "wave_height_ft") ==0) {
$waveHeight->getbyElementID("wave_height_ft");
echo $waveHeight->asXML();
}
}
function endElemHandler($parser, $name)
{
if (strcasecmp($name, "current_observation") ==0 ) {
echo "</div>";
}
}
$parser = xml_parser_create();
xml_set_element_handler($parser, startElemHandler, endElemHandler);
xml_parser_set_option($parser, XML_OPTION_CASE_FOLDING, 0);
$strXML = implode("",file("http://www.weather.gov/xml/current_obs/41009.xml"));
xml_parse($parser, $strXML);
xml_parser_free($parser);
?>
You have a few fundamental flaws with your code but you are essentially asking how to parse an XML file. I suggest using PHP's DOMDocument with DOMXPath to extract the data you need. Here is some example code:
$xml = file_get_contents('weather.xml');
$dom = new DOMDocument();
#$dom->loadHTML($xml);
$domx = new DOMXPath($dom);
$entries = $domx->evaluate("//wave_height_ft");
$arr = array();
foreach ($entries as $entry) {
$arr[] = '<' . $entry->tagName . '>' . $entry->nodeValue . '</' . $entry->tagName . '>';
}
print_r($arr);
$waveHeight seem not defined (not in scope) wher you try to use it!
you can
pass it as argument
use the global statement
create a class
In my UI there is a text area that has a list of URLs separated by new line.
I am getting this into a string and exploding it using newline character. I am getting this error:
Fatal error: Call to a member function find() on a non-object in C:\xampp\htdocs\apps\lt\track-them.php on line 34
$myurl = "http://mydomain.com/testpage1.html";
$str = "http://www.domain.com/page1.html
http://www.homepage.com/page2.html
http://www.internet.com/index.html";
$backlinkarr = explode("\n", $str);
echo '<table border=1>';
foreach ($backlinkarr as $backlinkitem) {
$backlinkitem = trim($backlinkitem);
$LinkCount = 0;
$html = file_get_html($backlinkitem);
foreach ($html->find('a') as $link) {
if ($link->href == $myurl) {
$LinkCount += $LinkCount + 1;
}
}
echo $backlinkitem . ' (' . $linkCount . ')';
}
on windows new line is \r\n
also, your error suggests that $html is not an object, meaning previous line fails. so, debug there with var_dump($backlinkitem) if it is correct and also you can use file_exists to verify if that file actually exists.