Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
The title says it all. Please give me some way for how to scrape AJAX loaded Division.
There is one website which has Product Grid Division. I want this division for scrap the data from it but the problem is when i get the page content data in another html page there is no product grid because it's loaded via AJAX after some time. For Scrapping i tried both Perl and CURL with PHP.
Thanks in advance.
From WWW::Mechanize::FAQ
Which modules work like Mechanize and have JavaScript support?
In no particular order: Gtk2::WebKit::Mechanize, Win32::IE::Mechanize,
WWW::Mechanize::Firefox, WWW::Scripter, WWW::Selenium
Also see: How do you scrape AJAX pages?
Using Selenium, e.g. through Selenium::Remote::Driver, you will be operating real browsers to access the site. Sites can be quite sensitive to subtle differences in browser behavior.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I'm working with PHP and I want to scrape some data from any website. But I have a problem. I scrape data but these items number are 48. But I know that page has 11K items. Rest of datas extend when you scroll and you get new bunch of datas (48 items).
I'm scraping with simple_html_dom. How can I manipulate scroll and get data ?
Thanks! :)
Sounds like the missing data is loaded via ajax.
Check the Network tab in the Developer Console (by pressing F12). Take a look at the URL which is being called (and the response), and edit it to your needs. Then call this URL instead of the one you are taking now.
It is impossible by this way.
But if you need to scrap this data you can send requests to endpoints which return lazily loaded data. You must research js code of target site.
p.s.
If you want to use really hard approach, you can research browser emulating.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I am trying to create an application that will have some similar looking pages. I am not sure what is the best way to do this.
Each page will have a header, side bar, content and footer.
The tricky part here is the sections will be similar but not the same. For example the header can have a combobox and different pages will have different data. One page will have a textbox.
The data will come from SQL server via PHP.
What would be the best way to handle this?
If you are using PHP then take a look into http://www.h2o-template.org/ - the templates will allow you to create similar structures and keeping them maintanable most importantly.
You can then have base template which encapsulates all of the similarities and extend sub templates from that which make each page unique.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
I need to include an external page into my website. How SEO friendly is this? And which will be the right code to use? I came a accross a few different codes and I'm not sure which to use...
Your question as is does not target a specific problem and no code is provided.
You can fetch anything you want and integrate into your website, using output buffers, file_get_contents(), SOAP, REST or cURL.
Search engines do not recognize the fetched content as external, but your page load time might increase if you do not buffer the content which decreases SEO value.
The better question is, does the site you want to fetch from allow using of their content, what licence does it have?
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
I know how to read and parse XML with PHP, but is there a way to make it auto advance through pages of XML data automatically? I work for a TV station that is featuring open houses via a real estate xml feed and need to show one or two houses at a time for a half hour without requiring someone to sit there and advance the pages themselves. Any thoughts would be appreciated!
You could have PHP output the necessary JavaScript to redirect to the next page.
Here's how to redirect to a new page: How can I make a redirect page in jQuery/JavaScript? combine that with setTimeout() to add a delay.
I'm not sure whether I would rely on this for a live TV feed, though - I'd be too scared of the embarrassment of the browser crashing, or an error popping up....
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
I need to create pagination and it should not have page numbers as if I have it then I need to load all data at page loading.(I was ordered to do so)
I am little bit lost. Don't know where to start Even a little help is highly appreciated.
using zend framework 2(MVC)
new to php and zend
Prefer not to use library
I am assuming, What you mean is that you cannot have page numbers listed with the results on your page, but just a forward and a back button. You keep track of what offset/limit results are being currently viewed by user, and depending on if the user clicks back/forward, you will send an appropriate request to server with offset and limit as parameters. The server based on php/Zend should return results based on this offset and limit.
I have had a precursory look at Zend Pagination and it seems to me that it should work for you.