Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I'm working with PHP and I want to scrape some data from any website. But I have a problem. I scrape data but these items number are 48. But I know that page has 11K items. Rest of datas extend when you scroll and you get new bunch of datas (48 items).
I'm scraping with simple_html_dom. How can I manipulate scroll and get data ?
Thanks! :)
Sounds like the missing data is loaded via ajax.
Check the Network tab in the Developer Console (by pressing F12). Take a look at the URL which is being called (and the response), and edit it to your needs. Then call this URL instead of the one you are taking now.
It is impossible by this way.
But if you need to scrap this data you can send requests to endpoints which return lazily loaded data. You must research js code of target site.
p.s.
If you want to use really hard approach, you can research browser emulating.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
The title says it all. Please give me some way for how to scrape AJAX loaded Division.
There is one website which has Product Grid Division. I want this division for scrap the data from it but the problem is when i get the page content data in another html page there is no product grid because it's loaded via AJAX after some time. For Scrapping i tried both Perl and CURL with PHP.
Thanks in advance.
From WWW::Mechanize::FAQ
Which modules work like Mechanize and have JavaScript support?
In no particular order: Gtk2::WebKit::Mechanize, Win32::IE::Mechanize,
WWW::Mechanize::Firefox, WWW::Scripter, WWW::Selenium
Also see: How do you scrape AJAX pages?
Using Selenium, e.g. through Selenium::Remote::Driver, you will be operating real browsers to access the site. Sites can be quite sensitive to subtle differences in browser behavior.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
I know how to read and parse XML with PHP, but is there a way to make it auto advance through pages of XML data automatically? I work for a TV station that is featuring open houses via a real estate xml feed and need to show one or two houses at a time for a half hour without requiring someone to sit there and advance the pages themselves. Any thoughts would be appreciated!
You could have PHP output the necessary JavaScript to redirect to the next page.
Here's how to redirect to a new page: How can I make a redirect page in jQuery/JavaScript? combine that with setTimeout() to add a delay.
I'm not sure whether I would rely on this for a live TV feed, though - I'd be too scared of the embarrassment of the browser crashing, or an error popping up....
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
Personally I would do this with PHP but maybe someone has an pro/contra for doing this with JavaScript:
I (will) fetch a few song details (about ten per page load) from Soundcloud
afterwards the statistic (played this week/total, liked this week/total etc.) will be displayed in a table
That's it - nothing special.
I can solve it like this:
let PHP do the stuff and render the whole page at once
This will of course take some time before the page is rendered.
Is there any advantage of doing this with JavaScript? I can only think of:
the page renders faster (with the disadvantage that the results may not be displayed instantly)
I'd say that doing it via JavaScript will actually take longer, as you'll have to connect again. Although, it may give an illusion of "being faster" (though I doubt it'll be noticeable).
I suppose it might depend on your usage cases too.
How are you handling authentication with SoundCloud?
I would honestly go the jQuery route.
Use a $.post() call, and within your success statement, loop through the json and append it to a table. This way, you can do something like pagination, which can update your table without needing to refresh the page.
Do it serverside and pre-load the page. The soundcloud API should not be the bottleneck here so reducing a request or two is worth it.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
Okay, bear with me. Part of my job is filling out online forms for customers who call in. In these forms there are all kinds of useful information that is hidden in input tags. I would like to make my own page that would extract and display all this useful information. I would also use this page to count the number of times I submit the form throughout the day.
The way I currently have it set up it is in a frameset. The left frame has links to the different forms we fill out, and when I click on one the external form is populated in the right frame. Javascript would be perfect for the job except that my frame is not located on the same server as the forms.
I stink at PHP. Not gonna lie. So i'm not looking for any hand-outs but any pointers would be great.
For example you may check out the forms on http://www.youcangetacar.com and you may use the pin code "A101" or you can check out what I have so far at http://customertrack.host-ed.me/
Make a page that cURL’s the page that is passed to it via url(curl.php?url=http://www.google.com) and then you dont have trouble with same domain policy.
Don’t forget to string replace relative urls with absolute urls.
eg, action="/submit/form.php" must become action="http://www.domain.com/submit/form.php"
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
I need to create pagination and it should not have page numbers as if I have it then I need to load all data at page loading.(I was ordered to do so)
I am little bit lost. Don't know where to start Even a little help is highly appreciated.
using zend framework 2(MVC)
new to php and zend
Prefer not to use library
I am assuming, What you mean is that you cannot have page numbers listed with the results on your page, but just a forward and a back button. You keep track of what offset/limit results are being currently viewed by user, and depending on if the user clicks back/forward, you will send an appropriate request to server with offset and limit as parameters. The server based on php/Zend should return results based on this offset and limit.
I have had a precursory look at Zend Pagination and it seems to me that it should work for you.