This question already has answers here:
Loading content as the user scrolls down
(2 answers)
Closed 4 years ago.
Is it possible to get data from items that are loaded on scroll with simplehtml parser?
My code is done and it works perfectly I don't need any help with that , I'm just asking for advice if this is even possible to acomplish, I know when the dom praser loads what ever it sees on the first load of the page but is it possible to load more?
Example:
The page that i am loading has 10 items on it. But when you scroll down it Loads 10 more. Or thats not possible?
simple-html-dom doesn't do this by itself, you need to study how the javascript on the page fetches new items, and re-implement that in PHP. the "Network" tab of the Developer Tools of chrome is of great help in doing this, rather than study the javascript itself, you can just study the requests created by the javascript when you scroll, i usually find that to be a much easier approach.
Related
This question already has an answer here:
Closed 10 years ago.
Possible Duplicate:
PHP echo-ing content as page loads
I have a php page that has to load a heavy table, so after the page request there is a lot of time to wait.
Is it possible to send the page initially empty and, during the php page loading, send to the client the results, so one can see what has been loaded so far? (not a loading bar).
Yes. It's called flushing the document early. At its simplest, output some amount of HTML, and then calling flush();. As with most things, the devil is in the details of exactly what and when things would happen.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
JavaScript for detecting browser language preference
How to getting browser current locale preference using javascript?
I have a wordpress website that I am in the works of localizing. I don't really have much php or javascript knowledge, and don't really know what I am looking for in terms of doing this.
I have a support tab that users can click to go to my helpdesk. I can create multiple tabs for different languages and get the code to display it.
So my question is: Is their an if then statement in javascript that I can use to help detect the language of the browser and display the correct tab? Or is their a better way of doing this? Since my localized domains format is "ru.domain.com" I could even have the if then statement check for the language code in the current page url. Any help would be appreciated.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
HTML Scraping in Php
Far from being web developer expert, so sorry in advance if I'm missing something basic:
I need to copy a table into mySql database using PHP; the table resides in a website which I don't own, however I have permission to copy and publish.
Manually when I watch this website in my web-browser I need to click on a link in the main website URL (I can't reach the final destination page link since it changes all time, however the main page link is static and the link to click is also static).
Example to such a content I need to copy from (just an example, this is not the real content):
http://www.flightstats.com/go/FlightStatus/flightStatusByAirport.do?airportCode=JFK&airportQueryType=0
Most people are going to ask what have you tried. Since you mentioned that you don't have much development experience, here are some tips on how to go about it - have to put it as an answer so it is easier to read.
What you're going to need to do is scraping.
Using PHP, you'd use the following functions at the very least
file_get_contents() - this function will read the data in the URL
preg_match_all - use of regular expressions will let you get the data you are looking for. Though some/many people will say that you should go through the DOM.
The data that is returned with preg_match_all can be stored into your MySQL table. Though because the data changes so frequently, you might be better off just scraping that section and storing the entire table as cache (though I do have to say I have no idea what you are trying to do on your site - so I could well be wrong).
This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
How do I render javascript from another site, inside a PHP application?
This is the site http://www.oferta.pl/strona_v2/gazeta_v2/ . This site is built totally on JavaScript. I want to scrape using PHP and curl. Currently I use DOMXPath. In the left menu there are some category to be selected. I see no 'form' there. How can I use curl to submit that form and scrap the output page?
I have used file_get_contents() only. It doesn't get all of the page. How can I proceed?
N.B : http://www.html-form-guide.com/php-form/php-form-submit.html I have found this example which have a 'form'. But my specified site has no 'form'.
You can not scrape it. Its possible. But its way too hard.
Simulate the http request by curl. Check every request it makes by ajax and try to simulate it.
Simulate Javascript executions (this part is almost impossible). Some requests contains values which are generated by Javascript. You need to do it in php. If they has some complicated algorithm implemented in JS you can invoke v8 javascript engine.
This question already has answers here:
Closed 11 years ago.
I'm looking for a PHP web-crawler to gather all the links to for a large site and tell me if the links are broken.
So far I've tried modifying an example on here myself. My question about the codeI've also tried grabbing phpDig but the site is down. any suggestions would be great on how I should proceed would be great.
EDIT
The problem isn't the grabbing of the links the issue of the scale I'm not sure if the script I modified is sufficient enough to grab what possibly be thousands of URL's as I tried setting the depth for the search link to 4 and the crawler timed out through the browser. Someone else mentioned something about killing processes as to not overload the server, could someone please elaborate on the issue.
Not a ready-to-use solution, but Simple HTML Dom parser is one of my favourite dom parsers.
It let's you use CSS selectors for finding nodes over the document, so you can easily find <a href="">'s.
With these hyperlinks's you can build your own crawler and check if the pages are still available.
You can find it here.