I'm trying to parse information from fonefinder.net. I was trying to use simplexmlload_file, but couldn't get the page to load successfully.
Now, I'm looking into Curl. But I'm not sure if this will work either.
I basically just want to take the html from the fonefinder page, and parse it to get phone carrier and city.
Is that possible? How?
SimpleXML will only work if the HTML is formatted correctly - and that is rarely the case ;)
You could do a simple cURL call to fetch the data and the easiest thing would probably be using a regular expression to get the information you need.
The solution however is not easy to supply you with, with nothing to go on. But this was an idea.
I Recommend using:
http://www.de.php.net/manual/en/function.file-get-contents.php to get the document
http://www.php.net/manual/en/domdocument.loadhtml.php to load it
http://www.php.net/manual/en/class.domxpath.php to get the information from it
Or use the search function here, that question must have been asked over and over, for example PHP: Fetch content from a html page using xpath()
Related
I want to get the content of this 1st and 2nd webpage https://www.goodreads.com/search?utf8=%E2%9C%93&q=%D8%B3%D9%8A%D8%A7%D8%B3%D8%A9&search_type=books
and then store it in my database and then make the list is searchable.so after I googled I found that I can do it like this
$url = "https://www.goodreads.com/search?utf8=%E2%9C%93&q=%D8%B3%D9%8A%D8%A7%D8%B3%D8%A9&search_type=books";
function get_links($url){
$input = file_get_contents($url);
echo $input;
}
get_links($url);
my problem is how can I get the 2nd page content also and how can I store these books in my database to the list searchable
The answer is not that easy...
Options
Getting the pages (Not recommended)
To get a later page you can send the "page argument" in your request:
e.g.:
https://www.goodreads.com/search?page=2&q=%D8%B3%D9%8A%D8%A7%D8%B3%D8%A9&search_type=books&tab=books&utf8=%E2%9C%93
But to get the Elements into a nice structure you need to parse the HTML you get which is realy hard.
Use the API (recommended)
At https://www.goodreads.com/api/index you can find the documentation for goodreads API which returns example as response and is easily parseable.
Parse XML in PHP
If you use the API you can parse the XML-response with SimpleXML.
See example 1 & 2: http://php.net/manual/en/simplexml.examples-basic.php
Saving to database
If you are a beginner you might read some tutorials about how to use mysql with php and PDO. But you may also have a look at RedBeanPHP which is an really easy to use ORM for databases.
For getting 2nd page you will have to append on the url ?page=2
In case that you need to get other pages, you can use a for loop if you don't know how to use for loop, please google php for loop
For the database, google php mysql insert
In PHP, I was wondering how to retrieve specific data from a google search. For example if I wanted to retrieve the price on https://www.google.com/#q=ps3&tbm=shop for the first result. I have experimented with curl and domdocs and am not going any where.
The simplest way to get the contents of a webpage in PHP is to use the file_get_contents function.
http://us2.php.net/file_get_contents
Once you have the data, you'll need to parse it. There have been plenty of great articles about how to approach this, so I'm not going to repeat them. Here's a link to a good one:
http://anchetawern.github.io/blog/2013/08/07/getting-started-with-web-scraping-in-php/
Hi friends i am working on JQ-GRID. I want to show Image in specific column, But i don't know how to attach image in JQ-GRID. Can anybody help me or please send me some links, thanks
jqgrid is a feature monster. I tell this everybody who asks about it.
When jqgrid loads, a function is called which actually gets the data you want to display.
This is normally an ajax call to your php. As a result set of this function, you can just use xml or json.
I prefer json, so I build my result array and do a echo json_encode($myarray)
jQuery("#your_grid_id").jqGrid({ url : '/ajax/getjqgriddata.php'})
Now displaying pictures, there are different ways you can do that. You can either generate a <img src="wherever/mypicture1.png"></img>-link and hand it over in your result, or encode your picture binary data with base64 and deliver it with your result.
A more addvanced way is to use an so called formatter and just returning a id for the image.
This depends on you, but I would suggest to get confident with jqgrid, experiment with returning -links to get a feeling how jqgrid works.
There is plenty of good documentation at:
http://trirand.com/blog/jqgrid/jqgrid.html
Just take a look at it.
Im guessing its php cURL, but Whats the best way to make a loop to scrape the DOM for info from a webpage that uses id's in the URL Query like (?ProductId=103) There is about 1200 pages. I need to find the innerHTML of the 9th span on each page. This info will just get stored in a mySQL table (id->value) for future scraping of this site.
Well curl might be faster (not sure), but if it is a one off thing, then I would just use file_get_contents
for($x=0;$x<1200;$x++){
$f = file_get_contents(URL . '?productId='.$x);
#do stuff to $f
}
Yes. Use cURL to retrieve the page, use a DOM parser like SimpleXML to get the info you need out of it.
cURL
to speed things up you could use multi_curl =>
https://stackoverflow.com/search?q=[php]+multi_curl
scraping
the scraping part has been answered before better => for example https://stackoverflow.com/questions/3885760/scraping-and-web-crawling-framework-php.
You should search => https://stackoverflow.com/search?q=[php]+web+scraping
mySQL
I don't know if you do, but you should be using PDO to make it safe(SQL-injections).
I am (trying) to build a web app, first one :)
I need to call a JSON object though, and I am using a simple search form to get a word that I will append to the call, but I need PHP to handle the object when it is returned. How can I do this?
Basically I had the form submit calling a function that, ideally, would go get the object and do what it needs to, then return the results formatted how I want. Does that make sense even?
Thank you!
PHP 5.x has some built in functions: json_decode() and json_encode(). They are worth looking into and should answer your question. If you have less then PHP 5.x the user comments at both of those pages should have alternatives to use in it's place.