In PHP, I was wondering how to retrieve specific data from a google search. For example if I wanted to retrieve the price on https://www.google.com/#q=ps3&tbm=shop for the first result. I have experimented with curl and domdocs and am not going any where.
The simplest way to get the contents of a webpage in PHP is to use the file_get_contents function.
http://us2.php.net/file_get_contents
Once you have the data, you'll need to parse it. There have been plenty of great articles about how to approach this, so I'm not going to repeat them. Here's a link to a good one:
http://anchetawern.github.io/blog/2013/08/07/getting-started-with-web-scraping-in-php/
Related
I'm beginner in PHP. I have some URL data like this https://ibb.co/7kQNTmq
I'd like to work with it. You can see there is a date (DD.MM.YY) and a number. Id like to create to some mathematical operation like arithmetic mean per month, etc.I know as the first step I need to get from that URL. How is that possible please? I learn PHP over a month so be easy please :) Also I'm trying work with Presenters (Nette)
Best
You can use file_get_contents method to get website/file content.
Some documentation: https://www.php.net/manual/en/function.file-get-contents.php
However, it's only work on GET requests. You need to use cURL if you want to send POST or etc.
I want to get the content of this 1st and 2nd webpage https://www.goodreads.com/search?utf8=%E2%9C%93&q=%D8%B3%D9%8A%D8%A7%D8%B3%D8%A9&search_type=books
and then store it in my database and then make the list is searchable.so after I googled I found that I can do it like this
$url = "https://www.goodreads.com/search?utf8=%E2%9C%93&q=%D8%B3%D9%8A%D8%A7%D8%B3%D8%A9&search_type=books";
function get_links($url){
$input = file_get_contents($url);
echo $input;
}
get_links($url);
my problem is how can I get the 2nd page content also and how can I store these books in my database to the list searchable
The answer is not that easy...
Options
Getting the pages (Not recommended)
To get a later page you can send the "page argument" in your request:
e.g.:
https://www.goodreads.com/search?page=2&q=%D8%B3%D9%8A%D8%A7%D8%B3%D8%A9&search_type=books&tab=books&utf8=%E2%9C%93
But to get the Elements into a nice structure you need to parse the HTML you get which is realy hard.
Use the API (recommended)
At https://www.goodreads.com/api/index you can find the documentation for goodreads API which returns example as response and is easily parseable.
Parse XML in PHP
If you use the API you can parse the XML-response with SimpleXML.
See example 1 & 2: http://php.net/manual/en/simplexml.examples-basic.php
Saving to database
If you are a beginner you might read some tutorials about how to use mysql with php and PDO. But you may also have a look at RedBeanPHP which is an really easy to use ORM for databases.
For getting 2nd page you will have to append on the url ?page=2
In case that you need to get other pages, you can use a for loop if you don't know how to use for loop, please google php for loop
For the database, google php mysql insert
Hi friends i am working on JQ-GRID. I want to show Image in specific column, But i don't know how to attach image in JQ-GRID. Can anybody help me or please send me some links, thanks
jqgrid is a feature monster. I tell this everybody who asks about it.
When jqgrid loads, a function is called which actually gets the data you want to display.
This is normally an ajax call to your php. As a result set of this function, you can just use xml or json.
I prefer json, so I build my result array and do a echo json_encode($myarray)
jQuery("#your_grid_id").jqGrid({ url : '/ajax/getjqgriddata.php'})
Now displaying pictures, there are different ways you can do that. You can either generate a <img src="wherever/mypicture1.png"></img>-link and hand it over in your result, or encode your picture binary data with base64 and deliver it with your result.
A more addvanced way is to use an so called formatter and just returning a id for the image.
This depends on you, but I would suggest to get confident with jqgrid, experiment with returning -links to get a feeling how jqgrid works.
There is plenty of good documentation at:
http://trirand.com/blog/jqgrid/jqgrid.html
Just take a look at it.
I'm trying to parse information from fonefinder.net. I was trying to use simplexmlload_file, but couldn't get the page to load successfully.
Now, I'm looking into Curl. But I'm not sure if this will work either.
I basically just want to take the html from the fonefinder page, and parse it to get phone carrier and city.
Is that possible? How?
SimpleXML will only work if the HTML is formatted correctly - and that is rarely the case ;)
You could do a simple cURL call to fetch the data and the easiest thing would probably be using a regular expression to get the information you need.
The solution however is not easy to supply you with, with nothing to go on. But this was an idea.
I Recommend using:
http://www.de.php.net/manual/en/function.file-get-contents.php to get the document
http://www.php.net/manual/en/domdocument.loadhtml.php to load it
http://www.php.net/manual/en/class.domxpath.php to get the information from it
Or use the search function here, that question must have been asked over and over, for example PHP: Fetch content from a html page using xpath()
I have a html site. In that site around 100 html files are available. i want to develop the search engine . If the user typing any word and enter search then i want to display the related contents with the keyword. Is't possible to do without using any server side scripting? And it's possible to implement by using jquery or javascript?? Please let me know if you have any ideas!!!
Advance thanks.
Possible? Yes. You can download all the files via AJAX, save their contents in an array of strings, and search the array.
The performance however would be dreadful. If you need full text search, then for any decent performance you will need a database and a special fulltext search engine.
3 means:
Series of Ajax indexing requests: very slow, not recommended
Use a DB to store key terms/page refernces and perform a fulltext search
Utilise off the shelf functionality, such as that offered by google
The only way this can work is if you have a list of all the pages on the page you are searching from. So you could do this:
pages = new Array("page1.htm","page2.htm"...)
and so on. The problem with that is that to search for the results, the browser would need to do a GET request for every page:
for (var i in pages)
$.get(pages[i], function (result) { searchThisPage(result) });
Doing that 100 times would mean a long wait for the user. Another way I can think of is to have all the content served in an array:
pages = {
"index" : "Some content for the index",
"second_page" : "Some content for the second page",
etc...
}
Then each page could reference this one script to get all the content, include the content for itself in its own content section, and use the rest for searching. If you have a lot of data, this would be a lot to load in one go when the user first arrives at your site.
The final option I can think of is to use the Google search API: http://code.google.com/apis/customsearch/v1/overview.html
Quite simply - no.
Client-side javascript runs in the client's browser. The client does not have any way to know about the contents of the documents within your domain. If you want to do a search, you'll need to do it server-side and then return the appropriate HTML to the client.
The only way to technically do this client-side would be to send the client all the data about all of the documents, and then get them to do the searching via some JS function. And that's ridiculously inefficient, such that there is no excuse for getting them to do so when it's easier, lighter-weight and more efficient to simply maintain a search database on the server (likely through some nicely-packaged third party library) and use that.
some useful resources
http://johnmc.co/llum/how-to-build-search-into-your-site-with-jquery-and-yahoo/
http://tutorialzine.com/2010/09/google-powered-site-search-ajax-jquery/
http://plugins.jquery.com/project/gss
If your site is allowing search engine indexing, then fcalderan's approach is definitely the simplest approach.
If not, it is possible to generate a text file that serves as an index of the HTML files. This would probably be rudimentarily successful, but it is possible. You could use something like the keywording in Toby Segaran's book to build a JSON text file. Then, use jQuery to load up the text file and find the instances of the keywords, unique the resultant filenames and display the results.