Perform google search programmatically using keywords - php

Question

Like we go on google and perform google search via keywords.
Can we do such query programaticaly? like
http://www.google.com/search?q=cupertino+american+food
After executing the query we should get all search result details for each link to store in database.
Exactly like some site provide REST api access, so that user can get bunch of results his query.
I don't have seen something like this possible with google or not.

Answer 1

Whatever technique you're going to use, Google will block your IP for bot-like search queries. And don't try with TOR proxy because all their IPs are always banned or challenged with captcha.
You have to use Google API in order to be compliant with Google's T&C. Also the result is much much better
https://developers.google.com/custom-search/json-api/v1/overview
The API is free if you have a CSE and has a limit of 100 queries per day. If you need more you'll be billed 5$ per 1000 queries

Answer 2

#mahtOrz: Okay, here's some rough code that will deliver a Json back to the console. Note that the api base search string is different than the one you have which is www.google.com/search?q=cupertino+american+food. You need to use the Google API base URL below. Do you have your APIkey and CxKey? If not, I can walk you through those steps too.
using System;
using System.Text;
using System.Net;
using System.IO;
using System.Web;
namespace GoogleSearchTest1
{
class Program
{
//Google keys
const string APIKey = "{your key here}";
const string CSEKey = "{your key here}";
//base url for the search query
const string GoogleBaseURL = "https://www.googleapis.com/customsearch/v1?"; //per Google documentation
public static void Main(string[] args)
{
string myQuery = "cupertino american food"; //put what you're searching for here
string result = submitSearch(myQuery);
Console.WriteLine(result);
string dummy = Console.ReadLine();
}
public static string submitSearch(string myQuery)
{
try
{
string final = string.Format(GoogleBaseURL+"key={0}&cx={1}&q={2}",
HttpUtility.UrlEncode(APIKey),
HttpUtility.UrlEncode(CSEKey),
HttpUtility.UrlEncode(myQuery));
final += "&alt=json";
WebRequest myRequest = WebRequest.Create(final);
HttpWebResponse myResponse = (HttpWebResponse)myRequest.GetResponse();
Stream myStream = myResponse.GetResponseStream();
StreamReader myReader = new StreamReader(myResponse.GetResponseStream(), System.Text.Encoding.UTF8);
string result = myReader.ReadToEnd();
myStream.Close();
myReader.Close();
myResponse.Close();
return result;
}
catch (Exception e)
{
//debug statement
}
return null;
}
}
}

Answer 3

use cUrl request, hand in hand with output buffering

Answer 4

#user123: I can offer some tips if you can work in C#? The API steps are extensive. Let me know!