flow of scraping a site - php

I want to scrape base on users' input but unlucky the site doesn't have good built-in search function. The webmaster of the site use google custom search to replace its default search function. The results are flawless, like user search shoe Kobe 2014, the google custom search will able to give the correct page, then I can scrape the page.
However, on scraping from the search engine result, it has many limitation, like Google's API has to pay if u more request, same goes to Bing API.

Have you tried contacting the site's owner? Afterall it's their information you're using.

Related

How to get Google Search keywords used to find my website in PHP without Google analytics?

We are trying to develop an analytics section in our site ( built in PHP). And we want to capture the search words which are used to find our website in search engines ( like Google.com) while it is redirected to our site.
Let's take an example, our site is "designsolution.com" and some people search in google with "best web designing company in Kanpur" and google.com showing our site on the first page. Then the user clicks on our site link and googles redirects the user to our site. Now, we want to know that the "best web designing company in Kanpur" used to search and find us.
Is there any way to get Google Search Terms/keywords in PHP? Any API service or any custom code? Please help.
Hello and welcome to Stackoverflow community.
When you click on a link in Google search result and you get redirected to your website, all data that Google sends to your website is just the Referer header parameter. So it's impossible to actualy get the search term using only PHP code. You can just findout that user came from Google search result.
Instead, you have to use Google webmasters and submit your website there so you can see what keywords used for finding your website.
Cheers,
I want to share my research here as I found many people also looking for it.
We can get Google Search keywords for our website using Google Webmaster console API
You need to sync or fetch data after Authorize Google Webmaster console API with your website.
Here is Search google search console API:
https://developers.google.com/webmaster-tools/search-console-api-original/v3/
Here is the sample code: https://developers.google.com/webmaster-tools/search-console-api-original/v3/samples
Another code library:
https://github.com/googleapis/google-api-php-client-services
Thanks

Pull Movies details from Google Play to a Webpage

I want to show google playstore movies in a webpage, so that uses can get info about the movie and can buy any movies from my website.
I haver read that Google does not expose any official API for this. Also tried available APIs from GitHub. Those are not working.
I dont want to read the DOM of google playstore page via cURL login and want to display it to my webpage.
There is no exposed Google Play API, but if you're willing to do just a little bit of DOM parsing you can do this in a pretty straightforward way.
If you get a google playstore page, and scan it for all the IDs of the videos (so, for example, things like '/store/movies/details/The_Fault_In_Our_Stars?id=iXwnxJu4q7s'), you can then take those IDs and query the YouTube API with them, since the Google Play ID is identical to the YouTube ID. If you built a DB of which IDs went with which titles, you could then use the YouTube API to pull the relevant metadata, display them on your page, and even allow rentals through your side (by using the YouTube Player API with one of these IDs, users who haven't rented or bought will see, in the video, a little button that lets them start the rental process). Similarly, You can embed the YouTube player with these IDs to let users watch through your page as well.

What is a good PHP analytics script?

I've created a completely custom website with a custom CMS. I would like to be able to use Google Analytics to get information such as visitor count, popular pages, visitor location(?), etc. I'm not into the analytics part of websites yet, so I've been looking for some easy ways to implement GA into the CMS. I'd love for the analytics dashboar to look similar to Blogger's analytics dashboard.
Does anyone have experience with a script that will get information from GA, and display it in a nice format?
Thanks!
By the way, I am very new to GA, so sorry if some of this doesn't make sense.
Piwik is a good option. It is open source and exposes a web API, so you can probably customize it to your needs.

Google adsense reference

My client has a Google Adsense account and one ad-campaign with several ads under it.
The client needs to identify the incoming users who come through clicking each ad.
The client uses Drupal PHP site.
Is there a possibility to track incoming users who come through Google ads?
Use Google Analytics. From the Features page:
Integrated with AdWords and AdSense
Optimize your AdWords performance with post-click data on your keywords, search queries, match type and more. AdSense reports show publishers which site content generates the most revenue.
You can add arguments to the urls used in the adds:
http://example.com
changed to
http://example.com?google=ad123
You can then in your Drupal check for the presence of the google argument in the $_REQUEST variable and act accordingly. This should be done in hook_boot.
http://code.google.com/apis/analytics/docs/tracking/eventTrackerGuide.html
You may use events, and you may add custom parameters to them, but they didn't work for me on the first shot and i gave up. If you figure out how to use them please post a story on how you did that:)

php class for seo

I recently use the great php class call : Google Analytics PHP API. I manage to get it working to get a lot of info and stats for my 25+ website that have analytic
Now i like to get more data like
The page rank the google placement
(where i am on google #325 or #520)
Google Page Rank
Google Indexed Pages
Traffic Rank
Inbound Links
DMOZ Directory
Yahoo! Directory
get a summary (my way) much more like http://websitegrader.com
do you know a php class to do that ?
I guarantee you're not going to find one "class" that does all that for you.
Manually checking your site's rank on Alexa is probably going to be your best source of information on top of what Google Analytics is already giving you.
Google Analytics does NOT require any sort of API.
The site you linked to asks for your Google username/password. I'm flagging this question. I encourage others to do the same.
you can get all from alexa api (Alexa Web Services)
http://aws.amazon.com/awis/
but not free
$0.00015 per request ($0.15 for 1,000 requests)
i thought in the past to build a website to arrange an Arab sites
but arrived at the checkpoint
somme website use awis
www.statbrain.com
www.websiteoutlook.com
www.websitevaluecalculator.com

Categories