Does anyone know of a piece of code that can run on a server that pipes the data from Apache logs into Google Analytics? I've got a bunch of websites that generate logs, but the users would likely object to injecting Google tracking codes into them. This might be a nice way to get the basics, what's being requested from where, and have it all sorted for me in with my other Google Analytics pages.
You can use the new measurement protocol (available for universal analytics account only) to implement a serverside solution.
Piping logs would probably not work very well (at least if you want to do a batch job - I don't think you can send a timestamp via the measurement protocol, so it would look as if all hits occured at the same time) but it shouldn't be necessary anyway, just create an url with the relevant parameters pointing to the google endpoint and sent it in the background via CURL (or similiar).
If you're in the European Union remember privacy guidelines still apply and you need to inform users and provide an opt-out link.
For non-Universal Analytics accounts, you can use php-ga - Server-Side Google Analytics Client -- it's essentially a server-side implementation of ga.js.
One caveat: If you want the location metrics to record something other than the location of your server, you'll need to log with a Google Analytics mobile tracking ID. Just replace the "UA" in the tracking ID with "MO", like "MO-12345678-1"
GA needs JavaScript, I think, so that various things like screen resolution can be grabbed. So even if this were possible, you'd be missing a good deal of info for some of your users, skewing your other percentages. Also, if your users are suspicious of Google, they probably would not want you to upload their IP addresses to GA.
With all that in mind, I wonder whether a self-hosted GA-like system would fit the bill? If so, try Piwik.
Related
I need to log details in PHP of analytics and usage.
I'm looking at various possibilities:
- Google Analytics server-side
- segment.io
- Just adding a record to a DB with PHP
My concern is how much additional processing this will take on my server. Of course Google Analytics' JavaScript implementation won't use anything on my server, but my server-side method of course will.
I also notice that on https://segment.io/docs/integrations/google-analytics they mention that "Server-side Google Analytics is being deprecated due to difficulty of use" - what does this mean?
So basically, I want to implement some basic analytics storing (count number of hits to a URL + some other basic info) server-side - what's the best way to do this considering all things? I only use the PHP language.
It seems that adding a record to the DB every page view might be a little too much.
Segment.io can actually give you the flexibility of all three of these. Using the php library https://segment.io/libraries/php you can start sending events from your server. The library is designed to queue and batch to maximize server efficiency.
Once the events leave your server, they'll go to Segment.io's servers. Once there, we can route the data to Google Analytics.
Additionally, you could use the "Webhooks" integration on Segment.io to set your own server as a receiving endpoint for the data in real-time, so that you could host your own analytics DB separately from the rest of your infrastructure quite easily/cleanly.
https://segment.io/docs/integrations/webhooks
I've a PHP project that use to work like a cake but now google map api is blocking it
I user a code as this one to get geolocation data from google map (address to coordinates transformation) with a low number of request/day.
$base_url = "http://maps.google.com/maps/geo?output=xml&key=".KEY."&q=".urlencode($address);
$xml = #simplexml_load_file($request_url);
with KEY ad my google API key but I started to get this error
We're sorry...
... but your computer or network may be sending automated queries. To protect our users, we can't process your request right now.
how can I fix that ? Google API guide is useless...
Most of Google Api's have rate limiting protections. You should read the terms of service and see the allowed policy.
Besides that, what you could technically do (and i'm almost certain you are running than code in a loop for multiple addresses) is set a random delay (more than 20s) between the calls to google api, limiting the load and your own evilness... You could also think about using curl to make requests with multiple user agents... its technically possible, although frowned upon :)
I'm looking to have a user enter an app ID on a website, save the information from their app (my sql database), and then display that information on the website.
If anyone would mind sharing the code/process that would be used to do this or are there tutorials that you can point me in the direction of learning how to do this?
If you could help me out at all I will be very grateful. Thanks.
You CANNOT do this via screen scraping. Read Apple's Terms Of Use
Your Use of the Site You may not use
any “deep-link”, “page-scrape”,
“robot”, “spider” or other automatic
device, program, algorithm or
methodology, or any similar or
equivalent manual process, to access,
acquire, copy or monitor any portion
of the Site or any Content, or in any
way reproduce or circumvent the
navigational structure or presentation
of the Site or any Content, to obtain
or attempt to obtain any materials,
documents or information through any
means not purposely made available
through the Site. Apple reserves the
right to bar any such activity.
What you need to do is investigate Apple's Partner Program which includes a program for developers and I believe would grant you access to an API where you would be able to directly query for and receive the info you wanted (such as app descriptions) to display on your site and perhaps even get a commission for sales you generate when people purchase something from apple via links on your site etc.
Odds are that the other site you see which is displaying such info from Apple's store, has an affiliate/partner arrangement with apple. (and if not, it's just a matter of time till they get blocked from the site, find cease and desist letters in the mail from apple's laywers, get sued, or some combination of those three.
You should really look at these first.
https://stackoverflow.com/questions/822380/how-legal-is-screen-scraping
https://stackoverflow.com/questions/396778/legalities-of-screen-scraping
Knowing Apple, they'll probably sue you. They have sued for less. Or IOW who haven't they sued?
If you want to save the information to your database then you might want to look at the program Appfetcher at http://www.altraware.com.
It uses the Xml that apple provides so there should be no legality issues.
We have a client that wants a store locator on their website. I've been asked to find a webservice that will allow us to send a zipcode as a request and have it return locations within x radius. We found this, but it's maintained by a single person, and doesn't look like it gets updated or supported very well. We're looking for something commercial, ideally that updates their zipcode database at least once per quarter, and that has a well-documented API with PHP accessibility. I won't say price isn't an object, but right now we just want some ideas, and my google-fu has failed me.
I've already posted this over on the webmasters forum, but thought I'd cover my bases and post here too.
I've repurposed this outstanding script to conquor this same challenge. It's free, has been very reliable, and is relatively quick.
In my script, I have addresses stored in the DB. So rather than show a page to enter addresses, I simply pass them as a string and let the magic happen.
He says it in the app, but ensure that if you go this route you get your own Google Maps API. It won't work with his!
If you want to go a bit less technical approach, here's a MySQL query you could run on your locations (you'd have to add lat/long to your DB or setup a GEOCODING service) to give you distance as the crow flies.
Google Maps has a geocoder as well and it geocodes to the specific address.
It's limited to x number of requests but that shouldn't be a big deal if your site is small and if you cache. You can get more requests if you pay.
It can be accessed via javascript or via PHP (and there are several prewritten PHP modules out there)
Link here:
http://code.google.com/apis/maps/documentation/javascript/v2/services.html
(I worked for a company that did upwards of 800,000 requests a day, so it's stable and fast :) )
PostcodeAnywhere has a Store Locator feature - I think it's pay per use, but I've used their other products before and they're very cheap.
http://www.postcodeanywhere.co.uk/store-locator-tool/
I have a PHP code that will return a image.
the link is given to 3rd party. so, i need to keep track where the php request coming from. Because the PHP only return the image, I cannot use the Javascript code for Google analytics.
I know that I can get the information from the access.log, but i think I can't dump the access.log to GA for analyzing, right?
so, is there a way that I can do in PHP (e.g. sending a CURL ), send somethig to Google Analytics for tracking?
In practice, what GA will do is to issue a HTTP GET for a 1-pixel sized GIF image, in which the GET parameters will contain the information to store in GA servers. If you figure out the format of the GET request, you may be able to store the information you want to. You can use any net monitoring tool or browser plugin of similar functionality (like Firebug, etc) to understand the parameters that are passed to GA servers. These are nowhere to be found in GA documentation, although the architecture of this process is.
In practice, what you're trying to accomplish is the same as enabling GA for a javascript-disabled client. By limiting the information you can provide to GA to the one that the server obtains from browser requests you won't be able to get some detailed info such as the screen resolution, etc. On the bright side, the information that won't be accessible by this method is actually very little (and probably of little significance) and the web is full of resources on using GA for the javascript-impaired that you can use as example, eventually adapting to PHP and to your particular case.
Galvanize is an open source project that does what Miguel is describing. This is the blog post introducing Galvanize.