Problem: I want to get my Google Analytics data into my own “backend system”
Solution: an example of how to download google analytics data without any realtime user intervention required (no Auth windows etc), using a Google Cloud service account and Go.
there are a number of pre-requisite steps that must be completed before we can get Go-ing. lets get the non-Go steps done first.
non-Go pre-requisite steps:
- ACCOUNT: you’ve got to have a Google Analytics account & property with valid, query-able data. additionally, because you’ll be querying the Core Reporting API using the Google APIs Client Library for Go, you’ll need a Google Cloud application to do it.
- PROJECT: create a Google Cloud project. to create one of those, you need to have an account on the Google Cloud Console, and have created a Project in that account.
- ANALYTICS API: turn on the Analytics API for your project. enable the Analytics API for the Project. find “Analytics” in the APIs & Auth > API’s submenu on the left, and switch it to ON.
- APP: create a Google Cloud Application. it’s time to create an Application! click “Register App” in the Registered Apps submenu. give your app a name (more of a description, really) and make it of type “Web Application”.
- CREDENTIALS. your headless application needs credentials (an email address & public/private key pair) so it can be authorized to query the Core Reporting API.
create them by expanding “Certificates” in the Registered Apps > YourAppName. you’ll see that there is no public key fingerprint, and email address says “Not generated yet”.
click “Generate Certificate”. you’ll be prompted to download the private key. save this and keep it safe (you can only download it once, ever).
it will also tell you the private key’s password; make a note of this, as you’ll need it when converting the .p12 key you just downloaded into a .pem key (which is what Google’s OAuth requires for service accounts) later.
- JSON: while making a note of your freshly generated service account email address (it looks like
8961649gibberishkjxcjhbsdv@developer.gserviceaccount.com), you should download the JSON file (this is your application’s “client secrets” file). you will set the variable gaServiceAcctSecretsFile to the location of the file later.
- PEM KEY: we need to convert the .p12 into a .pem key. ok, let’s convert the .p12 key into a .pem key, because that’s the key format required to sign the authentication requests.
$ openssl pkcs12 -in privatekeyfilename.p12 -nodes -nocerts > privatekeyfilename.pem
you’ll be asked for the Import password. enter the private key’s password you were given above. now you’ve got the PEM key! you will be setting the variable gaServiceAcctPEMKey to the location of this file later.
- ANALYTICS PERMISSIONS: now you have to grant the application appropriate permissions to query your Analytics account/property. within Google Analytics, in the Admin section specific to the account/property you want to query, give “Read & Analyze” permissions to the @developer.gserviceaccount.com address you made a note of earlier.
- ANALYTICS TABLE ID: okay, just one more step! the Core Reporting API requires us to identify the analytics dataset we want to query by using a TableID, not (as you might think), an analytics tracking ID.
we can find the TableID in the Google Analytics Query Explorer – which by the way, is fantastic.
login there, navigate to the specific account/property/profile, and you will see the TableID in the “ids” box. it will look something like:
ga:12345678
make a note of it, you setting the variable gaTableID to this value later.
w00t! that’s all of the pre-requisite steps done & dusted. let’s look at some Go code (finally).
Go pre-requisite steps:
in order to use the Go Google API, it must be installed. to install all of the Go Google API’s (Drive, BigQuery, Calendar, etc), you can just do:
$ go get golang.org/x/oauth2
$ go get google.golang.org/api/analytics/v3
once you’ve ensured that you’ve got the above packages safely import-able in your $GOPATH, you’re good to go.
no more pre-requisite steps. let’s talk code.
head on over to htps://github.com/rickt/analyticsdumper. full instructions and example run are there! but here’s the sauce since that’s why we’re all here 🙂
http://gist-it.appspot.com/http://github.com/rickt/analyticsdumper/blob/master/analyticsdumper.go
