Skip to content
rick tait's $HOME

rick tait's $HOME

rick tait's $HOME

  • Home
  • About
  • Tags
  • Code
  • Twitter
  • Github
  • Gists
  • Email

Tag: google api

HOW-TO: download Google Analytics data with Golang via the Google Core Reporting API using two-legged service account Google OAuth2 authentication

Problem: I want to get my Google Analytics data into my own “backend system”

Solution: an example of how to download google analytics data without any realtime user intervention required (no Auth windows etc), using a Google Cloud service account and Go.

there are a number of pre-requisite steps that must be completed before we can get Go-ing. lets get the non-Go steps done first.

non-Go pre-requisite steps:

  • ACCOUNT: you’ve got to have a Google Analytics account & property with valid, query-able data. additionally, because you’ll be querying the Core Reporting API using the Google APIs Client Library for Go, you’ll need a Google Cloud application to do it.
  • PROJECT: create a Google Cloud project. to create one of those, you need to have an account on the Google Cloud Console, and have created a Project in that account.
    image
  • ANALYTICS API: turn on the Analytics API for your project. enable the Analytics API for the Project. find “Analytics” in the APIs & Auth > API’s submenu on the left, and switch it to ON.
  • APP: create a Google Cloud Application. it’s time to create an Application! click “Register App” in the Registered Apps submenu. give your app a name (more of a description, really) and make it of type “Web Application”.
    image
  • CREDENTIALS. your headless application needs credentials (an email address & public/private key pair) so it can be authorized to query the Core Reporting API.create them by expanding “Certificates” in the Registered Apps > YourAppName. you’ll see that there is no public key fingerprint, and email address says “Not generated yet”.

    click “Generate Certificate”. you’ll be prompted to download the private key. save this and keep it safe (you can only download it once, ever).

    it will also tell you the private key’s password; make a note of this, as you’ll need it when converting the .p12 key you just downloaded into a .pem key (which is what Google’s OAuth requires for service accounts) later.

  • JSON: while making a note of your freshly generated service account email address (it looks like 8961649gibberishkjxcjhbsdv@developer.gserviceaccount.com), you should download the JSON file (this is your application’s “client secrets” file). you will set the variable gaServiceAcctSecretsFile to the location of the file later.
  • PEM KEY: we need to convert the .p12 into a .pem key. ok, let’s convert the .p12 key into a .pem key, because that’s the key format required to sign the authentication requests.
    $ openssl pkcs12 -in privatekeyfilename.p12 -nodes -nocerts > privatekeyfilename.pem

    you’ll be asked for the Import password. enter the private key’s password you were given above. now you’ve got the PEM key! you will be setting the variable gaServiceAcctPEMKey to the location of this file later.
    image

  • ANALYTICS PERMISSIONS: now you have to grant the application appropriate permissions to query your Analytics account/property. within Google Analytics, in the Admin section specific to the account/property you want to query, give “Read & Analyze” permissions to the @developer.gserviceaccount.com address you made a note of earlier.
    image
  • ANALYTICS TABLE ID: okay, just one more step! the Core Reporting API requires us to identify the analytics dataset we want to query by using a TableID, not (as you might think), an analytics tracking ID.we can find the TableID in the Google Analytics Query Explorer – which by the way, is fantastic.

    login there, navigate to the specific account/property/profile, and you will see the TableID in the “ids” box. it will look something like:
    ga:12345678
    make a note of it, you setting the variable gaTableID to this value later.

w00t! that’s all of the pre-requisite steps done & dusted. let’s look at some Go code (finally).

Go pre-requisite steps:

in order to use the Go Google API, it must be installed. to install all of the Go Google API’s (Drive, BigQuery, Calendar, etc), you can just do:

$ go get golang.org/x/oauth2
$ go get google.golang.org/api/analytics/v3

once you’ve ensured that you’ve got the above packages safely import-able in your $GOPATH, you’re good to go.

no more pre-requisite steps. let’s talk code.

head on over to htps://github.com/rickt/analyticsdumper. full instructions and example run are there! but here’s the sauce since that’s why we’re all here 🙂

UPDATED source:


package main
import (
"fmt"
"golang.org/x/oauth2"
"golang.org/x/oauth2/jwt"
"google.golang.org/api/analytics/v3"
"io/ioutil"
"log"
"time"
)
// constants
const (
datelayout string = "2006-01-02" // date format that Core Reporting API requires
)
// globals that you DON'T need to change
var (
enddate string = time.Now().Format(datelayout) // set end query date to today
startdate string = time.Now().Add(time.Hour * 24 * –1).Format(datelayout) // set start query date to yesterday
metric string = "ga:pageviews" // GA metric that we want
tokenurl string = "https://accounts.google.com/o/oauth2/token" // (json:"token_uri") Google oauth2 Token URL
)
// globals that you DO need to change
// populate these with values from the JSON secretsfile obtained from the Google Cloud Console specific to your app)
// example secretsfile JSON:
// {
// "web": {
// "auth_uri": "https://accounts.google.com/o/oauth2/auth",
// "token_uri": "https://accounts.google.com/o/oauth2/token",
// "client_email": "blahblahblahblah@developer.gserviceaccount.com",
// "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/blahblahblahblah@developer.gserviceaccount.com",
// "client_id": "blahblahblahblah.apps.googleusercontent.com",
// "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs"
// }
// }
var (
// CHANGE THESE!!!
gaServiceAcctEmail string = "blahblahblahblah@developer.gserviceaccount.com" // (json:"client_email") email address of registered application
gaServiceAcctPEMKey string = "./analyticsdumper.pem" // full path to private key file (PEM format) of your application from Google Cloud Console
gaTableID string = "ga:NNNNNNNN" // namespaced profile (table) ID of your analytics account/property/profile
)
// func: main()
// the main function.
func main() {
// load up the registered applications private key
pk, err := ioutil.ReadFile(gaServiceAcctPEMKey)
if err != nil {
log.Fatal("Error reading GA Service Account PEM key -", err)
}
// create a jwt.Config that we will subsequently use for our authenticated client/transport
// relevant docs for all the oauth2 & json web token stuff at https://godoc.org/golang.org/x/oauth2 & https://godoc.org/golang.org/x/oauth2/jwt
jwtc := jwt.Config{
Email: gaServiceAcctEmail,
PrivateKey: pk,
Scopes: []string{analytics.AnalyticsReadonlyScope},
TokenURL: tokenurl,
}
// create our authenticated http client using the jwt.Config we just created
clt := jwtc.Client(oauth2.NoContext)
// create a new analytics service by passing in the authenticated http client
as, err := analytics.New(clt)
if err != nil {
log.Fatal("Error creating Analytics Service at analytics.New() -", err)
}
// create a new analytics data service by passing in the analytics service we just created
// relevant docs for all the analytics stuff at https://godoc.org/google.golang.org/api/analytics/v3
ads := analytics.NewDataGaService(as)
// w00t! now we're talking to the core reporting API. the hard stuff is over, lets setup a simple query.
// setup the query, call the Analytics API via our analytics data service's Get func with the table ID, dates & metric variables
gasetup := ads.Get(gaTableID, startdate, enddate, metric)
// send the query to the API, get a big fat gaData back.
gadata, err := gasetup.Do()
if err != nil {
log.Fatal("API error at gasetup.Do() -", err)
}
// print out some nice things
fmt.Printf("%s pageviews for %s (%s) from %s to %s.\n", gadata.Rows[0], gadata.ProfileInfo.ProfileName, gadata.ProfileInfo.WebPropertyId, startdate, enddate)
return
}

view raw

analyticsdumper.go

hosted with ❤ by GitHub

March 7, 2014January 8, 2019 ricktdotorgcode, go, google analytics, google api, google cloud, oauth2, service accountLeave a comment
Blog at WordPress.com.
  • Follow Following
    • rick tait's $HOME
    • Already have a WordPress.com account? Log in now.
    • rick tait's $HOME
    • Customize
    • Follow Following
    • Sign up
    • Log in
    • Report this content
    • View site in Reader
    • Manage subscriptions
    • Collapse this bar