Google Analytics and Google BigQuery- Oh My!

May 17, 2013 By James Standen Leave a Comment

Google analytics with BigQuery means XL data Googles announcement at Google IO that session and hit data was going to be available in Google BigQuery marks a significant milestone in Google Analytics relentless drive into the Enterprise analytics market.

Google mentions a number of scenarios that become possible with this upcoming integration in the announcement post.

While all of these are exciting, one touches on an area where I see this new capability opening the flood gates- integrating with Data warehouses.

While of course it is possible to integrate huge amounts of Google Analytics data with data warehouses thanks to the existing API, and many of our customers are doing this already with sophisticated, automated integration into their enterprise systems using Analytics Canvas, the direct access to the most granular data in a platform like Google BigQuery is a game changer in terms of scale.

A new level of clarity through integration of data at the most granular level

What this integration between Google Analytics and BigQuery means is that now Google Analytics data can be tied to other enterprise data at a more granular level- and over a scope and scale that is necessary for the largest Google Analytics Premium customers.

This integration also makes Universal Analytics even more powerful and useful- tracking everything at a very granular level everywhere, with the customer at the center of it all will allow businesses to have a new clarity across their entire operation. The multiple digital and real world touch points they have with their customers and prospects will be visible and data scientists will have a field day.

Google has announced that this will be available “later this year” and will be available to Google Premium customers.

Stay tuned for more news- and to hear how Analytics Canvas will be taking advantage of the new capability.

Filed Under: Analytics Canvas Tool, Google Analytics Premium, Google BigQuery Tagged With: Google Analytics Premium, Google BigQuery

How to eliminate sampling by using the Google Analytics API

May 2, 2013 By James Standen Leave a Comment

Sampling can drastically reduce the accuracy of data queried from Google Analytics, but by using the API in many cases it can be eliminated.

Google Analytics is clearly an enterprise capable analytics platform. As higher and higher traffic websites take advantage of its features, Google has put in place some limits on how much data can be included in a query- if a query requires more than this, it implements sampling (called fast access mode in the Google Analytics user interface) this ensures that the user interface and API respond quickly.

Sampling provides an estimate based on a sample of visits in cases where Google has not pre-aggregated datasets. The most common cause of sampling is the use of advanced segments in a query.

In many cases, sampling only slightly affects the accuracy of the result- but in many others, it makes it impossible to get valid data.

The good news is, that Google has provided a fantastic API, and that lets tools like Analytics Canvas provide a solution.

How does it work?

Generally sampling kicks in at 500,000 visits in the query period. The way we can eliminate sampling is by breaking a query up into multiple queries by time period so that each individual query does not trigger sampling.

For example, imagine a site that gets 20,000 visits a day. Any query of more than a month in duration will trigger sampling. However, a query including 7 days will only involve 140,000 visits, and therefore the API will return exact data.

Analytics Canvas lets you automate this- we call it query partitioning.

Lets look at an example from one of our clients. We have done a query on a site for a period where there are almost 9 million visits. We’re looking for a specific source, so we have defined an advanced segment. This results in very heavy sampling- in fact less than 6% of the visits are actually included in the sample.

Lets do a comparison between the sampled data and unsampled data that we can get using Analytics Canvas. In this case, to make the query all we need to do is specify that we want to break the query up into 15 day increments to ensure that the exact data is retrieved. Analytics Canvas then automatically aggregates the results of all the different queries:

analytics-canvas-partitioning-setting-to-eliminate-sampling-from-google-analytics

Here is how sampled vs unsampled data sets compare:

comparison-of-sampled-vs-unsampled-google-analytics-data-and-accuracy

We can see that doing analysis on the sampled data alone could lead to very different conclusions and a significant amount of detail is lost due to the sampling process. While the number of pageviews is off by 8%, those pageviews are allocated by the sampling among about 400 pages when in fact, over two thousand different pages were viewed.

If you have a high volume site I’d strongly suggest you consider Google Analytics Premium where manual unsampled exports are possible. While a number of our customers subscribe to Premium, at $150,000 a year it’s not for everyone. But for companies that do serious business online it offers huge value.

So while there are limits on how much data can be included in a query, for both Premium and regular Google Analytics customers, Analytics Canvas can help you uncover those hidden insights.

Filed Under: Analytics Canvas Tool, Avoiding sampling, Google Analytics Premium Tagged With: fast access mode, Google Analytics API Limits, Google Analytics Premium

Analytics Canvas Advanced Funnel Analysis

April 25, 2013 By James Standen Leave a Comment

google-analytics-funnel-analysis

Funnel analysis is a key part of ecommerce analytics, and being able to understand how your visitors move through your cart and checkout process (and where they leave) is a key part of optimizing any website.

Google Analytics has some good functionality in this area, letting you see how many users are moving through your funnels for a given date range, as well as the excellent Goal Flow visualization that lets you see where visitors are coming from and going to around your funnel pages.

However, there are a number of types of analysis that are simply not possible within Google Analytics itself.

Pushing the limits of funnel analysis by using the Google Analytics API

visits-to-funnel-steps-mutliple-websites

Introducing the advanced funnel query in Analytics Canvas.

Define a funnel using events, custom variables, goals, pages
Look at your funnel on a segment by segment basis
See the evolution of exit rate by funnel step over time
Eliminate sampling in many cases for high traffic sites
Look at funnels accross multiple profiles and accounts at the same time
Automate your funnel analysis and integrate the data into your enterprise systems

Analytics Canvas lets you configure a funnel using an easy graphical interface, and then automatically creates all the API calls needed to give you the insight you need.

This new query lets you define funnels using events, custom variables, goals and pages. The result is a much more powerful way to understand how your visitors are interacting with your funnel and to have much more sophisticated funnels.

Here is an example of a series of funnel steps defined using the new feature:

Once you define the funnel, you get all the details for each step in a clear dataset with the following values:

The number of visits that entered the funnel at that step
The number of visits that entered from a previous step
The total number of visits that completed that step
The number of visits that went to a later step
The number of visits that exited at that step

Whats more, you can specify the period and frequency that you want to run the analysis, meaning you can analyze your funnel performance as it evolves over time.

Why would you want to do this? Because if you are making changes to your checkout process, or signup pages etc., then you want to know when the exit rate of a funnel step changes FAST, so you can fix any issues.

funnel-exit-rate-evolution-over-time

We’re looking forward to seeing the more sophisticated analysis that serious ecommerce sites will be able to do with this new functionality- give it a try on your funnel and discover the optimization you are missing.

Filed Under: Analytics Canvas Tool, ecommerce Tagged With: Analytics Canvas Tool, ecommerce funnel analysis

Analytics Canvas V1.6 Released- Cost upload, advanced funnels and easy access to hundreds of GA profiles

April 24, 2013 By James Standen Leave a Comment

We are pleased to announce the latest release of Analytics Canvas, with new features and upgrades that provide even more powerful access to your analytics data.

Google Analytics Cost upload and Bing Ads API connectivity
Advanced funnel Analysis
Enhanced GA profile management for those accessing hundreds of profiles at once
Advanced query partitioning to avoid sampling

Cost Upload into Google Analytics for online advertising ROI reporting

Sure, ad words might be a significant part of your overall digital marketing spend, but it’s not all of it. When Google Analytics introduced the ability to upload data INTO GA and created its Ads return on investment report, your ability to track cost and return from all of your advertising campaigns took a quantum leap. Analytics Canvas is the best way to take advantage of this functionality, particularly if you have lots of ad spend sources, and want to automate. You can learn more here with a helpful tutorial.

Advanced funnels for ecommerce analysis

analytics-canvas-v1_6_release
While Google Analytics provides funnel analysis, we’ve stepped up the bar by pushing the Google Analytics API to its fullest, and combining it with Analytics Canvas automation and database connectivity. If you want to better understand your funnel, you have to give this new capability a try.

Define your funnel steps using:

Events
Custom variables
Goals
Page path

Once you define the funnel, you get all the details for each step in a clear dataset, and can automatically generate funnel analysis monthly, or even daily, to track evolution over time.

Accessing and combining data from hundreds of Google analytics profiles at once

Many Analytics Canvas customers need to connect to dozens or even hundreds and hundreds of Google Analytics profiles. In V1.6 we’ve added even more features to help make this as easy as possible.

Filtered search, better options for multiple selection, editing and saving of profile lists make Analytics Canvas the choice of anyone with hundreds of websites that use Google Analytics. Check out our video tutorial that shows you how it works.

Sampling? No thanks!

If you try to query more than 500,000 sessions, Google Analytics will return sampled data. Analytics Canvas can automatically break the query down into smaller time frames, each of which is under the limit, so therefore avoiding sampling. Companies with serious web traffic can get access to raw data, and then pump it into databases using automation to integrate with their enterprise systems.

In version 1.6 partitioning is even easier to use, with more options for large numbers of queries and automation.

Filed Under: Analytics Canvas Tool, ecommerce, Google Analytics API Tagged With: Analytics Canvas Tool, ecommerce funnel analysis

Five ways to step up your use of the Google Analytics API

January 30, 2013 By James Standen Leave a Comment

Google Analytics is a powerful tool, and provides a wide range of excellent functionality right in the web interface.

But Google knows it hasn’t provided everything. That’s why they created the Google Analytics APIs.

Many readers might be using an excel based plugin to pull data using the API directly into a spreadsheet for manual manipulation. Often this is a great way to start.

Here are five ways to take your Analytics to the next level using Analytics Canvas:

1. Avoid sampling and get your raw data

Nobody likes sampled data that is “close but not quite right”- and in fact, if you have lots of traffic, or when analyzing highly segmented data the sampled results won’t even be close, because the sample size can be extremely small. The problem arises when you request a date range in the Google analytics UI that results in more than 500,000 visits being included. But with the API, because you can do mulitiple smaller queries, you can often avoid ever exceeding this limit- and you get the exact data. To see how to do this with Analytics Canvas, check out this blog post.

2. Combine data from lots of different profiles and accounts

While the Google Analytics UI gives you great access to a single profile, it lacks tools for combining data from multiple profiles and accounts. Using the API lets you do this quickly and easily, and saves you lots of tedious export, cut and paste and manual merge. Analytics Canvas can query even hundreds of Google Analytics profiles at once, and consolidate all the data into a single file or database table, saving hours of work every week.

3. Integrate your Google Analytics Data with data from other systems

Because the API lets you get data into another database, it then becomes possible to combine data from your systems, or data sets extracted from other vendors systems and connect it to your Google Analytics data. If you manage your tagging and code right, you can use Custom variables to link your Google Analytics data to your CRM data for customers that login to your site, and gain new insight as well as provide better service.
Having your Google Analytics data be a silo, disconnected from all your other systems should be a thing of the past- integration lets you deliver the service your customers expect, and understand what works across your entire enterprise. For retailers, connecting offline and online datasets can give a quantum leap in understanding through techniques like tracking point of sale coupon redemption, online/offline offers and buying behavior regardless of where it happens.

4. Break free of all limits in Segmenting and exploring the long tail

While Advanced segments are very powerful, there are always limitations, and only so far you can push things within the Google Analytics UI. It means that by leaving your data only in GA you are forgoing a whole world of more advanced segments and analysis.

With Analytics Canvas, it’s no longer necessary to have an SQL expert, or a database programmer to create very sophisticated analysis.

By being able to first get at ALL the data, unsampled, then create any number of filters, segmentation rules and subsets, without limitations in terms of numbers of dimensions or metrics, an Analyst armed with Analytics Canvas can go farther, faster, and without needing to specify everything to the IT department.

This more advanced, free form analysis is all the more powerful when you’ve integrated your Google Analytics data with data from other systems.

5. Automate, Automate, Automate!

On top of all the above, because the API can be automated, whether you build it yourself, or use tools such as Analytics Canvas, it makes it possible to have the data be extracted, formatted and delivered without having to have an army of under utilized analysts wasting time on cut and paste. Reporting is often necessary, but if you’re doing it purely manually, chances are you are paying far to much, and are missing out on true insight that your analysts could be discovering- if they had some better tools in their toolbox.

Get the free trial of Analytics Canvas now and discover how to take your use of the API past a simple Excel export.

Filed Under: Analytics Canvas Tool, Google Analytics API, Google Analytics Data Tagged With: Cross account query, fast access mode

Avoiding sampling in Google Analytics for your year end reporting

January 3, 2013 By James Standen 1 Comment

As 2012 is behind us, in many organisations it is time for year end reporting.

If you use Google Analytics, and had more than 500,000 visitors in 2012 and want to use an advanced segment in your reporting (and you should use segments, because segments rock) then it means it is also time to deal with sampling.

A query trying to see Non-paid search by source ignores 94% of the data on a site with about 9 M visitors in 2012

The evil yellow “fast access mode” box that informs you that GA is giving you an estimate only, based on a potentially quite small sample set. To do your reporting properly, you need to get unsampled data from Google Analytics- how?

One way is to just do lots of exports, each one on a time period small enough to avoid sampling. If you have a lot of visitors, then you might need to do dozens or even hundreds of exports, then add all that data up. This year, don’t waste all that time cutting and pasting. There is a solution to get unsampled data even if you have almost half a million visitors a day- by automatically generating all those queries using Analytics Canvas.

Avoid sampling and consolidate all your data exports into a one click operation

Analytics Canvas lets you avoid the nightmare this year. And because it has a 30 day, full function free trial you can even do your 2012 year end reporting for free.

In Analytics canvas, you can just select automatic partitioning, and it does all the work for you, making a query for every period, and then consolidating the data for you, ready to export into excel, or wherever you need it. Because its automatic, you can pick a period that is short enough that you KNOW there won’t be more than 500,000 visits, so no sampling.

We’ll show you an example- here is a query with a weekly partition- we’ve specified no more than 7 days per request, and set the date from Jan 1 2012 to Dec 31 2012. This results in 52 queries, and no sampling.

Do you need to have information for the entire year from Google Analytics and use an advanced segment? If you have lots of visits, then you need to watch for sampling- give Analytics Canvas a try for free, and discover how you can turn days of mindless work into a quick automated query.

Filed Under: Analytics Canvas Tool, Avoiding sampling, Google Analytics API Tagged With: fast access mode

Analytics Canvas at the Google Analytics Summit 2012

November 9, 2012 By James Standen Leave a Comment

Last week in Mountainview at the Computer science history museum, Google analytics certified partners, Google Analytics Premium customers, Googlers and a select group of five software vendors attended the annual Google Analytics Summit. Analytics Canvas was proud to have been invited again this year by Google to present our latest version and participate.

This was a particularly exciting summit, as Google made some very significant announcements regarding the future and direction of Google Analytics. Many of these are game changing, particularly for enterprise users. At a high level, I see two key points.

User Centricity Google Analytics is moving from a session centric to a USER (read customer) centric model
Data Import and linking It is now possible to load data INTO Google Analytics to enrich analysis, as well as to link external enterprise data sources to GA data based on user (customer) ids

First, rather than thinking about individual visits, the underlying model of Universal Analytics will be customer centric. Companies can build their analytics to revolve around customers, their actions, behaviors and the profitability attached to them. This means that web analyics data can become less isolated from other customer data and improves accross the enterprise visibility, bringing Google Analytics into the main stream of all tracking, not just web analytics.

The second point makes the first one actually work- because a large number of your customer interactions occur outside of your website, in places like customer relationship management systems (CRM) or enterprise resource planning systems (ERP), or point of sale (POS) to have a true global view of the customer, this data has to be tied into your Google Analytics Data.

Analytics Canvas will continue to provide the data integration and analysis capabilities needed, particularly by Google Analytics Premium customers to fully take advantage of all of the GA capabilities and to tie Google Analytics in ever more powerful ways to enterprise data. Companies that embrace these new capabilities are going to have visibility accross their entire operations and marketing process that will be a significant competitive advantage.

For a detailed overview of some of the new features that are bringing about these changes, Justin Cutroni from Google put together a great summary on his blog.

We’ll be blogging and talking much more about what all this means for integration going forward. To discuss this with us directly, feel free to contact us.

As an aside If you like computers then if you are ever in Mountainview you should definitely check out the museum- this is a place where there is a cray super computer across the room from a full size working model of one of Babbages difference engine designs- very cool for data folks.

Filed Under: Analytics Canvas Tool, Google Analytics API, Google Analytics Premium Tagged With: Analytics Canvas Tool, Google Analytics Premium

Announcing Google Analytics Cost Upload in Analytics Canvas

October 29, 2012 By admin Leave a Comment

For the very first time, it is now possible to UPLOAD data into GA.

Consolidation of cost data is key for anyone wanting to optimize their return on investment in digital marketing campaigns. Now for the first time, campaigns using AdWords, the Yahoo/Bing ad network, or any other other Ad system with exportable data can get a complete view of cost right in GA.

Analytics Canvas is pleased to announce the availability of a new Beta that implements this new functionality.

This beta version can connect directly to the Bing Ads API, pulling a wide range of report data directly onto the canvas and into GA. As well, using our simple column mapping function, you’ll be able to easily conform other data sets for uploading to Google Analytics.

Google has made a large step forward in making GA a platform for all Ad spending, and we’re Analytics Canvas is pleased to offer you ready and robust tools to take your analysis to the next level.

For more information on how this functionality works, and the options available to get all your cost data into Google Analytics, you can contact us here. Or, if you are already an Analytics Canvas customer, or have signed up for a trial key, you can download the beta and try it yourself.

In order to use the new cost upload functionality, you will need to get on the Google white list, to have the ability turned on in your chosen web properties.

We’ll have more information on how to do this in an upcoming blog post- in the mean time you can request your access from google here.

Download beta now

Filed Under: Analytics Canvas Tool, Google Analytics API, Google Analytics Data

Analytics Canvas V1.5 Released with Google Analytics Social Interactions Mobile Device Type and More

September 18, 2012 By James Standen Leave a Comment

Today Google announced more than forty new dimensions and metrics available in the Google analytics API. We are pleased to also announce the release of Analytics Canvas V1.5, that has full access to all of them.

We’ll be looking into the different metrics in more detail- but here’s our take on what we think is most interesting in these new data sets:

Offsite Social Activity

One of the key enhancements is that the data from the analytics social data hub is now available. The interesting aspect of this data is that it is not data about visits to your site but about people talking about, bookmarking, or linking back to your site. Want to see when your site is mentioned on Digg, bookmarked in Delicious, or saved on Pocket? Now that data is in Google analytics- and through the API, can be in your Analytics Canvas generated dashboards and reports.

These new dimensions and metrics do take a bit of getting used to- they are not visit based. In fact, if you try to use the Visit metric with a social activity dimension you’ll get an error. Because there wasn’t a visit- this is activity that is happening on other social websites, and being collected and delivered to you via the social hub. Which is, if you think about it, an interesting new capability- and by having the data available in the API, makes understanding what interactions are happening around your site and by extension your brand in a much wider way.

Mobile APP tracking

Of particular interest to mobile application developers using or thinking about using the Google Analytics SDK for iOS and android, now the App view and exception metrics are available. So if you are building apps, you can use Google analytics to collect lots of data, and the API to pull that into your reporting/backend systems for analyzing user behavior.

Mobile device information

Back in June, we blogged on how Google changed how it tracked its operating system dimension for mobile visits- suddenly, iPad and iPhone were gone, replaced by IOS.

Well, they are back- and better than ever, now you can track the exact model of the phone where it should be, on dedicated dimensions- you will have all the detail you need, if you want to know exactly which phones are visiting.

Plus more dimensions in the areas of site speed, and user timing- giving you full access to everything in Analytics Canvas via the API.

Result? Coming up on almost 500 dimensions and metrics available.

Looking at all the dimensions and metrics is impressive- Google has a complete interactive list that lets you select a dimension or metric, and shows you which other dimensions and metrics are compatible, to help you form queries.

Google Analytics has been evolving quickly over the last year, and Google’s obvious commitment to the API, and to delivering new data sets and sources through it means that it provides a compelling platform for enterprise analytics. With Analytics Canvas, it is possible to tap into that data, automate it, and report on it in entirely new ways.

To try the new dimensions and metrics out for yourself, sign up for a free trial. Or contact us to have us help you deliver an end to end solution.

Free Trial

Filed Under: Analytics Canvas Tool, Google Analytics API Tagged With: Analytics Canvas Tool, Google API

Analytics Canvas V1.4 Released with Multi Channel Funnel export

July 30, 2012 By James Standen Leave a Comment

Today Google announced that the Multi-channel funnel (MCF) data within Google analytics is now available to export with the API.

And today we are releasing V1.4 of Analytics Canvas, with full support.

What does this mean? It means you can now understand your conversions and how people are getting to your site like never before.

It also means now you can Automate your multi-channel funnel reporting with Analytics Canvas just as you can automate your core Google Analytics reports.

Google has exposed this awesome data by adding two new APIs: MCF Conversion Path and MCF Interactions.

These are new APIs, on top of the existing Core Reporting API. Each one has a specific set of dimensions and metrics that can be used to create queries on the multi-channel funnel data.

As part of Google’s trusted developer program, we’ve had access to these new APIs for a while and have already been taking advantage of the new power in test cases.

To learn more about how a top level consultancy is using Analytics Canvas and this new API, check out the Cardinal Path blog post where Stephane Hamel uses Analytics Canvas and Tableau to gain new insight into online ticket sales for Lollapalooza 2012.

Improve your online marketing performance

What does having access to multi channel funnel data do for you?

Now rather than a static snapshot of a given period of time, with Analytics Canvas we can show you how your multi-channel funnels are evolving over time. You can see the impact of campaign changes on conversion paths, and track them.

To have us show you what we can do to help you (or your clients) better understand how users are finding, interacting with and converting online, contact us now- we can deliver end to end data and dashboard solutions that take advantage of this new capability right now.

Or to try it out for yourself, get the free trial: