Lies, dammed lies and web analytics | March 3, 2011

At Clearleft we’re an incredibly business focussed agency. So we work closely with our stakeholders to understand their business needs, and then turn these into Key Performance Indicators to track. In the vast majority of cases, our clients KPIs increase after working with us. However on the rare occasion that things go in the other direction, we take it as a matter of professional pride to rectify the matter.

Thankfully we’ve only seen this happen on 4 occasions in our 6 year history. The first time this happened it was a temporary blip and righted it’s self naturally after a month or so. This can occasionally happen when new designs go live and users need to adjust to a new way of working.

The next two times this happened were more of a concern. Bounce rates had shot up, conversions had plummeted and our project sponsors were getting grief from the board. Both clients understandably panicked. One rolled their site back to the previous version while the other took versions of previous designs we’d supplied them, and rolled their own hybrid version.

The funny thing with both these stories was that it turned out there was nothing wrong with either site. Instead the clients analytics packages had been set up incorrectly and they were no longer tracking the whole picture. With one project they had accidentally started tracking spam “attacks” to the site, causing the bounce rate to seemingly go through the roof. In the other instance the client was only seeing some of the conversions due to the fact that several of the processes we now Ajax driven and weren’t registering as completed goals. Adding in some virtual page views and events in Google Analytics seemed to sort the problem.

Analytics are incredibly important, but easy to misread. In fact I always liken managing your website based on analytics to driving somewhere only with the aid of your GPS and not look out of the window. It’ll only give you partial view of your journey and doesn’t account for glitches in the system (I once knew somebody in the middle east with an in built and non-updatable GPS. Land reclamation meant that their drive home from work showed them driving through the sea!)

The final project is still ongoing, and we’re looking into all the various options to see how we can help. However from previous experience I wouldn’t be surprised if this was also analytics related.

So if you see a sudden and dramatic drop in your KPIs after launch it’s tempting to panic and roll back or make dramatic changes in the heat of the moment. Before doing this I urge you to take stock of your analytics setup and if necessary call in an expert. After all there are lies, dammed lies and web analytics.

Posted at March 3, 2011 6:42 PM

Comments

Stuart Frisby said on March 3, 2011 9:56 PM

Just a quick question Andy, how do you go about tracking those KPIs effectively in the long run, as analytics only paint part of the bigger picture, do you guys have tools or processes in place which allow you to pull together a broader, more complete analysis?

It’s great to have you back blogging regularly by the way!