Performance monitoring analysis. Who's doing it?

Performance monitoring analysis across digital properties often falls through the cracks due to unclear ownership or the assumption someone else is doing it. I've read stories how analysts have uncovered shopping cart browser anomalies that have saved companies millions in revenue, that's all well and good however with a proper performance monitoring framework in place this should get picked up outside of ad-hoc analysis.

What is performance monitoring analysis?
Performance monitoring analysis also known as "is crap broken/working analysis" is the regular (daily/weekly) review of your important digital metrics. You'd expect your Technology department to be monitoring system health and performance, however, there'll be all kinds of things they can't see or aren't concerned with. If you're in e-commerce you can normally assume the top line metrics are being reviewed regularly so if revenue falls off a cliff then alarm bells start to ring. But what about everything else that's a dimension or two deep? It's quite possible that all sorts of discrepancies, bugs and blips are about and you just wouldn't know about it.

What should be monitored and by who?
It's not efficient to centralise the monitoring so I'd suggest a decentralised approach which effectively distributes analysis across the business with appropriate business/product owners. So the million dollar question - who are the owners and what are they monitoring? To understand the "what" it generally helps to split your business into commercial and platform products, let's take an online bank as an example and for brevity take a simplified view of their commercial, platform products and metrics;

Commercial products	Metrics	Dimensions
Bank accounts A primary product of a bank and online application forms would be a major source of new business.	Application form visitors, Completed applications, Form completion ratio	Traffic source, Account type, Nationality
Mortgages	Mortgage amount, Application form visitors, Completed applications, Form completion ratio	Traffic source, Mortgage type
Insurance	Insurance amount, Application form visitors, Completed applications, Form completion ratio	Traffic source, Insurance type
Platform products
Website Seeing as browser compatibility can be the cause of many anomalies you'd want browser versions available as dimensions with the main KPIs.	Visitors, Completed forms, Logins, Error pages	Browser type, Browser version, Operating system
Website accounts This is the account login for customers to check balances and make transfers.	Logins, Average time, Transfers	Account type, Location
Mobile accounts Same as the above but for mobile.	Install rate, Uninstall rate, Crash rate, Logins, Transfers	Operating system, Account type
Online forms I'd envisage this being an aggregate view of all the lead generation forms, the data being reported will have overlap with some of the previous products listed but I don't see that as an issue, there could be more detailed metrics being reported such as form validation errors. In addition, there could be non-commercial products such as mortgage calculators as part of this view.	Form Visitors, Form error rate, Form completion rate, Abandonment rate	Form type, Traffic source
Analytics All of our commercial and digital products are being monitored by our analytics software which means we need to monitor the monitoring system to be sure of the data integrity.	Unclassified and unknown KPIs, Completed forms, Visitors, Form completion rate	Browser type, Browser version

How to help
Identify who the metric owners are, sometimes this is easy and sometimes not - platform products are more abstract than commercial so if there's not already a product owner see where you get by nominating someone, quite often people are happy to be better informed with relevant data around an area they have a working interest. Once you have a list of products and owners you can get to work on developing the dashboards, this is where tools such as Adobe's Report Builder can really help; pulling data into Excel and scheduling reports is a breeze which means your product owners don't need to login/learn your analytics package.

Anomalies should be caught sooner with a good reporting framework in place, and with a distributed analytical network working across the business it's possible that relevant optimisation suggestions are generated too.

Right, that's it - thanks for reading. If you have any comments, questions or feedback please leave them below. And you can follow new posts from this blog on Twitter, Email or RSS.