Let me propose a maturity scale and suggest where things are... Level 1) Are you recording metrics? That's the first basic level of maturity, without recordin

Let me propose a maturity scale and suggest where things are... Level 1) Are you recording metrics? That's the first basic level of maturity, without recording data you cannot have data from which to support any decisions. Without data you are blind. The assumption here is that you are able to record everything that you need to... recording 3 metrics isn't enough, you need to be able to get to all your data and record it over time. Level 2) Are you able to visualise your raw data? This is the basic dashboard. Whether you build something pretty or use an existing software package it's just the ability to see the data that you are collecting. This relies on people to figure out the inter-connectedness of data and to make inferences from it. Basic graphs and trend analysis fit in here. Level 3) Are you able to visualise insights in your data? This is about changing your dashboard or reports such that you can start to consider metrics that are abstractions from the raw data but important to you. Think of cohort charts, revenue relationships to user behaviour, UX funnels, engagement charts, etc. Level 4) Are you able to record exceptions in your reporting? As in, can you create thresholds against a metric such that if the metric goes above or below the threshold an event is triggered to record it. Exception tracking coupled with the insights in level 3 should give you the ability to do things like release a new feature and see the resulting effect on certain metrics and be alerted when those effects are outside of some normal range. Level 5) Are you able to visualise your exceptional events? And here we are at the newsfeed of data, the events which stand-out are highlighted and all of the other noise drops away. This is great for producing a list of things you might want to know or react to... but you can't get here without going through (and still having) levels 1-4 and it's important to understand that different people in your organisation still need all those prior levels (try telling your sysadmin that he can't have log files or munin and that instead he's got a news feed).