6 ms·

One of the things that really hit home at my last job and that I see missing from a lot of these dashboards is comparative context. So I walk in in the morning

by weego 10y ago

One of the things that really hit home at my last job and that I see missing from a lot of these dashboards is comparative context. So I walk in in the morning and we have a dashboard for vists and signups / payments and it shows lots of visits and no payments. Is that normal? Is that expected or concerning? Without showing what it was yesterday at the same time or on the same day last week, or weekly/monthly avg how do I know how to feel about anything I'm looking at. I know the answer to that can be "you should know your product" but I'm not sure that answer is enough.

AznHisoka 10y ago
you know.. I have been an user of some of these paid dashboards and you hit on a very good point! historical context is something I always felt was missing. i end up having to store the history myself and comparing it but it would make sense for these dashboards to be "smart" and actually do a little more than just display the current data.
- jaxn 10y ago
  You can do this with Klipfolio. That is the one I am currently using.
Gys 10y ago
I thought it was just me. I even emailed a commercial supplier and they replied (more or less) that its not a high priority.
dotBen 10y ago
What you are looking for is a BI (business intelligence) tool. Those typically include dashboard but also look at trends - both automatically and programmatically via your own scripting. Looker is probably my favorite currently but it depends on your needs really.
- weego 10y ago
  I know what you're saying, I'm often bad at meandering past the point I'm thinking of with a non-focused example. I guess in a nutshell I feel like these dashboard products, while well meaning and often very well crafted are just throwing things at your eye sockets. Showing data because you have data is not a solution in of itself.
  - dotBen 10y ago
    I dunno, there's probably room for all of them. There's lots of uses for dashboards that are not BI-driven - Castor even suggested their service could be used for conference room information etc. I think the problem is dashboards on their own are pretty low value - ie $40/y for Castor. But that doesn't create a very interesting or particularly sustainable business.
michaelbuckbee 10y ago
When I've built dashboards I usually try and show that by having barcharts of averages for different time periods. Example: MRR+ Today, MRR+/Day over last 7 days, MRR+/Day over last 30 days, etc. It's still not perfect, but it adds at least an element of that "at a glance" context.
- jaxn 10y ago
  I like area + line charts for this. The area fill is the comp period and the line is the current. One of my favorite ways to do this (that most charting packages struggle with) is to use an area full for last year, a solid line for this year, and a dotted line for the future based on current trajectory. You can compare to multiple periods using stacked areas (particularly when you can assume previous periods had growth).
- deleted 10y ago
  [deleted]
j_s 10y ago
I would like to know "am I above the ___th percentile for any timeout metric" right now vs. last week / month / etc. I am just getting started with Graphite and wondering how to do this across all of the timeout metrics at the same time on the same graph. (It would probably be an all-or-nothing scenario but there is a chance only a portion of the system is affected by something...)
dsshanley 10y ago
I'm the founder of Notion (usenotion.com) and one of the reasons we built Notion is we experienced the gaps with just a TV-style dashboard. Historical trends and context were definitely missing. We even deployed Tableau (so $$$) across my last company to find no one really used it. We're trying to tackle this entire experience by building a collaborative data tool that's easy enough for everyone to actually use. It's a good fit for technical leaders that are running non-tech teams or teams that simply don't have the time to spend on connecting all the data sources: Intercom, Github, JIRA, etc.
curun1r 10y ago
It's hard to include that information in a meaningful way on a dashboard. It's much easier to alert on that when a metric deviates by a certain percentage. We used a service called StatHat that did this automatically, but it could easily be written on top of an in-house data source like Influx or Prometheus. Dashboards are about visualizing how your application works over time, not whether your application is failing. You should know as soon as possible when something has gone wrong and not be waiting until you come in to work in the morning to figure it out. For us, a key stat trending up or down by more than 10% was sent to PagerDuty for someone to investigate. This didn't happen often, but when it did, it was frequently due to something that we were glad we investigated promptly (e.g. abuse/fraud, misbehaving release, etc). It was really rare that we just got that much more or less usage organically.
- tw04 10y ago
  >It's hard to include that information in a meaningful way on a dashboard. How is it hard to show a comparison of current state vs. average state? That's literally about as simple as it gets, and something I would expect as basic functionality out of any dashboard product I was using.
  - dagw 10y ago
    But I don't care just about current vs. average. I care about current vs. 10 minutes ago vs. this morning vs. this time yesterday vs. same day last week vs. average. Now this can be shown in a dashboard with sufficient cleverness, but I find that most/all dashboards lack this.
nobrains 10y ago
The best dashboards are those that have actionable indicators. Graphs or indicators that, if beyond a certain range, will make you act to fix something.
sh87 10y ago
I've never understood the lure of dashboards. Ok First, there are those, who care enough about the data, and for them dashboards are just not enough. Not enough data, not exactly what you are looking for and there's the element of ambiguity behind the large red bar on the chart. Second, there are those who don't care enough and are honest enough to admit it, they would rather talk to someone who understood the data to get a sense of what's going on every once in a while. Third, are those in the gray area in between these 2. They don't / can't care enough about the data but they also won't/can't admit/accept it. Dashboards, to them give an illusion of control and being in the know. And from my experience there are more of this type than the other two. So clearly Dashboards are a 'thing'! I monitor my stuff based on scripts that I own and write myself Excel, Python and shell scripts. What I need is a reliable notification mechanism that's easy peasy to setup and move around. I could imagine a dashboard of this historical data that I have hand-picked but I don't think I will ever get there. So, yeah I'll build this dashboard and keep people happy while I keep tabs based on what I can trust.
- swalsh 10y ago
  Dashboards aren't things you pull up once in a while. They work best when they are things that you can just glance at, and instantly know some important metric you care about. In our office we have giant screens all around (because screens are dirt cheap now) each team puts the metrics they care about on them. It's really not unusual for us to look at it, and be like "whoa, that metric is kind of spiking, let's take a look". If you're in phase 1 of development, or even the first half of phase 2... you don't need one. If you're in phase 3 (maintence) you need one.
jaxn 10y ago
This is spot on. A KPI is not a number, but rather a number, a direction, and a target. Also, gauges are almost always a terrible visual indicator. Even car dashboards are starting to understand this. Fuel gauges are good, speedometers are not.
- ams6110 10y ago
  Why are speedometers not good? Speed is analog, seems appropriate to display it in analog. And you can tell at a glance if speed is increasing or decreasing by the direction the dial/indicator is moving.
  - dagw 10y ago
    For what it's worth my current car has both an analog and digital speedometer, and I never look at the analog speedometer anymore. The digital is just so much quicker and easier to read.
    - epynonymous 10y ago
      same here, i'm venturing a guess that you drive a mini?
    - untangle 10y ago
      Look at the HUDs of modern fighters. (I'm familiar with the F/A-18). All digital (except angle of attack, if memory serves). Much easier to parse.
  - jaxn 10y ago
    Sometimes, going 50mph is slow, sometimes it will get you arrested. The position on the dial is completely irrelevant, and is thus noise. The gas gauge however has a permanent meaning. Empty is bad. The Tachometer is also useful if you are driving a manual since you want to keep from redlining and keep from stalling. On almost every business dashboard I have seen, gauges are more like speedometers. People love to put revenue in a gauge. More is better and they usually have a goal that they want to hit. But what happens if they max out the gauge? Do they stop sales? Or do they adjust the gauge to go higher (thus making it even more meaningless and difficult to read).

by weego 10y ago

AznHisoka 10y ago

you know.. I have been an user of some of these paid dashboards and you hit on a very good point! historical context is something I always felt was missing. i end up having to store the history myself and comparing it but it would make sense for these dashboards to be "smart" and actually do a little more than just display the current data.

jaxn 10y ago
You can do this with Klipfolio. That is the one I am currently using.

Gys 10y ago

I thought it was just me. I even emailed a commercial supplier and they replied (more or less) that its not a high priority.

dotBen 10y ago

What you are looking for is a BI (business intelligence) tool. Those typically include dashboard but also look at trends - both automatically and programmatically via your own scripting. Looker is probably my favorite currently but it depends on your needs really.

weego 10y ago
I know what you're saying, I'm often bad at meandering past the point I'm thinking of with a non-focused example. I guess in a nutshell I feel like these dashboard products, while well meaning and often very well crafted are just throwing things at your eye sockets. Showing data because you have data is not a solution in of itself.
- dotBen 10y ago
  I dunno, there's probably room for all of them. There's lots of uses for dashboards that are not BI-driven - Castor even suggested their service could be used for conference room information etc. I think the problem is dashboards on their own are pretty low value - ie $40/y for Castor. But that doesn't create a very interesting or particularly sustainable business.

michaelbuckbee 10y ago

When I've built dashboards I usually try and show that by having barcharts of averages for different time periods. Example: MRR+ Today, MRR+/Day over last 7 days, MRR+/Day over last 30 days, etc. It's still not perfect, but it adds at least an element of that "at a glance" context.

jaxn 10y ago
I like area + line charts for this. The area fill is the comp period and the line is the current. One of my favorite ways to do this (that most charting packages struggle with) is to use an area full for last year, a solid line for this year, and a dotted line for the future based on current trajectory. You can compare to multiple periods using stacked areas (particularly when you can assume previous periods had growth).
deleted 10y ago
[deleted]

j_s 10y ago

I would like to know "am I above the ___th percentile for any timeout metric" right now vs. last week / month / etc. I am just getting started with Graphite and wondering how to do this across all of the timeout metrics at the same time on the same graph. (It would probably be an all-or-nothing scenario but there is a chance only a portion of the system is affected by something...)

dsshanley 10y ago

I'm the founder of Notion (usenotion.com) and one of the reasons we built Notion is we experienced the gaps with just a TV-style dashboard. Historical trends and context were definitely missing. We even deployed Tableau (so $$$) across my last company to find no one really used it. We're trying to tackle this entire experience by building a collaborative data tool that's easy enough for everyone to actually use. It's a good fit for technical leaders that are running non-tech teams or teams that simply don't have the time to spend on connecting all the data sources: Intercom, Github, JIRA, etc.

curun1r 10y ago

It's hard to include that information in a meaningful way on a dashboard. It's much easier to alert on that when a metric deviates by a certain percentage. We used a service called StatHat that did this automatically, but it could easily be written on top of an in-house data source like Influx or Prometheus. Dashboards are about visualizing how your application works over time, not whether your application is failing. You should know as soon as possible when something has gone wrong and not be waiting until you come in to work in the morning to figure it out. For us, a key stat trending up or down by more than 10% was sent to PagerDuty for someone to investigate. This didn't happen often, but when it did, it was frequently due to something that we were glad we investigated promptly (e.g. abuse/fraud, misbehaving release, etc). It was really rare that we just got that much more or less usage organically.

tw04 10y ago
>It's hard to include that information in a meaningful way on a dashboard. How is it hard to show a comparison of current state vs. average state? That's literally about as simple as it gets, and something I would expect as basic functionality out of any dashboard product I was using.
- dagw 10y ago
  But I don't care just about current vs. average. I care about current vs. 10 minutes ago vs. this morning vs. this time yesterday vs. same day last week vs. average. Now this can be shown in a dashboard with sufficient cleverness, but I find that most/all dashboards lack this.

nobrains 10y ago

The best dashboards are those that have actionable indicators. Graphs or indicators that, if beyond a certain range, will make you act to fix something.

sh87 10y ago

I've never understood the lure of dashboards. Ok First, there are those, who care enough about the data, and for them dashboards are just not enough. Not enough data, not exactly what you are looking for and there's the element of ambiguity behind the large red bar on the chart. Second, there are those who don't care enough and are honest enough to admit it, they would rather talk to someone who understood the data to get a sense of what's going on every once in a while. Third, are those in the gray area in between these 2. They don't / can't care enough about the data but they also won't/can't admit/accept it. Dashboards, to them give an illusion of control and being in the know. And from my experience there are more of this type than the other two. So clearly Dashboards are a 'thing'! I monitor my stuff based on scripts that I own and write myself Excel, Python and shell scripts. What I need is a reliable notification mechanism that's easy peasy to setup and move around. I could imagine a dashboard of this historical data that I have hand-picked but I don't think I will ever get there. So, yeah I'll build this dashboard and keep people happy while I keep tabs based on what I can trust.

swalsh 10y ago
Dashboards aren't things you pull up once in a while. They work best when they are things that you can just glance at, and instantly know some important metric you care about. In our office we have giant screens all around (because screens are dirt cheap now) each team puts the metrics they care about on them. It's really not unusual for us to look at it, and be like "whoa, that metric is kind of spiking, let's take a look". If you're in phase 1 of development, or even the first half of phase 2... you don't need one. If you're in phase 3 (maintence) you need one.

jaxn 10y ago

This is spot on. A KPI is not a number, but rather a number, a direction, and a target. Also, gauges are almost always a terrible visual indicator. Even car dashboards are starting to understand this. Fuel gauges are good, speedometers are not.

ams6110 10y ago
Why are speedometers not good? Speed is analog, seems appropriate to display it in analog. And you can tell at a glance if speed is increasing or decreasing by the direction the dial/indicator is moving.
- dagw 10y ago
  For what it's worth my current car has both an analog and digital speedometer, and I never look at the analog speedometer anymore. The digital is just so much quicker and easier to read.
  - epynonymous 10y ago
    same here, i'm venturing a guess that you drive a mini?
  - untangle 10y ago
    Look at the HUDs of modern fighters. (I'm familiar with the F/A-18). All digital (except angle of attack, if memory serves). Much easier to parse.
- jaxn 10y ago
  Sometimes, going 50mph is slow, sometimes it will get you arrested. The position on the dial is completely irrelevant, and is thus noise. The gas gauge however has a permanent meaning. Empty is bad. The Tachometer is also useful if you are driving a manual since you want to keep from redlining and keep from stalling. On almost every business dashboard I have seen, gauges are more like speedometers. People love to put revenue in a gauge. More is better and they usually have a goal that they want to hit. But what happens if they max out the gauge? Do they stop sales? Or do they adjust the gauge to go higher (thus making it even more meaningless and difficult to read).