3 ms·
Out of curiosity, are you watching percentiles or averages in your metrics?
by kevan 9y ago
Out of curiosity, are you watching percentiles or averages in your metrics?
- pspeter3 9y agoWe used to track percentiles but discovered Asanas struggled to empathize with them. We switched to defining semantic performance bands (eg < 100ms, < 300ms, < 1s, < 3s, < 5s, < 10s). For each action, we track the percentage per band. Asanas seemed to empathize better with time as the independent variable and percentage as the dependent variable.
- f1notformula1 9y agoCan you elaborate on what an Asana is and what "empathize" means? I'm not an Asana user, so perhaps I'm unfamiliar with the jargon.
- btilly 9y agoI would also suggest that you do the same with a metric for "slowest action in 15 minute period of user input". 99% of actions are fast but you have hundreds of actions over 15 minutes, people will notice those slow ones.
- pspeter3 9y agoThanks for the advice! We believe that users remember the slowest actions as well. Your strategy sounds like it will help identify the long tail of rare yet slow actions.
- btilly 9y agoHere is another tip to track down that long tail. "Tracer bullets". Have a flag that can be put on an action that causes it to be logged very verbosely. In a way that travels through your system. And then aggregate those logs, process them, and make them available. Then randomly tag a small fraction of actions with that flag. Like 0.1%. Because you do this relatively rarely, you can make verbose logs without overwhelming your system. When you receive a complaint about foo occasionally being slow your first stop is to see if you've got a concrete action that was slow, look at the picture from the logs, and try to figure out why. This will greatly help in reproducing rare problems or, if you can't, at least identifying where they come from deep in your system.