4 ms·
It might be, assuming two things: 1) That you actually care about this metric. I don't, I do not get paid by the number of minutes spent on pages, I get paid b
by jwr 6y ago
It might be, assuming two things:
1) That you actually care about this metric. I don't, I do not get paid by the number of minutes spent on pages, I get paid by the number of signed-up subscribers who use my software to make their workflow easier. I can (and prefer to) use bandit testing to measure the performance of redesigns.
2) that it can be reliably measured, which I don't think it can.
- TomGullen 6y agoWe are a SaaS company also, and time spent reading manual/tutorial pages is important to us. With regards to point two, being reliably measured sounds to me like perfection is the enemy of adequate. Perhaps in low volumes you can't measure certain stats like this reliabily but in large volumes I think it's useful.
- Arnt 6y agoTom Gilb once said that "everything can be measured so accurately that the result is better than having no measurement at all" or words to that effect. I'm sure he phrased it better. In this case, some users aren't measured, which drags down the accuracy of the measurement. Even if the numbers are off by quite a few per cent, I can easily see how some site operators might benefit from knowing e.g. that visitors close one tutorial much quicker than the others.
- srrr 6y agoAs long as your measurement is a random sample everything is okay. Even if it is not, it is much more information than you had before. You just need to keep it in mind when evaluating conclusions. We are not talking about drug tails here and no one dies if the measurement is not 100% accurate. I am able to measure everything 100% accurate. But this is really really expensive. It's a trade-of.