3 ms·
But it's important not to forget that the results of these tests were (intentionally) limited to certain variables. I recently made the argument that you shoul
by yummyfajitas 14y ago
But it's important not to forget that the results of these tests were (intentionally) limited to certain variables.
I recently made the argument that you should cook up a single metric which captures all variables of interest, and use it everywhere. Put all variables of interest into them, and carefully define what tradeoffs you are willing to make.
http://www.chrisstucchio.com/blog/2013/metrics_manifesto.html http://www.chrisstucchio.com/blog/2013/metrics_manifesto.htm...
As for the tired analogy, founders intuition gives you the idea of cars. A/B testing can still demonstrate that cars are superior.
- darkxanthos 14y agoSince you're proposing a calculated metric, it's advisable to just log all three and do the calculation afterwards. That way if the business decides that number is silly at some point it can be changed and your results recalculated. Also trying to come up with a simple formula that relates different units of business value can be VERY difficult. Any literature on perspectives around that?
- yummyfajitas 14y agoAlso trying to come up with a simple formula that relates different units of business value can be VERY difficult. Deciding what you actually want and what tradeoffs you are willing to make can be hard. Then again, if you don't know what you want you are unlikely to achieve it.