2 ms·

And this is why the field of design gets shit on so much. Jakob Nielsen might have a PhD, but he’s much better at PR than he is at conducting actual research. I

And this is why the field of design gets shit on so much. Jakob Nielsen might have a PhD, but he’s much better at PR than he is at conducting actual research. It’s mind boggling to me that people will still quote his opinion pieces 20 years later as if there was any actual science behind them. There’s a few other fields that happen to conduct a lot of studies around human behavior. It turns out, there’s actually an established process for calculating the confidence level of your findings: https://www.calculator.net/sample-size-calculator.html https://www.calculator.net/sample-size-calculator.html Is 3 or 5 better than nothing? Sure! But don’t trick yourself into thinking your findings actually mean anything because a Danish dude drew a graph 20 years ago.

Vinnl 7y ago
So again, a large margin of error and a low confidence level are fine, because it doesn't matter if in reality, 80% of your users run into something rather than the 60% that came out of user testing. In fact, we don't care about the specific numbers at all. All we want to know is what things are unclear to many users. Often, you test a design on five users, and all five of them miss the same thing - that gives you plenty of confidence that you'll want to try something else there. User testing is not a statistical test, and it's not a way to learn about human behaviour (so indeed, in that sense it doesn't "mean" anything); it's to give insight into your designs. I'd highly recommend everyone to take part in conducting user research sometime, as it will quickly become obvious just how easy it is to miss "obvious" things in something you're closely involved with, and how you need to have just few people interact with it to find out which.