8 ms·
A real user might be worse. A program is less flexible (maybe) and more consistent (definitely) than a meat space CBL. The goal is not realism but a kind of re
by CaffeineLD50 2y ago
A real user might be worse. A program is less flexible (maybe) and more consistent (definitely) than a meat space CBL.
The goal is not realism but a kind of ready made "you must be this tall to ride the rollercoaster" threshold.
Discovering edge cases with dodgy human users has its value, but that's a different value.
- gffrd 2y agoA real user will be worse … but that’s kinda the point. The most valuable thing you learn in usability/research is not if your experience works, but the way it’ll be misinterpreted, abused, and bent to do things it wasn’t designed to.
- cpeterso 2y agoEnter "Drunk User Testing". Host a happy hour event and give some buzzed users some scenarios to test. https://www.newyorker.com/magazine/2018/04/30/an-open-bar-for-beta-testers https://www.newyorker.com/magazine/2018/04/30/an-open-bar-fo... https://uxpamagazine.org/boozeability/ https://uxpamagazine.org/boozeability/
- Tepix 2y agoMore consistent? That's not a given with LLMs unless you set the temperature to 0.
- CaffeineLD50 2y agoYou are right. LLMs are totally random and useless. Thanks for playing.
- Tepix 2y agoYou seem to disagree. Here's an interesting study where the researchers used an OpenAI-LLM-based tool to grade student papers and by grading them 10 times in a row, they got vastly different results: https://rainermuehlhoff.de/en/fobizz-AI-grading-assistant-test-studie/ https://rainermuehlhoff.de/en/fobizz-AI-grading-assistant-te... Quote: "The results reveal significant shortcomings: The tool’s numerical grades and qualitative feedback are often random and do not improve even when its suggestions are incorporated."