3 ms·
This is what happens when you don't have psychologists or similar quantitative people who handle human data on your team. I've studied psychology and CS and th
by mettamage 7y ago
This is what happens when you don't have psychologists or similar quantitative people who handle human data on your team.
I've studied psychology and CS and the difference in how they treat ratings is stark, CS GoT nothing on us! (yes, I did that, I'll get myself of stage now)
A psychologist is paranoid when it comes to quantifying human data. How paranoid you ask? I'll try to capture the sentiment, but I'm sure it's way worse than the following paragraph.
Psychologist: Should I go for a simple yes/no? Well that could useful, but it could also be that I leave too much data on the table. Let me look at tons of research on this topic (hello Google Scholar! Long time no see). Well, if I do 3 stars, then the middle option also becomes a neutral option, which means I'm skewing the data, so I might have to go to 4 possible choices then. Then again, people are more used to the 5 star rating, so that has a plus for usability, but then you'd have the same issue as a 3 choice rating. Maybe I should just use a 7-likert scale type of thing? And obviously, it's also the case that this is an ordinal scale. The property of 1 < 2 < 3 and so on still holds, but the property of 1 + 1 = 2 doesn't. Do people really mean with "2" twice as big compared to a "1"? Are they able to quantify such relationships on any scale? Perhaps quantitatively minded people on a 3 choice scale, but we'd need to do studies about that, but the average user probably not -- still need to do studies about that (and hello Google Scholar, here we meet again in the same paragraph).
CS or UX person with no research background: ah yea, let's do a 5 star rating, that seems cool :)
It's on you Uber.
It's really on you.
Hire psychologists / socially minded statisticians.
- crooked-v 7y agoI'm a fan of HappyOrNot for dead-simple intuitive feedback: https://www.happy-or-not.com/en/smiley-terminal/ https://www.happy-or-not.com/en/smiley-terminal/
- hnzix 7y agoUntil schoolkids repeatedly mash the frowny face button for lulz.
- walterbell 7y agoWe need version numbers for game-theoretic universes, e.g. the effects of companies which launch into a market whose assumptions and practices will soon be overturned by said company. The second-order feedback loops are adversarial (kids, competitors, city-states, policy) and subject to data analysis, but the public is more likely to receive marketing narratives than peer-reviewed scholarship.
- simongr3dal 7y agoI always press the frowny button in grocery shops, not necesarily because there were long lines or because the shelves weren't stocked adequatly, but if I press the frowny button I assume the place will get even better. I disagree with those buttons because you can't really find the cause of a problem in a giant macihnery with just a single output. So the feedback is probably close to useless.
- hnzix 7y agoThe buttons are generally used to punish the clerks on duty. All you are doing is griefing minimum-wage employees.
- p10_user 7y agoCompanies don’t care about this. They just need some arbitrary scale (5 stars, 10 point rating) and anything less than perfect is an excuse to put someone on notice. Always handy to point to when needing an excuse to trim numbers or not increase benefits. And since nobody is perfect, you can find some arbitrary cutoff that lots of people will be below (“sorry you’re averaging 4.4 stars, we can’t give EVERYONE bonuses now can we”) there’s always room for excuses.
- throwawaymath 7y agoUber's research teams already employ psychologists and statisticians. They also employ various other types of cross-disciplinary researchers.
- mettamage 7y agoIf they had a say on the 5 star rating, then fair enough. In that case, my comment is simply a fun distinction between how two different perspectives look at data from people. If not, it's still on Uber. But like other comments said, there are other considerations that Uber needs to take into account (business, brand, etc.).
- rainonmoon 7y agoWell, your comment is also a pretty disparaging and reductive idea of what UX is actually like. I'm sorry you've met people in UX by title only, because what you described is nothing like the actual field. If someone claiming to be in UX isn't doing research and testing, it's not UX.
- mettamage 7y agoFair point, that’s is what I have seen so far.
- ajross 7y agoYeah, so... trying to keep the snark under control long enough to ask a genuine question. Can you give a quick exposition for why the giant paragraph is actually, y'know, better than "that seems cool"? I mean, I buy that psychologists will spend lots of time thinking about a fundamentally intractable problem in their field, in the same way that the programmers are going to obsess over irrelevant design issues (and they do!). But it's bad when psychologists do it for the same reason it's bad when programmers do it. Just pick something and move on, unless you've got a clear and persuasive argument to the contrary.
- 4rgento 7y agoI really like your comment. As a scientist I'd say that a pragmatic solution (widely used in experimental sciences) is to calibrate your measures. Want to use a yes/no? Fine (maybe 7 stars give better feedback, maybe not). But what should be the business decision based on the answer? How valuable is a yes? How costly a no?