6 ms·

I have no idea what "4.9 level" means. When I used to use uber (before it stopped working reliably in the UK about 5 years ago) they would ask me "how many star

by ta1243 3y ago

I have no idea what "4.9 level" means. When I used to use uber (before it stopped working reliably in the UK about 5 years ago) they would ask me "how many stars" on a range of 1-5. There was no indication of what this meant or how to judge it. Presumably the idea is that the star ratings are relative and follow a normal distribution, so most are in the middle (say about 70% between 2/could be better and 4/really good, with about 15% being 5/excellent and 15% being 1 terrible.

iainmerrick 3y ago
The ratings actually mean: 5 stars = OK, 1-4 stars = bad. Yes, it's a bad system.
- ta1243 3y ago
  Which isn't explained on various systems. Feels to me that if you want that rating system applied consistently you need to only ask one question -- "Was the service acceptable Yes/No". If more than 50% of your replies are "this service was above average", then you have a meaningless system.
  - iainmerrick 3y ago
    It's a bad system, but pretty much every other online rating system is uncalibrated too (they just have different social expectations than the "always give 5 stars" that most people use for taxis and deliveries). There are plenty of things that could be done, like using the algorithm that was developed for Netflix's prize contest, but nobody seems to be doing them.
  - nurple 3y ago
    A lot of the same topics in this thread sound similar to when Netflix changed their star system to a binary good/bad, and why. https://news.ycombinator.com/item?id=13893672 https://news.ycombinator.com/item?id=13893672
  - NoMoreNicksLeft 3y ago
    No system built for a society that (sometimes) values kindness over honesty will ever give you honest answers. My wife still wants to leave big tips when the service is lousy "they're counting on it to be able to make rent". With stars or similar ratings systems, even cheapskates can be kind. Have you ever seen a thread where someone starts talking about "brutal honesty", and how people react to that term? >If more than 50% of your replies are "this service was above average", then you have a meaningless system. Only true mathematically if ratings are for internal comparison only. It could be that the service at restaurant A is consistently above average, because the service at restaurant B is consistently below average. Only their aggregate ratings should show it split half and half.
- zarzavat 3y ago
  Does anyone know if Uber rescales the star ratings? For example if you have someone like GP who consistently gives out 3s or 4s, do those get rescaled back up to 5s based on the user’s average?
lupusreal 3y ago
Most of the general population have no idea what a normal distribution is. You have to put yourself into a common mindset to figure out the meaning of things designed for the general population. It doesn't need to be explained because half the population is too dumb to understand a written explaination anyway. 5 means they did good, anything less means you have complaints that you think are serious enough to warrant effective "demotion" of the driver.
- ta1243 3y ago
  So stupid system with no instructions on how to calibrate. How do you say someone did a better than expected job if the "baseline" is 5? If they expect the normal to be 5, why give 1-4 as options. Why not "good" and "bad" Not to mention the cultural differences between countries. While America loves being over-the-top with fake "oh you are so awesome", other countries (say Japan and the UK) are far more reserved.
  - acdha 3y ago
    > If they expect the normal to be 5, why give 1-4 as options. Why not "good" and "bad" Some places do use a thumbs up/down rating but I think a lot of companies like the stress the 1-5 or 1-10 ratings put on workers. If you view your workers as easily replaceable, stress is a feature and the false precision lets you say that any particular outcome was data driven.
  - zztop44 3y ago
    Japan has the opposite problem on restaurant review websites, where ~3.0 stars means OK and ~3.65 stars or above means absolutely bloody fantastic.
    - themaninthedark 3y ago
      So they use the system as intended?
  - kergonath 3y ago
    I think a 3-states scale would be good. Something like “happy”, “not happy”, and ”feared for your life”. You’d want a way to identify outliers. But using a quantitative scale for this is not a good idea.

by ta1243 3y ago

iainmerrick 3y ago

The ratings actually mean: 5 stars = OK, 1-4 stars = bad. Yes, it's a bad system.

ta1243 3y ago
Which isn't explained on various systems. Feels to me that if you want that rating system applied consistently you need to only ask one question -- "Was the service acceptable Yes/No". If more than 50% of your replies are "this service was above average", then you have a meaningless system.
- iainmerrick 3y ago
  It's a bad system, but pretty much every other online rating system is uncalibrated too (they just have different social expectations than the "always give 5 stars" that most people use for taxis and deliveries). There are plenty of things that could be done, like using the algorithm that was developed for Netflix's prize contest, but nobody seems to be doing them.
- nurple 3y ago
  A lot of the same topics in this thread sound similar to when Netflix changed their star system to a binary good/bad, and why. https://news.ycombinator.com/item?id=13893672 https://news.ycombinator.com/item?id=13893672
- NoMoreNicksLeft 3y ago
  No system built for a society that (sometimes) values kindness over honesty will ever give you honest answers. My wife still wants to leave big tips when the service is lousy "they're counting on it to be able to make rent". With stars or similar ratings systems, even cheapskates can be kind. Have you ever seen a thread where someone starts talking about "brutal honesty", and how people react to that term? >If more than 50% of your replies are "this service was above average", then you have a meaningless system. Only true mathematically if ratings are for internal comparison only. It could be that the service at restaurant A is consistently above average, because the service at restaurant B is consistently below average. Only their aggregate ratings should show it split half and half.
zarzavat 3y ago
Does anyone know if Uber rescales the star ratings? For example if you have someone like GP who consistently gives out 3s or 4s, do those get rescaled back up to 5s based on the user’s average?

lupusreal 3y ago

Most of the general population have no idea what a normal distribution is. You have to put yourself into a common mindset to figure out the meaning of things designed for the general population. It doesn't need to be explained because half the population is too dumb to understand a written explaination anyway. 5 means they did good, anything less means you have complaints that you think are serious enough to warrant effective "demotion" of the driver.

ta1243 3y ago
So stupid system with no instructions on how to calibrate. How do you say someone did a better than expected job if the "baseline" is 5? If they expect the normal to be 5, why give 1-4 as options. Why not "good" and "bad" Not to mention the cultural differences between countries. While America loves being over-the-top with fake "oh you are so awesome", other countries (say Japan and the UK) are far more reserved.
- acdha 3y ago
  > If they expect the normal to be 5, why give 1-4 as options. Why not "good" and "bad" Some places do use a thumbs up/down rating but I think a lot of companies like the stress the 1-5 or 1-10 ratings put on workers. If you view your workers as easily replaceable, stress is a feature and the false precision lets you say that any particular outcome was data driven.
- zztop44 3y ago
  Japan has the opposite problem on restaurant review websites, where ~3.0 stars means OK and ~3.65 stars or above means absolutely bloody fantastic.
  - themaninthedark 3y ago
    So they use the system as intended?
- kergonath 3y ago
  I think a 3-states scale would be good. Something like “happy”, “not happy”, and ”feared for your life”. You’d want a way to identify outliers. But using a quantitative scale for this is not a good idea.