4 ms·

Asking the crowd: does anyone know of an in-depth comparison of rating systems (thumbs, 3 stars, 5 stars, 1-10, etc)? Also obligatory Black Mirror reference: N

by mLuby 8y ago

Asking the crowd: does anyone know of an in-depth comparison of rating systems (thumbs, 3 stars, 5 stars, 1-10, etc)? Also obligatory Black Mirror reference: Nosedive S3E1

alleyshack 8y ago
I don't know where the specific article wandered off to, but about six months ago I read a thinkpiece talking about those "happy face/frowny face" buttons that have popped up. The article talked a lot about how the binary up/down, with no ability to comment on specific people/processes/whatever, actually improved both overall ratings and the businesses' ability to address negative ratings considerably. If I find it again I'll update this comment. Edit: This article has less detail than I remember, but covers the gist (and makes the same Black Mirror reference): https://www.newyorker.com/magazine/2018/02/05/customer-satisfaction-at-the-push-of-a-button https://www.newyorker.com/magazine/2018/02/05/customer-satis...
- naravara 8y ago
  >The article talked a lot about how the binary up/down, with no ability to comment on specific people/processes/whatever, actually improved both overall ratings and the businesses' ability to address negative ratings considerably I guess that makes sense. Individual end-users only ever see the tip of the ice-berg when interacting with a large organization, so they are likely to blame the parts of it that they interact with while ignoring the deeper root-causes. If you let them blame the customer service rep's attitude or the size that their vegetables are chopped then you wind up trying to over-optimize around making the reps smile or definining exactly how to chop the veggies. But if all you know is that people aren't happy with the experience you send a process minded person who is familiar with the whole stack of how things get done to figure out why the service reps are in a foul mood or how come irregularly chopped veggies are winding up on peoples' plates.
combatentropy 8y ago
I don't know, but I prefer binary. I agree with the author about numeric ratings. If service meets my expectations, then I want to put a 3 or 4, which to me means average or above average. A 5 means not only were all expectations met but they surprised me by going way beyond my expectations. In fact this is how Yelp describes it: 1 - Eek! Methinks not. 2 - Meh. I've experienced better. 3 - A-OK. 4 - Yay! I'm a fan. 5 - Woohoo! As good as it gets! But the subjects of review seem to interpret the ratings like this: 1 - A black hole. 2 - Don't bother. 3 - If you're desperate. 4 - Could be better. 5 - Met all expectations. To compound the problem, many people actually rate like this, causing what the article calls "reputation inflation." The result is that I end up having to use this higher scale when looking for service, because enough people have been rating that way. This is why I would prefer a binary up or down vote, with the option to comment. My 1-2 would would translate to No, and 3-5 to Yes.
- agglomerative 8y ago
  There's a name for this. I forget what it is, but I remember having the principle explained to me, and thinking to myself: "Holy fucking shit, this is some seriously out-of-touch economic high-finance quantit=ative bullshit." Like, the kind that always expects all graphs to go "up and to the right." Basically, you sit down at a restaurant, order a meal, and on the way out, the host or hostess ask you to mark off checkboxes rating the experience on a scall of 0 to 11. Eleven being the Spinal Tap interpretation, and Zero being the hospital visit. We now face three realities at this point, completely disconnected and untethered from one another, doomed to resolve as if it were some Cellular Automaton rule set from hell. On the customer's side of things, maybe they let their small child mark the check boxes to satisfy the child's curiosity, maybe it's the waitress' mom dropping by for a visit to take pictures at their first summer job. Maybe it's an actual normal customer, a random person with with a pulse, fresh off the street. Anything goes. Straight A's or all balls. The wait staff, have no control and don't actually know how they're being judged. By what criteria, or rules. They just know what it means to wait tables. They've been in a restaurant before, and they know what middle of the road is. Unless specifically advised of the level of service, they observe their surroundings, and follow their instinct, drawing on prior experience and whatever they learn as they go. The analyst that receives the feedback however, operates according to rules both alien and strange to normal people. Only 9, 10 and 11 are positive ratings. All others are an insult to the business, and all parties associated with an 8 rating or less must be eliminated from the system. They don't want to see paying customers that aren't ecstatic, nor do they want employees delivering service that catches a middling rating. To them, a 9 represents a danger zone, threatening profits with a backslide. The person who explained this principle to me was my boss' boss, and it was in this moment that I knew the ship I was on was aimed at an iceberg, and that I needed to escape. He was referring to our OKR process, while simultaneously explaining this principle in relation to our review process. It was a very "Steve Ballmer/Stack Ranking/Cut the Weakest Link" sort of discussion, and I stuck my thumb out and found another job, for better pay, and less demand within weeks.
  - Rotten194 8y ago
    was it the net promoter score? https://en.m.wikipedia.org/wiki/Net_Promoter https://en.m.wikipedia.org/wiki/Net_Promoter

by mLuby 8y ago

Asking the crowd: does anyone know of an in-depth comparison of rating systems (thumbs, 3 stars, 5 stars, 1-10, etc)? Also obligatory Black Mirror reference: Nosedive S3E1

alleyshack 8y ago

I don't know where the specific article wandered off to, but about six months ago I read a thinkpiece talking about those "happy face/frowny face" buttons that have popped up. The article talked a lot about how the binary up/down, with no ability to comment on specific people/processes/whatever, actually improved both overall ratings and the businesses' ability to address negative ratings considerably. If I find it again I'll update this comment. Edit: This article has less detail than I remember, but covers the gist (and makes the same Black Mirror reference): https://www.newyorker.com/magazine/2018/02/05/customer-satisfaction-at-the-push-of-a-button https://www.newyorker.com/magazine/2018/02/05/customer-satis...

naravara 8y ago
>The article talked a lot about how the binary up/down, with no ability to comment on specific people/processes/whatever, actually improved both overall ratings and the businesses' ability to address negative ratings considerably I guess that makes sense. Individual end-users only ever see the tip of the ice-berg when interacting with a large organization, so they are likely to blame the parts of it that they interact with while ignoring the deeper root-causes. If you let them blame the customer service rep's attitude or the size that their vegetables are chopped then you wind up trying to over-optimize around making the reps smile or definining exactly how to chop the veggies. But if all you know is that people aren't happy with the experience you send a process minded person who is familiar with the whole stack of how things get done to figure out why the service reps are in a foul mood or how come irregularly chopped veggies are winding up on peoples' plates.

combatentropy 8y ago

I don't know, but I prefer binary. I agree with the author about numeric ratings. If service meets my expectations, then I want to put a 3 or 4, which to me means average or above average. A 5 means not only were all expectations met but they surprised me by going way beyond my expectations. In fact this is how Yelp describes it: 1 - Eek! Methinks not. 2 - Meh. I've experienced better. 3 - A-OK. 4 - Yay! I'm a fan. 5 - Woohoo! As good as it gets! But the subjects of review seem to interpret the ratings like this: 1 - A black hole. 2 - Don't bother. 3 - If you're desperate. 4 - Could be better. 5 - Met all expectations. To compound the problem, many people actually rate like this, causing what the article calls "reputation inflation." The result is that I end up having to use this higher scale when looking for service, because enough people have been rating that way. This is why I would prefer a binary up or down vote, with the option to comment. My 1-2 would would translate to No, and 3-5 to Yes.

agglomerative 8y ago
There's a name for this. I forget what it is, but I remember having the principle explained to me, and thinking to myself: "Holy fucking shit, this is some seriously out-of-touch economic high-finance quantit=ative bullshit." Like, the kind that always expects all graphs to go "up and to the right." Basically, you sit down at a restaurant, order a meal, and on the way out, the host or hostess ask you to mark off checkboxes rating the experience on a scall of 0 to 11. Eleven being the Spinal Tap interpretation, and Zero being the hospital visit. We now face three realities at this point, completely disconnected and untethered from one another, doomed to resolve as if it were some Cellular Automaton rule set from hell. On the customer's side of things, maybe they let their small child mark the check boxes to satisfy the child's curiosity, maybe it's the waitress' mom dropping by for a visit to take pictures at their first summer job. Maybe it's an actual normal customer, a random person with with a pulse, fresh off the street. Anything goes. Straight A's or all balls. The wait staff, have no control and don't actually know how they're being judged. By what criteria, or rules. They just know what it means to wait tables. They've been in a restaurant before, and they know what middle of the road is. Unless specifically advised of the level of service, they observe their surroundings, and follow their instinct, drawing on prior experience and whatever they learn as they go. The analyst that receives the feedback however, operates according to rules both alien and strange to normal people. Only 9, 10 and 11 are positive ratings. All others are an insult to the business, and all parties associated with an 8 rating or less must be eliminated from the system. They don't want to see paying customers that aren't ecstatic, nor do they want employees delivering service that catches a middling rating. To them, a 9 represents a danger zone, threatening profits with a backslide. The person who explained this principle to me was my boss' boss, and it was in this moment that I knew the ship I was on was aimed at an iceberg, and that I needed to escape. He was referring to our OKR process, while simultaneously explaining this principle in relation to our review process. It was a very "Steve Ballmer/Stack Ranking/Cut the Weakest Link" sort of discussion, and I stuck my thumb out and found another job, for better pay, and less demand within weeks.
- Rotten194 8y ago
  was it the net promoter score? https://en.m.wikipedia.org/wiki/Net_Promoter https://en.m.wikipedia.org/wiki/Net_Promoter