4 ms·

> Second, I think these systems ignore player incentives. I think that global competitive ranking encourages the worst of playerbases, that skill is an arbitrar

> Second, I think these systems ignore player incentives. I think that global competitive ranking encourages the worst of playerbases, that skill is an arbitrary mechanism to optimize on instead of something like how close each match was, or player-reported satisfaction levels, or variety of play-styles, or any other of a dozen other metrics. I think you're totally missing the point. Computing a global competitive rating isn't intended to optimize for skill. Rating is a proxy for skill, sure, but generally it's being used in matchmaking because (a) rating gap is a not-completely-terrible proxy for closeness of matches (because skill gap is a decent proxy for closeness of matches), which contributes to player satisfaction, and (b) you can derive it from easy-to-measure data and while there are fiddly details, everyone is roughly on the same page as to what you're trying to approximate. It happens that, once you compute that, it's really tempting to sort and report. Perhaps that's a mistake and invites the worst of people's instincts to make numbers go more and bigger. This still has the constraint that all the players are playing the same game, but as long as you can somehow avoid ascribing value to rank placement then many forms of cheating just stop mattering; the cheaters self-sort themselves into a tier where they don't interact with anyone who can't deal with them.

danShumway 7y ago
I think my argument there would be that despite common assumptions, rating is actually a very poor proxy for skill, and an even worse proxy for a good match-up. Similarly skilled players won't always like to play with each other. Nor do I think that win-percentages are the only metric we have to work with -- most multiplayer games have reams of much better data to sort through, particularly in games with high variability of play-styles, player types, and team compositions and setups (games like League/Hearthstone/Overwatch, etc...) That being said, I would be more amiable to a system that used player skill as a matchmaking tool if it didn't report those numbers to players. However, I'm not sure I would say that sorting and reporting is usually a side effect or minor mistake on top of those systems, given that many of these games have seasons, rewards, an entire meta-game built on top of their rankings. The rankings aren't just serving the purpose of getting players matched together -- they're serving as player motivators. The responses that players have can't be seen as a purely accidental side-effect when designers are going out of their way to present this information to the player and to encourage the player to pay attention to it. Hearthstone is a good example here. Hearthstone has brilliant design, but its ladder system is horrible and dumb. It explicitly encourages net-decking and grinding instead of experimentation. Fast decks are objectively better than slow ones with the same win percentage, because ranking is largely determined by win percentage combined with the number of games you can fit into a season. Season rewards are then tied to rank, so the best way to get new cards if you want to experiment is to build a boring deck and grind. This is despite the fact that for most players, knowing within the first few turns whether or not they're going to win is not fun. So of course people bot Hearthstone, because before you can build creative fun decks and play against interesting people with interesting strategies, you have to pay the "fun tax" of grinding on the ladder. Everything brilliant about Hearthstone is bogged down by being tied to this kind of toxic experience. There's no way to say this about such a popular game without sounding naive, and I'll take the criticism, but I do not believe for one second it would be difficult to build a better public ranking system for Hearthstone, especially given the ludicrous amount of player data they have. Even something as simple as balancing rank with deck variety so players got fewer rewards the more times they played the same cards would go a huge way towards getting rid of net-decking. These systems do increase engagement among very invested players, so maybe that's the only intention. But to heck with engagement -- is laddering at the higher ranks of Hearthstone fun? People do it, people say they enjoy it. Again, I'll take the criticism a statement like this warrants, but I don't believe them. I don't believe the experience of grinding a single 55% win-rate deck for hours and hours is an experience that is worth having or producing. And when I think about why players do that, the only cause that comes to my mind is, "the game tells them to."
- celticmusic 7y ago
  > I think my argument there would be that despite common assumptions, rating is actually a very poor proxy for skill you can't know that without knowing the specifics of how that rating is generated. I promise you ELO for chess is a hell of a lot more accurate than ELO for league.
  - danShumway 7y ago
    I do know how the specifics of how these rankings are calculated because they're publicly visible[0]. I'm not arguing that behind the scenes Hearthstone's matchmaking isn't sophisticated enough (although it probably isn't), I'm arguing that publicly, the public mechanisms for determining rank are really bad and encourage bad play habits. Hearthstone ranking is not a good proxy for skill, it's a good proxy for the amount of time you have to play and the speed of your deck. Objectively, the best strategy to get good ranks at Hearthstone is to build a fast, >50% win-rate deck and then to grind. That's not a good proxy for skill. [0]: https://hearthstone.gamepedia.com/Ranked https://hearthstone.gamepedia.com/Ranked
    - celticmusic 7y ago
      I'd love to see the details on League of Legends ranking algorithm.
      - danShumway 7y ago
        Sure! https://dotesports.com/league-of-legends/news/league-of-legends-ranking-system-explained-17171 https://dotesports.com/league-of-legends/news/league-of-lege... > As players win games in ranked, they gain League Points, also more commonly known as LP. They also lose LP when they are defeated. Upon reaching 100 total LP in a given division, players enter a promotion series to try and get to the next available division. > Players must win a majority of their games in order to advance to the next division. Promotion series within certain ranks, like Bronze III to II, are a best-of-three series. From one rank to the next, such as Silver I to Gold V, promotion series are best-of-five instead. > If a player doesn’t win that amount of games, then they’ll need to get back to 100 LP to try again. ---- Yes, the exact details of how your MMR is calculated is kept secret, but it doesn't matter if you still know exactly what you need to optimize for: winning sequential games, often in chunks, and going out of your way to avoid losing streaks which can result in demotions. Arguing that hiding the exact MMR formula means that players won't alter their behaviors to fit its approximate contours is like arguing that because Google's page-rank algorithm isn't public that SEO is not a thing. (Some of) the consequences of this setup include: A) losing matches becomes much more costly, which exasperates the problems League has with feeders and trolls, and lends itself to a more toxic environment. B) Experimentation and learning new heroes is best done off of the ladder until you feel like you're competitive with that hero. Experimenting on the ladder is not only bad for your rank, it's also likely to get you reported by other teammates as a feeder (see the toxicity mentioned above). Players are effectively optimizers. If you give them a system with a clear success metric (ie, a public rank going up), they will optimize their play-style to fit that system. Designers often assume players will stop doing this at some point. They won't grind in an RPG because that's boring. They won't bot a competitive game because that's not fun for other players. Most players don't think in those terms during gameplay (myself included). Thinking that way is hard. As a designer, if you want a player to optimize for creating a fun experience for themselves and others, you have to explicitly build your publicly-facing systems to encourage that behavior.

danShumway 7y ago

I think my argument there would be that despite common assumptions, rating is actually a very poor proxy for skill, and an even worse proxy for a good match-up. Similarly skilled players won't always like to play with each other. Nor do I think that win-percentages are the only metric we have to work with -- most multiplayer games have reams of much better data to sort through, particularly in games with high variability of play-styles, player types, and team compositions and setups (games like League/Hearthstone/Overwatch, etc...) That being said, I would be more amiable to a system that used player skill as a matchmaking tool if it didn't report those numbers to players. However, I'm not sure I would say that sorting and reporting is usually a side effect or minor mistake on top of those systems, given that many of these games have seasons, rewards, an entire meta-game built on top of their rankings. The rankings aren't just serving the purpose of getting players matched together -- they're serving as player motivators. The responses that players have can't be seen as a purely accidental side-effect when designers are going out of their way to present this information to the player and to encourage the player to pay attention to it. Hearthstone is a good example here. Hearthstone has brilliant design, but its ladder system is horrible and dumb. It explicitly encourages net-decking and grinding instead of experimentation. Fast decks are objectively better than slow ones with the same win percentage, because ranking is largely determined by win percentage combined with the number of games you can fit into a season. Season rewards are then tied to rank, so the best way to get new cards if you want to experiment is to build a boring deck and grind. This is despite the fact that for most players, knowing within the first few turns whether or not they're going to win is not fun. So of course people bot Hearthstone, because before you can build creative fun decks and play against interesting people with interesting strategies, you have to pay the "fun tax" of grinding on the ladder. Everything brilliant about Hearthstone is bogged down by being tied to this kind of toxic experience. There's no way to say this about such a popular game without sounding naive, and I'll take the criticism, but I do not believe for one second it would be difficult to build a better public ranking system for Hearthstone, especially given the ludicrous amount of player data they have. Even something as simple as balancing rank with deck variety so players got fewer rewards the more times they played the same cards would go a huge way towards getting rid of net-decking. These systems do increase engagement among very invested players, so maybe that's the only intention. But to heck with engagement -- is laddering at the higher ranks of Hearthstone fun? People do it, people say they enjoy it. Again, I'll take the criticism a statement like this warrants, but I don't believe them. I don't believe the experience of grinding a single 55% win-rate deck for hours and hours is an experience that is worth having or producing. And when I think about why players do that, the only cause that comes to my mind is, "the game tells them to."

celticmusic 7y ago
> I think my argument there would be that despite common assumptions, rating is actually a very poor proxy for skill you can't know that without knowing the specifics of how that rating is generated. I promise you ELO for chess is a hell of a lot more accurate than ELO for league.
- danShumway 7y ago
  I do know how the specifics of how these rankings are calculated because they're publicly visible[0]. I'm not arguing that behind the scenes Hearthstone's matchmaking isn't sophisticated enough (although it probably isn't), I'm arguing that publicly, the public mechanisms for determining rank are really bad and encourage bad play habits. Hearthstone ranking is not a good proxy for skill, it's a good proxy for the amount of time you have to play and the speed of your deck. Objectively, the best strategy to get good ranks at Hearthstone is to build a fast, >50% win-rate deck and then to grind. That's not a good proxy for skill. [0]: https://hearthstone.gamepedia.com/Ranked https://hearthstone.gamepedia.com/Ranked
  - celticmusic 7y ago
    I'd love to see the details on League of Legends ranking algorithm.
    - danShumway 7y ago
      Sure! https://dotesports.com/league-of-legends/news/league-of-legends-ranking-system-explained-17171 https://dotesports.com/league-of-legends/news/league-of-lege... > As players win games in ranked, they gain League Points, also more commonly known as LP. They also lose LP when they are defeated. Upon reaching 100 total LP in a given division, players enter a promotion series to try and get to the next available division. > Players must win a majority of their games in order to advance to the next division. Promotion series within certain ranks, like Bronze III to II, are a best-of-three series. From one rank to the next, such as Silver I to Gold V, promotion series are best-of-five instead. > If a player doesn’t win that amount of games, then they’ll need to get back to 100 LP to try again. ---- Yes, the exact details of how your MMR is calculated is kept secret, but it doesn't matter if you still know exactly what you need to optimize for: winning sequential games, often in chunks, and going out of your way to avoid losing streaks which can result in demotions. Arguing that hiding the exact MMR formula means that players won't alter their behaviors to fit its approximate contours is like arguing that because Google's page-rank algorithm isn't public that SEO is not a thing. (Some of) the consequences of this setup include: A) losing matches becomes much more costly, which exasperates the problems League has with feeders and trolls, and lends itself to a more toxic environment. B) Experimentation and learning new heroes is best done off of the ladder until you feel like you're competitive with that hero. Experimenting on the ladder is not only bad for your rank, it's also likely to get you reported by other teammates as a feeder (see the toxicity mentioned above). Players are effectively optimizers. If you give them a system with a clear success metric (ie, a public rank going up), they will optimize their play-style to fit that system. Designers often assume players will stop doing this at some point. They won't grind in an RPG because that's boring. They won't bot a competitive game because that's not fun for other players. Most players don't think in those terms during gameplay (myself included). Thinking that way is hard. As a designer, if you want a player to optimize for creating a fun experience for themselves and others, you have to explicitly build your publicly-facing systems to encourage that behavior.