6 ms·

The implementation of SBMM in Call of Duty is particularly bad. Usually, you would expect such a system to match you with players of similar skill. But, that's

by jswny 6y ago

The implementation of SBMM in Call of Duty is particularly bad. Usually, you would expect such a system to match you with players of similar skill. But, that's NOT how it works in CoD. Instead, the SBMM in CoD matches you with a bunch of players who are far worse than you for a game or two. Then, it matches you with a bunch of players who are far better than you for a game or two, and this cycle continues over and over again. This means that you will do really well for a game or two, and then get destroyed for a game or two. Eventually, this averages out to the target of every player having around a 1.0 kill/death ratio. In theory, SBMM is supposed to help lower-skilled players, but the implementation in CoD is so bad it is more of a roller coaster than a bunch of matches against similarly-skilled players.

asdfasgasdgasdg 6y ago
I do perceive some "swinginess" in my performance in Black Ops Cold War, but I'm not positive that it's actually real. Gamblers at the roulette table also perceive trends where in fact there is only random variance. It's also worth noting that this is only half of the complaints that you see on message boards about this game. The other half, or maybe the majority, is people saying, "I just want to be able to get high and use off-meta weapons and not lose to sweats (i.e. people who are trying) all night." This type of complaint is fundamentally a request to be matched against lower skilled players, and has nothing to do with the swinginess of matchmaking.
- strken 6y ago
  It sounds like a request to be matched appropriately to expected performance at times when they're deliberately sabotaging their ability, which is not that unreasonable except for the near-impossibility of doing so. They might even be able to achieve it by having a separate account for messing around while they're high and using dumb weapons.
  - harimau777 6y ago
    Something that could help would be to allow players to choose what server they play on. The server could then establish rules against tactics that are regarded as effective but unsportsmanlike. Casual servers could also choose to implement more aggressive rebalancing systems between rounds. Finally, players could choose to select servers that they know to have a more casual community.
nemothekid 6y ago
This feels like the same argument that's been used in League, DoTA and Overwatch that implies the system gives you really bad teammates, and then good teammates. I'm trying to find where I read this, but essentially even minor differences in skill can translate to feeling like you are "curbstomping" the opponent.
- ngngngng 6y ago
  > I'm trying to find where I read this It says so in the article we're commenting on
- LoSboccacc 6y ago
  It's inevitable, the player distribution isn't uniform and as you move forward the gradient steepens. Most sbmm as they call them suffers from oscillations, even games with a large player base, where you'd think there's enough to balance things out. It also flattens the base, as people have to figure out what works and what not without having skilled player in the mix to learn from
- doublerabbit 6y ago
  I have noticed the same with Overwatch ever since they implemented the new match making system. Myself, somewhat a player who plays for fun and is okay ranked player will be put in a team with low experienced players paired against a team with three or four-extremely ranked high ranked players with one player low ranked. This ends up in a slaughter and it's not that my team sucks, it's just we are overthrown by experience. This turns the game in to toxicity when the other team players then start bragging "ez" and such.
duskwuff 6y ago
I haven't played the game, but based on what I'm reading about it, there's a couple of confounding factors: 1. There are a lot of different game modes for CoD -- Cold War has eleven game modes. This means that the pool of players for each mode will be smaller, making fair matchmaking more difficult. 2. Some of those game modes have unusually large teams -- two of the signature game modes in Cold War have ten or more players to a team. This makes it more difficult for a matchmaking algorithm to fairly assess any individual player's contribution to their team's win/loss. It also makes it more likely that one or more teams will contain a player whose skill has been poorly assessed, meaning that their team has an unfair advantage or disadvantage.
- asdfasgasdgasdg 6y ago
  I think that point one is true in many games, but cold war is probably not one of them. There are probably tens or hundreds of thousands of people queuing for a game of black ops cold war at any given point in time. Even if you divide by ten different queues, there should be enough granularity to give decent matchups to people at all skill levels. And I don't know if I would call combined arms or dirty bomb signature modes. 6v6 is the bread and butter of the annual call of duty.
  - duskwuff 6y ago
    > Even if you divide by ten different queues, there should be enough granularity to give decent matchups to people at all skill levels. Could be. I don't know what the numbers look like, and I wouldn't be surprised if the player base is being split up in other ways (e.g. console vs. PC, geographic regions, etc) at the same time. > And I don't know if I would call combined arms or dirty bomb signature modes. By "signature modes", I mean that those game modes were created specifically for CoD: Cold War, rather than ones which are shared by many FPS games, or carried over from previous games in the series.
hogFeast 6y ago
That isn't how SBMM works. If you play a team with higher ELO, the amount of ELO you win/lose changes based on the skill differential. The eventual averaging is not how the algo finds your skill level. If you play a worse team than you are expected to win with X probability, that is what you are judged against. Also, just to be clear, no implementation is this simplistic. Most models are Bayesian and so incorporate uncertainty around the estimate of skill. And most also include your individual stats (this is game-dependent but I believe Overwatch has a class/map-based ELO that you are rated against...as an example). I agree with your general point though. Most SBMM systems do not feel close games every time. Usually, someone is smurfing, someone is throwing, someone won't come on comms, etc. (yes, I play Overwatch and I am still angry about it).
deevolution 6y ago
it sounds like a CoD socialist utopia simulator where every player has an equal outcome regardless of skill: 1.0 k/d. k/d creeping above 1.0? Let's distribute those gains to other players by curbstomping you for a bit. K/d falling below 1.0? Here's a couple of easy and free "k/d UBI" matches to boost you up. Who could have guessed this results in a mediocre gameplay experience?
defertoreptar 6y ago
Sounds like the matchmaking needs to factor in the standard deviation of the players' skill, like a Sharpe ratio for skill.
stunt 6y ago
It's just impossible to fix it. You have to put players with different ranks in a server. If you just sort everyone by their skills, you get good teams that crush the rest of server. It will be annoying for low rank players, and boring for high rank players. There are servers tiers, but they can't always fill a server with same rank players. How many players are online on the same region that want to join a server at a given time... So there will be always some randomness to it. As long as you sometimes get to play with good teams and win, it will keep you in the game. You will probably even play longer that way.
saddestcatever 6y ago
> matches you with a bunch of players who are far worse than you for a game or two. Then, it matches you with a bunch of players who are far better than you for a game or two What are the chances this is an 'engagement' hack? I wonder if players are more likely to continue playing if they have ups and downs. In a perfect matchmaking system, would players get bored? I have to wonder if this is intentional, or just the side effect of either perception or an imperfect system.
- filoleg 6y ago
  >I wonder if players are more likely to continue playing if they have ups and downs. If the ups and downs were fairly close, then I could see that. But that's not what people have been complaining about with regards to the new CoD game. The issue was that a lot of matches end up being disproportionally one-sided one way or the other. And that's not fun at all. While an occasional slam-dunk game is fun (or maybe even most slam-dunk games are fun, i don't know), I cannot find anyone who would have an increased desire to play the game after they get absolutely annihilated by a vastly superior team that they cannot do absolutely anything against. And that's a sign of a bad SBMM system. A "perfect" SBMM system would always put you in matches where you are either slightly better or slightly worse than the other team. Turns out, that's the scenario that optimizes for engagement the most (at least from personal observations across a ton of different video games and real-life competitive sports I participated in). If I won a game by a tiny margin, I feel proud and exhilarated. If I lost a game by a tiny margin, I feel incentivized to play more, because "only if I did this one small thing differently, we would have won, let's play one more game and see". If I won by a giant margin, it feels good too, so I might as well continue playing more. But if I lost by a giant margin, it drops all desire I had to continue this gaming session, because no amount of incremental improvements to my gameplay I can do within a day that would have let me somehow close the 30-100 point gap in that game (so, in a sense, it feels like i was set up for a game that i stand zero chance of winning, and that's pretty demotivating). Losing by a tiny bit is a learning moment that makes you feel like you can win if you try a bit better or improve in one specific aspect a little bit. You learn from those kinds of losses, and the feeling of improvement is good. Losing by a giant margin makes you learn nothing. Just like in basketball or chess. Playing against someone who is slightly better than you are is fun. Playing against someone who is a pro (while you are a beginner) isn't fun, because of the sheer width of the skill gap. There is no "victory carrot" hanging in front of you to motivate you to continue playing, because that "victory carrot" is located on the other continent, and you aren't keen on swimming across the pacific ocean in one go. So, at least in this case, the incentive of the matchmaking system being good aligns with the increased engagement. Or so I believe, based upon my personal observations and anecdata.
Twisell 6y ago
I think is globally harder to balance SBMM for games that involve teamwork because bad partners can make you loose even when opposing team is adequately balanced. With Dota2 (a while ago might have been fixed) my complain was more than SBMM wasn't tight enough and unfortunately due to game dynamics a single sub-par player in the team could give a big advantage to the adversaries (and I totally found myself in this quite unpleasant role). So maybe i'd say elo hell exist in some way as when the matchmaking algorithm doesn't have enough data to match you adequately it must train itself by matching you using a pretty loose standard deviation. I guess the success of "good matchmaking" actually depend on how this training sequence is more or less gracefully handled from the player perspective.
- Ntrails 6y ago
  I rarely have a game of Dota2 where the spread of skill in my team is the reason I've lost. Indeed, I can't think of a time. I've lost to smurfs occasionally. I've lost because people have tilted and abandoned or chain fed. I've lost because internet has died. I have even lost despite playing at my best and having a good game... Mostly, 8-9 times out of 10, I've lost because I did not have enough impact on the game. I did not do enough to win, and to help my team win. But yeah. Wide spreads in matchmaking really only happen with very extreme skill levels, on rare game modes / timezones, and where explicitly playing as a party (and thus forcing it)
  - Twisell 6y ago
    >I've lost because people have tilted and abandoned or chain fed. Was mostly thinking about that. Everyone can chain feed when in bad shape. But it's more likely that the lowest skilled player of the team do that. However its true that a poor hero choice in the setup can be all it takes. I guess more than SBMM what ultimately made me go away from this game is the average game length. My lifestyle is now incompatible with potential 1hour+ game and i didn't want to become "the selfish quitter".
castlecrasher2 6y ago
It was the same in Destiny 2; the game would do everything in its power to keep you at 50% win rate which generally meant win, lose, repeat.

by jswny 6y ago

asdfasgasdgasdg 6y ago

I do perceive some "swinginess" in my performance in Black Ops Cold War, but I'm not positive that it's actually real. Gamblers at the roulette table also perceive trends where in fact there is only random variance. It's also worth noting that this is only half of the complaints that you see on message boards about this game. The other half, or maybe the majority, is people saying, "I just want to be able to get high and use off-meta weapons and not lose to sweats (i.e. people who are trying) all night." This type of complaint is fundamentally a request to be matched against lower skilled players, and has nothing to do with the swinginess of matchmaking.

strken 6y ago
It sounds like a request to be matched appropriately to expected performance at times when they're deliberately sabotaging their ability, which is not that unreasonable except for the near-impossibility of doing so. They might even be able to achieve it by having a separate account for messing around while they're high and using dumb weapons.
- harimau777 6y ago
  Something that could help would be to allow players to choose what server they play on. The server could then establish rules against tactics that are regarded as effective but unsportsmanlike. Casual servers could also choose to implement more aggressive rebalancing systems between rounds. Finally, players could choose to select servers that they know to have a more casual community.

nemothekid 6y ago

This feels like the same argument that's been used in League, DoTA and Overwatch that implies the system gives you really bad teammates, and then good teammates. I'm trying to find where I read this, but essentially even minor differences in skill can translate to feeling like you are "curbstomping" the opponent.

ngngngng 6y ago
> I'm trying to find where I read this It says so in the article we're commenting on
LoSboccacc 6y ago
It's inevitable, the player distribution isn't uniform and as you move forward the gradient steepens. Most sbmm as they call them suffers from oscillations, even games with a large player base, where you'd think there's enough to balance things out. It also flattens the base, as people have to figure out what works and what not without having skilled player in the mix to learn from
doublerabbit 6y ago
I have noticed the same with Overwatch ever since they implemented the new match making system. Myself, somewhat a player who plays for fun and is okay ranked player will be put in a team with low experienced players paired against a team with three or four-extremely ranked high ranked players with one player low ranked. This ends up in a slaughter and it's not that my team sucks, it's just we are overthrown by experience. This turns the game in to toxicity when the other team players then start bragging "ez" and such.

duskwuff 6y ago

I haven't played the game, but based on what I'm reading about it, there's a couple of confounding factors: 1. There are a lot of different game modes for CoD -- Cold War has eleven game modes. This means that the pool of players for each mode will be smaller, making fair matchmaking more difficult. 2. Some of those game modes have unusually large teams -- two of the signature game modes in Cold War have ten or more players to a team. This makes it more difficult for a matchmaking algorithm to fairly assess any individual player's contribution to their team's win/loss. It also makes it more likely that one or more teams will contain a player whose skill has been poorly assessed, meaning that their team has an unfair advantage or disadvantage.

asdfasgasdgasdg 6y ago
I think that point one is true in many games, but cold war is probably not one of them. There are probably tens or hundreds of thousands of people queuing for a game of black ops cold war at any given point in time. Even if you divide by ten different queues, there should be enough granularity to give decent matchups to people at all skill levels. And I don't know if I would call combined arms or dirty bomb signature modes. 6v6 is the bread and butter of the annual call of duty.
- duskwuff 6y ago
  > Even if you divide by ten different queues, there should be enough granularity to give decent matchups to people at all skill levels. Could be. I don't know what the numbers look like, and I wouldn't be surprised if the player base is being split up in other ways (e.g. console vs. PC, geographic regions, etc) at the same time. > And I don't know if I would call combined arms or dirty bomb signature modes. By "signature modes", I mean that those game modes were created specifically for CoD: Cold War, rather than ones which are shared by many FPS games, or carried over from previous games in the series.

hogFeast 6y ago

That isn't how SBMM works. If you play a team with higher ELO, the amount of ELO you win/lose changes based on the skill differential. The eventual averaging is not how the algo finds your skill level. If you play a worse team than you are expected to win with X probability, that is what you are judged against. Also, just to be clear, no implementation is this simplistic. Most models are Bayesian and so incorporate uncertainty around the estimate of skill. And most also include your individual stats (this is game-dependent but I believe Overwatch has a class/map-based ELO that you are rated against...as an example). I agree with your general point though. Most SBMM systems do not feel close games every time. Usually, someone is smurfing, someone is throwing, someone won't come on comms, etc. (yes, I play Overwatch and I am still angry about it).

deevolution 6y ago

it sounds like a CoD socialist utopia simulator where every player has an equal outcome regardless of skill: 1.0 k/d. k/d creeping above 1.0? Let's distribute those gains to other players by curbstomping you for a bit. K/d falling below 1.0? Here's a couple of easy and free "k/d UBI" matches to boost you up. Who could have guessed this results in a mediocre gameplay experience?

defertoreptar 6y ago

Sounds like the matchmaking needs to factor in the standard deviation of the players' skill, like a Sharpe ratio for skill.

stunt 6y ago

It's just impossible to fix it. You have to put players with different ranks in a server. If you just sort everyone by their skills, you get good teams that crush the rest of server. It will be annoying for low rank players, and boring for high rank players. There are servers tiers, but they can't always fill a server with same rank players. How many players are online on the same region that want to join a server at a given time... So there will be always some randomness to it. As long as you sometimes get to play with good teams and win, it will keep you in the game. You will probably even play longer that way.

saddestcatever 6y ago

> matches you with a bunch of players who are far worse than you for a game or two. Then, it matches you with a bunch of players who are far better than you for a game or two What are the chances this is an 'engagement' hack? I wonder if players are more likely to continue playing if they have ups and downs. In a perfect matchmaking system, would players get bored? I have to wonder if this is intentional, or just the side effect of either perception or an imperfect system.

filoleg 6y ago
>I wonder if players are more likely to continue playing if they have ups and downs. If the ups and downs were fairly close, then I could see that. But that's not what people have been complaining about with regards to the new CoD game. The issue was that a lot of matches end up being disproportionally one-sided one way or the other. And that's not fun at all. While an occasional slam-dunk game is fun (or maybe even most slam-dunk games are fun, i don't know), I cannot find anyone who would have an increased desire to play the game after they get absolutely annihilated by a vastly superior team that they cannot do absolutely anything against. And that's a sign of a bad SBMM system. A "perfect" SBMM system would always put you in matches where you are either slightly better or slightly worse than the other team. Turns out, that's the scenario that optimizes for engagement the most (at least from personal observations across a ton of different video games and real-life competitive sports I participated in). If I won a game by a tiny margin, I feel proud and exhilarated. If I lost a game by a tiny margin, I feel incentivized to play more, because "only if I did this one small thing differently, we would have won, let's play one more game and see". If I won by a giant margin, it feels good too, so I might as well continue playing more. But if I lost by a giant margin, it drops all desire I had to continue this gaming session, because no amount of incremental improvements to my gameplay I can do within a day that would have let me somehow close the 30-100 point gap in that game (so, in a sense, it feels like i was set up for a game that i stand zero chance of winning, and that's pretty demotivating). Losing by a tiny bit is a learning moment that makes you feel like you can win if you try a bit better or improve in one specific aspect a little bit. You learn from those kinds of losses, and the feeling of improvement is good. Losing by a giant margin makes you learn nothing. Just like in basketball or chess. Playing against someone who is slightly better than you are is fun. Playing against someone who is a pro (while you are a beginner) isn't fun, because of the sheer width of the skill gap. There is no "victory carrot" hanging in front of you to motivate you to continue playing, because that "victory carrot" is located on the other continent, and you aren't keen on swimming across the pacific ocean in one go. So, at least in this case, the incentive of the matchmaking system being good aligns with the increased engagement. Or so I believe, based upon my personal observations and anecdata.

Twisell 6y ago

I think is globally harder to balance SBMM for games that involve teamwork because bad partners can make you loose even when opposing team is adequately balanced. With Dota2 (a while ago might have been fixed) my complain was more than SBMM wasn't tight enough and unfortunately due to game dynamics a single sub-par player in the team could give a big advantage to the adversaries (and I totally found myself in this quite unpleasant role). So maybe i'd say elo hell exist in some way as when the matchmaking algorithm doesn't have enough data to match you adequately it must train itself by matching you using a pretty loose standard deviation. I guess the success of "good matchmaking" actually depend on how this training sequence is more or less gracefully handled from the player perspective.

Ntrails 6y ago
I rarely have a game of Dota2 where the spread of skill in my team is the reason I've lost. Indeed, I can't think of a time. I've lost to smurfs occasionally. I've lost because people have tilted and abandoned or chain fed. I've lost because internet has died. I have even lost despite playing at my best and having a good game... Mostly, 8-9 times out of 10, I've lost because I did not have enough impact on the game. I did not do enough to win, and to help my team win. But yeah. Wide spreads in matchmaking really only happen with very extreme skill levels, on rare game modes / timezones, and where explicitly playing as a party (and thus forcing it)
- Twisell 6y ago
  >I've lost because people have tilted and abandoned or chain fed. Was mostly thinking about that. Everyone can chain feed when in bad shape. But it's more likely that the lowest skilled player of the team do that. However its true that a poor hero choice in the setup can be all it takes. I guess more than SBMM what ultimately made me go away from this game is the average game length. My lifestyle is now incompatible with potential 1hour+ game and i didn't want to become "the selfish quitter".

castlecrasher2 6y ago

It was the same in Destiny 2; the game would do everything in its power to keep you at 50% win rate which generally meant win, lose, repeat.