6 ms·

What Happens When Baseball-Stats Nerds Run a Pro Team

132 points by zonotope 11y ago

cortesoft 10y ago
The idea that a baseball team could ever go undefeated for even half a season is ridiculous. No amount of data analysis can make you do that. That would be like saying that a perfect poker player could win every hand. Even if your opponent was playing with their cards face up, sometimes he is gonna hit his inside straight on the river.
- mathattack 10y ago
  I think they were aiming for very good (keeping up their 26-11 first half record) rather than undefeated. The story as a whole shows good humility.
  - cortesoft 10y ago
    They said right in the article: "We addressed the players and laid out our goals for the season: to lead the league in the first half, and in the second half, to go undefeated."
- legohead 10y ago
  I don't follow baseball. Are you saying that a large part of baseball is luck, and you are bound to lose regardless of how skilled your team is?
  - supercanuck 10y ago
    Yes, injuries are usually attributed to bad luck and can impact a season mightily
  - mwhite 10y ago
    Enough of it's luck that it's unlikely that a team would go undefeated for half of a season, but it's also the fact that it's hard to get a team with only the best players; most teams have a mixture of player ability. If you put the best major league team in the minor leagues they'd probably have a 90%+ winning percentage. (Highest MLB season winning percentage is 76%, longest winning streak is 26 games out of 162, including a tie.)
    - daveguy 10y ago
      Two additional points. 1) The mixture of player ability is mostly due to cost penalties for paying players a higher salary (although Yankees just pay it). Not as distributed as (american) football, but still distributed. 2) Note the longest winning streak is 26 games out of 162... out of 162. Longest basketball streak is 33 games (not enough for an undefeated 82 game season). The ability to have a 162 game streak that starts on game one and lasts through the baseball season is essentially impossible. It is possible to have an undefeated season in american football where the season is 16 games (but it is rare -- hasn't happened since pre-NFL 1937, 1948). Baseball has an order of magnitude more games per season. EDIT: I knew that wasn't right (that's what I get for haphazardly scanning Wikipedia for facts). A couple others since recently 2007 Patriots and 1972 Dolphins also had undefeated seasons in NFL. Still rare, and still a fraction of the number of games. Point is: luck is only part of it, no matter the sport staying undefeated is difficult -- especially for a season when season means 162 games! Thanks for pointing out the mistake everyone. Sorry for the confusion!
      - tomlor 10y ago
        2007 Patriots went 16-0 https://en.wikipedia.org/wiki/2007_New_England_Patriots_season https://en.wikipedia.org/wiki/2007_New_England_Patriots_seas...
        datawrangler 10y ago
        The 1972 Dolphins went 14-0 and also won the Super Bowl [1].
        cubano 10y ago
        They actually went 17-0 and won the Super Bowl.
        datawrangler 10y ago
        I know--I was only counting the regular season. :)
  - cortesoft 10y ago
    Yes, there is a lot of luck. For one thing, the gap between very bad players and very good players is fairly small in an absolute sense; for example, a player who gets a hit 25% of the time would have trouble staying in the league, but a player who gets a hit 30% of the time has a good chance of being an all-star. A 5% increase in your success rate separates the very worst players from the very best. There is a great saying about baseball: Every team is going to win 54 games and every team is going to lose 54 games; it is what you do with the other 54 games that counts. (There are 162 games in a season)
    - kriro 10y ago
      Isn't a 20% skill difference pretty meaningful in most endeavors? Maybe not the difference between unemployment and rock star but being 20% better at sales for example can be quite lucrative.
      - thaumasiotes 10y ago
        Your 20% is a relative difference, not an absolute difference. The difference between hitting 0.5% of pitches and 0.6% of pitches isn't meaningful at all despite being the same relative difference.
        kriro 10y ago
        The league wide batting average in the MLB was .253 in 2013 (first Google result, I'm assuming it doesn't vary all that much between seasons). While you are correct that hitting 0.6% compared to 0.5% doesn't mean much the numbers I responded to were meaningful in an MLB context (.250 vs. .300) so I think it's justified to use relative differences. In fact I think it's more insightful than using absolute numbers. Speaking of a 5% increase in that context is confusing (at least to me :D) when in reality the skill is roughly 20% above league average. A programmer who is 20% above average will certainly command a premium as will a salesperson.
        thaumasiotes 10y ago
  - mmanfrin 10y ago
    No, but everyone has bad days, and the players are all human. A small error can lead to points which can change the outcome of the game.
  - umanwizard 10y ago
    Practically everything in the world more complex than basic physics is non-deterministic, especially who wins games of skill. Things occur or don't occur with probabilities. Even if you have a 95% chance of winning each game (unrealistic IMO), the probability of winning 81 games in a row is under 2%. This is less apparent in other sports than in baseball, not because the outcomes of those games are more deterministic, but because fewer games are played per season.
    - deleted 10y ago
      [deleted]
    - closed 10y ago
      You are a bayesian analysis with strong priors on the chance of winning. I'm not objecting to your critique (as I don't know much about sports), I just keep thinking of P(data | model).
      - deleted 10y ago
        [deleted]
      - umanwizard 10y ago
        Well, there is no sport or other organized competitive activity I can think of where the best in the world beats the second best in the world more than 95% of the time.
        largote 10y ago
        Maybe sprinting?
        markeroon 10y ago
        Good call. 800m-10km track races too.
        umanwizard 10y ago
        Good point. I'd guess that's true of stuff like weightlifting also. These sports that are about pushing your body as far as it can go in one dimension are a lot simpler and "purer" than a team game with lots of moving parts like baseball.
  - ericdykstra 10y ago
    Off the top of my head, here's a list of reasons that may tip the scale of an outcome in favor of the worse overall team in a baseball contest, whether luck or other factors. (the first two are by far the most important, and are in general order of how much variability each factors to a game): - sequencing (3 walks, 3 strikeouts, and a home run in an inning can end with 1 or 4 runs depending on sequencing) - ball-in-play luck (batters have little-to-no control over whether their line drive finds a glove for an out or a gap for a double) - pitcher/hitter matchups favoring one team or another - other luck over a single game (2+ errors in a game, 3+ home runs in a game, etc) that are well above the average for any team, but any team will have many of these games over the course of a season - umpire strike zone favoring one team's pitchers - umpire calls consistency - park/weather favoring one team's players If you need any explanation on any of these I'd be happy to provide it. I'll add more if I think of them, but this should give you a pretty good idea of why the best teams in baseball usually end up with a less than 60% win record.
  - ubernostrum 10y ago
    cortesoft is on track by pointing out how small the gaps are (in absolute terms) between bad, average, good and very good in baseball. And just as importantly, the gaps have been shrinking. Stephen Jay Gould famously wrote about this in explaining why we don't see .400 batting averages anymore (that's a hit 4 out of every 10 times a player bats; it last happened in the major leagues in 1941). The average major-league player of today is significantly better than the average player of seventy years ago, so there are fewer opportunities for the best player in the league to take advantage of large numbers of truly-outmatched opponents and achieve inflated performance numbers. The highest batting average in the modern era of baseball is .426 by Nap Lajoie in 1901. 2015's highest batting average in the major leagues was .338, and it was .341 the year before; even an average of .360 or better is now rare (only happened eight times since the year 2000). And overall team performance is affected in the same way. The 1906 Chicago Cubs own the best win percentage in major-league history at .763 (116 wins out of 152 games played). That year the worst team (the Boston Americans, predecessors of today's Boston Red Sox) went 49-105, a win percentage of .318. Last year (2015) the best win percentage was .617 while the worst was .389. Only four major-league teams failed to win at least 70 games last year, and over half managed to win at least 81 games (81-81 being an even .500 win percentage with the season now 162 games long), while at the top there were seven teams with at least 90 wins, and four of them had at least 95 wins.
- pimlottc 10y ago
  For those curious, they ended up going 18-22 in the second half of the season. This was still good enough for a second place record of 44-33 overall, out of four teams. In the playoffs against the first place San Rafael Pacifics, they came close but lost 4 games to 3. http://papbl_wtt.bbstats.pointstreak.com/standings.html?leagueid=1275&seasonid=29496 http://papbl_wtt.bbstats.pointstreak.com/standings.html?leag...
mathattack 10y ago
I like that this captures something very important about Data Science. The data may give the truth, but it's the story that makes people change. The story taps our emotions, which cause us to change behavior faster than hard data. The chief analyst at a prior employer once commented that he liked people "with degrees that suggest storytelling" in addition to their math/science/engineering pedigrees.
- allisthemoist 10y ago
  Because what is a story really but condensed data?
  - deleted 10y ago
    [deleted]
  - rpgmaker 10y ago
    A story can have a lot of noise too.
    - RobertoG 10y ago
      not the good ones
  - mathattack 10y ago
    The story taps emotions, and impacts decisions better than just raw data. This is why politicians try to attach names and faces to policies. Good leaders use the data to inform the decisions, but stories to explain them.
kmnc 10y ago
It would be interesting to see some scouting models based purely on psychological and personel relationship data. What kind of metrics are best at predicting team chemistry. Sports analytics is going to complete change most sports, we are just getting started.
- kobayashi 10y ago
  IIRC, the Blue Jays already take that into account (or at least they did under the previous GM, Alex Anthopolous).
- drumdance 10y ago
  A different angle is player health. The Golden State Warriors are known for resting top players when they're in Denver because they found it can take a player up to a week to fully recover from the effects of playing at altitude.
mmanfrin 10y ago
This reminds me of Football Manager (video game series). It's a turn based moneyball-sort-of game for soccer/football. Players have stats, potentials (which need to be scouted), training, etc. But there are other relevant details like how a player is likely to get along with other players that affect game outcomes.
- wolfgke 10y ago
  > But there are other relevant details like how a player is likely to get along with other players that affect game outcomes. Time for creating mathematical models for predictors of how people are getting along... :-)
fallous 10y ago
Expertise is knowledge within a problem domain. Art, and more importantly wisdom, is in the implementation and execution.
tempestn 10y ago
This is a great lesson for hackers in general. Many of us, myself included, have a tendency to expect data to automatically win arguments. I mean, if something is irrefutably true with the data to prove it, that's the end of the discussion, right? But just as in this case, in reality, it's not even the beginning. Whether you're trying to convince your manager to go in a particular direction, or you're trying to motivate your employees to rally behind a cause, the data will help you decide where to go, but you need the right story to get people to follow you there. And just because it's a "story" doesn't mean it's some BS you make up to get people to agree with you. It's more about considering the other person's point of view (their poorly-documented public API, if you like), and building an interface between that and your data.
- pimlottc 10y ago
  The best manager I ever had was great at this. He was a former developer who understood the tech but knew how to speak the executives' language. You could convince him with a good tech story and then he'd turn around and sell to his bosses with a good business story.
  - danieltillett 10y ago
    Domain translators are undervalued. Anyone who can effectively translate the different needs and desires between groups is worth their weight in gold.
    - ethbro 10y ago
      Agreed! I think this also cuts to the fact that data doesn't win arguments. Data + time spent studying and understanding data wins. Most people can't or don't want to take time (it's inefficient). So they make decisions based on estimates / stories / approximations / biases / gut and the world keeps spinning. Which means your goal should be to most accurately frame your data in terms of their stories (translating as much or little as they require).
- thaumasiotes 10y ago
SixSigma 10y ago
No matter how good your analysis, data collection only told you what happened in the past.
vwelling 10y ago
For us nerds who want to effect change, and who are convinced data should be all you need, the book "Switch: How to Change Things When Change Is Hard" by Chip and Dan Heath is a highly recommended read. It'll tell you all about the story, or as they call it: the elephant.

132 points by zonotope 11y ago

The idea that a baseball team could ever go undefeated for even half a season is ridiculous. No amount of data analysis can make you do that. That would be like saying that a perfect poker player could win every hand. Even if your opponent was playing with their cards face up, sometimes he is gonna hit his inside straight on the river.

mathattack 10y ago
I think they were aiming for very good (keeping up their 26-11 first half record) rather than undefeated. The story as a whole shows good humility.
- cortesoft 10y ago
  They said right in the article: "We addressed the players and laid out our goals for the season: to lead the league in the first half, and in the second half, to go undefeated."
legohead 10y ago
I don't follow baseball. Are you saying that a large part of baseball is luck, and you are bound to lose regardless of how skilled your team is?
- supercanuck 10y ago
  Yes, injuries are usually attributed to bad luck and can impact a season mightily
- mwhite 10y ago
  Enough of it's luck that it's unlikely that a team would go undefeated for half of a season, but it's also the fact that it's hard to get a team with only the best players; most teams have a mixture of player ability. If you put the best major league team in the minor leagues they'd probably have a 90%+ winning percentage. (Highest MLB season winning percentage is 76%, longest winning streak is 26 games out of 162, including a tie.)
  - daveguy 10y ago
    Two additional points. 1) The mixture of player ability is mostly due to cost penalties for paying players a higher salary (although Yankees just pay it). Not as distributed as (american) football, but still distributed. 2) Note the longest winning streak is 26 games out of 162... out of 162. Longest basketball streak is 33 games (not enough for an undefeated 82 game season). The ability to have a 162 game streak that starts on game one and lasts through the baseball season is essentially impossible. It is possible to have an undefeated season in american football where the season is 16 games (but it is rare -- hasn't happened since pre-NFL 1937, 1948). Baseball has an order of magnitude more games per season. EDIT: I knew that wasn't right (that's what I get for haphazardly scanning Wikipedia for facts). A couple others since recently 2007 Patriots and 1972 Dolphins also had undefeated seasons in NFL. Still rare, and still a fraction of the number of games. Point is: luck is only part of it, no matter the sport staying undefeated is difficult -- especially for a season when season means 162 games! Thanks for pointing out the mistake everyone. Sorry for the confusion!
    - tomlor 10y ago
      2007 Patriots went 16-0 https://en.wikipedia.org/wiki/2007_New_England_Patriots_season https://en.wikipedia.org/wiki/2007_New_England_Patriots_seas...
      - datawrangler 10y ago
        The 1972 Dolphins went 14-0 and also won the Super Bowl [1].
        cubano 10y ago
        They actually went 17-0 and won the Super Bowl.
        datawrangler 10y ago
        I know--I was only counting the regular season. :)
- cortesoft 10y ago
  Yes, there is a lot of luck. For one thing, the gap between very bad players and very good players is fairly small in an absolute sense; for example, a player who gets a hit 25% of the time would have trouble staying in the league, but a player who gets a hit 30% of the time has a good chance of being an all-star. A 5% increase in your success rate separates the very worst players from the very best. There is a great saying about baseball: Every team is going to win 54 games and every team is going to lose 54 games; it is what you do with the other 54 games that counts. (There are 162 games in a season)
  - kriro 10y ago
    Isn't a 20% skill difference pretty meaningful in most endeavors? Maybe not the difference between unemployment and rock star but being 20% better at sales for example can be quite lucrative.
    - thaumasiotes 10y ago
      Your 20% is a relative difference, not an absolute difference. The difference between hitting 0.5% of pitches and 0.6% of pitches isn't meaningful at all despite being the same relative difference.
      - kriro 10y ago
        The league wide batting average in the MLB was .253 in 2013 (first Google result, I'm assuming it doesn't vary all that much between seasons). While you are correct that hitting 0.6% compared to 0.5% doesn't mean much the numbers I responded to were meaningful in an MLB context (.250 vs. .300) so I think it's justified to use relative differences. In fact I think it's more insightful than using absolute numbers. Speaking of a 5% increase in that context is confusing (at least to me :D) when in reality the skill is roughly 20% above league average. A programmer who is 20% above average will certainly command a premium as will a salesperson.
        thaumasiotes 10y ago
- mmanfrin 10y ago
  No, but everyone has bad days, and the players are all human. A small error can lead to points which can change the outcome of the game.
- umanwizard 10y ago
  Practically everything in the world more complex than basic physics is non-deterministic, especially who wins games of skill. Things occur or don't occur with probabilities. Even if you have a 95% chance of winning each game (unrealistic IMO), the probability of winning 81 games in a row is under 2%. This is less apparent in other sports than in baseball, not because the outcomes of those games are more deterministic, but because fewer games are played per season.
  - deleted 10y ago
    [deleted]
  - closed 10y ago
    You are a bayesian analysis with strong priors on the chance of winning. I'm not objecting to your critique (as I don't know much about sports), I just keep thinking of P(data | model).
    - deleted 10y ago
      [deleted]
    - umanwizard 10y ago
      Well, there is no sport or other organized competitive activity I can think of where the best in the world beats the second best in the world more than 95% of the time.
      - largote 10y ago
        Maybe sprinting?
        markeroon 10y ago
        Good call. 800m-10km track races too.
        umanwizard 10y ago
        Good point. I'd guess that's true of stuff like weightlifting also. These sports that are about pushing your body as far as it can go in one dimension are a lot simpler and "purer" than a team game with lots of moving parts like baseball.
- ericdykstra 10y ago
  Off the top of my head, here's a list of reasons that may tip the scale of an outcome in favor of the worse overall team in a baseball contest, whether luck or other factors. (the first two are by far the most important, and are in general order of how much variability each factors to a game): - sequencing (3 walks, 3 strikeouts, and a home run in an inning can end with 1 or 4 runs depending on sequencing) - ball-in-play luck (batters have little-to-no control over whether their line drive finds a glove for an out or a gap for a double) - pitcher/hitter matchups favoring one team or another - other luck over a single game (2+ errors in a game, 3+ home runs in a game, etc) that are well above the average for any team, but any team will have many of these games over the course of a season - umpire strike zone favoring one team's pitchers - umpire calls consistency - park/weather favoring one team's players If you need any explanation on any of these I'd be happy to provide it. I'll add more if I think of them, but this should give you a pretty good idea of why the best teams in baseball usually end up with a less than 60% win record.
- ubernostrum 10y ago
  cortesoft is on track by pointing out how small the gaps are (in absolute terms) between bad, average, good and very good in baseball. And just as importantly, the gaps have been shrinking. Stephen Jay Gould famously wrote about this in explaining why we don't see .400 batting averages anymore (that's a hit 4 out of every 10 times a player bats; it last happened in the major leagues in 1941). The average major-league player of today is significantly better than the average player of seventy years ago, so there are fewer opportunities for the best player in the league to take advantage of large numbers of truly-outmatched opponents and achieve inflated performance numbers. The highest batting average in the modern era of baseball is .426 by Nap Lajoie in 1901. 2015's highest batting average in the major leagues was .338, and it was .341 the year before; even an average of .360 or better is now rare (only happened eight times since the year 2000). And overall team performance is affected in the same way. The 1906 Chicago Cubs own the best win percentage in major-league history at .763 (116 wins out of 152 games played). That year the worst team (the Boston Americans, predecessors of today's Boston Red Sox) went 49-105, a win percentage of .318. Last year (2015) the best win percentage was .617 while the worst was .389. Only four major-league teams failed to win at least 70 games last year, and over half managed to win at least 81 games (81-81 being an even .500 win percentage with the season now 162 games long), while at the top there were seven teams with at least 90 wins, and four of them had at least 95 wins.
pimlottc 10y ago
For those curious, they ended up going 18-22 in the second half of the season. This was still good enough for a second place record of 44-33 overall, out of four teams. In the playoffs against the first place San Rafael Pacifics, they came close but lost 4 games to 3. http://papbl_wtt.bbstats.pointstreak.com/standings.html?leagueid=1275&seasonid=29496 http://papbl_wtt.bbstats.pointstreak.com/standings.html?leag...

mathattack 10y ago

I like that this captures something very important about Data Science. The data may give the truth, but it's the story that makes people change. The story taps our emotions, which cause us to change behavior faster than hard data. The chief analyst at a prior employer once commented that he liked people "with degrees that suggest storytelling" in addition to their math/science/engineering pedigrees.

allisthemoist 10y ago
Because what is a story really but condensed data?
- deleted 10y ago
  [deleted]
- rpgmaker 10y ago
  A story can have a lot of noise too.
  - RobertoG 10y ago
    not the good ones
- mathattack 10y ago
  The story taps emotions, and impacts decisions better than just raw data. This is why politicians try to attach names and faces to policies. Good leaders use the data to inform the decisions, but stories to explain them.

kmnc 10y ago

It would be interesting to see some scouting models based purely on psychological and personel relationship data. What kind of metrics are best at predicting team chemistry. Sports analytics is going to complete change most sports, we are just getting started.

kobayashi 10y ago
IIRC, the Blue Jays already take that into account (or at least they did under the previous GM, Alex Anthopolous).
drumdance 10y ago
A different angle is player health. The Golden State Warriors are known for resting top players when they're in Denver because they found it can take a player up to a week to fully recover from the effects of playing at altitude.

mmanfrin 10y ago

This reminds me of Football Manager (video game series). It's a turn based moneyball-sort-of game for soccer/football. Players have stats, potentials (which need to be scouted), training, etc. But there are other relevant details like how a player is likely to get along with other players that affect game outcomes.

wolfgke 10y ago
> But there are other relevant details like how a player is likely to get along with other players that affect game outcomes. Time for creating mathematical models for predictors of how people are getting along... :-)

fallous 10y ago

Expertise is knowledge within a problem domain. Art, and more importantly wisdom, is in the implementation and execution.

tempestn 10y ago

This is a great lesson for hackers in general. Many of us, myself included, have a tendency to expect data to automatically win arguments. I mean, if something is irrefutably true with the data to prove it, that's the end of the discussion, right? But just as in this case, in reality, it's not even the beginning. Whether you're trying to convince your manager to go in a particular direction, or you're trying to motivate your employees to rally behind a cause, the data will help you decide where to go, but you need the right story to get people to follow you there. And just because it's a "story" doesn't mean it's some BS you make up to get people to agree with you. It's more about considering the other person's point of view (their poorly-documented public API, if you like), and building an interface between that and your data.

pimlottc 10y ago
The best manager I ever had was great at this. He was a former developer who understood the tech but knew how to speak the executives' language. You could convince him with a good tech story and then he'd turn around and sell to his bosses with a good business story.
- danieltillett 10y ago
  Domain translators are undervalued. Anyone who can effectively translate the different needs and desires between groups is worth their weight in gold.
  - ethbro 10y ago
    Agreed! I think this also cuts to the fact that data doesn't win arguments. Data + time spent studying and understanding data wins. Most people can't or don't want to take time (it's inefficient). So they make decisions based on estimates / stories / approximations / biases / gut and the world keeps spinning. Which means your goal should be to most accurately frame your data in terms of their stories (translating as much or little as they require).
thaumasiotes 10y ago

SixSigma 10y ago

No matter how good your analysis, data collection only told you what happened in the past.

vwelling 10y ago

For us nerds who want to effect change, and who are convinced data should be all you need, the book "Switch: How to Change Things When Change Is Hard" by Chip and Dan Heath is a highly recommended read. It'll tell you all about the story, or as they call it: the elephant.