5 ms·

Dataset: Ten Years of NFL Plays Analyzed, Visualized, Quizzified

75 points by glaugh 13y ago

cubecul 13y ago
"You got 2 out of 5 answers correct. When you try this quiz with a sorry quiz-taker like you, that’s the result you’re going to get." Sassy. Jesus Christ.
- lejohnq 13y ago
  Richard Sherman [1] must have written this quiz. [1] http://www.youtube.com/watch?v=yjOkTib5eVQ http://www.youtube.com/watch?v=yjOkTib5eVQ
- glaugh 13y ago
  OP here Made the message a bit friendlier for future folks, thanks for the feedback :)
  - hansy 13y ago
    "You got 4 out of 5 answers correct. When you try this quiz with a sorry quiz-taker like you, that’s the result you’re going to get. [That's a joke, we think you did fine :)]"
  - cubecul 13y ago
    Haha it was totally fine and funny, just a little surprising. Didn't expect to see something with so much attitude in a page like that.
burntsushi 13y ago
If you're willing to limit yourself to the last five years, you can avoid the pain of parsing those free-form text descriptions with nflgame[1] or nfldb[2]. Disclaimer: I am the author of those tools. We've been slowly building up a small community of people using it. In fact, I'm currently in a small fantasy playoff league that's running off nflgame. They also include live updates while games are playing. I'd be curious to see if you get any substantially different results using structured data (nflgame gets it from NFL.com's JSON feed) as opposed to parsing the text descriptions. [1] - https://github.com/BurntSushi/nflgame https://github.com/BurntSushi/nflgame [2] - https://github.com/BurntSushi/nfldb https://github.com/BurntSushi/nfldb
- glaugh 13y ago
  OP here. That is awesome. I'd definitely recommend people use this over the download at the bottom of the blog post, it's really painful to parse the free text (there's a lot of weird edge cases). I'll add a link to this stuff in the post. edit: clarity update: link added to post
  - burntsushi 13y ago
    > it's really painful to parse the free text I can only imagine. I've tried motivating myself to do it a few times so I could increase the amount of data in nfldb (I believe they are available in one form or another all the way back to 1999), but it's a rather daunting task when there are so many statistical categories.[1] [1] - https://github.com/BurntSushi/nfldb/wiki/Statistical-categories https://github.com/BurntSushi/nfldb/wiki/Statistical-categor...
  - burntsushi 13y ago
    > update: link added to post Thank you so much for helping to spread the word. I really appreciate it!
- jmduke 13y ago
  Oh man, this is just the best -- I can't wait to play around with this.
  - burntsushi 13y ago
    Come on IRC/FreeNode on #nflgame if you have any questions or need help.
- weixiyen 13y ago
  Wow, is this being updated each year to account for any changes to json/xml schema by NFL.com?
  - burntsushi 13y ago
    It's JSON. I've been an active maintainer since I released it two years ago. Proof is in the issue tracker and the wiki. :-) (But there haven't been much---if any---changes to the JSON feed's structure. At most, they add some statistical categories.)
jib 13y ago
The 4th and 2 on the 2 argument bothers me every time someone brings it up. To me it feels like its an incorrect use of statistics. EV isn't the end all - kicking the field goal has way lower variance. If your goal is winning consistently then giving up high variance high EV plays for low variance slightly lower EV plays is often the right choice, as a football season is made up of a very low number of discrete events. About 45% of all games finish with a spread of 7 or less according to a quick search. Making a play that has a close to 50% chance of making you be down 3 points is costing you a lot of the margin if you think you are a close favorite. You can't win more in football but you can lose a sure win, so if you believe you are say a 3-4 point favorite then the right play is to take the field goal every time - giving up the safe points means you take half the games and make them a crap shoot.
- gojomo 13y ago
  Indeed, the pure point-differential EV isn't enough. And beyond variance, there are mental factors involved: cascading micro-rewards from scoring, thinkability of a win or loss, social dominance. I believe the NYTimes "4th Down Bot" actually performs a likelihood-of-winning analysis, as well.
- m_myers 13y ago
  The question specifies "early in the game"; the correct call can and will vary in a late game or late first half situation (and of course depends on the strengths and weaknesses of the teams involved). As Bill Barnwell of Grantland is fond of saying, in the first half, your only goal is to maximize your expected points. You don't know exactly how many points you're going to need in the game until the end.
  - loganfrederick 13y ago
    The Bill Barnwell aphorism reads pretty empty to me. You could just as easily flip it to: "In the first half, your only goal is to minimize your opponent's expected points. You don't know exactly how many points they will need until the end." His phrase doesn't really have any impact on the argument being made here unless you really dissect the relative importance of offense versus defense in the game, which requires a lot more evidence to make a case one way or the other.
gojomo 13y ago
Fun stuff! You might want to make it clear you want the decisions most likely to succeed, not the decision most common among professional coaches (who are presumably optimizing some other form of career-stability-against-criticism).
vacri 13y ago
5/5. Looks like I choose the right thing to do each time... except... which run should I be calling :)
neovive 13y ago
Do coaches have access to real-time data on the field or in the cooridnator's box? If so, if they followed the datasets exactly, would the results reverse themselves over time due to adjustments by offense or vice versa?
- JackFr 13y ago
  There is some game theory involved. If everyone ALWAYS ran for two point conversions, the defense could just ignore the pass and put 11 in the box and stuff the run. The its most likely that the optimal solution is a mixed strategy; you're going to need to mix it up to keep defenses honest.
socrates1998 13y ago
This sounds really good and NFL teams would definitely want something like this, but even coaches would wary about using mostly data. For example, let's say you are the coach of the Patriots and you have been running the ball very successfully the past few games, even winning games because of your running game. And it's 4th down and 2 yards to go against the Broncos. The data says, run the ball. Especially since you have done it well in the past. However, you forgot to take into account the stud defensive tackle that has just started playing really well for the Broncos. So, you try to run the ball and you lose the game. This is just one example of the inability of data to deal with match-ups and schemes. As both a person who likes data and coached football, I would love the integration of the two, but football has too many variables. If you have all this data, you are actually going to make the wrong decision because the matchup is bad for your team. Matchups and schemes trump data.
- JackFr 13y ago
  Additionally, it could be that the statistics show a result that is an artifact of coaches decisions. If a particular decision is chosen by the pro coaches extremely rarely, but has a moderately better expected result, it could be that the typical expected result is very poor, but the only times coaches make the decision is when they observe a decisive matchup or a special situation -- making it appear statistically like that decision is the better one. (Like looking at basketball shooting percentages, and wondering why centers don't take all the shots given how high their FG%s are.)
  - socrates1998 13y ago
    That is an excellent point. Coaches do what they think will be successful. If they can't run the ball well, then they don't run the ball, skewing the data. Also, tendencies change from week to week. You really need a deep understanding of football to get the right, correct data for that week. This is why data hasn't dominated football, you need to have tons of knowledge before anything else.
    - the_watcher 13y ago
jackschultz 13y ago
Hey guys, I did this a few months ago. At least attempted to organize the data from the descriptions on Advanced NFL Stats. (https://github.com/jackschultz/nfl-data https://github.com/jackschultz/nfl-data). Turns out there were tons of special cases. Just curious as to how you decided to organize the data.
viveksodera 13y ago
Numberfire (www.numberfire.com) may have this data, if not more, for their fantasy football tool.
MengYuanLong 13y ago
I am going to a group discussion tomorrow about first-refusal rights in the NFL and happened to do a brief naive analysis of extra point vs 2 point conversions earlier today. In brief, the EV of a 2 point conversion was .91 while an extra point was .99. That said, for every 19 extra point attempts there was only one two point conversion attempt. Frankly, I am all for variance so I am rooting for the more ambitious two-point attempt.
market_hacker 13y ago
I think there may be a problem with this kind of analysis - it seems to me that the "riskier" plays (2 point conversion, going for it, etc.) - are more likely attempted when coaches think they will work - not randomly. To really do a fair analysis of expectancy you would need trials where the play selection is chosen randomly. Anyone else agree with me?

75 points by glaugh 13y ago

"You got 2 out of 5 answers correct. When you try this quiz with a sorry quiz-taker like you, that’s the result you’re going to get." Sassy. Jesus Christ.

lejohnq 13y ago
Richard Sherman [1] must have written this quiz. [1] http://www.youtube.com/watch?v=yjOkTib5eVQ http://www.youtube.com/watch?v=yjOkTib5eVQ
glaugh 13y ago
OP here Made the message a bit friendlier for future folks, thanks for the feedback :)
- hansy 13y ago
  "You got 4 out of 5 answers correct. When you try this quiz with a sorry quiz-taker like you, that’s the result you’re going to get. [That's a joke, we think you did fine :)]"
- cubecul 13y ago
  Haha it was totally fine and funny, just a little surprising. Didn't expect to see something with so much attitude in a page like that.

burntsushi 13y ago

If you're willing to limit yourself to the last five years, you can avoid the pain of parsing those free-form text descriptions with nflgame[1] or nfldb[2]. Disclaimer: I am the author of those tools. We've been slowly building up a small community of people using it. In fact, I'm currently in a small fantasy playoff league that's running off nflgame. They also include live updates while games are playing. I'd be curious to see if you get any substantially different results using structured data (nflgame gets it from NFL.com's JSON feed) as opposed to parsing the text descriptions. [1] - https://github.com/BurntSushi/nflgame https://github.com/BurntSushi/nflgame [2] - https://github.com/BurntSushi/nfldb https://github.com/BurntSushi/nfldb

glaugh 13y ago
OP here. That is awesome. I'd definitely recommend people use this over the download at the bottom of the blog post, it's really painful to parse the free text (there's a lot of weird edge cases). I'll add a link to this stuff in the post. edit: clarity update: link added to post
- burntsushi 13y ago
  > it's really painful to parse the free text I can only imagine. I've tried motivating myself to do it a few times so I could increase the amount of data in nfldb (I believe they are available in one form or another all the way back to 1999), but it's a rather daunting task when there are so many statistical categories.[1] [1] - https://github.com/BurntSushi/nfldb/wiki/Statistical-categories https://github.com/BurntSushi/nfldb/wiki/Statistical-categor...
- burntsushi 13y ago
  > update: link added to post Thank you so much for helping to spread the word. I really appreciate it!
jmduke 13y ago
Oh man, this is just the best -- I can't wait to play around with this.
- burntsushi 13y ago
  Come on IRC/FreeNode on #nflgame if you have any questions or need help.
weixiyen 13y ago
Wow, is this being updated each year to account for any changes to json/xml schema by NFL.com?
- burntsushi 13y ago
  It's JSON. I've been an active maintainer since I released it two years ago. Proof is in the issue tracker and the wiki. :-) (But there haven't been much---if any---changes to the JSON feed's structure. At most, they add some statistical categories.)

jib 13y ago

The 4th and 2 on the 2 argument bothers me every time someone brings it up. To me it feels like its an incorrect use of statistics. EV isn't the end all - kicking the field goal has way lower variance. If your goal is winning consistently then giving up high variance high EV plays for low variance slightly lower EV plays is often the right choice, as a football season is made up of a very low number of discrete events. About 45% of all games finish with a spread of 7 or less according to a quick search. Making a play that has a close to 50% chance of making you be down 3 points is costing you a lot of the margin if you think you are a close favorite. You can't win more in football but you can lose a sure win, so if you believe you are say a 3-4 point favorite then the right play is to take the field goal every time - giving up the safe points means you take half the games and make them a crap shoot.

gojomo 13y ago
Indeed, the pure point-differential EV isn't enough. And beyond variance, there are mental factors involved: cascading micro-rewards from scoring, thinkability of a win or loss, social dominance. I believe the NYTimes "4th Down Bot" actually performs a likelihood-of-winning analysis, as well.
m_myers 13y ago
The question specifies "early in the game"; the correct call can and will vary in a late game or late first half situation (and of course depends on the strengths and weaknesses of the teams involved). As Bill Barnwell of Grantland is fond of saying, in the first half, your only goal is to maximize your expected points. You don't know exactly how many points you're going to need in the game until the end.
- loganfrederick 13y ago
  The Bill Barnwell aphorism reads pretty empty to me. You could just as easily flip it to: "In the first half, your only goal is to minimize your opponent's expected points. You don't know exactly how many points they will need until the end." His phrase doesn't really have any impact on the argument being made here unless you really dissect the relative importance of offense versus defense in the game, which requires a lot more evidence to make a case one way or the other.

gojomo 13y ago

Fun stuff! You might want to make it clear you want the decisions most likely to succeed, not the decision most common among professional coaches (who are presumably optimizing some other form of career-stability-against-criticism).

vacri 13y ago

5/5. Looks like I choose the right thing to do each time... except... which run should I be calling :)

neovive 13y ago

Do coaches have access to real-time data on the field or in the cooridnator's box? If so, if they followed the datasets exactly, would the results reverse themselves over time due to adjustments by offense or vice versa?

JackFr 13y ago
There is some game theory involved. If everyone ALWAYS ran for two point conversions, the defense could just ignore the pass and put 11 in the box and stuff the run. The its most likely that the optimal solution is a mixed strategy; you're going to need to mix it up to keep defenses honest.

socrates1998 13y ago

This sounds really good and NFL teams would definitely want something like this, but even coaches would wary about using mostly data. For example, let's say you are the coach of the Patriots and you have been running the ball very successfully the past few games, even winning games because of your running game. And it's 4th down and 2 yards to go against the Broncos. The data says, run the ball. Especially since you have done it well in the past. However, you forgot to take into account the stud defensive tackle that has just started playing really well for the Broncos. So, you try to run the ball and you lose the game. This is just one example of the inability of data to deal with match-ups and schemes. As both a person who likes data and coached football, I would love the integration of the two, but football has too many variables. If you have all this data, you are actually going to make the wrong decision because the matchup is bad for your team. Matchups and schemes trump data.

JackFr 13y ago
Additionally, it could be that the statistics show a result that is an artifact of coaches decisions. If a particular decision is chosen by the pro coaches extremely rarely, but has a moderately better expected result, it could be that the typical expected result is very poor, but the only times coaches make the decision is when they observe a decisive matchup or a special situation -- making it appear statistically like that decision is the better one. (Like looking at basketball shooting percentages, and wondering why centers don't take all the shots given how high their FG%s are.)
- socrates1998 13y ago
  That is an excellent point. Coaches do what they think will be successful. If they can't run the ball well, then they don't run the ball, skewing the data. Also, tendencies change from week to week. You really need a deep understanding of football to get the right, correct data for that week. This is why data hasn't dominated football, you need to have tons of knowledge before anything else.
  - the_watcher 13y ago

jackschultz 13y ago

Hey guys, I did this a few months ago. At least attempted to organize the data from the descriptions on Advanced NFL Stats. (https://github.com/jackschultz/nfl-data https://github.com/jackschultz/nfl-data). Turns out there were tons of special cases. Just curious as to how you decided to organize the data.

viveksodera 13y ago

Numberfire (www.numberfire.com) may have this data, if not more, for their fantasy football tool.

MengYuanLong 13y ago

I am going to a group discussion tomorrow about first-refusal rights in the NFL and happened to do a brief naive analysis of extra point vs 2 point conversions earlier today. In brief, the EV of a 2 point conversion was .91 while an extra point was .99. That said, for every 19 extra point attempts there was only one two point conversion attempt. Frankly, I am all for variance so I am rooting for the more ambitious two-point attempt.

market_hacker 13y ago

I think there may be a problem with this kind of analysis - it seems to me that the "riskier" plays (2 point conversion, going for it, etc.) - are more likely attempted when coaches think they will work - not randomly. To really do a fair analysis of expectancy you would need trials where the play selection is chosen randomly. Anyone else agree with me?