7 ms·

Analyze a Soccer Game Using Tensorflow Object Detection and OpenCV (2018)

80 points by wilsonfiifi 8y ago

PaulRobinson 8y ago
I remember when this article was knocking around during last Summer, and came away thinking "nope, that's not how to analy[s|z]e a [football|soccer] game". I have people inside a Premier League club just waiting for me to be able to throw OpenCV at an archive of video and turn it into something useful they can do something with, and have had for a couple of years now. I think in 2019 we might get there, but it isn't quite this. Object detection isn't enough, and drawing paths isn't enough. You need to know which player is which, where on the pitch they are, the phase of the game (knowing who has possession is enough, but knowing current score and minutes of play elapsed is also helpful), how long the player has been on the pitch, etc., etc. At that point you can then start to do some real analysis. It's doable, but not trivial, and this is only the start. Good work on the object detection though. If anybody reading this thinks they know how to do the above and is interested in having an interesting conversation about it, feel free to ping me...
- siquick 8y ago
  There's a part of me that hopes the former Spurs and Leeds keeper Paul Robinson traded in football for a tech career.
  - lordnacho 8y ago
    If that's him he's going to get really annoyed at me: https://www.youtube.com/watch?v=Zi8nzTjJXOI https://www.youtube.com/watch?v=Zi8nzTjJXOI
- johnwyles 8y ago
  I would say from what I have seen in articles surfacing now and then that the NBA has some of the best data analysis and breakdown of players and such. You might look into the companies or efforts doing this for an analog. These guys (SportsVU) popped up last time I read about 5 years ago (looks like they did some of the recent WC as well?): https://www.stats.com/2018-world-cup/ https://www.stats.com/2018-world-cup/ https://en.wikipedia.org/wiki/SportVU https://en.wikipedia.org/wiki/SportVU
  - hassaanm 8y ago
    https://www.secondspectrum.com/ https://www.secondspectrum.com/
- deleted 8y ago
  [deleted]
- atmontague 8y ago
  It's interesting to hear that PL clubs are looking into this. I've been wondering what type of data clubs have been using and how they've been using it. Football (soccer) in general seems like it could benefit from CV; from identifying when a player is offside to analyzing player performance.
- lordnacho 8y ago
  > Object detection isn't enough, and drawing paths isn't enough. You need to know which player is which, where on the pitch they are, the phase of the game (knowing who has possession is enough, but knowing current score and minutes of play elapsed is also helpful), how long the player has been on the pitch, etc., etc. The players have numbers on them on several sides, and you have the priors of what position they play. And then there's skin tone and other features you might be able to use. Time and score has got to be something that's already there? What kinds of things do they want to turn it into?
- jimfleming 8y ago
  I have experience designing these kinds of computer vision systems for other applications. What you describe is very doable. I'd start by isolating each component you want to identify and track. Then break that list down into multiple models that run in multiple passes over the video. While doing this try and group tasks that use the same inputs together, even if the outputs are different. This will help identify what exactly you need to isolate for each model. For example, some models may only need to see the top left of the video which can simplify things. You'll also probably want a context model to determine which frames should be seen by other models. Avoid going all-in with a single end-to-end deep model right away. There are lots of details to work out and it will be easier to iterate as separate components. Ultimately you may eventually end up with a single model to leverage feature sharing and improve results and those details will be relevant.
- kamarg 8y ago
  Do PL players wear those tracker bra looking things like players in the MLS do? If so, could you combine that data to determine who's who?
  - mhh__ 8y ago
    Some do, but I don't think it's available to the FA etc.(It's just for the club to track the players for their own analysis)
  - pbalau 8y ago
    Those things don't provide location information. On top of that, getting location info with the accuracy needed in cases like this is extremely hard.
  - santino79 8y ago
    The majority of PL clubs use them for internal player analysis. A company local to me called STATSports (https://statsports.com/soccer/ https://statsports.com/soccer/) provide the tech for big clubs all over the world including 17 of the 20 English Premier League clubs.
- lbotos 8y ago
  The NBA is already doing this for Basketball: https://moderndata.plot.ly/nba-player-movement-using-plotly-animations/ https://moderndata.plot.ly/nba-player-movement-using-plotly-... I imagine it's slightly harder in football stadiums due to size, but that's a money problem for the amount of cameras.
- ams6110 8y ago
  I saw this and wanted to try to adapt it to hockey, and had the same sort of thoughts. It's a start, but not nearly enough to produce useful game video breakdown for a coach.
- proofofconcept 8y ago
  These problems are all tractable using traditional computer vision methods. There's a sizable body of published academic research from the past couple of decades that addresses tackling these and other computer vision problems in the specific applied context of video analysis of soccer games. Considering the amount of knowledge out there I'm kind of surprised that any clubs at the premier league level are still in a position where they aren't able to extract useful information from their library of game footage. With so much money at stake I figured that even if clubs weren't developing their own analytics tools in-house there would be no shortage of third party tools out there by now. >I have people inside a Premier League club just waiting for me to be able to throw OpenCV at an archive of video How extensive is the footage in their archives? i.e. is it just what winds up in the broadcasts, do they keep every second of footage of every game taken by every camera in the stadium, or is it something in-between?
- ctoth 8y ago
  Combine this with some text generation magic and you suddenly make blind people very happy. Iterate a bit and sports commentators start getting nervous.
t3hprofit 8y ago
"sportsball". I love it.
- johnwyles 8y ago
  I saw that too! They can label the team names but cannot name the other objects "Players" and "Football"?
brookview 8y ago
Unfortunately this is not a 'REAL' analysis at all. Just a homework-ish object detection. It is like equalizing some fact based data reports in an Excel sheet to an advanced neural network model.
nik736 8y ago
Where is the analysis?
nbpalomino 8y ago
Really interesting application of Tensorflow and OpenCV, Kudos for choosing Peru's match in World Cup 2018 after 36 years of waiting.

80 points by wilsonfiifi 8y ago

I remember when this article was knocking around during last Summer, and came away thinking "nope, that's not how to analy[s|z]e a [football|soccer] game". I have people inside a Premier League club just waiting for me to be able to throw OpenCV at an archive of video and turn it into something useful they can do something with, and have had for a couple of years now. I think in 2019 we might get there, but it isn't quite this. Object detection isn't enough, and drawing paths isn't enough. You need to know which player is which, where on the pitch they are, the phase of the game (knowing who has possession is enough, but knowing current score and minutes of play elapsed is also helpful), how long the player has been on the pitch, etc., etc. At that point you can then start to do some real analysis. It's doable, but not trivial, and this is only the start. Good work on the object detection though. If anybody reading this thinks they know how to do the above and is interested in having an interesting conversation about it, feel free to ping me...

siquick 8y ago
There's a part of me that hopes the former Spurs and Leeds keeper Paul Robinson traded in football for a tech career.
- lordnacho 8y ago
  If that's him he's going to get really annoyed at me: https://www.youtube.com/watch?v=Zi8nzTjJXOI https://www.youtube.com/watch?v=Zi8nzTjJXOI
johnwyles 8y ago
I would say from what I have seen in articles surfacing now and then that the NBA has some of the best data analysis and breakdown of players and such. You might look into the companies or efforts doing this for an analog. These guys (SportsVU) popped up last time I read about 5 years ago (looks like they did some of the recent WC as well?): https://www.stats.com/2018-world-cup/ https://www.stats.com/2018-world-cup/ https://en.wikipedia.org/wiki/SportVU https://en.wikipedia.org/wiki/SportVU
- hassaanm 8y ago
  https://www.secondspectrum.com/ https://www.secondspectrum.com/
deleted 8y ago
[deleted]
atmontague 8y ago
It's interesting to hear that PL clubs are looking into this. I've been wondering what type of data clubs have been using and how they've been using it. Football (soccer) in general seems like it could benefit from CV; from identifying when a player is offside to analyzing player performance.
lordnacho 8y ago
> Object detection isn't enough, and drawing paths isn't enough. You need to know which player is which, where on the pitch they are, the phase of the game (knowing who has possession is enough, but knowing current score and minutes of play elapsed is also helpful), how long the player has been on the pitch, etc., etc. The players have numbers on them on several sides, and you have the priors of what position they play. And then there's skin tone and other features you might be able to use. Time and score has got to be something that's already there? What kinds of things do they want to turn it into?
jimfleming 8y ago
I have experience designing these kinds of computer vision systems for other applications. What you describe is very doable. I'd start by isolating each component you want to identify and track. Then break that list down into multiple models that run in multiple passes over the video. While doing this try and group tasks that use the same inputs together, even if the outputs are different. This will help identify what exactly you need to isolate for each model. For example, some models may only need to see the top left of the video which can simplify things. You'll also probably want a context model to determine which frames should be seen by other models. Avoid going all-in with a single end-to-end deep model right away. There are lots of details to work out and it will be easier to iterate as separate components. Ultimately you may eventually end up with a single model to leverage feature sharing and improve results and those details will be relevant.
kamarg 8y ago
Do PL players wear those tracker bra looking things like players in the MLS do? If so, could you combine that data to determine who's who?
- mhh__ 8y ago
  Some do, but I don't think it's available to the FA etc.(It's just for the club to track the players for their own analysis)
- pbalau 8y ago
  Those things don't provide location information. On top of that, getting location info with the accuracy needed in cases like this is extremely hard.
- santino79 8y ago
  The majority of PL clubs use them for internal player analysis. A company local to me called STATSports (https://statsports.com/soccer/ https://statsports.com/soccer/) provide the tech for big clubs all over the world including 17 of the 20 English Premier League clubs.
lbotos 8y ago
The NBA is already doing this for Basketball: https://moderndata.plot.ly/nba-player-movement-using-plotly-animations/ https://moderndata.plot.ly/nba-player-movement-using-plotly-... I imagine it's slightly harder in football stadiums due to size, but that's a money problem for the amount of cameras.
ams6110 8y ago
I saw this and wanted to try to adapt it to hockey, and had the same sort of thoughts. It's a start, but not nearly enough to produce useful game video breakdown for a coach.
proofofconcept 8y ago
These problems are all tractable using traditional computer vision methods. There's a sizable body of published academic research from the past couple of decades that addresses tackling these and other computer vision problems in the specific applied context of video analysis of soccer games. Considering the amount of knowledge out there I'm kind of surprised that any clubs at the premier league level are still in a position where they aren't able to extract useful information from their library of game footage. With so much money at stake I figured that even if clubs weren't developing their own analytics tools in-house there would be no shortage of third party tools out there by now. >I have people inside a Premier League club just waiting for me to be able to throw OpenCV at an archive of video How extensive is the footage in their archives? i.e. is it just what winds up in the broadcasts, do they keep every second of footage of every game taken by every camera in the stadium, or is it something in-between?
ctoth 8y ago
Combine this with some text generation magic and you suddenly make blind people very happy. Iterate a bit and sports commentators start getting nervous.

t3hprofit 8y ago

"sportsball". I love it.

johnwyles 8y ago
I saw that too! They can label the team names but cannot name the other objects "Players" and "Football"?

brookview 8y ago

Unfortunately this is not a 'REAL' analysis at all. Just a homework-ish object detection. It is like equalizing some fact based data reports in an Excel sheet to an advanced neural network model.

nik736 8y ago

Where is the analysis?

nbpalomino 8y ago

Really interesting application of Tensorflow and OpenCV, Kudos for choosing Peru's match in World Cup 2018 after 36 years of waiting.