Well, thresholding based on colours is really as basic as it gets. Of course, a lot of the problems come from the quality of the video you're working with. As

Well, thresholding based on colours is really as basic as it gets. Of course, a lot of the problems come from the quality of the video you're working with. As for automated player stat tracking, it's probably not easy, but I think it is doable. Computer vision and object tracking is definitely already being applied to american football, although I'm not sure if it's being applied to the ball and players (it's being applied to the field and stadium to show scores and yard lines). Although, there would be a lot of huge problems, such as video feed quality, the video feed itself (you probably don't want one with tv channel overlays, lots of zoomed-in shots, replays, etc., which would make it difficult to find a good feed), and probably more problems which will pop up as you try to code the program. It would probably be useful to have some kind of partnership with whatever authority in charge of match broadcasting. Examples of improvements: pre-process the image and adjust the colour balance, etc. You can track more than just colour. Relax the colour thresholds, and include thresholds for area, density, etc. You might also want to automatically calculate the threshold values, rather than hand coding them in. The wiki page for segmentation ( http://en.wikipedia.org/wiki/Segmentation_(image_processing) http://en.wikipedia.org/wiki/Segmentation_(image_processing) ) lists plenty of alternatives to colour thresholding too. Since a video is just a series of images, it would make sense to use previous images to help track anything in a video. A Kalman filter ( http://en.wikipedia.org/wiki/Kalman_filter http://en.wikipedia.org/wiki/Kalman_filter ) with condensation tracking ( http://en.wikipedia.org/wiki/Condensation_algorithm http://en.wikipedia.org/wiki/Condensation_algorithm ) can statistically predict object positions and should also be able to cope with fast changes in direction. Also useful for frames where the ball disappears behind a player or something. Stuff like image moments ( http://en.wikipedia.org/wiki/Image_moment http://en.wikipedia.org/wiki/Image_moment ) can describe objects despite movement, rotation and scaling, so that would be useful if the camera zooms in/out. I'm probably also missing out a whole bunch of more advanced vision stuff. I'm no expert, sorry. I don't think soccernomics would be quite as useful as moneyball though, since players in soccer depend on each other way too much for individual stats to mean much, while in baseball, the players pretty much play as individuals, which is why it's so easy to track stats and compare players. Edit: Also, to see some examples of the problem I mentioned in my previous post, the video identifies a circle inside the penalty box as the ball. And the ball is actually yellow, but that's only clear when the camera zooms in, so nothing is being detected during those shots. I feel like I may be getting a bit too critical of the program, so apologies to the author. I really do think it's quite an awesome demo despite everything I mentioned.