4 ms·

Show HN: World Cup 2018 Predictions with Bayesian ML

26 points by lum 8y ago

lum 8y ago
Short technical explanation: We've extended a pretty cool skill-based model of comparison outcomes called TrueSkill Through Time [1], developed by Microsoft Research. Compared to TrueSkill, our model is able to learn the dynamics of football teams' skill in a more flexible way, and therefore we achieve slightly more accurate predictions. It also makes for great visualizations! For those of you who know the Elo rating system: our model is similar but beats Elo by ~10% [2] Our goal is to use the kickoff.ai platform to continuously evaluate and improve our models. We hope to become the FiveThirtyEight of football! [3] Links to our Euro 2016 model & show HN: https://news.ycombinator.com/item?id=11868863 https://news.ycombinator.com/item?id=11868863 http://euro2016.kickoff.ai http://euro2016.kickoff.ai [1] https://www.microsoft.com/en-us/research/publication/trueskill-through-time-revisiting-the-history-of-chess/ https://www.microsoft.com/en-us/research/publication/trueski... [2] in terms of log-loss, compared to a baseline that gives probability 1/3 to each outcome. [3] Actually, FiveThirtyEight also has football predictions (albeit only for club competitions): https://projects.fivethirtyeight.com/soccer-predictions/ https://projects.fivethirtyeight.com/soccer-predictions/
- ajra 8y ago
  Awesome project, I love the use of TrueSkill Through Time here! Really interesting that it beats out Elo as well. FiveThirtyEight actually posted their WC predictions earlier today [1]. Finding where the two models diverge is a fun exercise -- they have Morocco as a 40% favorite tomorrow while your model has Iran as a 46% favorite. It would be interesting to test the model by applying the Kelly Criterion [2] when you have an odds advantage to a fake initial $1000 bankroll and seeing where you end up at the end of the tournament. If Iran does end up being a 46% favorite tomorrow then that Kelly test will start to look really good, really quickly - the market is only giving Iran a ~25% chance right now! [1] https://projects.fivethirtyeight.com/2018-world-cup-predictions/matches/ https://projects.fivethirtyeight.com/2018-world-cup-predicti... [2] http://www.elem.com/~btilly/kelly-criterion/ http://www.elem.com/~btilly/kelly-criterion/ for the background, http://www.albionresearch.com/kelly/ http://www.albionresearch.com/kelly/ for a useful calculator
  - lum 8y ago
    Thanks! Regarding the Kelly Criterion—that's a great idea, I'll look into it! I think we were off to a good start after Morocco vs Iran (for which we had significantly different predictions and got it right), but lost quite a bit with Danemark vs Peru (we had Peru as favorite).
noso 8y ago
It will be interesting to see the Portugal v Spain result, now the manager has been sacked. Looks really good and I love this stuff. Great work!
gakos 8y ago
Sweet project!
senatorobama 8y ago
What did you predict for ARG v ISL?
- noso 8y ago
  I think it was: ARG 73% to win for a draw 19% ISL 8% to win
  - lum 8y ago
    Yes, exactly!

26 points by lum 8y ago

lum 8y ago

Short technical explanation: We've extended a pretty cool skill-based model of comparison outcomes called TrueSkill Through Time [1], developed by Microsoft Research. Compared to TrueSkill, our model is able to learn the dynamics of football teams' skill in a more flexible way, and therefore we achieve slightly more accurate predictions. It also makes for great visualizations! For those of you who know the Elo rating system: our model is similar but beats Elo by ~10% [2] Our goal is to use the kickoff.ai platform to continuously evaluate and improve our models. We hope to become the FiveThirtyEight of football! [3] Links to our Euro 2016 model & show HN: https://news.ycombinator.com/item?id=11868863 https://news.ycombinator.com/item?id=11868863 http://euro2016.kickoff.ai http://euro2016.kickoff.ai [1] https://www.microsoft.com/en-us/research/publication/trueskill-through-time-revisiting-the-history-of-chess/ https://www.microsoft.com/en-us/research/publication/trueski... [2] in terms of log-loss, compared to a baseline that gives probability 1/3 to each outcome. [3] Actually, FiveThirtyEight also has football predictions (albeit only for club competitions): https://projects.fivethirtyeight.com/soccer-predictions/ https://projects.fivethirtyeight.com/soccer-predictions/

ajra 8y ago
Awesome project, I love the use of TrueSkill Through Time here! Really interesting that it beats out Elo as well. FiveThirtyEight actually posted their WC predictions earlier today [1]. Finding where the two models diverge is a fun exercise -- they have Morocco as a 40% favorite tomorrow while your model has Iran as a 46% favorite. It would be interesting to test the model by applying the Kelly Criterion [2] when you have an odds advantage to a fake initial $1000 bankroll and seeing where you end up at the end of the tournament. If Iran does end up being a 46% favorite tomorrow then that Kelly test will start to look really good, really quickly - the market is only giving Iran a ~25% chance right now! [1] https://projects.fivethirtyeight.com/2018-world-cup-predictions/matches/ https://projects.fivethirtyeight.com/2018-world-cup-predicti... [2] http://www.elem.com/~btilly/kelly-criterion/ http://www.elem.com/~btilly/kelly-criterion/ for the background, http://www.albionresearch.com/kelly/ http://www.albionresearch.com/kelly/ for a useful calculator
- lum 8y ago
  Thanks! Regarding the Kelly Criterion—that's a great idea, I'll look into it! I think we were off to a good start after Morocco vs Iran (for which we had significantly different predictions and got it right), but lost quite a bit with Danemark vs Peru (we had Peru as favorite).

noso 8y ago

It will be interesting to see the Portugal v Spain result, now the manager has been sacked. Looks really good and I love this stuff. Great work!

gakos 8y ago

Sweet project!

senatorobama 8y ago

What did you predict for ARG v ISL?

noso 8y ago
I think it was: ARG 73% to win for a draw 19% ISL 8% to win
- lum 8y ago
  Yes, exactly!