4 ms·

Unfortunately, all such efforts are limited by their central assumption - that skill is orderable. This is demonstrably untrue - skill at chess (to use their e

Unfortunately, all such efforts are limited by their central assumption - that skill is orderable. This is demonstrably untrue - skill at chess (to use their example) occurs on several axes, the most obvious of which are opening theory, tactics, and positional play. Players may excel only in certain regions of that space, and it's quite easy to set up a player cycle A->B->C->A, in which each player is more likely to win against one player and lose against another. A players observed 'skill' therefore will depend on any biases in the population at large rather heavily (at the lower levels of chess, the population has overwhelmingly studied opening theory and some tactics) Because of their non-dimensionality, skill ranking algorithms are universally limited to expressing how likely one is to win against an average person of a given skill ranking, rather than the likely outcome of the match about to be played. Sports match prediction techniques are all domain-specific, precisely for this reason (and because substantial sums of money are riding on their predictive effectiveness).

imurray 15y ago
There are models that do learn several latent features for each player [e.g. 1]. And I recently supervised a Master's project motivated by exactly your observation[2]. Models with multiple features per player do make better predictions, but by less than I'd hoped. A result from the Master's project, which surprised me, was quite how much details of the approximate inference mattered — much more so than the choice of model. [1] http://www.cs.toronto.edu/~rpa/adams-dahl-murray-2010a.shtml http://www.cs.toronto.edu/~rpa/adams-dahl-murray-2010a.shtml [2] http://homepages.inf.ed.ac.uk/imurray2/projects/2011_marius_stanescu_msc.pdf http://homepages.inf.ed.ac.uk/imurray2/projects/2011_marius_...
grifaton 15y ago
You can even observe cycles with just two players. In Cambridge (UK), the River Cam is too narrow for side-by-side racing over more than a few hundred yards. Instead, a popular kind of racing has developed called "Bumps". Crews line up along the river with about a boat and a half's length between each other. At a signal, all crews start rowing, and if one boat manages to touch the boat in front ("bumping"), both crews pull over to the side, and the next day they switch places in the starting line up. It's not uncommon to see two boats trade places repeatedly -- this could happen because crew A has a much stronger start than crew B, but that crew B have much more stamina than crew A, so A will always chase down B relatively quickly, but B will always manage to catch A over the course of a longer race.

imurray 15y ago

There are models that do learn several latent features for each player [e.g. 1]. And I recently supervised a Master's project motivated by exactly your observation[2]. Models with multiple features per player do make better predictions, but by less than I'd hoped. A result from the Master's project, which surprised me, was quite how much details of the approximate inference mattered — much more so than the choice of model. [1] http://www.cs.toronto.edu/~rpa/adams-dahl-murray-2010a.shtml http://www.cs.toronto.edu/~rpa/adams-dahl-murray-2010a.shtml [2] http://homepages.inf.ed.ac.uk/imurray2/projects/2011_marius_stanescu_msc.pdf http://homepages.inf.ed.ac.uk/imurray2/projects/2011_marius_...

grifaton 15y ago

You can even observe cycles with just two players. In Cambridge (UK), the River Cam is too narrow for side-by-side racing over more than a few hundred yards. Instead, a popular kind of racing has developed called "Bumps". Crews line up along the river with about a boat and a half's length between each other. At a signal, all crews start rowing, and if one boat manages to touch the boat in front ("bumping"), both crews pull over to the side, and the next day they switch places in the starting line up. It's not uncommon to see two boats trade places repeatedly -- this could happen because crew A has a much stronger start than crew B, but that crew B have much more stamina than crew A, so A will always chase down B relatively quickly, but B will always manage to catch A over the course of a longer race.