If you're interested in evaluating and rating/ranking agents, it might be worthwhile checking out DeepMind's multidimensional Elo rating system (https://arxiv.o

If you're interested in evaluating and rating/ranking agents, it might be worthwhile checking out DeepMind's multidimensional Elo rating system (https://arxiv.org/abs/1806.02643 https://arxiv.org/abs/1806.02643) which attempts to solve some of the issues with Elo and Glicko. Most notably, the ability to handle non-transitive interactions (like rock, paper, scissors) and the presence of redundant duplications of matches that might erroneously inflate ratings. Shameless plug, I've created an R implementation of it here: https://dclaz.github.io/mELO/ https://dclaz.github.io/mELO/