3 ms·
There are tweaks to move choice like that, but it's all evaluation time. None of that happens in training. Zero times in the RL loop does katago see a two stone
by kadoban 1mo ago
There are tweaks to move choice like that, but it's all evaluation time. None of that happens in training. Zero times in the RL loop does katago see a two stone game against a weaker player.
You can kind of tweak towards play this metric or that, but it's not the same.
- Alpha3031 1mo agoMaybe not to the extent you mentioned earlier (I definitely agree that it's not "focused" on this) but there is definitely asymmetric playout training in the training data (see comments on lightvector/KataGo issues #39 and #162 mentioning it for example), that is presumably how they got the tweak for playoutDoublingAdvantage (i.e. actually having a few thousand of the millions upon millions of training games be games where playouts have been doubled for advantage).
- kadoban 1mo agoYeah, thanks, I forgot that existed. That definitely weakens my point quite a bit. It's still not _quite_ the same thing because an advantage in playouts is not a great model for how a stronger/weaker player dynamic actually works, but it is something for sure, way more than the nothing that I said exists.