7 ms·
If the value/policy model is predictive with a dataset containing only amateur games, but fails to generalize to unseen data with professional games, that seems
by emcq 11y ago
If the value/policy model is predictive with a dataset containing only amateur games, but fails to generalize to unseen data with professional games, that seems like a case of overfitting to a dataset only containing amateur games. In this case the expected value network may be different for amateur games than professional games.
Is there something I'm missing?
- argonaut 11y agoSorry, I'm being a little academic. Overfitting is when the model fits to noise or error. Overfitting is not synonymous with "inability to generalize beyond the train and test distribution." For all we know AlphaGo has perfectly fit amateur games, but professional games are on a whole different level
- tel 11y agoAll depends on how you define your data sets I suppose. IGS games include some professional games which, if it is the case that AG is perfectly trained on the amateur mode, were smoothed away.
- argonaut 11y agoAgain, the only way to tell if overfitting specifically (and not other factors that are more likely) is the issue is performance on a held out test set.
- tel 11y agoThat's the only empirical way, yes. We can also just talk about what it would mean in theory, though. In this case, we'd say that AlphaGo is well trained to the training data set sampling distribution but that may be far from the actual world game distribution.
- argonaut 11y agoThat does not relate to overfitting. That problem is closer to "domain adaptation."
- ktRolster 11y agoUnlikely. Some of AlphaGo's moves were lousy moves, even for an amateur.
- argonaut 11y agoYou are implying overfitting is the only reason a model ever performs poorly. That is not true. Just because a model does poorly does not mean overfitting must be the issue.
- etherealmachine 11y agoThe value/policy model includes a few hundred thousand amateur games, and a few hundred million games of self-play. Once AlphaGo beat Fan Hui those would have been games of self-play versus the equivalent of a professional. So overfitting is probably not a problem. I think it's a basic incentive mismatch - MCTS algorithms tend to like close games, whereas humans will try crazy moves when losing to throw off their opponent.
- argonaut 11y agoNeither of those are really evidence overfitting isn't a problem. You tell whether overfitting is a problem by evaluating performance on a held-out test set.
- moistgorilla 11y agoWouldn't a million self play games exacerbate overfitting by learning it's own play style which it initially learned from amateur games? I