11 ms·
So how is it that tree-search algorithms like AlphaZero or MuZero don't drift ? I understand that the searching part allows to reduce the epsilon errors, but wi
by lairv 3y ago
So how is it that tree-search algorithms like AlphaZero or MuZero don't drift ? I understand that the searching part allows to reduce the epsilon errors, but with tree search of bounded depth, drifting should still occur right ?
- polygamous_bat 3y agoOne primary difference is between a discrete (think chess/go moves) and a continuous (think steering wheel input) control space. Even theoretically, the error in one is bounded linearly while the other can have an exponential error with respect to control horizon.
- YeGoblynQueenne 3y agoIs that the important difference? The article is talking about sensor error if I read that right, so there's not much opportunity for that in a game environment. It's not like one needs sensors to know the state of a board game, or even an atari game. The world is really its own _perfect_ model. That's basically why all the RL approaches for game playing don't work nearly as well in the real world: because in the real world policy is a function of state, action, error.