3 ms·
Well, they're not really playing text-based adventures. The fun and challenge of most adventures is that you don't necessarily know all the possible actions, an
by jcranmer 11y ago
Well, they're not really playing text-based adventures. The fun and challenge of most adventures is that you don't necessarily know all the possible actions, and you have to guess what possible state transitions might exist. Many adventures make this more devilish by purposefully being coy about stuff (e.g., one game had a puzzle solved with "use stick on door"--the stick is a stick of dynamite).
What the paper studied was not that sort of text-based adventure, from what I can tell. Their AIs had a list of all possible actions, which is the sort of thing that could easily by solved by a simple reinforcement learner given a few thousand runs. Instead of making it more difficult by removing the list of actions, they made it more difficult by varying the success condition. In effect, what they've shown is that their algorithm does a better job of figuring out what the success condition is given a textual description, not that their algorithm does a better job of figuring out plausible actions.
- marssaxman 11y agoPeople do that sort of thing on purpose? That was the aspect of text adventures that ultimately drove me away from the whole format!
- eli_gottlieb 11y ago>Well, they're not really playing text-based adventures. The fun and challenge of most adventures is that you don't necessarily know all the possible actions, and you have to guess what possible state transitions might exist. Making the environment only partially observable is fairly normal for RL papers. Of course, having a binary success condition encoded as, "Did you reach state Accept of this very extremely large, partially-observable deterministic (potentially pushdown instead of finite) state machine?" does make it hard to use RL, since last I heard, RL requires that the reward feature be continuously valued and that it actually vary when incremental progress towards the goal is made.