Interesting idea and good results, but I don't think there's any natural language work being done here. The descriptive sentences are only there as human readab

Interesting idea and good results, but I don't think there's any natural language work being done here. The descriptive sentences are only there as human readable labels for features extracted from templates, and the actual performance increase is coming from these custom features and giving the agent intermediate rewards. But I still like idea, on x86 'INC' is just a human readable label for 0xCD, and we instruct our processor to do a series of tasks. The difference is on an x86, we specify the instructions and we get the reward (yay). Here you specify the meta-instructions and the agent is rewarded for completing those. The Atari game's end goal reward is sort of irrelevant to the agent, its real goal is completing each meta-instruction we have given it. Only the instruction giver knows the end-goal and the sequence of instructions to give. Perhaps you could teach a second agent to be the instruction giver, the first agent the instruction performer (gets rewarded for completing subtasks only), and together they solve the problem. The job of the first agent still seems intractable for a game like Montezuma's revenge. The real natural language work will be if you can actually use human descriptive sentences to create the subtasks without making hand-made templates.