While I have yet to read the paper, I don't see how this is a good idea. One of the fundamental ideas of RL is MDPs, and that is still around because problems t

While I have yet to read the paper, I don't see how this is a good idea. One of the fundamental ideas of RL is MDPs, and that is still around because problems that aren't approximable as MDPs are very hard. With MDPs, your goal is to understand the state and action values so that you don't have care about the history or sequences. If you throw MDPs away you are getting rid of a very useful inductive bias and might just be making your problem a lot harder.