14 ms·
Fair enough. It would be interesting/instructive to construct (relatively simple) examples where we can see that they’re broken :-)
by ssivark 6y ago
Fair enough. It would be interesting/instructive to construct (relatively simple) examples where we can see that they’re broken :-)
- xamuel 6y agoEasy to come up with examples using exotic money-related constructions. Suppose there's something called a "superdollar". If you have a superdollar, you can use it to create an arbitrary number of dollars for yourself, any time you want, which you can trade for goods and services. If you want, you can also trade the superdollar itself. Now picture an environment with two buttons, one of which always rewards you one dollar, and the other of which always rewards you one superdollar. Shoe-horning this environment into traditional RL, you'd have to assign the superdollar button some finite reward, say a million. But then you would mislead the traditional-RL-agent into thinking a million dollars was as good as one superdollar, which clearly is not true.