I feel that the use of moving objects weakened the case in the example, such as requiring object persistence priors. The paper also did not report theoretical m

I feel that the use of moving objects weakened the case in the example, such as requiring object persistence priors. The paper also did not report theoretical maximum scores. Instead, I would have liked to see a planning scenario: 1. Objects are randomly placed such that all circles are reachable without hitting exes. 2. Objects do not move. 3. The objective function is the total points collected, discounted by time. In this case, which is essentially the traveling salesman problem[1], it is possible to analytically compute the optimal strategy using discrete optimization methods (e.g. dynamic programming[2]). The optimal strategy gives us an absolute basis for algorithm evaluation. Next, divide the playing space into groups (maybe via spectral clustering[3]). Use dynamic programming to determine the order in which to visit groups, and automated planning for the sequence of moves to reach the groups. After group planning comes step planning. Build a state predictor that maps a the current state and a set of steps to a new state, where the steps have been determined through automated planning to reach an object. Again, use something like dynamic programming to determine the optimal series of plans to visit all the objects within the group. This would be a hybrid system for using NN-derived features to fuel discrete optimization and automated planning algorithms. Such an approach should learn faster than DQNs and generate good solutions faster than dynamic programming. In the more general case of an Atari game, I can see clustering frames into game states, mapping moves to those states, and using automated planning to determine the series of states, and therefore moves, to optimize score. [1]: https://en.wikipedia.org/wiki/Traveling_salesman_problem https://en.wikipedia.org/wiki/Traveling_salesman_problem [2]: https://en.wikipedia.org/wiki/Dynamic_programming https://en.wikipedia.org/wiki/Dynamic_programming [3]: http://scikit-learn.org/stable/modules/clustering.html#spectral-clustering http://scikit-learn.org/stable/modules/clustering.html#spect...