3 ms·
I agree with you overall, but while it was relatively simple, DQN was never "elegant". The elegant and most important part of DQN was to use a CNN for learning
by WnZ39p0Dgydaz1 7y ago
I agree with you overall, but while it was relatively simple, DQN was never "elegant". The elegant and most important part of DQN was to use a CNN for learning a good image representation - but CNNs are completely unrelated to RL.
The sole contribution of DQN was to introduce two ugly band-aids, experience replay and a target network, to make the CNN training stable in an RL setting.
So, DQN = CNN (elegant and simple) + ugly band aid
- pabloem 7y agoJust a side note. Probably not very fair to attribue the introduction of experience replay to DQN authors. This technique was popularized in 1992 by Lin[1] and used later in many works to improve performance. The difference, perhaps, is that in DQN, experience replay is not only a way to improve performance, but it's a must to obtain stability. [1] http://www.incompleteideas.net/lin-92.pdf http://www.incompleteideas.net/lin-92.pdf