I don't think it will work well unless you have several actors. IIRC, one of the main factors giving A3C an edge compared to previous solutions, is that multipl

I don't think it will work well unless you have several actors. IIRC, one of the main factors giving A3C an edge compared to previous solutions, is that multiple learners from different episodes contribute experience at the same time. This avoids weights being updated too much in one direction. This is similar to what using batches does in supervised learning.