This is my personal opinion, it is summarized from experience. It is probably arguable though. ES needs to learn from a population, the size of which is non-tr

This is my personal opinion, it is summarized from experience. It is probably arguable though. ES needs to learn from a population, the size of which is non-trivial for hard problems. To better evaluate population fitness, each individual performs multiple roll-outs and report the average scores as its fitness. Therefore, when an ES algorithm and an RL algorithm are both guaranteed to learn, ES is probably not as sample efficient as (some) RL algorithms. On the other hand, ES tends to be stable and robust, one needs less trials to determine if the algorithm/hyper-parameters/network work for an agent when compared with RL algorithms. So if we count the samples needed for one to be confident that her/his method works reliably, ES is better.