In reinforcement learning problems, significant improvement has been found from approximating the distribution of rewards for a particular action, rather than j

In reinforcement learning problems, significant improvement has been found from approximating the distribution of rewards for a particular action, rather than just estimating the mean reward. DeepMind's research suggests that a similar model is at work in the brains of mice, mediated by dopamine. Each neuron has a different level of optimism/pessimism when interpreting new data, which leads them to capture different part of the reward distribution. Roughly, one neuron will converge on the 20th percentile reward, while another converges on the 90th percentile (technically, each neuron tracks an expectile rather than a percentile, but the intuition is the same). This is a significant break from the previous model, which said that all dopamine neurons roughly just estimate the mean.