What's the difference between the critic model and the reward model? In the diagram they show both. EDIT: Is the idea that the critic model learns via the PPO

What's the difference between the critic model and the reward model? In the diagram they show both. EDIT: Is the idea that the critic model learns via the PPO process and gives a value estimate to prefixes of the responses?