3 ms·

Author does give concrete examples of precisely how and why these models end up RL'd to be sycophants.