3 ms·

I wouldn't call that "too insane." As others have pointed out, you can get similar results from fine-tuning the RLHF.