Isn't it even a little more fraught than that? I.e, the sophistication and worth of these models are directly relative to the crude and subjective human labor o

Isn't it even a little more fraught than that? I.e, the sophistication and worth of these models are directly relative to the crude and subjective human labor of feedback upon them. That is, the RLHF part?