2 ms·
> But humans can have a discussion about what is natural and what is not, and decide to mitigate it. (1) accuracy or fit for purpose, not natural, is the issue
by dragonwriter 3y ago
> But humans can have a discussion about what is natural and what is not, and decide to mitigate it.
(1) accuracy or fit for purpose, not natural, is the issue.
(2) evaluation, not having a discussion, is the issue.
(3) LLMs can definitely do evaluation, and probably can also have discussions and decide to, if given a proper prompt and access to ground truth in the first case, and if also used in a properly constructed agent framework, for the second.
(4) if you did all that, and used LLM decisions and not human decisions to guide retraining, I suspect with current LLMs you’d actually have made the problem worse not better, because you’d reduce the influence of higher-capacity and more diverse humans on the training cycle and increase the influence of less diverse, lower capacity LLMs.