2 ms·
Always the same trick of not answering the question and deflecting to „what about humans“. Can you folks not evaluate LLMs as the system they are, without vague
by dgellow 3mo ago
Always the same trick of not answering the question and deflecting to „what about humans“. Can you folks not evaluate LLMs as the system they are, without vague gestures at how a different system behaves?
- Certhas 3mo agoEvaluating LLMs is incredibly difficult. They are categorically different from any other system we have intuition about. That said, I read the question I am replying to as a rhetorical one. If it was meant as a genuine question, curious about the question of meta knowledge, then I misread. Certainly the question is extremely interesting, for both LLMs and humans! But it's also obviously a very difficult one, as we don't even have a clear theory on how "knowing" works in the base case.