3 ms·

Can't they technically be supplemented to report their confidence level along with their output? Afaict there's ways to do it.

• _heimdall 3y ago
  But what does a confidence level even mean for an LLM? "Truthfulness" would be limited by the original training set, compounded by the fact that the LLM loses context when effectively compressing the original dataset down to the LLM's final model. A confidence level could be helpful to know how much competing data was considered by the LLM when predicting each word, but it would say nothing of truthfulness or even accuracy relative to the original training data.