My article goes on how predictive entropy, measuring uncertainty in transformer models’ token predictions—drives hallucinations in LLMs, leading to outputs like

My article goes on how predictive entropy, measuring uncertainty in transformer models’ token predictions—drives hallucinations in LLMs, leading to outputs like fabricated facts or events. I’m curious if broader uncertainty (e.g., epistemic uncertainty from knowledge gaps) is the real bottleneck for AGI from the public perspective and fellow researchers. Current LLMs struggle with novel scenarios, suggesting a need for adaptive systems, like stateful architectures and uncertainty handling in someway. What do you think: is mastering uncertainty beyond entropy the key to AGI? Any promising approaches?