"[Hallucinations] don't form from out-of-distribution samples — they arise due to spurious attractor states on the Energy Landscape. In other words, 'noisy bump

"[Hallucinations] don't form from out-of-distribution samples — they arise due to spurious attractor states on the Energy Landscape. In other words, 'noisy bumps' in the loss. So in the physics approach, to characterize the training error, we ask how sensitive is the free energy to noise. This is why in statistical physics, the training and generalization errors are not defined distributionally. That is, we don't define them using a training set and a test set. Instead, the errors are found by using the free energy as a generating function and taking the appropriate partial derivative." https://www.linkedin.com/posts/charlesmartin14_talktochuck-theaiguy-activity-7307260688509349889-RHvF/ https://www.linkedin.com/posts/charlesmartin14_talktochuck-t...