2 ms·

You could be right that it’s a fundamental difference in what’s going on, but each word is a new iteration/token and most errors are more than a single token.

by Retric 2y ago

You could be right that it’s a fundamental difference in what’s going on, but each word is a new iteration/token and most errors are more than a single token. I’m assuming it’s like a converging vs diverging series in how well the LLM recovers from problems. Where the boundary between the two states is arbitrarily small, but OpenAI etc has fine tuned the system so you generally see the converging / confabulation types of errors even if the difference is just a slight change in the scale of error.

Nevermark 2y ago
> I’m assuming it’s like a converging vs diverging series in how well the LLM recovers from problems. That's a really good point. Similar causes diverging in effect, a phase change or not, depending on whether a critical feedback threshold is hit or not. That could certainly be a factor. With increasing doses of some drugs, humans do progress from being just a bit wonky, to full on lost in space! But I think confabulation might be on a different continuum too. We fill in recalled memories with made up details all the time. Our memory storage is lossy, associative and overlapping. Our recall is always an imperfect combination of actual incidence memory filled out by similar memories. We must confabulate to recall. Statistically, sometimes we over confabulate.