4 ms·

If you want to call context "memory", then sure, but that's not what anyone means when they say the word. We don't build our world model fresh with every senten

by timr 3y ago

If you want to call context "memory", then sure, but that's not what anyone means when they say the word. We don't build our world model fresh with every sentence someone says to us, nor do we have to communicate our complete knowledge of conversational state to another human by repeating the entire prior conversation with every new exchange. It's obviously different in a fundamental way. > My bet is on learned world models because I’m not convinced there’s magic in our physical world. You don't need to bet, and it has nothing to do with "magic". They quite literally have no ability to have a "world model" -- it's just a text generator, producing tokens. There's no working set memory, other than the text you pass into it. It should hopefully be obvious to you that when you write, you're not simply emitting one word at a time. You have a complete mental model of whatever you're discussing, stored in working memory, and it's persistent. We also update that model with every interaction we have. The point of my post was that as soon as you take on a harder problem than simulating language, the lack of intelligence slaps you in the face. It turns out that understandable, coherent free-text responses is not magic, and the surprising result is that human language is regular enough that you can statistically simulate "intelligence" with a few hundred million free parameters.

famouswaffles 3y ago
>We don't build our world model fresh with every sentence someone says to us Neither do LLMs. The state for the current text perhaps. Definitely not the entire world model(s) which is learnt from the training process and stored in its weights. >They quite literally have no ability to have a "world model" You keep repeating this so let's get one thing straight. You're wrong. You're just wrong. I'm not trying to convince you of my opinion. This has been empirically observed and tested multiple times. https://www.neelnanda.io/mechanistic-interpretability/othello https://www.neelnanda.io/mechanistic-interpretability/othell... https://adamkarvonen.github.io/machine_learning/2024/01/03/chess-world-models.html https://adamkarvonen.github.io/machine_learning/2024/01/03/c... You're literally saying absolute nonsense with a high level of confidence. When an LLM does this, somehow it's a "hallucination". Why are you different ?
baq 3y ago
You keep saying LLMs just produce tokens as if it was trivial, whereas it’s the key result that those models produce tokens at all (on that we seem to agree, at least) and how they do that is more or less unknown. You have a hidden assumption somewhere that the mechanism doesn’t matter, but in fact the opposite is true: it’s the only thing that matters. Yes the model is static and doesn’t update weights all the time, so what? I’m not saying it’s a sentient being, I’m saying it has world models discovered and embedded in the parameters and a working (as in, writeable, short term) memory.

by timr 3y ago

famouswaffles 3y ago

>We don't build our world model fresh with every sentence someone says to us Neither do LLMs. The state for the current text perhaps. Definitely not the entire world model(s) which is learnt from the training process and stored in its weights. >They quite literally have no ability to have a "world model" You keep repeating this so let's get one thing straight. You're wrong. You're just wrong. I'm not trying to convince you of my opinion. This has been empirically observed and tested multiple times. https://www.neelnanda.io/mechanistic-interpretability/othello https://www.neelnanda.io/mechanistic-interpretability/othell... https://adamkarvonen.github.io/machine_learning/2024/01/03/chess-world-models.html https://adamkarvonen.github.io/machine_learning/2024/01/03/c... You're literally saying absolute nonsense with a high level of confidence. When an LLM does this, somehow it's a "hallucination". Why are you different ?

baq 3y ago

You keep saying LLMs just produce tokens as if it was trivial, whereas it’s the key result that those models produce tokens at all (on that we seem to agree, at least) and how they do that is more or less unknown. You have a hidden assumption somewhere that the mechanism doesn’t matter, but in fact the opposite is true: it’s the only thing that matters. Yes the model is static and doesn’t update weights all the time, so what? I’m not saying it’s a sentient being, I’m saying it has world models discovered and embedded in the parameters and a working (as in, writeable, short term) memory.