4 ms·

This is one of those comments that begs for actually using LLMs. Yes, an advanced LLM like GPT-4 will absolutely show evidence of an abstracted world model, cor

This is one of those comments that begs for actually using LLMs. Yes, an advanced LLM like GPT-4 will absolutely show evidence of an abstracted world model, correctly answering questions about where person X thinks object Y is, or what relation two people have to each other, or playing a text adventure game while clearly following an internal map it has created for itself.

graypegg 4y ago
So I get that the evidence is that you can do insanely complex things with consistency, like playing a text adventure. You pick up a thing and it stays in your inventory etc, ergo, it’s modelled a character with an inventory in a text adventure. vvv But, coming at this as *respectfully and curiously* as I can here: ^^^ I guess I’m a little more skeptical about attributing magic to something that I know is only working from words as a source. Like if it’s seen NOUN really close to VERB a lot, it can “assume” (read: bias output) that NOUN will VERB in the context it sees it in, that’s an abstraction! But a weak one right? Language is pretty inaccurate. The fact that NLP was a whole booming research field a few years ago tells me that just parsing language consistently is a difficult problem. So, I know I can’t give you any convincing argument there since we’re already mentioning how people are giving untestable definitions of “reasoning” and “abstraction” in this thread. And im no better in that respect here, BUT I guess my best inkling of what feels “off” is a lack of precision, inherent to the medium in which it operates which makes it feel like a true abstraction is impossible in an LLM.
- cjbprime 4y ago
  > something that I know is only working from words as a source. Your intuition about the architecture might be causing the distrust. Yes, the ends of the model are words in and words out (although GPT-4 is now multi-modal and can accept images as input too), but that doesn't mean the calculations in the middle are dealing in words. The "language" in "large language model" is misleading. It's a large generalized model that imports and exports language.
  - graypegg 4y ago
    That is totally possible, the explosion of stuff around this space has kind of blindsided me, so I don’t have much intuition for precisely how these models are built. It all still feels rather “attribute magic to complexity” to say that there’s some human element skimmed off the patterns of our language, hidden in the coefficients somewhere. I’m also not a magician, so really who am I to say :)

graypegg 4y ago

So I get that the evidence is that you can do insanely complex things with consistency, like playing a text adventure. You pick up a thing and it stays in your inventory etc, ergo, it’s modelled a character with an inventory in a text adventure. vvv But, coming at this as *respectfully and curiously* as I can here: ^^^ I guess I’m a little more skeptical about attributing magic to something that I know is only working from words as a source. Like if it’s seen NOUN really close to VERB a lot, it can “assume” (read: bias output) that NOUN will VERB in the context it sees it in, that’s an abstraction! But a weak one right? Language is pretty inaccurate. The fact that NLP was a whole booming research field a few years ago tells me that just parsing language consistently is a difficult problem. So, I know I can’t give you any convincing argument there since we’re already mentioning how people are giving untestable definitions of “reasoning” and “abstraction” in this thread. And im no better in that respect here, BUT I guess my best inkling of what feels “off” is a lack of precision, inherent to the medium in which it operates which makes it feel like a true abstraction is impossible in an LLM.

cjbprime 4y ago
> something that I know is only working from words as a source. Your intuition about the architecture might be causing the distrust. Yes, the ends of the model are words in and words out (although GPT-4 is now multi-modal and can accept images as input too), but that doesn't mean the calculations in the middle are dealing in words. The "language" in "large language model" is misleading. It's a large generalized model that imports and exports language.
- graypegg 4y ago
  That is totally possible, the explosion of stuff around this space has kind of blindsided me, so I don’t have much intuition for precisely how these models are built. It all still feels rather “attribute magic to complexity” to say that there’s some human element skimmed off the patterns of our language, hidden in the coefficients somewhere. I’m also not a magician, so really who am I to say :)