4 ms·

>the most massive LLMs do not even attempt to abstract and manipulate even basic concepts then how does this work? H: translate the following sentence to in

by knome 4y ago

>the most massive LLMs do not even attempt to abstract and manipulate even basic concepts then how does this work? H: translate the following sentence to instead use olympic curling related metaphors ( do not explicitly mention curling, and do not use similes in forming the translation ): man:"I've been spinning my tires all day without getting anywhere" woman:"running out of gas?" man:"nah, I'm gassed up, geared up, oiled and revving. I just can't seem to get in the right lane, you know?" woman:"anything knocking?" man:"nah, I'm tuned just fine. I can't seem to get my GPS to connect, you know?" woman:"take a few laps without a destination, see if you can figure out the lay of the land" man:"you know, that might help. thanks barbara" R: man: "I've been sweeping the ice all day without hitting the target." . woman: "Losing momentum?" . man: "Nah, I've got the energy, the technique, and the focus. I just can't seem to find the right path, you know?" . woman: "Anything coming in your way?" . man: "Nah, I'm on point with my aim. I just can't seem to get my strategy to align, you know?" . woman: "Maybe glide a few times without aiming specifically, see if you can figure out the ice conditions." . man: "You know, that might help. Thanks, Barbara." I find it unlikely that these larger models do anything less than abstract and manipulate concepts. ( my apologies to anyone that has seen me paste this same example more than once, I find it a very good and succinct example for the topic )

toss1 4y ago
Ask it about actual concepts of specific domains, not merely idioms. If anything, these are idiom engines. In the middle of the road, it outputs very sensible, grammatical, and mostly accurate text. It looks impressive, and can be genuinely helpful. As the same questions get into more obscure stuff, it starts making errors, then on anything near advanced cases, it hallucinates/confabulates/lies. Even with simple math, you get this: Q: what is 17 plus 34? A: 17 plus 34 equals 51. (correct) Q: what is 297438 plus 927465? A: 297438 plus 927465 equals 1224903. (correct) Q: what is 38460 times 2398756? A: 38460 times 2398756 equals 9218928960. (wrong, correct is 92256155760) It is merely outputting a very grammatical word salad that has the same relationship to the actual concepts as a broken clock to the time (right twice a day; this is just better with insanely more parameters). In the areas where it has ingested massive amounts and has billions of relevant vectors, it'll output good stuff. But in the edges, where the input is more scarce, it'll just guess, and that is were you see that there is no abstraction there. I've observed it doing the same thing with legal and scientific topics that I have or know experts. When there is a great body of text that it has ingested, the predictions and grammar engine and idiom are amazing, sometimes astonishing, and often useful. But the kinds of errors show that it has no shred of a concept. It if did, it would not have the kind of problems with facts. We would not get entirely hallucinated/confabulated references to entire papers, authors, APIs. etc., including math answers. We'd get something like "I should have that knowledge, but it is out of scope". Your example is very cool, but pretty much the same as translating from English to French, it's just car idiom to curling idiom. When the LLM translates "Car" to "voiture", it is not abstracting the concept of a car, it is making a prediction. When we translate as novices in a second language, we have to hunt around for the vocabulary equivalent, and when we speak a second language, we aren't translating, but speaking from our own concepts. LLMs are not human, and they learn and produce nothing like humans, but humans have an extremely powerful tendency to anthropomorphize and infer qualities that are not there. Also, from an entirely different angle, in every paper on LLM technology I've read or skimmed, I've seen zero reference to any technology that would abstract and wield concepts. So, I find it extremely unlikely that these models even come close to anything resembling use of abstractions.
- wcoenen 4y ago
  > I've seen zero reference to any technology that would abstract and wield concepts. ChatGPT (and I assume other LLMs as well) uses an embeddings layer. This is how it can translate concepts between domains. It is working with abstractions in a high dimensional semantic space. https://openai.com/blog/new-and-improved-embedding-model https://openai.com/blog/new-and-improved-embedding-model
  - toss1 4y ago
    Thanks, that's good news, and looks like a step in the right direction. they say: >>Embeddings are numerical representations of concepts converted to number sequences, which make it easy for computers to understand the relationships between those concepts. Beyond that claim, there is nearly no information on how this works, and the examples give hints of not any kind of concept, but an additional abstraction layer of clustering words with similar meanings. It is not the abstraction of an understanding, but a more far-reaching thesaurus lookup. (edit) It's an astonishingly good map of words, and this makes it better. The clusters may sort of map resembling concepts, but the kinds of errors it makes show that it's not actually wielding concepts to think. That said, a journey of a thousand miles begins with a single step, and this looks like a good step.

by knome 4y ago

toss1 4y ago

Ask it about actual concepts of specific domains, not merely idioms. If anything, these are idiom engines. In the middle of the road, it outputs very sensible, grammatical, and mostly accurate text. It looks impressive, and can be genuinely helpful. As the same questions get into more obscure stuff, it starts making errors, then on anything near advanced cases, it hallucinates/confabulates/lies. Even with simple math, you get this: Q: what is 17 plus 34? A: 17 plus 34 equals 51. (correct) Q: what is 297438 plus 927465? A: 297438 plus 927465 equals 1224903. (correct) Q: what is 38460 times 2398756? A: 38460 times 2398756 equals 9218928960. (wrong, correct is 92256155760) It is merely outputting a very grammatical word salad that has the same relationship to the actual concepts as a broken clock to the time (right twice a day; this is just better with insanely more parameters). In the areas where it has ingested massive amounts and has billions of relevant vectors, it'll output good stuff. But in the edges, where the input is more scarce, it'll just guess, and that is were you see that there is no abstraction there. I've observed it doing the same thing with legal and scientific topics that I have or know experts. When there is a great body of text that it has ingested, the predictions and grammar engine and idiom are amazing, sometimes astonishing, and often useful. But the kinds of errors show that it has no shred of a concept. It if did, it would not have the kind of problems with facts. We would not get entirely hallucinated/confabulated references to entire papers, authors, APIs. etc., including math answers. We'd get something like "I should have that knowledge, but it is out of scope". Your example is very cool, but pretty much the same as translating from English to French, it's just car idiom to curling idiom. When the LLM translates "Car" to "voiture", it is not abstracting the concept of a car, it is making a prediction. When we translate as novices in a second language, we have to hunt around for the vocabulary equivalent, and when we speak a second language, we aren't translating, but speaking from our own concepts. LLMs are not human, and they learn and produce nothing like humans, but humans have an extremely powerful tendency to anthropomorphize and infer qualities that are not there. Also, from an entirely different angle, in every paper on LLM technology I've read or skimmed, I've seen zero reference to any technology that would abstract and wield concepts. So, I find it extremely unlikely that these models even come close to anything resembling use of abstractions.

wcoenen 4y ago
> I've seen zero reference to any technology that would abstract and wield concepts. ChatGPT (and I assume other LLMs as well) uses an embeddings layer. This is how it can translate concepts between domains. It is working with abstractions in a high dimensional semantic space. https://openai.com/blog/new-and-improved-embedding-model https://openai.com/blog/new-and-improved-embedding-model
- toss1 4y ago
  Thanks, that's good news, and looks like a step in the right direction. they say: >>Embeddings are numerical representations of concepts converted to number sequences, which make it easy for computers to understand the relationships between those concepts. Beyond that claim, there is nearly no information on how this works, and the examples give hints of not any kind of concept, but an additional abstraction layer of clustering words with similar meanings. It is not the abstraction of an understanding, but a more far-reaching thesaurus lookup. (edit) It's an astonishingly good map of words, and this makes it better. The clusters may sort of map resembling concepts, but the kinds of errors it makes show that it's not actually wielding concepts to think. That said, a journey of a thousand miles begins with a single step, and this looks like a good step.