5 ms·

I've seen you frequently argue similar points about LLMs, but matching a statistical distribution of text is imitation, which necessitates confabulation on the

I've seen you frequently argue similar points about LLMs, but matching a statistical distribution of text is imitation, which necessitates confabulation on the part of LLMs. Namely, humans have underlying causes that affect their writing, like being tired after a long period without rest, frustrated at a traffic jam they are stuck in, or even making typos because they are using a crappy input device like a mobile phone. If an LLM matches the distribution of text, it might superficially make similar mistakes a human would: introducing a typo that might be common on a phone keyboard. If asked, its reasoning will likely be that the typo was due to a phone keyboard, or maybe another common reason humans give for their typos. Though it's super unlikely that it will give the true reason: that it's been trained on text that exhibited this property. That's a fundamental difference between an LLM doing an exceedingly competent job at pattern matching human behavior and real human behavior (unless maybe you're a human with schizophrenia). That doesn't mean current LLMs aren't useful, but it does mean there is a very significant gap between them and the idea of an AGI. As a NLP researcher, I can confidently say we currently don't know how to imbue agency (as in embodied causal reasoning as I described above) into LLMs. There are definitely differences of opinions on how difficult that step is and if we are close, but it is a major limitation of current LLMs that can't be ignored.

Grimblewald 3y ago
What does schizophrenia have to do with it? You might be getting confused by the use of term "hallucination" in the LLM space it doesnt mean what it is conventionally used to mean. Also look at a defention for AGI from before LLMs and they do a good job fitting the bill. Theyre not quite going to replace all humans everywhere forever, but that was never part of the definition. They match early definitions, especially multimodal models, theres no denying they match what people envisioned agi to be. You can ask fronteir models in plain language to do a task and it can break down the task, formulate a plan of attack, research if need be, sythesise knowledge and apply the solution. If that is all "just word prediction" then so are we. Just because we understand how it is built doesnt make it any less impressive. More importantly, we simply dont understand LLMs yet. How theyre built, we know, yes, why they work so stupidly well, we do not know. Importantly, we cant just change definitions to move goal posts because we feel uncomfortable. Agi defenition https://www.gartner.com/en/information-technology/glossary/artificial-general-intelligence-agi https://www.gartner.com/en/information-technology/glossary/a...
famouswaffles 3y ago
>causes that affect their writing, like being tired after a long period without rest, frustrated at a traffic jam they are stuck in There is nothing special about fatigue or frustration that make it any less modellable than any other implicit structure present in the dataset that it models just fine. >Though it's super unlikely that it will give the true reason It's Super Unlikely humans will give the "true reason" for anything they do. There's a fair bit of research that stated reasons for the decisions we make are often(always?) just post-hoc rationalizations even if you believe otherwise. >very significant gap between them and the idea of an AGI. What is this idea of AGI that there exists a significant gap still ? It certainly isn't the idea of being Artificial and Generally Intelligent. >I can confidently say we currently don't know how to imbue agency (as in embodied causal reasoning I don't see the difference between what you've described and examples like these. https://innermonologue.github.io/ https://innermonologue.github.io/ https://tidybot.cs.princeton.edu/ https://tidybot.cs.princeton.edu/
- stale2002 3y ago
  > What is this idea of AGI that there exists a significant gap still? Because an LLM doesn't simulate the brain. It's a completely different model that simulates something much different. > There is nothing special about fatigue or frustration that make it any less modellable than any other implicit structure Fatigue isn't being modeled. Instead, the predictive text as the result of that fatigue is being models. You are confusing output to input. To make this more clear, imagine a coin flip modeler. A model that just picked a random seed, is completely different from a model that is doing physics calculations based on a coin flipping in the air. Even if both models only output "heads" or "tails" . Same argument applies to language models.
  - famouswaffles 3y ago
    >Because an LLM doesn't simulate the brain. That's pretty moot. You don't need a human brain anyone than a plane needs feathers and to flap wings to fly. >Fatigue isn't being modeled. Instead, the predictive text as the result of that fatigue is being models. Emotion is definitely being modelled. There's nothing random about anything that's happened. Train on protein sequences alone and biological structure and function emerge in the inner layers alone. It doesn't matter that those things are not explicitly stated in the data. Because they implicitly structure it, it gets learnt. https://www.pnas.org/doi/full/10.1073/pnas.2016239118 https://www.pnas.org/doi/full/10.1073/pnas.2016239118 Similarity, emotion is evidently being modelled to a high degree. https://arxiv.org/abs/2307.11760 https://arxiv.org/abs/2307.11760
krainboltgreene 3y ago
> If an LLM matches the distribution of text, it might superficially make similar mistakes a human would It's also important to point out (IMHO) that the current distribution of text that almost every LLM is based on is almost certainly a slice of the Internet which has a very very specific lean to in terms of culture, language, tone, and availability.

Grimblewald 3y ago

What does schizophrenia have to do with it? You might be getting confused by the use of term "hallucination" in the LLM space it doesnt mean what it is conventionally used to mean. Also look at a defention for AGI from before LLMs and they do a good job fitting the bill. Theyre not quite going to replace all humans everywhere forever, but that was never part of the definition. They match early definitions, especially multimodal models, theres no denying they match what people envisioned agi to be. You can ask fronteir models in plain language to do a task and it can break down the task, formulate a plan of attack, research if need be, sythesise knowledge and apply the solution. If that is all "just word prediction" then so are we. Just because we understand how it is built doesnt make it any less impressive. More importantly, we simply dont understand LLMs yet. How theyre built, we know, yes, why they work so stupidly well, we do not know. Importantly, we cant just change definitions to move goal posts because we feel uncomfortable. Agi defenition https://www.gartner.com/en/information-technology/glossary/artificial-general-intelligence-agi https://www.gartner.com/en/information-technology/glossary/a...

famouswaffles 3y ago

>causes that affect their writing, like being tired after a long period without rest, frustrated at a traffic jam they are stuck in There is nothing special about fatigue or frustration that make it any less modellable than any other implicit structure present in the dataset that it models just fine. >Though it's super unlikely that it will give the true reason It's Super Unlikely humans will give the "true reason" for anything they do. There's a fair bit of research that stated reasons for the decisions we make are often(always?) just post-hoc rationalizations even if you believe otherwise. >very significant gap between them and the idea of an AGI. What is this idea of AGI that there exists a significant gap still ? It certainly isn't the idea of being Artificial and Generally Intelligent. >I can confidently say we currently don't know how to imbue agency (as in embodied causal reasoning I don't see the difference between what you've described and examples like these. https://innermonologue.github.io/ https://innermonologue.github.io/ https://tidybot.cs.princeton.edu/ https://tidybot.cs.princeton.edu/

stale2002 3y ago
> What is this idea of AGI that there exists a significant gap still? Because an LLM doesn't simulate the brain. It's a completely different model that simulates something much different. > There is nothing special about fatigue or frustration that make it any less modellable than any other implicit structure Fatigue isn't being modeled. Instead, the predictive text as the result of that fatigue is being models. You are confusing output to input. To make this more clear, imagine a coin flip modeler. A model that just picked a random seed, is completely different from a model that is doing physics calculations based on a coin flipping in the air. Even if both models only output "heads" or "tails" . Same argument applies to language models.
- famouswaffles 3y ago
  >Because an LLM doesn't simulate the brain. That's pretty moot. You don't need a human brain anyone than a plane needs feathers and to flap wings to fly. >Fatigue isn't being modeled. Instead, the predictive text as the result of that fatigue is being models. Emotion is definitely being modelled. There's nothing random about anything that's happened. Train on protein sequences alone and biological structure and function emerge in the inner layers alone. It doesn't matter that those things are not explicitly stated in the data. Because they implicitly structure it, it gets learnt. https://www.pnas.org/doi/full/10.1073/pnas.2016239118 https://www.pnas.org/doi/full/10.1073/pnas.2016239118 Similarity, emotion is evidently being modelled to a high degree. https://arxiv.org/abs/2307.11760 https://arxiv.org/abs/2307.11760

krainboltgreene 3y ago

> If an LLM matches the distribution of text, it might superficially make similar mistakes a human would It's also important to point out (IMHO) that the current distribution of text that almost every LLM is based on is almost certainly a slice of the Internet which has a very very specific lean to in terms of culture, language, tone, and availability.