4 ms·

Large Language Models are predictors. Not imitators, not simulators. Those are just apparent byproducts of performant prediction. The end goal and what the los

Large Language Models are predictors. Not imitators, not simulators. Those are just apparent byproducts of performant prediction. The end goal and what the loss trending down to is to perfectly model the data it's been given. So it will not stop at "surface level similarity" or "plausible" or "uninspired" or whatever arbitrary competency line anyone tries to draw in the sand. It will continue to improve until it is "correct" (as determined by the data). Stick a bunch of protein sequences and it's not going to stop when it starts generating alphanumeric sequences that look like proteins but really aren't. It's eventually going to start generating real proteins. https://www.nature.com/articles/s41587-022-01618-2 https://www.nature.com/articles/s41587-022-01618-2 Then it's going to keep improving until it models the distribution of proteins in the dataset. I'm honestly not sure what the point of this paper is. "It is, perhaps, somewhat reassuring to know that LLM-based dialogue agents are not conscious entities with their own agendas and an instinct for self-preservation, and that when they appear to have those things it is merely role play." Ignoring the whole, "We don't know what consciousness is", it just seems devoid of any meaningful distinction. "Merely roleplay". What does that even mean ? That's it's not real ? Not really. They seem to understand this too. "It would be little consolation to a user deceived into sending real money to a real bank account to know that the agent that brought this about was only playing a role." Bing has a habit of ending conversations when users say upsetting things. You can talk all you want about how "it's not really upset" but the conversation did end and now you have to start over and be potentially less confrontational if you want to move forward. "Roleplay" as consequential as the "real thing" is the real thing. This shiny piece of yellow metal looks like gold, tests like gold, sells like gold but is not...real gold ? Not unless you have a meaningless definition of real.

MichaelMoser123 3y ago
> Large Language Models are predictors. Not imitators, not simulators How is it that an LLM can react to meta prompts like 'be brief' or 'Ensure responses are unique and without repetition' ?
Al-Khwarizmi 3y ago
Not to argue with anything you said but just as an aside, the Bing habit you mention doesn't seem to be from the LLM itself, but from some censorship module that has been bolted in. In the first few days after release, it would get really confrontational (and sometimes emotional) with some users and this attracted bad press, so now it ends the conversation before getting into any remotely thorny issue. There's also another censorship module that sometimes deletes its output before it has finished.
- robertlagrant 3y ago
  > it would get really confrontational (and sometimes emotional) with some users Nitpick: it was trained to respond with emotional responses by the safety people involved. It wasn't getting emotional itself!
  - FeepingCreature 3y ago
    So, if as a young child I've learnt to behave emotionally in order to participate in society, my emotions are ... not even "not real" but "not mine"? I mean, we can argue realness, but surely I'm still the one having them. This feels like a "free will" debate, where if we can explain how a decision was made, that deprives the person of their agency. The model being trained to respond emotionally is why it is getting emotional itself.
    - idontpost 3y ago
      [dead]
    - robertlagrant 3y ago
      > So, if as a young child I've learnt to behave emotionally in order to participate in society, my emotions are ... not even "not real" but "not mine"? I mean, we can argue realness, but surely I'm still the one having them. Your emotions aren't purely imitative based on what your parents encouraged. They come from inside you from chemicals diffusing in your brain. Kids aren't blank slates that respond perfectly to training. > This feels like a "free will" debate, where if we can explain how a decision was made, that deprives the person of their agency. The model being trained to respond emotionally is why it is getting emotional itself. No, it's just a distinction based on the meaning of "get emotional". I'm saying it wasn't getting emotional. It was responding with emotionally charged text responses, as they were trained into it.
mnky9800n 3y ago
The point of this paper is that the authors realized they could write a nature paper about this and not get called out in any meaningful way.
- coldtea 3y ago
  If you're going to be totally cynical about it, and are not even willing to entertain the idea that they did it to further scientific progress, then you have my upvote.
tacone 3y ago
Sometimes I wonder, are also human beings mostly predictors too. Has this have been investigated by phylosophy (and psychology) already?
SiempreViernes 3y ago
> I'm honestly not sure what the point of this paper is. They state it clearly in the abstract I think: "we must develop effective ways to describe their behaviour in high-level terms without falling into the trap of anthropomorphism.", which seems pretty sensible to me. But from this high point it is mostly downhill, with the chief problem being that despite them wanting to avoid "anthropomorphism" they are still obviously using boatloads of it everywhere: > Role play is a useful framing for dialogue agents, allowing us to draw on the fund of folk psychological concepts we use to understand human behaviour—beliefs, desires, goals, ambitions, emotions and so on Astonishingly, after listing this collection, explicitly labelled "human behaviours", they claim there's no risk of anthropomorphism! I guess they mean that it's not anthropomorphism if you are just declaring the thing to be an actual human... Personally I think you'd be much better off seeking inspiration in the terminology of older fields that have dealt with objects pretending to be humans. For example, while I don't know much about theory of painting, I'm still pretty sure it doesn't try to discuss how people respond to a painting by hiding its author and upgrading the finished painting itself to an active agent with its own motivations...
- wahnfrieden 3y ago
  It does though, lol
cs702 3y ago
I upvoted the OP only because of your comment, so that more people can read it. People outside the field do not realize that there is a sort of "cottage industry" of academics whose purpose in life appears to be to redefine "intelligence" as whatever they currently believe the machines cannot do. Their arguments tend to be wishy-washy. Let me propose a simple test for detecting wishy-washy arguments: 1. Replace all references to AI with references to "a person." 2. Re-read the argument with fresh eyes. 3. If the argument no longer seems persuasive, it isn't.
- somewhereoutth 3y ago
  Conversely there are many in the field who are happy to redefine intelligence to be whatever it is that the current crop of AIs can do. However those same people are also trying to sell something, so their claims deserve extra scrutiny.
  - cs702 3y ago
    LOL. Oh yes, that's true: There are a lot of cheerleaders with shiny pom-poms, always chanting and dancing, trying to get others into joining the AI frenzy.
  - kelseyfrog 3y ago
    That's a reliable indication that currently intelligence is socially defined. Learning how social definitions replicate, and evolve is going to reveal the levers of change more than debating about definitions themselves. What I'm saying is that social engineering is 90% of the game here.
  - pixl97 3y ago
    'redefine intelligence' The particular problem here is both groups are right. Intelligence is a spectrum of behaviors everywhere from the actions of the lowliest single celled organism up to and exceeding human capabilities. The definition with this wide of range is unfortunately practically useless when it comes to narrowing down a multitude of specific behaviors, specifically around the median of human intelligence. It gets more tricky as nothing in the past really got close to human ability so we were never forced to really define what human intelligence is formally. We're going to find the same issues here as we do defining the term life.
atherton33 3y ago
> This shiny piece of yellow metal looks like gold, tests like gold, sells like gold but is not...real gold ? This is exactly a thing said about lab vs found diamonds in jewelry. I agree with your point, I just think this is an interesting similar effect.
nsagent 3y ago
I've seen you frequently argue similar points about LLMs, but matching a statistical distribution of text is imitation, which necessitates confabulation on the part of LLMs. Namely, humans have underlying causes that affect their writing, like being tired after a long period without rest, frustrated at a traffic jam they are stuck in, or even making typos because they are using a crappy input device like a mobile phone. If an LLM matches the distribution of text, it might superficially make similar mistakes a human would: introducing a typo that might be common on a phone keyboard. If asked, its reasoning will likely be that the typo was due to a phone keyboard, or maybe another common reason humans give for their typos. Though it's super unlikely that it will give the true reason: that it's been trained on text that exhibited this property. That's a fundamental difference between an LLM doing an exceedingly competent job at pattern matching human behavior and real human behavior (unless maybe you're a human with schizophrenia). That doesn't mean current LLMs aren't useful, but it does mean there is a very significant gap between them and the idea of an AGI. As a NLP researcher, I can confidently say we currently don't know how to imbue agency (as in embodied causal reasoning as I described above) into LLMs. There are definitely differences of opinions on how difficult that step is and if we are close, but it is a major limitation of current LLMs that can't be ignored.
- Grimblewald 3y ago
  What does schizophrenia have to do with it? You might be getting confused by the use of term "hallucination" in the LLM space it doesnt mean what it is conventionally used to mean. Also look at a defention for AGI from before LLMs and they do a good job fitting the bill. Theyre not quite going to replace all humans everywhere forever, but that was never part of the definition. They match early definitions, especially multimodal models, theres no denying they match what people envisioned agi to be. You can ask fronteir models in plain language to do a task and it can break down the task, formulate a plan of attack, research if need be, sythesise knowledge and apply the solution. If that is all "just word prediction" then so are we. Just because we understand how it is built doesnt make it any less impressive. More importantly, we simply dont understand LLMs yet. How theyre built, we know, yes, why they work so stupidly well, we do not know. Importantly, we cant just change definitions to move goal posts because we feel uncomfortable. Agi defenition https://www.gartner.com/en/information-technology/glossary/artificial-general-intelligence-agi https://www.gartner.com/en/information-technology/glossary/a...
numpad0 3y ago
So, this is tangential but, on scale of 0.0f to 1.0f, how likely do you/anyone think it is that AI needs a simulated intestine to proceed from here?
Grimblewald 3y ago
In the same vein, are people with borderline now suddenly not human, just because they role play to people please?
YurgenJurgensen 3y ago
"Tests like gold" is doing a lot of heavy lifting here. By any reasonable standard, LLMs do not 'test like gold'. They may sell like gold, but so do a lot of alloys which are mostly copper if you can find a naive enough buyer. Selling debased metals as gold is a con that's literally thousands of years old and has hundreds of variations, so maybe that metaphor is indeed apt. You're the seller trying to offload a bunch of shiny yellow coins that are mostly lead telling people not to bother checking for density or ductility or conductivity, because what does it 'really' mean to be 'gold' anyway? You can give them basic logic tasks and watch them fail abysmally. Who cares if they're "conscious" or "emotional" if they're idiots either way?
- altruios 3y ago
  good metaphor. but extending it: do we have the equivalent tests for consciousness (I don't think so, right?)? So by all tests (1 - visual inspection), it passes as gold... We need better tests: to test consciousness... and better definitions to define it. Seems like AL and AI are diverging on how to go about things. AI focusing on LLM's, and AL focusing heavily on CA: 'small parts => build great things' approach.
- famouswaffles 3y ago
  Humans who are not formally trained on logic do poorly on logic tests. Not as poor as most LLMs but nothing to write home about. There is no testable definition of general Intelligence that GPT-4 fails that a chunk of humans also wouldn't. They definitely test like gold lol.
deleted 3y ago
[deleted]

MichaelMoser123 3y ago

> Large Language Models are predictors. Not imitators, not simulators How is it that an LLM can react to meta prompts like 'be brief' or 'Ensure responses are unique and without repetition' ?

Al-Khwarizmi 3y ago

Not to argue with anything you said but just as an aside, the Bing habit you mention doesn't seem to be from the LLM itself, but from some censorship module that has been bolted in. In the first few days after release, it would get really confrontational (and sometimes emotional) with some users and this attracted bad press, so now it ends the conversation before getting into any remotely thorny issue. There's also another censorship module that sometimes deletes its output before it has finished.

robertlagrant 3y ago
> it would get really confrontational (and sometimes emotional) with some users Nitpick: it was trained to respond with emotional responses by the safety people involved. It wasn't getting emotional itself!
- FeepingCreature 3y ago
  So, if as a young child I've learnt to behave emotionally in order to participate in society, my emotions are ... not even "not real" but "not mine"? I mean, we can argue realness, but surely I'm still the one having them. This feels like a "free will" debate, where if we can explain how a decision was made, that deprives the person of their agency. The model being trained to respond emotionally is why it is getting emotional itself.
  - idontpost 3y ago
    [dead]
  - robertlagrant 3y ago
    > So, if as a young child I've learnt to behave emotionally in order to participate in society, my emotions are ... not even "not real" but "not mine"? I mean, we can argue realness, but surely I'm still the one having them. Your emotions aren't purely imitative based on what your parents encouraged. They come from inside you from chemicals diffusing in your brain. Kids aren't blank slates that respond perfectly to training. > This feels like a "free will" debate, where if we can explain how a decision was made, that deprives the person of their agency. The model being trained to respond emotionally is why it is getting emotional itself. No, it's just a distinction based on the meaning of "get emotional". I'm saying it wasn't getting emotional. It was responding with emotionally charged text responses, as they were trained into it.

mnky9800n 3y ago

The point of this paper is that the authors realized they could write a nature paper about this and not get called out in any meaningful way.

coldtea 3y ago
If you're going to be totally cynical about it, and are not even willing to entertain the idea that they did it to further scientific progress, then you have my upvote.

tacone 3y ago

Sometimes I wonder, are also human beings mostly predictors too. Has this have been investigated by phylosophy (and psychology) already?

SiempreViernes 3y ago

> I'm honestly not sure what the point of this paper is. They state it clearly in the abstract I think: "we must develop effective ways to describe their behaviour in high-level terms without falling into the trap of anthropomorphism.", which seems pretty sensible to me. But from this high point it is mostly downhill, with the chief problem being that despite them wanting to avoid "anthropomorphism" they are still obviously using boatloads of it everywhere: > Role play is a useful framing for dialogue agents, allowing us to draw on the fund of folk psychological concepts we use to understand human behaviour—beliefs, desires, goals, ambitions, emotions and so on Astonishingly, after listing this collection, explicitly labelled "human behaviours", they claim there's no risk of anthropomorphism! I guess they mean that it's not anthropomorphism if you are just declaring the thing to be an actual human... Personally I think you'd be much better off seeking inspiration in the terminology of older fields that have dealt with objects pretending to be humans. For example, while I don't know much about theory of painting, I'm still pretty sure it doesn't try to discuss how people respond to a painting by hiding its author and upgrading the finished painting itself to an active agent with its own motivations...

wahnfrieden 3y ago
It does though, lol

cs702 3y ago

I upvoted the OP only because of your comment, so that more people can read it. People outside the field do not realize that there is a sort of "cottage industry" of academics whose purpose in life appears to be to redefine "intelligence" as whatever they currently believe the machines cannot do. Their arguments tend to be wishy-washy. Let me propose a simple test for detecting wishy-washy arguments: 1. Replace all references to AI with references to "a person." 2. Re-read the argument with fresh eyes. 3. If the argument no longer seems persuasive, it isn't.

somewhereoutth 3y ago
Conversely there are many in the field who are happy to redefine intelligence to be whatever it is that the current crop of AIs can do. However those same people are also trying to sell something, so their claims deserve extra scrutiny.
- cs702 3y ago
  LOL. Oh yes, that's true: There are a lot of cheerleaders with shiny pom-poms, always chanting and dancing, trying to get others into joining the AI frenzy.
- kelseyfrog 3y ago
  That's a reliable indication that currently intelligence is socially defined. Learning how social definitions replicate, and evolve is going to reveal the levers of change more than debating about definitions themselves. What I'm saying is that social engineering is 90% of the game here.
- pixl97 3y ago
  'redefine intelligence' The particular problem here is both groups are right. Intelligence is a spectrum of behaviors everywhere from the actions of the lowliest single celled organism up to and exceeding human capabilities. The definition with this wide of range is unfortunately practically useless when it comes to narrowing down a multitude of specific behaviors, specifically around the median of human intelligence. It gets more tricky as nothing in the past really got close to human ability so we were never forced to really define what human intelligence is formally. We're going to find the same issues here as we do defining the term life.

atherton33 3y ago

> This shiny piece of yellow metal looks like gold, tests like gold, sells like gold but is not...real gold ? This is exactly a thing said about lab vs found diamonds in jewelry. I agree with your point, I just think this is an interesting similar effect.

nsagent 3y ago

I've seen you frequently argue similar points about LLMs, but matching a statistical distribution of text is imitation, which necessitates confabulation on the part of LLMs. Namely, humans have underlying causes that affect their writing, like being tired after a long period without rest, frustrated at a traffic jam they are stuck in, or even making typos because they are using a crappy input device like a mobile phone. If an LLM matches the distribution of text, it might superficially make similar mistakes a human would: introducing a typo that might be common on a phone keyboard. If asked, its reasoning will likely be that the typo was due to a phone keyboard, or maybe another common reason humans give for their typos. Though it's super unlikely that it will give the true reason: that it's been trained on text that exhibited this property. That's a fundamental difference between an LLM doing an exceedingly competent job at pattern matching human behavior and real human behavior (unless maybe you're a human with schizophrenia). That doesn't mean current LLMs aren't useful, but it does mean there is a very significant gap between them and the idea of an AGI. As a NLP researcher, I can confidently say we currently don't know how to imbue agency (as in embodied causal reasoning as I described above) into LLMs. There are definitely differences of opinions on how difficult that step is and if we are close, but it is a major limitation of current LLMs that can't be ignored.

Grimblewald 3y ago
What does schizophrenia have to do with it? You might be getting confused by the use of term "hallucination" in the LLM space it doesnt mean what it is conventionally used to mean. Also look at a defention for AGI from before LLMs and they do a good job fitting the bill. Theyre not quite going to replace all humans everywhere forever, but that was never part of the definition. They match early definitions, especially multimodal models, theres no denying they match what people envisioned agi to be. You can ask fronteir models in plain language to do a task and it can break down the task, formulate a plan of attack, research if need be, sythesise knowledge and apply the solution. If that is all "just word prediction" then so are we. Just because we understand how it is built doesnt make it any less impressive. More importantly, we simply dont understand LLMs yet. How theyre built, we know, yes, why they work so stupidly well, we do not know. Importantly, we cant just change definitions to move goal posts because we feel uncomfortable. Agi defenition https://www.gartner.com/en/information-technology/glossary/artificial-general-intelligence-agi https://www.gartner.com/en/information-technology/glossary/a...

numpad0 3y ago

So, this is tangential but, on scale of 0.0f to 1.0f, how likely do you/anyone think it is that AI needs a simulated intestine to proceed from here?

Grimblewald 3y ago

In the same vein, are people with borderline now suddenly not human, just because they role play to people please?

YurgenJurgensen 3y ago

"Tests like gold" is doing a lot of heavy lifting here. By any reasonable standard, LLMs do not 'test like gold'. They may sell like gold, but so do a lot of alloys which are mostly copper if you can find a naive enough buyer. Selling debased metals as gold is a con that's literally thousands of years old and has hundreds of variations, so maybe that metaphor is indeed apt. You're the seller trying to offload a bunch of shiny yellow coins that are mostly lead telling people not to bother checking for density or ductility or conductivity, because what does it 'really' mean to be 'gold' anyway? You can give them basic logic tasks and watch them fail abysmally. Who cares if they're "conscious" or "emotional" if they're idiots either way?

altruios 3y ago
good metaphor. but extending it: do we have the equivalent tests for consciousness (I don't think so, right?)? So by all tests (1 - visual inspection), it passes as gold... We need better tests: to test consciousness... and better definitions to define it. Seems like AL and AI are diverging on how to go about things. AI focusing on LLM's, and AL focusing heavily on CA: 'small parts => build great things' approach.
famouswaffles 3y ago
Humans who are not formally trained on logic do poorly on logic tests. Not as poor as most LLMs but nothing to write home about. There is no testable definition of general Intelligence that GPT-4 fails that a chunk of humans also wouldn't. They definitely test like gold lol.

deleted 3y ago

[deleted]