6 ms·

Language models are nearly AGIs but we don't notice

26 points by ctoth 4y ago

simpleintheory 4y ago
Part of the issue is that we don’t really understand consciousness in humans. I think that’s needed before we can actually see what’s happening with AI. This article reminds me of that Google engineer that called LaMDA sentient.
- kwhitefoot 4y ago
  Intelligence doesn't presuppose consciousness. At least it is plausible that it doesn't. See Blindsight by Peter Watts
- boxed 4y ago
  We don't understand decision making in insects. We're enormously far from understanding humans.
- idiotsecant 4y ago
  I'm not sure that we can ever 'understand' intelligence in humans, let alone something as fluffy as consciousness simply because the systems are so vast, interconnected, and inseparable. It seems more and more apparent that the networks and structures that cause 'us' to emerge are so deeply non-hierarchical and interconnected that labeling parts in a way where we can say this-does-that and that-does-this is not a meaningful idea. It's not like a car engine where a component does one thing, has an input from this system, and produces an output to this system. It's more like a weather system where small deviation in one part of the system can have small and large changes everywhere else, which in turn cause changes of their own, and on and on. I suspect that AGI will first come about as a mishmash of 'expert systems' with some currently incomprehensible glue allowing them all to communicate effectively. I further guess that it's development will be incremental in nature - taking tiny pieces of things that work and putting them together with novel techniques that also worked somewhere else until eventually you get something that thinks back at you.
- grantcas 4y ago
  [dead]
yarg 4y ago
90% of the way there, 90% of the way to go. Just because it gets close to what you think is the turing test, doesn't make it an AGI.
constantcrying 4y ago
The author restricts general to mean restricted to text input and output. Which is obviously ridicolous. By their very nature, being restricted to text input and output these models can not be general. The response to that objection also seems pretty bad. Sure, some humans do not have the prerequisities to do e.g. image recognition, but they have the intellectual ability to do so. A person detached from all their senses and means to express himself is not less intelligent than GPT-3, just because GPT-3 has the ability to consume input and produce output. In fact, by the standards of the article, such a person would be outclassed by any computer programm. And for the second response, saying that GPT-3 could potentially also deal with images or sound, if presented in the right format is just proof that it is not general. Saying maybe it could do more things if we train it on more things is just blind speculation. It could very easily be the case that if you tried to implement sound input in GPT-3, it would significantly worsen it performance on all other tasks. Besides, doesn't GPT-3 still struggle very much with continuity and producing output that is coherent over long spans of text? I am not sure if there is any one good test to judge the literary abilities of AI, but surely long form writing has to be included before it is judged as equivalent...
- profile53 4y ago
  While I agree with most of your point, > By their very nature, being restricted to text input and output these models can not be general. Humans only have five types of I/O, yet are general intelligences. I don’t think having limited I/O means is enough to restrict something from being generally intelligent
  - sudosysgen 4y ago
    GPT's IO is not text, its bytes. It can't handle different byte structures, humans can.
    - yarg 4y ago
      The doesn't seem too relevant - the most important part (in terms of whether or not you're dealing with an intelligence) is how the data is process - not how it's imported.
Imnimo 4y ago
Suggesting that PaLM is human level at common sense reasoning because it does well at the Winograd benchmark is insane. Not least of all because there is significant test set leakage for that benchmark in web corpus training data.
andrepd 4y ago
It's trivially easy to trip up GPT-3 at this stage. What is hard, and requires very careful prompt doctoring, is to make it seem human. A parrot can mimic speech without understanding it, current language models are qualitatively the same. A fancy chatbot but a chatbot nonetheless.
dinkblam 4y ago
The author claims we do "keep shifting the bar", but that simply isn't true. The same criteria for A(G)I still applies for 70 years now. The Turing Test has not been passed by GPT-3 or any other system yet.
- kelseyfrog 4y ago
  I don't know about you, but recently when I bring up the idea of Turning testing AI, I get push back along the lines of, "Turning tests can be gamed." People instead suggest that Winnograd schema are better because they incorporate generalized knowledge. Now that we're getting human level Winnograd scores, we're getting suggestions that images, audio, and video are requirements. If this doesn't look like moving the goalposts, I don't know what is.
  - drowsspa 4y ago
    I think it's a case of "we'll know it when we see it". I agree it looks like moving the goalposts, but I think most people would agree that current LMs just don't feel like AGIs.
    - wizeman 4y ago
      > I think it's a case of "we'll know it when we see it". I agree it looks like moving the goalposts, but I think most people would agree that current LMs just don't feel like AGIs. Heh, that sounds like a dangerous slippery slope. One day we're going to have real, "conscious" (whatever that means) AGI robots walking around us and people are still going to say "they don't feel like AGIs" (which is going to be used as a justification for cruelty, slavery, etc). Perhaps because they wouldn't behave exactly like us (or even if they would!).
      - kelseyfrog 4y ago
        There's the anchored functionalists, the evasive functionalists, and the metaphysical impossiblists. The anchored functionalists, have a threshold at which they will accept AGI and stick to it. The evasive functionalists will move their functional threshold because they feel uneasy about accepting an existence of AGI and the metaphysical impossibilists will never accept AGI on metaphysical grounds ie: it's incompatible with the way the universe is constructed/reality. I agree, there's going to be a point where AGI ends up existing, modifying social relations, and reshaping society and we're still going to be having this conversation. It feels as beyond the pale as someone who would today who would believe in a flat earth, epicycles upon epicycles in order to support non-existence.
FartyMcFarter 4y ago
So what problems do language models solve to a human-like level or higher? I think answering that question should be required as part of any claim that a system is an AGI or nearly there.
- make3 4y ago
  https://arxiv.org/abs/2109.01652 https://arxiv.org/abs/2109.01652 https://arxiv.org/abs/2206.04615 https://arxiv.org/abs/2206.04615
- dinosaurdynasty 4y ago
  They're (probably) better than humans at text prediction https://www.lesswrong.com/posts/htrZrxduciZ5QaCjw/language-models-seem-to-be-much-better-than-humans-at-next https://www.lesswrong.com/posts/htrZrxduciZ5QaCjw/language-m... (which is what they are trained on, so maybe not unreasonable). There's also the idea that GPT-3/etc can produce text that's difficult for humans to distinguish from human-generated text at the lowest levels of quality (think like time cube), which is closer to AGI than say simplistic Markov generators. (How close is anyone's guess)
deleted 4y ago
[deleted]
adampwells 4y ago
it seems that 'making a better' GPT-3 or similar model is like climbing higher up a tall tree and claiming you are closer to the moon... technically true, but...
deleted 4y ago
[deleted]
renewiltord 4y ago
It would be fun to prompt the next language model with "You are connected to a Linux bash terminal. Your next command will be executed on that bash console. Type what you want", feed that into an appropriate VM and then reprompt with "The stderr was $X and the stdout was $Y. Your next command will be executed on the console" and loop. I shall try it on the next one when it comes out. AI alignment be damned. Let's let the baby play with a bomb and see how close it is to AGI if we let it drive itself.
deleted 4y ago
[deleted]
seydor 4y ago
we should also notice that intelligence is not necessarily useful
deleted 4y ago
[deleted]
tombakt 4y ago
Until a language model can develop a generalized solution to a real-world phenomena, it's not even close to AGI. The current iteration of ML algorithms are useful, yes, but not intelligent.
- whatwherewhy 4y ago
  What is a generalized solution to a real-world phenomena? Github Copilot solved my business problem by itself just as I would've done. Is that real-world enough and the solution generalized enough?
  - qayxc 4y ago
    > Github Copilot solved my business problem by itself just as I would've done. Is that real-world enough and the solution generalized enough? No, it isn't. Co-Pilot is unable to provide a rationalisation for the generated code and is incapable of assessing its security or performance properties. It's a very advanced auto-complete that just happens to have been specialised on auto-completing code.
    - dogcomplex 4y ago
      I mean... it can generate comment summaries explaining any block of code, or an entire class. It can generate comprehensive unit tests and usability tests. It can tell you the Big-O operational complexity of any code you (or it) writes, and it can put those all together. Is that not good enough for you..?
      - qayxc 4y ago
        It can't do that. It can try to do that and it might work well in some cases. It is definitely incapable of doing that for any code you throw at it. It also cannot generate comprehensive unit tests - it can can generate unit tests. The definition of "comprehensive" is way too subjective. > Is that not good enough for you..? No, it indeed isn't, but maybe that's because I actually have at least some idea of what happens behind the scenes and how the system works. From the implementation details, I can tell you for a fact that none of what you described can be done by the system in the general case - and especially not correctly. It's hit and miss depending on the input and that's basically a design limitation.
bilsbie 4y ago
I don’t understand why we don’t talk about this kind of stuff more. It’s really amazing.
Jack000 4y ago
LMs aren't AGI, but they show that the search for architecture is essentially settled. What the scaling laws demonstrate is that any architectural improvements you could find manually can be superseded by a slightly larger transformer. LMs lack crucial elements of of human like intelligence - long term memory, short term memory, episodic memory, proprioceptive awareness etc. The task is now to implement these things using transformers.
- krackers 4y ago
  >long term memory, short term memory Doesn't that mean the architecture isn't settled? The current mechanism of appending the output of the model back into the prompt feels like a bit of a hack. I'm only a layman here but it seems transformer models can only propagate information layer-wise, adding some ability to propagate information across time like RNNs do might be useful for achieving longer-term coherence.
avalys 4y ago
There are some activities of daily life that I do without thinking (most of them, in fact). Moving my limbs while walking up the stairs, driving to the corner store and stopping at a traffic light, recognizing a stop sign at a glance, reading the numerals off a mailbox, etc. These are all things that, after you initially learn to do them, get wired up into the fabric of your brain well enough that you can execute them without really thinking about them. They become instinctual. Interestingly, these are all things that deep neural networks have turned out to be quite good at - instinctual things. There is also a category of written and oral communication that I do without thinking - instinctually. I don't think deliberately about the choice of each word when writing this post, and certainly not when speaking out loud. Idioms and turns of phrase emerge without deliberate thought or intent. If my social circle has started using certain terminology habitually (e.g. here in the Bay Area, people have been using the adjective "super" a lot, as in "that's super cool"), I'll find myself using it as well without making any deliberate attempt to do so, sometimes to my own chagrin. And even when the topic is something nominally intellectual, I'll sometimes find myself simply regurgitating the general opinion on this topic that I last read from a trusted source. This seems to exemplify what GPT3 does - a sort of instinctual written communication - and I don't believe it's an example of intelligence any more than a human being able to recognize a stop sign in 100 ms is a sign of intelligence. GPT3 a great pattern-matching engine and it applies pattern-matching on the human language to interpolate a response that is consistent with the patterns it observed. I don't see any evidence that GPT3 can critically evaluate the information it is is pattern-matching - that it can go beyond what literate humans do instinctually. Don't get me wrong - GPT3 is surprising and amazing. But not because it signifies anything about AGI. What's amazing to me about GPT3 is it reveals how much ordinary human written and oral communication is instinctual in the same sense that visual processing and image recognition is.
togaen 4y ago
clickbait nonsense
aaron695 4y ago
inawarminister 4y ago
You know, being able to do all things human can do... with text input and text output? Why does that reminds me of Unix Philosophy where everything is just text files? I wonder if someone has experimented with one of the LLMs to get them to do Unix sysadmin jobs. It's not exactly outputting source codes, but Bash commands should be similar enough right?

26 points by ctoth 4y ago

Part of the issue is that we don’t really understand consciousness in humans. I think that’s needed before we can actually see what’s happening with AI. This article reminds me of that Google engineer that called LaMDA sentient.

kwhitefoot 4y ago
Intelligence doesn't presuppose consciousness. At least it is plausible that it doesn't. See Blindsight by Peter Watts
boxed 4y ago
We don't understand decision making in insects. We're enormously far from understanding humans.
idiotsecant 4y ago
I'm not sure that we can ever 'understand' intelligence in humans, let alone something as fluffy as consciousness simply because the systems are so vast, interconnected, and inseparable. It seems more and more apparent that the networks and structures that cause 'us' to emerge are so deeply non-hierarchical and interconnected that labeling parts in a way where we can say this-does-that and that-does-this is not a meaningful idea. It's not like a car engine where a component does one thing, has an input from this system, and produces an output to this system. It's more like a weather system where small deviation in one part of the system can have small and large changes everywhere else, which in turn cause changes of their own, and on and on. I suspect that AGI will first come about as a mishmash of 'expert systems' with some currently incomprehensible glue allowing them all to communicate effectively. I further guess that it's development will be incremental in nature - taking tiny pieces of things that work and putting them together with novel techniques that also worked somewhere else until eventually you get something that thinks back at you.
grantcas 4y ago
[dead]

yarg 4y ago

90% of the way there, 90% of the way to go. Just because it gets close to what you think is the turing test, doesn't make it an AGI.

constantcrying 4y ago

The author restricts general to mean restricted to text input and output. Which is obviously ridicolous. By their very nature, being restricted to text input and output these models can not be general. The response to that objection also seems pretty bad. Sure, some humans do not have the prerequisities to do e.g. image recognition, but they have the intellectual ability to do so. A person detached from all their senses and means to express himself is not less intelligent than GPT-3, just because GPT-3 has the ability to consume input and produce output. In fact, by the standards of the article, such a person would be outclassed by any computer programm. And for the second response, saying that GPT-3 could potentially also deal with images or sound, if presented in the right format is just proof that it is not general. Saying maybe it could do more things if we train it on more things is just blind speculation. It could very easily be the case that if you tried to implement sound input in GPT-3, it would significantly worsen it performance on all other tasks. Besides, doesn't GPT-3 still struggle very much with continuity and producing output that is coherent over long spans of text? I am not sure if there is any one good test to judge the literary abilities of AI, but surely long form writing has to be included before it is judged as equivalent...

profile53 4y ago
While I agree with most of your point, > By their very nature, being restricted to text input and output these models can not be general. Humans only have five types of I/O, yet are general intelligences. I don’t think having limited I/O means is enough to restrict something from being generally intelligent
- sudosysgen 4y ago
  GPT's IO is not text, its bytes. It can't handle different byte structures, humans can.
  - yarg 4y ago
    The doesn't seem too relevant - the most important part (in terms of whether or not you're dealing with an intelligence) is how the data is process - not how it's imported.

Imnimo 4y ago

Suggesting that PaLM is human level at common sense reasoning because it does well at the Winograd benchmark is insane. Not least of all because there is significant test set leakage for that benchmark in web corpus training data.

andrepd 4y ago

It's trivially easy to trip up GPT-3 at this stage. What is hard, and requires very careful prompt doctoring, is to make it seem human. A parrot can mimic speech without understanding it, current language models are qualitatively the same. A fancy chatbot but a chatbot nonetheless.

dinkblam 4y ago

The author claims we do "keep shifting the bar", but that simply isn't true. The same criteria for A(G)I still applies for 70 years now. The Turing Test has not been passed by GPT-3 or any other system yet.

kelseyfrog 4y ago
I don't know about you, but recently when I bring up the idea of Turning testing AI, I get push back along the lines of, "Turning tests can be gamed." People instead suggest that Winnograd schema are better because they incorporate generalized knowledge. Now that we're getting human level Winnograd scores, we're getting suggestions that images, audio, and video are requirements. If this doesn't look like moving the goalposts, I don't know what is.
- drowsspa 4y ago
  I think it's a case of "we'll know it when we see it". I agree it looks like moving the goalposts, but I think most people would agree that current LMs just don't feel like AGIs.
  - wizeman 4y ago
    > I think it's a case of "we'll know it when we see it". I agree it looks like moving the goalposts, but I think most people would agree that current LMs just don't feel like AGIs. Heh, that sounds like a dangerous slippery slope. One day we're going to have real, "conscious" (whatever that means) AGI robots walking around us and people are still going to say "they don't feel like AGIs" (which is going to be used as a justification for cruelty, slavery, etc). Perhaps because they wouldn't behave exactly like us (or even if they would!).
    - kelseyfrog 4y ago
      There's the anchored functionalists, the evasive functionalists, and the metaphysical impossiblists. The anchored functionalists, have a threshold at which they will accept AGI and stick to it. The evasive functionalists will move their functional threshold because they feel uneasy about accepting an existence of AGI and the metaphysical impossibilists will never accept AGI on metaphysical grounds ie: it's incompatible with the way the universe is constructed/reality. I agree, there's going to be a point where AGI ends up existing, modifying social relations, and reshaping society and we're still going to be having this conversation. It feels as beyond the pale as someone who would today who would believe in a flat earth, epicycles upon epicycles in order to support non-existence.

FartyMcFarter 4y ago

So what problems do language models solve to a human-like level or higher? I think answering that question should be required as part of any claim that a system is an AGI or nearly there.

make3 4y ago
https://arxiv.org/abs/2109.01652 https://arxiv.org/abs/2109.01652 https://arxiv.org/abs/2206.04615 https://arxiv.org/abs/2206.04615
dinosaurdynasty 4y ago
They're (probably) better than humans at text prediction https://www.lesswrong.com/posts/htrZrxduciZ5QaCjw/language-models-seem-to-be-much-better-than-humans-at-next https://www.lesswrong.com/posts/htrZrxduciZ5QaCjw/language-m... (which is what they are trained on, so maybe not unreasonable). There's also the idea that GPT-3/etc can produce text that's difficult for humans to distinguish from human-generated text at the lowest levels of quality (think like time cube), which is closer to AGI than say simplistic Markov generators. (How close is anyone's guess)

deleted 4y ago

[deleted]

adampwells 4y ago

it seems that 'making a better' GPT-3 or similar model is like climbing higher up a tall tree and claiming you are closer to the moon... technically true, but...

deleted 4y ago

[deleted]

renewiltord 4y ago

It would be fun to prompt the next language model with "You are connected to a Linux bash terminal. Your next command will be executed on that bash console. Type what you want", feed that into an appropriate VM and then reprompt with "The stderr was $X and the stdout was $Y. Your next command will be executed on the console" and loop. I shall try it on the next one when it comes out. AI alignment be damned. Let's let the baby play with a bomb and see how close it is to AGI if we let it drive itself.

deleted 4y ago

[deleted]

seydor 4y ago

we should also notice that intelligence is not necessarily useful

deleted 4y ago

[deleted]

tombakt 4y ago

Until a language model can develop a generalized solution to a real-world phenomena, it's not even close to AGI. The current iteration of ML algorithms are useful, yes, but not intelligent.

whatwherewhy 4y ago
What is a generalized solution to a real-world phenomena? Github Copilot solved my business problem by itself just as I would've done. Is that real-world enough and the solution generalized enough?
- qayxc 4y ago
  > Github Copilot solved my business problem by itself just as I would've done. Is that real-world enough and the solution generalized enough? No, it isn't. Co-Pilot is unable to provide a rationalisation for the generated code and is incapable of assessing its security or performance properties. It's a very advanced auto-complete that just happens to have been specialised on auto-completing code.
  - dogcomplex 4y ago
    I mean... it can generate comment summaries explaining any block of code, or an entire class. It can generate comprehensive unit tests and usability tests. It can tell you the Big-O operational complexity of any code you (or it) writes, and it can put those all together. Is that not good enough for you..?
    - qayxc 4y ago
      It can't do that. It can try to do that and it might work well in some cases. It is definitely incapable of doing that for any code you throw at it. It also cannot generate comprehensive unit tests - it can can generate unit tests. The definition of "comprehensive" is way too subjective. > Is that not good enough for you..? No, it indeed isn't, but maybe that's because I actually have at least some idea of what happens behind the scenes and how the system works. From the implementation details, I can tell you for a fact that none of what you described can be done by the system in the general case - and especially not correctly. It's hit and miss depending on the input and that's basically a design limitation.

bilsbie 4y ago

I don’t understand why we don’t talk about this kind of stuff more. It’s really amazing.

Jack000 4y ago

LMs aren't AGI, but they show that the search for architecture is essentially settled. What the scaling laws demonstrate is that any architectural improvements you could find manually can be superseded by a slightly larger transformer. LMs lack crucial elements of of human like intelligence - long term memory, short term memory, episodic memory, proprioceptive awareness etc. The task is now to implement these things using transformers.

krackers 4y ago
>long term memory, short term memory Doesn't that mean the architecture isn't settled? The current mechanism of appending the output of the model back into the prompt feels like a bit of a hack. I'm only a layman here but it seems transformer models can only propagate information layer-wise, adding some ability to propagate information across time like RNNs do might be useful for achieving longer-term coherence.

avalys 4y ago

There are some activities of daily life that I do without thinking (most of them, in fact). Moving my limbs while walking up the stairs, driving to the corner store and stopping at a traffic light, recognizing a stop sign at a glance, reading the numerals off a mailbox, etc. These are all things that, after you initially learn to do them, get wired up into the fabric of your brain well enough that you can execute them without really thinking about them. They become instinctual. Interestingly, these are all things that deep neural networks have turned out to be quite good at - instinctual things. There is also a category of written and oral communication that I do without thinking - instinctually. I don't think deliberately about the choice of each word when writing this post, and certainly not when speaking out loud. Idioms and turns of phrase emerge without deliberate thought or intent. If my social circle has started using certain terminology habitually (e.g. here in the Bay Area, people have been using the adjective "super" a lot, as in "that's super cool"), I'll find myself using it as well without making any deliberate attempt to do so, sometimes to my own chagrin. And even when the topic is something nominally intellectual, I'll sometimes find myself simply regurgitating the general opinion on this topic that I last read from a trusted source. This seems to exemplify what GPT3 does - a sort of instinctual written communication - and I don't believe it's an example of intelligence any more than a human being able to recognize a stop sign in 100 ms is a sign of intelligence. GPT3 a great pattern-matching engine and it applies pattern-matching on the human language to interpolate a response that is consistent with the patterns it observed. I don't see any evidence that GPT3 can critically evaluate the information it is is pattern-matching - that it can go beyond what literate humans do instinctually. Don't get me wrong - GPT3 is surprising and amazing. But not because it signifies anything about AGI. What's amazing to me about GPT3 is it reveals how much ordinary human written and oral communication is instinctual in the same sense that visual processing and image recognition is.

togaen 4y ago

clickbait nonsense

aaron695 4y ago

inawarminister 4y ago

You know, being able to do all things human can do... with text input and text output? Why does that reminds me of Unix Philosophy where everything is just text files? I wonder if someone has experimented with one of the LLMs to get them to do Unix sysadmin jobs. It's not exactly outputting source codes, but Bash commands should be similar enough right?