13 ms·

Three things that LLMs have made us rethink

58 points by chmaynard 3y ago

jfengel 3y ago
1. We always knew the Turing test was insufficient. 2. The Chinese Room argument can invite itself back out, as always. It was never more than an intuition pump and not a very good one. 3. Universal Grammar doesn't really figure into it one way or the other. LLMs don't work the way we do, despite being "neural". They get their basic grammar from a very different place. Chomsky's arguments are doing fine, though this is a great time to look at them through new lenses.
- dosinga 3y ago
  ad 3) If the claim of Chomsky (followers) is that you can only learn human language if you have the universal grammar, then the discovery of a system that learns human languages without such a grammar does seem like rather relevant to the discussion
  - furyofantares 3y ago
    That isn't the claim.
    - markisus 3y ago
      But many people did believe that. From the article, > I personally remember very heated arguments on the MIT campus between Chomsky followers and computer scientists working on statistical models in the early 2000’s. The arguments came up in computer science faculty hiring meetings. The Chomskians claimed that because of the need for universal grammar there would never be anything useful that came out of statistical approaches to language. ChatGPT has proved them wrong. (And yes, I personally shouted angrily at Chomskians in some of those hiring meetings.)
      - foobarqux 3y ago
        Chomsky himself never said that, he said that it wouldn't produce anything useful for understanding how language works in humans.
  - foobarqux 3y ago
    It's completely irrelevant, as he points out LLMs can learn any "language" even ones that humans can't, so therefore it doesn't tell you anything about humans. A related argument is that you can predict the weather using a neural net but that doesn't mean that the weather works like a neural network and it doesn't tell you anything about how weather does work.
dosinga 3y ago
Quite interesting that Turing thought 125MB would be all it would take to reach AGI. To him it must have seemed such a large amount of storage
- pryelluw 3y ago
  It’s interesting that more advanced machine learning is developed might prove him right. Who knows what sort of discovery might come as a result of entities that are able to “think” in different ways.
- whiplash451 3y ago
  It is quite possible that we end up reaching GPT4-level performance with 125MB of weights in the end.
- o11c 3y ago
  The Turing test is necessary but not sufficient for AGI. That said, I have yet to see a modern AI that actually passes the Turing test either: > an average interrogator will not have more than 70 per cent, chance of making the right identification after five minutes of questioning. LLMs can answer some questions fine, but within five minutes you're likely to find breakage even without consciously trying.
  - famouswaffles 3y ago
    >That said, I have yet to see a modern AI that actually passes the Turing test either: This is not saying much when modern state of the art LLMs are being delibrately trained not to pass the Turing Test.
- bastawhiz 3y ago
  For his time, it must have seemed like such an unfathomable amount of storage. He had no frame of reference for how much data that really is. Today, we see the size of every file on our devices and have an intuition for how big things are (or can be). Only in the last thirty years did we really develop this sense.
- 3y ago
willprice89 3y ago
I've always felt the Chinese Room thought experiment was founded on a complete misunderstanding of bilingualism. The only "algorithm" that can effectively transform text between English and Chinese would be one that understands the source and target texts like a human. So the Chinese Room is really just a more complicated Turing Test - a computer can only convince an observer that it "understands Chinese" if it is also able to convince an observer that it understands the conversation the same way as a human, in either language.
- bilsbie 3y ago
  I thought so too but LLM’s sort of made me change my mind.
  - willprice89 3y ago
    I think LLM's support my theory. The reason they are good at translation is because they are already good at simulating understanding in all of the languages they translate between.
- JadeNB 3y ago
  > The only "algorithm" that can effectively transform text between English and Chinese would be one that understands the source and target texts like a human. How do you know?
  - pixl97 3y ago
    Or the corollary viewpoint, that if ChatGPT can do this as the previous poster says, then it is a type of AGI.
    - Jensson 3y ago
      Yeah, people used to think that since only humans can do arithmetics it was the peak of intelligence, and computers can do arithmetics better than humans, then AGI was just around the corner. Now people argue that since only humans have complex language it was the peak of intelligence, and now computers can do that language like humans, then AGI is just around the corner.
bilsbie 3y ago
I wish I could find more discussion like this. There’s so much of this to discuss with LLM’s in the scene. Honestly I don’t understand why people aren’t talking about AI all the time. Nothing is like we would have expected.
- dgreensp 3y ago
  Exactly! Nothing is like we would have expected. The seeming “understanding” and coherency of LLMs is a surprise, and it doesn’t seem like anyone knows exactly what it means about human language and cognition. It’s possible that by analyzing what LLMs are actually doing we will discover a “code” underlying language that is sort of like a universal grammar, or at least gain some insight into how they stitch together and combine remembered snippets of language so sensically. While what ChatGPT does seems more like “open-ended talking” than thinking, reasoning, and responding to its external and internal situation (which does not really exist, in the absence of plugins—there is just the talking), that talking can be used to (for example) plan a series of actions.
stavros 3y ago
The most important thing LLMs made me reconsider is the nature of thought itself. What does it mean to think when a computer can do it more convincingly than many of the humans I know?
- almostnormal 3y ago
  Humans are easily convinced (fooled) by clever use of language. Politics, tests, interviews, everything. There has always been a difference between actual understanding and just great presentation. LLMs are so far ahead in their use of language that given a complete lack of understanding they are more capable than humans to fool the listener. But there's nothing substantially new, some people have mastered that skill, too.
  - stavros 3y ago
    What's "understanding", and how do you know LLMs lack it?
    - foobarqux 3y ago
      It's clear that any sane definition of understanding would exclude the types of failures that you regularly encounter when using LLMs.
      - stavros 3y ago
        Interesting how we need to define understanding in such a way that it doesn't apply to LLMs.
        foobarqux 3y ago
        I think it's extremely misleading to use a definition that allows for routinely logically contradicting yourself and induces the impression that it doesn't do that. People are regularly doing this in the titles of papers --- ascribing words like "emergence" and "world models" with definitions that don't match their informal usage -- purposely to mislead people into mistakenly applying conventional intuitions and make the paper look like it proves more than it does.
AndyNemmity 3y ago
Chomsky on the topic: "It simply shows they’re irrelevant. A two-year-old child or two, three-year-old child has basically mastered the essentials of language with very sparse data. We want to understand language learning, cognition. We want to see how that works. The fact that some program that scans extraordinary amounts of data gets something superficially similar to what a two-year-old child does basically tells us nothing." and "It’s like asking whether submarines swim. You want to call that swimming? Okay, submarine swim. What the programs are doing is, again, scanning huge amounts of data, finding statistical regularities enough so that you can make a fair guess as to what the next word will be in some sequence. Is that thinking? Does submarine swim? It’s the same question."
- jongjong 3y ago
  I feel like people who play down the significance of AI are lacking a critical piece of understanding about how scalability changes things. Even if it was only as smart as a dog, if you can duplicate it without limit in silicon, that is a highly transformative tech because electricity is cheaper than food. You can solve more problems per unit of currency. Maybe it would be constrained to certain simple problems but still, the AI would completely dominate that problem space and push humans out of it
  - lacrimacida 3y ago
    >AI would completely dominate that problem space and push humans out of it This type of AI needs to be trained on human data. If humans are pushed out I don’t see it progressing too far
    - xyzzy123 3y ago
      I feel like AlphaGo is a counter-example - it learned by playing against itself. That said, go is an extremely constrained problem space. It's possible that in science, mathematics, games etc that AI could keep advancing without human input or feedback (and not hit a plateau where more human input is needed) because the universe or whatever axioms you choose provide all the necessary structure you need to keep making "progress". But for cultural artefacts - music / art / literature etc perhaps not. At least in the near term while we provide data, power and compute perhaps the most important thing humans do is set goals and rewards.
markisus 3y ago
Along these lines, Gary Marcus said that "you can't build a ladder to the moon" [1]. In other words, neural nets do well on simple things, but general intelligence is such a high bar that neural nets (and statistical methods in general) can never reach it, without the addition of the sort of secret sauce that allows humans to really make sense of the world. I used to be sympathetic to this viewpoint. But these days I'm more and more willing to believe that we can actually build a ladder to the moon. [1] https://www.quantamagazine.org/common-sense-comes-closer-to-computers-20200430/ https://www.quantamagazine.org/common-sense-comes-closer-to-...
- ren_engineer 3y ago
  I mean even Gary Marcus is now an AI doomer who went from mocking these models to saying they need to be banned or research slowed for safety reasons. Why's he so scared of a ladder? A lot of people having to eat crow due to how quickly things changed in just a few years
- MichaelZuo 3y ago
  > Gary Marcus said that "you can't build a ladder to the moon" [1]. It is physically possible to build a ladder all the way to the Moon. A structure made out of close-to-ideal carbon nanotubes would have sufficient strength. It would just require an unbelievable complex carbon nanotube 'weaving' system that can somehow operate and move directly between the Earth and Moon. Which is quite an interesting parallel to the complexity of the human brain when I think about the analogy.
  - HPsquared 3y ago
    How would it behave with the relative movement and rotation of the moon/earth? Would it maintain a stable position? I imagine it would wrap around the earth if it was fixed to the surface. But if it's free floating, wouldn't it want to fall one way or the other? Or spin somehow? EDIT: this would be a good one for Randall Munroe's "What If" EDIT EDIT: of course: https://what-if.xkcd.com/157/ https://what-if.xkcd.com/157/
    - 3y ago
BoppreH 3y ago
I think this is a good place to ask: are there any tasks left that LLMs still fail significantly more often than average humans? Not "sometimes it writes buggy code", or "it refuses to discuss ethics", but getting something wrong in a way that people on the street wouldn't. So far I've only seen examples based on tokenization ("what's the third letter in word?"), and the fact that answers are generated sequentially ("write a sentence that mentions its own word count"). But those feel unsatisfactory, like asking a human to decode ASCII. I've personally seen some impressive feats from simple LLMs, like writing sensible code in a new language based only on a few surrounding lines. That shows a level of expertise beyond the common line "it's just mixing sentences from the training data".
- dartos 3y ago
  Creative writing. LLMs tend to overly stick to tropes or create plot holes Complex problem solving. There are only so many confounding variables that next token generation can handle Anything requiring recent knowledge. RAG exists, but in context learning only gets you so far.
  - jameshart 3y ago
    The average person on the street sticks to tropes and creates plot holes, too.
  - famouswaffles 3y ago
    He asked for a comparison with the average human. Even with the "helpful assistant" RLHF hampering GPT-4's fiction writing abilities, i do not think it is significantly worse than average human.
- Jensson 3y ago
  > So far I've only seen examples based on tokenization ("what's the third letter in word?"), and the fact that answers are generated sequentially ("write a sentence that includes the count of words in itself"). But those feel unsatisfactory because those are not cognitive limitations. It could solve those, the reason it can't is that it doesn't understand its own limitations, ie it has no self awareness. Humans plan things based around their own limitations, if an AI model can't even do that how can it be considered an AGI?
notabennett 3y ago
About half the Wikipedia article on Universal Grammar is devoted to arguments against it -- that not only is there no genetic or biological evidence for it, but that there can be no evidence for it, since it's fundamentally unfalsifiable.
Jack000 3y ago
I think a lot of the confusion on whether LLMs can think stems from the fact that LLMs are purely models of language and solve intelligence as a kind of accidental side-effect. The real problem that an LLM is trying to solve is to create a model that can enumerate all meaningful sequences of words. This is just an insane way of approaching the problem of intelligence on the face of it. There's a huge difference between a model of language and an intelligent agent that uses language to communicate. What LLMs show is that the hardest problem - of how to get emergent capabilities at scale from huge quantities of data - is solved. To get more human-like thinking, all that is needed is to find the right pre-training task that more closely aligns with agentic behavior. This is still a huge problem but it's an engineering problem and not one of linguistic theory or philosophy.
- famouswaffles 3y ago
  What we feed these huge LLMs is not just language, but text. and an enormous amount of it. The transformer is an arbitrary sequence to sequence modeller. Think about what is contained (explicitly and implicitly) in all the text we can feed a model. It's not just language, but a projection of the world as humans see it. GPT-3.5 Instruct Turbo can play valid chess at about 1800 ELO, no doubt because of the chess games described in PGN in the training set. Does Chess suddenly become a language ability because it was expressed in Text ? No
  - Jack000 3y ago
    Chess is a great example because it highlights the subtle difference between LLMs and agents. What GPT3.5 does is not quite playing chess but creating realistic chess moves that a human might make. An LLM could play chess though, all it needs is grounding (by feeding it the current board state) and agency (RF to reward the model for winning games)
    - famouswaffles 3y ago
      >What GPT3.5 does is not quite playing chess but creating realistic chess moves that a human might make. No it's playing games. And if you're not at about the level I spoke of, you will lose repeatedly over whatever number or stretch of games you imagine. https://github.com/adamkarvonen/chess_gpt_eval https://github.com/adamkarvonen/chess_gpt_eval 3.5 Instruct (different model from regular 3.5 that can't play) can play chess. There's no trick. Any other framing seems like a meaningless distinction. The goal is to model the chess games and there's no better way to do that than to learn to play the game. >all it needs is grounding (by feeding it the current board state) The Model is already constructing a board state to play the game. https://www.neelnanda.io/mechanistic-interpretability/othello https://www.neelnanda.io/mechanistic-interpretability/othell... >agency (RF to reward the model for winning games) Predict the next token loss is already rewarding models for winning when the side they are predicting wins. And when the preceeding text says x side wins and it's playing as x side then the loss is rewarding it to do everything it can to win. I agree different goals and primary rewards led to this ability to play and with it , slight manifestations(GPT can probably modulate level of play better than any other machine or human) but it is nonetheless playing.
tetrisgm 3y ago
Of course LLMs can be intelligent. We've trained billions of humans to be automatons and simply work within the confines of school and work. Never create original thought. Why would a machine not be able to emulate this? The question implies a difference between human automatons who perform jobs and follow rules, vs intelligence in the sense of philosophy or generating original ideas beyond the confines of training. It's like saying most humans aren't intelligent. Absurd. LLMs are absolutely capable of this. The counterpoint tends to be around emotion and empathy. Besides being a very clear shifting of goalposts, this is already solved. We have several intelligent humans who have limited empathy and emotions. Are we to argue they aren't intelligent? Some of our most brilliant people, and many people who just live their normal lives, are excluded from the premise of the question. I suspect it is rooted in a fear that human intelligence isn't exceptional. AI models are already intelligent, the question is what humans we're comparing them to.

58 points by chmaynard 3y ago

1. We always knew the Turing test was insufficient. 2. The Chinese Room argument can invite itself back out, as always. It was never more than an intuition pump and not a very good one. 3. Universal Grammar doesn't really figure into it one way or the other. LLMs don't work the way we do, despite being "neural". They get their basic grammar from a very different place. Chomsky's arguments are doing fine, though this is a great time to look at them through new lenses.

dosinga 3y ago
ad 3) If the claim of Chomsky (followers) is that you can only learn human language if you have the universal grammar, then the discovery of a system that learns human languages without such a grammar does seem like rather relevant to the discussion
- furyofantares 3y ago
  That isn't the claim.
  - markisus 3y ago
    But many people did believe that. From the article, > I personally remember very heated arguments on the MIT campus between Chomsky followers and computer scientists working on statistical models in the early 2000’s. The arguments came up in computer science faculty hiring meetings. The Chomskians claimed that because of the need for universal grammar there would never be anything useful that came out of statistical approaches to language. ChatGPT has proved them wrong. (And yes, I personally shouted angrily at Chomskians in some of those hiring meetings.)
    - foobarqux 3y ago
      Chomsky himself never said that, he said that it wouldn't produce anything useful for understanding how language works in humans.
- foobarqux 3y ago
  It's completely irrelevant, as he points out LLMs can learn any "language" even ones that humans can't, so therefore it doesn't tell you anything about humans. A related argument is that you can predict the weather using a neural net but that doesn't mean that the weather works like a neural network and it doesn't tell you anything about how weather does work.

dosinga 3y ago

Quite interesting that Turing thought 125MB would be all it would take to reach AGI. To him it must have seemed such a large amount of storage

pryelluw 3y ago
It’s interesting that more advanced machine learning is developed might prove him right. Who knows what sort of discovery might come as a result of entities that are able to “think” in different ways.
whiplash451 3y ago
It is quite possible that we end up reaching GPT4-level performance with 125MB of weights in the end.
o11c 3y ago
The Turing test is necessary but not sufficient for AGI. That said, I have yet to see a modern AI that actually passes the Turing test either: > an average interrogator will not have more than 70 per cent, chance of making the right identification after five minutes of questioning. LLMs can answer some questions fine, but within five minutes you're likely to find breakage even without consciously trying.
- famouswaffles 3y ago
  >That said, I have yet to see a modern AI that actually passes the Turing test either: This is not saying much when modern state of the art LLMs are being delibrately trained not to pass the Turing Test.
bastawhiz 3y ago
For his time, it must have seemed like such an unfathomable amount of storage. He had no frame of reference for how much data that really is. Today, we see the size of every file on our devices and have an intuition for how big things are (or can be). Only in the last thirty years did we really develop this sense.
3y ago

willprice89 3y ago

I've always felt the Chinese Room thought experiment was founded on a complete misunderstanding of bilingualism. The only "algorithm" that can effectively transform text between English and Chinese would be one that understands the source and target texts like a human. So the Chinese Room is really just a more complicated Turing Test - a computer can only convince an observer that it "understands Chinese" if it is also able to convince an observer that it understands the conversation the same way as a human, in either language.

bilsbie 3y ago
I thought so too but LLM’s sort of made me change my mind.
- willprice89 3y ago
  I think LLM's support my theory. The reason they are good at translation is because they are already good at simulating understanding in all of the languages they translate between.
JadeNB 3y ago
> The only "algorithm" that can effectively transform text between English and Chinese would be one that understands the source and target texts like a human. How do you know?
- pixl97 3y ago
  Or the corollary viewpoint, that if ChatGPT can do this as the previous poster says, then it is a type of AGI.
  - Jensson 3y ago
    Yeah, people used to think that since only humans can do arithmetics it was the peak of intelligence, and computers can do arithmetics better than humans, then AGI was just around the corner. Now people argue that since only humans have complex language it was the peak of intelligence, and now computers can do that language like humans, then AGI is just around the corner.

bilsbie 3y ago

I wish I could find more discussion like this. There’s so much of this to discuss with LLM’s in the scene. Honestly I don’t understand why people aren’t talking about AI all the time. Nothing is like we would have expected.

dgreensp 3y ago
Exactly! Nothing is like we would have expected. The seeming “understanding” and coherency of LLMs is a surprise, and it doesn’t seem like anyone knows exactly what it means about human language and cognition. It’s possible that by analyzing what LLMs are actually doing we will discover a “code” underlying language that is sort of like a universal grammar, or at least gain some insight into how they stitch together and combine remembered snippets of language so sensically. While what ChatGPT does seems more like “open-ended talking” than thinking, reasoning, and responding to its external and internal situation (which does not really exist, in the absence of plugins—there is just the talking), that talking can be used to (for example) plan a series of actions.

stavros 3y ago

The most important thing LLMs made me reconsider is the nature of thought itself. What does it mean to think when a computer can do it more convincingly than many of the humans I know?

almostnormal 3y ago
Humans are easily convinced (fooled) by clever use of language. Politics, tests, interviews, everything. There has always been a difference between actual understanding and just great presentation. LLMs are so far ahead in their use of language that given a complete lack of understanding they are more capable than humans to fool the listener. But there's nothing substantially new, some people have mastered that skill, too.
- stavros 3y ago
  What's "understanding", and how do you know LLMs lack it?
  - foobarqux 3y ago
    It's clear that any sane definition of understanding would exclude the types of failures that you regularly encounter when using LLMs.
    - stavros 3y ago
      Interesting how we need to define understanding in such a way that it doesn't apply to LLMs.
      - foobarqux 3y ago
        I think it's extremely misleading to use a definition that allows for routinely logically contradicting yourself and induces the impression that it doesn't do that. People are regularly doing this in the titles of papers --- ascribing words like "emergence" and "world models" with definitions that don't match their informal usage -- purposely to mislead people into mistakenly applying conventional intuitions and make the paper look like it proves more than it does.

AndyNemmity 3y ago

Chomsky on the topic: "It simply shows they’re irrelevant. A two-year-old child or two, three-year-old child has basically mastered the essentials of language with very sparse data. We want to understand language learning, cognition. We want to see how that works. The fact that some program that scans extraordinary amounts of data gets something superficially similar to what a two-year-old child does basically tells us nothing." and "It’s like asking whether submarines swim. You want to call that swimming? Okay, submarine swim. What the programs are doing is, again, scanning huge amounts of data, finding statistical regularities enough so that you can make a fair guess as to what the next word will be in some sequence. Is that thinking? Does submarine swim? It’s the same question."

jongjong 3y ago
I feel like people who play down the significance of AI are lacking a critical piece of understanding about how scalability changes things. Even if it was only as smart as a dog, if you can duplicate it without limit in silicon, that is a highly transformative tech because electricity is cheaper than food. You can solve more problems per unit of currency. Maybe it would be constrained to certain simple problems but still, the AI would completely dominate that problem space and push humans out of it
- lacrimacida 3y ago
  >AI would completely dominate that problem space and push humans out of it This type of AI needs to be trained on human data. If humans are pushed out I don’t see it progressing too far
  - xyzzy123 3y ago
    I feel like AlphaGo is a counter-example - it learned by playing against itself. That said, go is an extremely constrained problem space. It's possible that in science, mathematics, games etc that AI could keep advancing without human input or feedback (and not hit a plateau where more human input is needed) because the universe or whatever axioms you choose provide all the necessary structure you need to keep making "progress". But for cultural artefacts - music / art / literature etc perhaps not. At least in the near term while we provide data, power and compute perhaps the most important thing humans do is set goals and rewards.

markisus 3y ago

Along these lines, Gary Marcus said that "you can't build a ladder to the moon" [1]. In other words, neural nets do well on simple things, but general intelligence is such a high bar that neural nets (and statistical methods in general) can never reach it, without the addition of the sort of secret sauce that allows humans to really make sense of the world. I used to be sympathetic to this viewpoint. But these days I'm more and more willing to believe that we can actually build a ladder to the moon. [1] https://www.quantamagazine.org/common-sense-comes-closer-to-computers-20200430/ https://www.quantamagazine.org/common-sense-comes-closer-to-...

ren_engineer 3y ago
I mean even Gary Marcus is now an AI doomer who went from mocking these models to saying they need to be banned or research slowed for safety reasons. Why's he so scared of a ladder? A lot of people having to eat crow due to how quickly things changed in just a few years
MichaelZuo 3y ago
> Gary Marcus said that "you can't build a ladder to the moon" [1]. It is physically possible to build a ladder all the way to the Moon. A structure made out of close-to-ideal carbon nanotubes would have sufficient strength. It would just require an unbelievable complex carbon nanotube 'weaving' system that can somehow operate and move directly between the Earth and Moon. Which is quite an interesting parallel to the complexity of the human brain when I think about the analogy.
- HPsquared 3y ago
  How would it behave with the relative movement and rotation of the moon/earth? Would it maintain a stable position? I imagine it would wrap around the earth if it was fixed to the surface. But if it's free floating, wouldn't it want to fall one way or the other? Or spin somehow? EDIT: this would be a good one for Randall Munroe's "What If" EDIT EDIT: of course: https://what-if.xkcd.com/157/ https://what-if.xkcd.com/157/
  - 3y ago

BoppreH 3y ago

I think this is a good place to ask: are there any tasks left that LLMs still fail significantly more often than average humans? Not "sometimes it writes buggy code", or "it refuses to discuss ethics", but getting something wrong in a way that people on the street wouldn't. So far I've only seen examples based on tokenization ("what's the third letter in word?"), and the fact that answers are generated sequentially ("write a sentence that mentions its own word count"). But those feel unsatisfactory, like asking a human to decode ASCII. I've personally seen some impressive feats from simple LLMs, like writing sensible code in a new language based only on a few surrounding lines. That shows a level of expertise beyond the common line "it's just mixing sentences from the training data".

dartos 3y ago
Creative writing. LLMs tend to overly stick to tropes or create plot holes Complex problem solving. There are only so many confounding variables that next token generation can handle Anything requiring recent knowledge. RAG exists, but in context learning only gets you so far.
- jameshart 3y ago
  The average person on the street sticks to tropes and creates plot holes, too.
- famouswaffles 3y ago
  He asked for a comparison with the average human. Even with the "helpful assistant" RLHF hampering GPT-4's fiction writing abilities, i do not think it is significantly worse than average human.
Jensson 3y ago
> So far I've only seen examples based on tokenization ("what's the third letter in word?"), and the fact that answers are generated sequentially ("write a sentence that includes the count of words in itself"). But those feel unsatisfactory because those are not cognitive limitations. It could solve those, the reason it can't is that it doesn't understand its own limitations, ie it has no self awareness. Humans plan things based around their own limitations, if an AI model can't even do that how can it be considered an AGI?

notabennett 3y ago

About half the Wikipedia article on Universal Grammar is devoted to arguments against it -- that not only is there no genetic or biological evidence for it, but that there can be no evidence for it, since it's fundamentally unfalsifiable.

Jack000 3y ago

I think a lot of the confusion on whether LLMs can think stems from the fact that LLMs are purely models of language and solve intelligence as a kind of accidental side-effect. The real problem that an LLM is trying to solve is to create a model that can enumerate all meaningful sequences of words. This is just an insane way of approaching the problem of intelligence on the face of it. There's a huge difference between a model of language and an intelligent agent that uses language to communicate. What LLMs show is that the hardest problem - of how to get emergent capabilities at scale from huge quantities of data - is solved. To get more human-like thinking, all that is needed is to find the right pre-training task that more closely aligns with agentic behavior. This is still a huge problem but it's an engineering problem and not one of linguistic theory or philosophy.

famouswaffles 3y ago
What we feed these huge LLMs is not just language, but text. and an enormous amount of it. The transformer is an arbitrary sequence to sequence modeller. Think about what is contained (explicitly and implicitly) in all the text we can feed a model. It's not just language, but a projection of the world as humans see it. GPT-3.5 Instruct Turbo can play valid chess at about 1800 ELO, no doubt because of the chess games described in PGN in the training set. Does Chess suddenly become a language ability because it was expressed in Text ? No
- Jack000 3y ago
  Chess is a great example because it highlights the subtle difference between LLMs and agents. What GPT3.5 does is not quite playing chess but creating realistic chess moves that a human might make. An LLM could play chess though, all it needs is grounding (by feeding it the current board state) and agency (RF to reward the model for winning games)
  - famouswaffles 3y ago
    >What GPT3.5 does is not quite playing chess but creating realistic chess moves that a human might make. No it's playing games. And if you're not at about the level I spoke of, you will lose repeatedly over whatever number or stretch of games you imagine. https://github.com/adamkarvonen/chess_gpt_eval https://github.com/adamkarvonen/chess_gpt_eval 3.5 Instruct (different model from regular 3.5 that can't play) can play chess. There's no trick. Any other framing seems like a meaningless distinction. The goal is to model the chess games and there's no better way to do that than to learn to play the game. >all it needs is grounding (by feeding it the current board state) The Model is already constructing a board state to play the game. https://www.neelnanda.io/mechanistic-interpretability/othello https://www.neelnanda.io/mechanistic-interpretability/othell... >agency (RF to reward the model for winning games) Predict the next token loss is already rewarding models for winning when the side they are predicting wins. And when the preceeding text says x side wins and it's playing as x side then the loss is rewarding it to do everything it can to win. I agree different goals and primary rewards led to this ability to play and with it , slight manifestations(GPT can probably modulate level of play better than any other machine or human) but it is nonetheless playing.

tetrisgm 3y ago

Of course LLMs can be intelligent. We've trained billions of humans to be automatons and simply work within the confines of school and work. Never create original thought. Why would a machine not be able to emulate this? The question implies a difference between human automatons who perform jobs and follow rules, vs intelligence in the sense of philosophy or generating original ideas beyond the confines of training. It's like saying most humans aren't intelligent. Absurd. LLMs are absolutely capable of this. The counterpoint tends to be around emotion and empathy. Besides being a very clear shifting of goalposts, this is already solved. We have several intelligent humans who have limited empathy and emotions. Are we to argue they aren't intelligent? Some of our most brilliant people, and many people who just live their normal lives, are excluded from the premise of the question. I suspect it is rooted in a fear that human intelligence isn't exceptional. AI models are already intelligent, the question is what humans we're comparing them to.