4 ms·

There's billions of years of compressed knowledge in those 4 year olds. Lots of useful priors.

ak_111 4y ago
You basically landed on Chomsky's universal grammar. And this only proves the chatgpt critics: we have no idea what those priors are, how they evolved, why they are so effective and thus we are not even sure they exist. Until this is demonstrated I think it is very fair to say chatgpt is applying very different reasoning to what humans are applying. Also language is a fairly recent development in human evolution (only 60-70 generations ago) which makes it much more puzzling how a mechanism that is so efficient and effective could evolve so quickly, let alone pondering how actual languages evolved (almost instantly all over the world) given how hard it is to construct an artificial one.
- alcover 4y ago
  60-70 generations ago More like 1000+ considering the Chauvet painters certainly had speech.
  - ak_111 4y ago
    let's even make it 10,000+ generation, this still makes it quite magical to see how these priors could evolve to make language acquisition so trivial to humans relative to chatgpt. Chatgpt requires on the order of gazillion of epochs and tokens, and can still confidently express elementary mistake that a 4 year old doesn't.
    - alcover 4y ago
      Yes, per Wikipedia 10000 seems like it. Plus the human brain and anatomy was ready for it (even neanderthal is thought to have spoken). I don't see GPT's blunders as mistakes. They are to us for sure but would not be to another GPT instance in that it would produce the same continuation to a prompt and thus agree.
      - ak_111 4y ago
        Plus the human brain and anatomy was ready for it We have no idea how evolution "readied" a deeply complex organ like the brain over many thousands of years, then almost instantly repurposed it for language acquisition and generation. To further hypothesise that what it was "readying" was something that trains from data in a way similar to how chatgpt is trained from data makes it even more astonishing and until this is demonstrated it is more scientific to not accept this hypothesis.
- Blikkentrekker 4y ago
  Universal grammar only speaks to the ability to parse and form sentences, not to internal logic. That ChatGPT is capable of producing human-like sentences from pattern recognition without any universal grammar baked in, even if the underlying reasoning might be flawed, goes against the argument of something such as universal grammar existing. At the very least, it shows that a neural net is capable of parsing and producing coherent grammar without any assistance from universal grammar. It does not prove that humans don't have it, but it does make a compelling case that it's clearly not required for humans to have it either.
  - ak_111 4y ago
    That ChatGPT is capable of producing human-like sentences from pattern recognition without any universal grammar baked in You didn't address or missed the main point: chatgpt requires something in the order of a trillion tokens to be capable of producing what you mentioned in one language. There are 40 months old babies that are fairly conversant in both Chinese and English, and are able to detect sarcasm with something like 0.0000001% of the tokens, doesn't that give you pause that part of language acquisition is innate to humans and is not entirely acquired the way chatgpt is taught?
    - Blikkentrekker 4y ago
      It requires so much to be able to generate answers tot hings asked about a wide variety of subjects, not to be able to generate grammatically valid sentences and parse them, which other a.i.'s also did before with far fewer tokens.

You basically landed on Chomsky's universal grammar. And this only proves the chatgpt critics: we have no idea what those priors are, how they evolved, why they are so effective and thus we are not even sure they exist. Until this is demonstrated I think it is very fair to say chatgpt is applying very different reasoning to what humans are applying. Also language is a fairly recent development in human evolution (only 60-70 generations ago) which makes it much more puzzling how a mechanism that is so efficient and effective could evolve so quickly, let alone pondering how actual languages evolved (almost instantly all over the world) given how hard it is to construct an artificial one.

alcover 4y ago
60-70 generations ago More like 1000+ considering the Chauvet painters certainly had speech.
- ak_111 4y ago
  let's even make it 10,000+ generation, this still makes it quite magical to see how these priors could evolve to make language acquisition so trivial to humans relative to chatgpt. Chatgpt requires on the order of gazillion of epochs and tokens, and can still confidently express elementary mistake that a 4 year old doesn't.
  - alcover 4y ago
    Yes, per Wikipedia 10000 seems like it. Plus the human brain and anatomy was ready for it (even neanderthal is thought to have spoken). I don't see GPT's blunders as mistakes. They are to us for sure but would not be to another GPT instance in that it would produce the same continuation to a prompt and thus agree.
    - ak_111 4y ago
      Plus the human brain and anatomy was ready for it We have no idea how evolution "readied" a deeply complex organ like the brain over many thousands of years, then almost instantly repurposed it for language acquisition and generation. To further hypothesise that what it was "readying" was something that trains from data in a way similar to how chatgpt is trained from data makes it even more astonishing and until this is demonstrated it is more scientific to not accept this hypothesis.
Blikkentrekker 4y ago
Universal grammar only speaks to the ability to parse and form sentences, not to internal logic. That ChatGPT is capable of producing human-like sentences from pattern recognition without any universal grammar baked in, even if the underlying reasoning might be flawed, goes against the argument of something such as universal grammar existing. At the very least, it shows that a neural net is capable of parsing and producing coherent grammar without any assistance from universal grammar. It does not prove that humans don't have it, but it does make a compelling case that it's clearly not required for humans to have it either.
- ak_111 4y ago
  That ChatGPT is capable of producing human-like sentences from pattern recognition without any universal grammar baked in You didn't address or missed the main point: chatgpt requires something in the order of a trillion tokens to be capable of producing what you mentioned in one language. There are 40 months old babies that are fairly conversant in both Chinese and English, and are able to detect sarcasm with something like 0.0000001% of the tokens, doesn't that give you pause that part of language acquisition is innate to humans and is not entirely acquired the way chatgpt is taught?
  - Blikkentrekker 4y ago
    It requires so much to be able to generate answers tot hings asked about a wide variety of subjects, not to be able to generate grammatically valid sentences and parse them, which other a.i.'s also did before with far fewer tokens.