3 ms·

> I doubt any biological system can ever learn all the information gpt4 has learned The fact that human brains can become competent at language with training d

> I doubt any biological system can ever learn all the information gpt4 has learned The fact that human brains can become competent at language with training datasets with a size of a tiny fraction of common crawl points to how much more efficient they are

akasakahakada 3y ago
Until you realize that ChatGPT is also fluent in 20 other languages by including minimal data into the training dataset.
- namaria 3y ago
  Minimal as in the amount a person needs to learn the language or minimal as in tiny if compared to English in common crawl but still orders of magnitude more than a person goes through for infancy language acquisition? I think you know the difference.
  - skybrian 3y ago
    It’s anecdotal, but here [1] is an example of AI learning to translate a new language faster than a human could learn to do it, from a similar amount of data, using Claude 3 Opus. Translating a language isn’t quite the same as understanding it, but this is still impressively data-efficient. [1] https://twitter.com/hahahahohohe/status/1765088860592394250 https://twitter.com/hahahahohohe/status/1765088860592394250
    - namaria 3y ago
      Did you read the "added context"? "The author has later responded and apologized a flawed / biased methodology which led them to believe that Opus has no prior knowledge of Circassian (i.e. wasn't trained on it) Apparently it was, and is able to speak Circassian just not perfectly"
      - skybrian 3y ago
        I didn't see that! Thanks!
akasakahakada 3y ago
Bro you are proven wrong. Baby FOV camera footage as the training dataset of LLM, the result is not bad. https://www.nyu.edu/about/news-publications/news/2024/february/ai-learns-through-the-eyes-and-ears-of-a-child.html https://www.nyu.edu/about/news-publications/news/2024/februa...
- Jensson 3y ago
  The result is bad, that is just an image recognition model we know those can be trained with relatively little words and just a bunch of images, the model didn't learn grammar or sentences or relationships like a baby would. The only models we have that can do grammar and sentences and relationships are LLMs with massive amounts of data.

akasakahakada 3y ago

Until you realize that ChatGPT is also fluent in 20 other languages by including minimal data into the training dataset.

namaria 3y ago
Minimal as in the amount a person needs to learn the language or minimal as in tiny if compared to English in common crawl but still orders of magnitude more than a person goes through for infancy language acquisition? I think you know the difference.
- skybrian 3y ago
  It’s anecdotal, but here [1] is an example of AI learning to translate a new language faster than a human could learn to do it, from a similar amount of data, using Claude 3 Opus. Translating a language isn’t quite the same as understanding it, but this is still impressively data-efficient. [1] https://twitter.com/hahahahohohe/status/1765088860592394250 https://twitter.com/hahahahohohe/status/1765088860592394250
  - namaria 3y ago
    Did you read the "added context"? "The author has later responded and apologized a flawed / biased methodology which led them to believe that Opus has no prior knowledge of Circassian (i.e. wasn't trained on it) Apparently it was, and is able to speak Circassian just not perfectly"
    - skybrian 3y ago
      I didn't see that! Thanks!

akasakahakada 3y ago

Bro you are proven wrong. Baby FOV camera footage as the training dataset of LLM, the result is not bad. https://www.nyu.edu/about/news-publications/news/2024/february/ai-learns-through-the-eyes-and-ears-of-a-child.html https://www.nyu.edu/about/news-publications/news/2024/februa...

Jensson 3y ago
The result is bad, that is just an image recognition model we know those can be trained with relatively little words and just a bunch of images, the model didn't learn grammar or sentences or relationships like a baby would. The only models we have that can do grammar and sentences and relationships are LLMs with massive amounts of data.