4 ms·

> Moreover, words are already compressed data with a lot of lost information That's exactly the point, 99.9% of the information processed by the 6 year old has

> Moreover, words are already compressed data with a lot of lost information That's exactly the point, 99.9% of the information processed by the 6 year old has went into training his visual cortex, a subsystem not necessary for reasoning. This will eventually be used 8 hours a day for transforming visual representation of text on a computer screen into a compressed textual form which the inner rational loop can easily manipulate. The holy grail of AI, for anything but artistic creation, is just this inner loop.

TheRoque 2y ago
This sounds like a huge oversimplification. A lot of knowledge doesn't come in the form of words, animals, early humans, kids, all don't have speech systems yet are capable of reasoning. You can't become Mozart just by reading the theory of his work, you can't become an expert at anything just by reading the theory in fact, even though it lifts you up. There's definitely things that you know and that aren't in the form of words.
- cornholio 2y ago
  > There's definitely things that you know and that aren't in the form of words. But are they necessary for reasoning? Records of deaf-blinds with IQs a few standard deviations above average seem to suggest even highly restricted sensory inputs can allow for rational thought. It stretches credulity that you could convey more than a few bits of meaning per second to a child purely by hand gestures and touch, let alone the exabytes claimed. Of course someone like Yann Lecun will say scale is the only solution, he leads one of the juggernauts that uses scale as a competitive moat. We shouldn't expect AI researchers in positions of power to be any less vane, power hungry or human than any other humans.
  - TheRoque 2y ago
    This is on point that you mention IQ, since in IQ itself, only a single test includes words. The rest is about pattern recognition, working memory, spatial capabilities and other stuff I don't remember. So in other words, the metric you use as a benchmark for good reasoning barely measures the literacy
    - cornholio 2y ago
      Okay, but that's a markedly different problem than the information density of the training data. It's: "can a rational machine trained only with natural language descriptions of the world form complex representations of it and solve associated problems described only in natural language?". For you and me, a verbal description of a pattern test such as "You are given seven squares. Square A contains a filled cross in the upper left position, an empty circle in the middle... etc." would send us grasping for pen and paper. But with training many people can play mental chess, there are famous blind theoretical physicists and so on. This all seems to suggest that natural language could be called "Turing complete", to bastardize the term, for describing the real world to a rational entity and formulating solutions to real world problems.

TheRoque 2y ago

This sounds like a huge oversimplification. A lot of knowledge doesn't come in the form of words, animals, early humans, kids, all don't have speech systems yet are capable of reasoning. You can't become Mozart just by reading the theory of his work, you can't become an expert at anything just by reading the theory in fact, even though it lifts you up. There's definitely things that you know and that aren't in the form of words.

cornholio 2y ago
> There's definitely things that you know and that aren't in the form of words. But are they necessary for reasoning? Records of deaf-blinds with IQs a few standard deviations above average seem to suggest even highly restricted sensory inputs can allow for rational thought. It stretches credulity that you could convey more than a few bits of meaning per second to a child purely by hand gestures and touch, let alone the exabytes claimed. Of course someone like Yann Lecun will say scale is the only solution, he leads one of the juggernauts that uses scale as a competitive moat. We shouldn't expect AI researchers in positions of power to be any less vane, power hungry or human than any other humans.
- TheRoque 2y ago
  This is on point that you mention IQ, since in IQ itself, only a single test includes words. The rest is about pattern recognition, working memory, spatial capabilities and other stuff I don't remember. So in other words, the metric you use as a benchmark for good reasoning barely measures the literacy
  - cornholio 2y ago
    Okay, but that's a markedly different problem than the information density of the training data. It's: "can a rational machine trained only with natural language descriptions of the world form complex representations of it and solve associated problems described only in natural language?". For you and me, a verbal description of a pattern test such as "You are given seven squares. Square A contains a filled cross in the upper left position, an empty circle in the middle... etc." would send us grasping for pen and paper. But with training many people can play mental chess, there are famous blind theoretical physicists and so on. This all seems to suggest that natural language could be called "Turing complete", to bastardize the term, for describing the real world to a rational entity and formulating solutions to real world problems.