4 ms·

I disagree. On pure # of words, sure. But for humans, language is actually just a compressed version of reality perceived through multiple senses / prediction

I disagree. On pure # of words, sure. But for humans, language is actually just a compressed version of reality perceived through multiple senses / prediction + observation cycles / model paradigms / scales / contexts/ social cues, etc. and we get full access to the entire thing. So a single sentence is wrapped in orders of magnitude more data. We also get multiple modes of interconnected feedback. How to describe this? Let me use an analogy. In poker, different properties a player has statistically take different amounts of data to reach convergence: Some become evident in 10s of hands, some take 100s of hands, and some take 1000s, and some even take 10,000s before you get over 90% confidence. ....And yet, if you let a good player see your behavior on just one single hand that goes to showdown, a good human player will be able to estimate your playing style, skill, and where your stats will converge to with remarkable accuracy. They get to see how you acted pre-flop, on the flop, turn, and river, with the rich context of position, pot-size, and what the other players were doing during those times, along with the stakes and location you're playing at, what you're wearing, how you move and handle your chips, etc. etc.

numpad0 4y ago
We also eat. It feels to me that better food with divergent micronutrients has positive performance implications. Maybe I’m just schizophrenic, but to me it just feels that way.
- fshbbdssbbgdd 4y ago
  Try training the model with free-range, locally-sourced electrons.
marcosdumay 4y ago
It's less data anyway. There's no way to add up the data a person senses and get into a volume any similar to what's on the internet. But it may be better data.
- thom 4y ago
  To quantify this a little, the human sensory system has been estimated to generate on the order of 11m bits per second of data. So 1-2 megabytes a second for most of your life. That’s probably in the region of a day of YouTube. But it’s clear that a lot of human cognition is directed towards novelty, and humans are able to run experiments interactively, not just consume data (see schemas in child development etc). So, you take your baby AI and instead of training it just on a static corpus, you put it in a simulator. And then when you again hit a wall where you conclude data is the problem, you give them a robot body and plug them into the internet bidirectionally. Some would argue this would be a mistake.

numpad0 4y ago

We also eat. It feels to me that better food with divergent micronutrients has positive performance implications. Maybe I’m just schizophrenic, but to me it just feels that way.

fshbbdssbbgdd 4y ago
Try training the model with free-range, locally-sourced electrons.

marcosdumay 4y ago

It's less data anyway. There's no way to add up the data a person senses and get into a volume any similar to what's on the internet. But it may be better data.

thom 4y ago
To quantify this a little, the human sensory system has been estimated to generate on the order of 11m bits per second of data. So 1-2 megabytes a second for most of your life. That’s probably in the region of a day of YouTube. But it’s clear that a lot of human cognition is directed towards novelty, and humans are able to run experiments interactively, not just consume data (see schemas in child development etc). So, you take your baby AI and instead of training it just on a static corpus, you put it in a simulator. And then when you again hit a wall where you conclude data is the problem, you give them a robot body and plug them into the internet bidirectionally. Some would argue this would be a mistake.