6 ms·

So a staple belief about hyperintelligence in the last couple of decades was that there would be no obvious ceiling on how smart a recursively self-improving AI

So a staple belief about hyperintelligence in the last couple of decades was that there would be no obvious ceiling on how smart a recursively self-improving AI could get. The assumption was that intelligence was a property of some (possibly very complex) algorithm, which could be refined without end. But in the AI models we have now, the intelligence is an emergent property of the training data, while the algorithm is fairly simple. So my question is, to what extent is the growth in 2023-style AI capped by the availability of training data? At some point soon, we'll have fed these models with every document ever written; what's the next step up from there?

alanbernstein 4y ago
Feed all documents, audio, images, video, 3d models, and all life experiences to a single model that understands how to interpret and synthesize all of these things. Then see if you still need to ask the question.
- marginalia_nu 4y ago
  How are you going to acquire all life experiences?
  - idlewords 4y ago
    Killbots at the door demanding to know how your semester in Prague changed you as a person
  - pixl97 4y ago
    You've not heard about the mandatory neurolink's yet?
  - alanbernstein 4y ago
    That was just a nod to the idea that a fully functional AGI might need to "grow up in the world". Consuming static digital media is one thing, but experiencing the physical world from the perspective of a physical body that you can control is different.
  - jareklupinski 4y ago
    Doorbell cameras already stream everyone who comes and goes from your home, and can probably hear things in a decent enough radius. Some people have cameras and microphones inside their own businesses (CCTV) or home (nanny cams), possibly with lax security settings. Endgame, just turn on everyone's smartphone camera and microphone for five minutes.
- realce 4y ago
  INSUFFICIENT DATA FOR MEANINGFUL ANSWER
mdp2021 4y ago
> AI models we have now, the intelligence ... an emergent property Exception raised on several terms. «AI models»: they are built (roughly) on the "problem" (task) of "reply something relevant and convincing". Which satisfies the technical contextual definition of "intelligence" ("solving a problem") but not the intrinsic one ("say something through intelligence"). «emergent»: the specific "intelligence" was already there in the training data - it does not emerge but it is repeated. > to what extent is the growth in 2023-style AI capped by the availability of training data An intelligent entity does not need to read the whole of past intellectual production: there is no cap. Fed data is already overly plentiful. It does achieve "relevant and convincing", does not it? It is the jump to "reliable" (and beyond) that is missing.
- idlewords 4y ago
  Emergent in this case means that the property is not present in the individual elements but shows up in the whole. There was no hint ahead of time that (for example) training neural networks on hundreds of billions of images would create qualitatively different results than training on millions, but here we are.
dwohnitmok 4y ago
Have your thoughts changed since https://idlewords.com/talks/superintelligence.htm https://idlewords.com/talks/superintelligence.htm or are they still the same?
- idlewords 4y ago
  On AI hard takeoff/alignment risk (the stuff that talk was about) I feel exactly the same. I am very surprised at the unreasonable effectiveness of large language models at doing... whatever it is they're doing, but I stand behind everything I said in that talk.
  - dwohnitmok 4y ago
    Would any hypothetical advancement cause you to rethink your position?
    - idlewords 4y ago
      Well yeah, of course. One big one would be if the pathway to improving AI was not data limited (as it is now) but cleverness limited. Like say there was a cleverness function K at the heart of an AI system that was devilishly complex to reason about but we knew could improve with tweaking. Then the recursive self-improvement scenario that keeps AI catastrophists up at night would look a lot more realizable, the first person to rig K to optimize itself into a slightly better K' really might be playing with fire. That's not the pattern I see right now with large language model systems, which seem to be capped by how much material they can train on, a bound that is not easy to raise. But that's the kind of development that would make me change my view a lot. Otherwise what's even the purpose of thinking about this stuff, if nothing can change your mind about it?
      - dwohnitmok 4y ago
        Would you be worried if the amount of data required to train LLMs went down significantly, but scaling benefits continue to hold? I.e. if it turns out that by more efficiently using data there are 2-3 more orders of magnitude's worth of training possible. What if LLMs can be used to train themselves on how to collect data from more diverse and plentiful sources?

alanbernstein 4y ago

Feed all documents, audio, images, video, 3d models, and all life experiences to a single model that understands how to interpret and synthesize all of these things. Then see if you still need to ask the question.

marginalia_nu 4y ago
How are you going to acquire all life experiences?
- idlewords 4y ago
  Killbots at the door demanding to know how your semester in Prague changed you as a person
- pixl97 4y ago
  You've not heard about the mandatory neurolink's yet?
- alanbernstein 4y ago
  That was just a nod to the idea that a fully functional AGI might need to "grow up in the world". Consuming static digital media is one thing, but experiencing the physical world from the perspective of a physical body that you can control is different.
- jareklupinski 4y ago
  Doorbell cameras already stream everyone who comes and goes from your home, and can probably hear things in a decent enough radius. Some people have cameras and microphones inside their own businesses (CCTV) or home (nanny cams), possibly with lax security settings. Endgame, just turn on everyone's smartphone camera and microphone for five minutes.
realce 4y ago
INSUFFICIENT DATA FOR MEANINGFUL ANSWER

mdp2021 4y ago

> AI models we have now, the intelligence ... an emergent property Exception raised on several terms. «AI models»: they are built (roughly) on the "problem" (task) of "reply something relevant and convincing". Which satisfies the technical contextual definition of "intelligence" ("solving a problem") but not the intrinsic one ("say something through intelligence"). «emergent»: the specific "intelligence" was already there in the training data - it does not emerge but it is repeated. > to what extent is the growth in 2023-style AI capped by the availability of training data An intelligent entity does not need to read the whole of past intellectual production: there is no cap. Fed data is already overly plentiful. It does achieve "relevant and convincing", does not it? It is the jump to "reliable" (and beyond) that is missing.

idlewords 4y ago
Emergent in this case means that the property is not present in the individual elements but shows up in the whole. There was no hint ahead of time that (for example) training neural networks on hundreds of billions of images would create qualitatively different results than training on millions, but here we are.

dwohnitmok 4y ago

Have your thoughts changed since https://idlewords.com/talks/superintelligence.htm https://idlewords.com/talks/superintelligence.htm or are they still the same?

idlewords 4y ago
On AI hard takeoff/alignment risk (the stuff that talk was about) I feel exactly the same. I am very surprised at the unreasonable effectiveness of large language models at doing... whatever it is they're doing, but I stand behind everything I said in that talk.
- dwohnitmok 4y ago
  Would any hypothetical advancement cause you to rethink your position?
  - idlewords 4y ago
    Well yeah, of course. One big one would be if the pathway to improving AI was not data limited (as it is now) but cleverness limited. Like say there was a cleverness function K at the heart of an AI system that was devilishly complex to reason about but we knew could improve with tweaking. Then the recursive self-improvement scenario that keeps AI catastrophists up at night would look a lot more realizable, the first person to rig K to optimize itself into a slightly better K' really might be playing with fire. That's not the pattern I see right now with large language model systems, which seem to be capped by how much material they can train on, a bound that is not easy to raise. But that's the kind of development that would make me change my view a lot. Otherwise what's even the purpose of thinking about this stuff, if nothing can change your mind about it?
    - dwohnitmok 4y ago
      Would you be worried if the amount of data required to train LLMs went down significantly, but scaling benefits continue to hold? I.e. if it turns out that by more efficiently using data there are 2-3 more orders of magnitude's worth of training possible. What if LLMs can be used to train themselves on how to collect data from more diverse and plentiful sources?