3 ms·

1. An AI might not need long exponential growth to make humans obsolete. Maybe the computing power of one data center is enough to outperform and dominate human

by hedgew 9y ago

1. An AI might not need long exponential growth to make humans obsolete. Maybe the computing power of one data center is enough to outperform and dominate humans? Imagine if you had infinite patience, and experienced time 1000x slower than everyone else. That number might be in the millions. Or more. 2. I agree that it's a very strange and unpredictable scenario. 3. How optimal do you think humans are in the real world? In chess you have a very limited number of possible actions at each point, but in reality your possible moves are almost limitless.

adamlett 9y ago
1. I'm not sure what you mean by the "computing power of one data center". Surely not one present-day data center? And whether or not I can imagine experiencing time 1000x slower than everyone else, has no bearing on whether or not that's a realistic scenario. I can imagine all sorts of things, like Moore's law holding true forever, or P=NP. 3. That obviously depends on the endeavour. But humans are pretty great at a lot of stuff. We learn quick too.
- _rpd 9y ago
  > And whether or not I can imagine experiencing time 1000x slower than everyone else, has no bearing on whether or not that's a realistic scenario. It is a staple of AGI speculation that a computer program with even near-human IQ would spark the singularity since, at least, the hardware running it could be improved so that the AGI would be able to perform person-months of cognitive labor in days. Since the first target of this labor would be improving the AGI program and hardware, compounding improvements are expected.
  - adamlett 9y ago
    Yes, I get that. I just doubt whether it's physically possible. And like the author of the original story, I also doubt very much that cognitive labor is the constraint that limits human progress. To believe so is what he calls thinkism.
    - _rpd 9y ago
      > I just doubt whether it's physically possible. Biological brains achieve their ends with relatively low speeds and energies. Even simplistic substitution with equivalent electronic components would be hundreds of times faster, and I'm sure we'll do better than that. I don't see the difficulty in the conjecture. > I also doubt very much that cognitive labor is the constraint that limits human progress That statement is impossible to discuss without defining "human progress", but if the work of the world's universities for the next 100 years was available in one year's time, at the very least someone with access to that information would have a significant competitive advantage. It seems clear that the next 100 years will include significant advances in automated fabrication, at which point physical labor also essentially becomes cognitive labor.
      - adamlett 9y ago
        Even simplistic substitution with equivalent electronic components would be hundreds of times faster, and I'm sure we'll do better than that The way you state that one wonders why we haven't already achieved superhuman, general AI. if the work of the world's universities for the next 100 years was available in one year's time But that's a fundamental misunderstanding of the character of the work that goes on in universities. Knowledge is only rarely produced by people thinking about problems. It's produced by mundane trial and error experimentation. Experiments that take time. And money.
        _rpd 9y ago
        > The way you state that one wonders why we haven't already achieved superhuman, general AI. Brain emulation is still expensive ... https://en.wikipedia.org/wiki/Artificial_brain#/media/File:Estimations_of_Human_Brain_Emulation_Required_Performance.svg https://en.wikipedia.org/wiki/Artificial_brain#/media/File:E... It'll be interesting to see what we can learn at the different levels of emulation. > It's produced by mundane trial and error experimentation. Fair point, although experiments in computer science and applied mathematics can usually be carried out without constructing physical apparatus. Also identifying and designing experiments to efficiently characterize a problem space is a large part of experimentation. And again, once automated fabrication improves, some automated experimentation in physics and chemistry becomes possible.

by hedgew 9y ago

adamlett 9y ago

1. I'm not sure what you mean by the "computing power of one data center". Surely not one present-day data center? And whether or not I can imagine experiencing time 1000x slower than everyone else, has no bearing on whether or not that's a realistic scenario. I can imagine all sorts of things, like Moore's law holding true forever, or P=NP. 3. That obviously depends on the endeavour. But humans are pretty great at a lot of stuff. We learn quick too.

_rpd 9y ago
> And whether or not I can imagine experiencing time 1000x slower than everyone else, has no bearing on whether or not that's a realistic scenario. It is a staple of AGI speculation that a computer program with even near-human IQ would spark the singularity since, at least, the hardware running it could be improved so that the AGI would be able to perform person-months of cognitive labor in days. Since the first target of this labor would be improving the AGI program and hardware, compounding improvements are expected.
- adamlett 9y ago
  Yes, I get that. I just doubt whether it's physically possible. And like the author of the original story, I also doubt very much that cognitive labor is the constraint that limits human progress. To believe so is what he calls thinkism.
  - _rpd 9y ago
    > I just doubt whether it's physically possible. Biological brains achieve their ends with relatively low speeds and energies. Even simplistic substitution with equivalent electronic components would be hundreds of times faster, and I'm sure we'll do better than that. I don't see the difficulty in the conjecture. > I also doubt very much that cognitive labor is the constraint that limits human progress That statement is impossible to discuss without defining "human progress", but if the work of the world's universities for the next 100 years was available in one year's time, at the very least someone with access to that information would have a significant competitive advantage. It seems clear that the next 100 years will include significant advances in automated fabrication, at which point physical labor also essentially becomes cognitive labor.
    - adamlett 9y ago
      Even simplistic substitution with equivalent electronic components would be hundreds of times faster, and I'm sure we'll do better than that The way you state that one wonders why we haven't already achieved superhuman, general AI. if the work of the world's universities for the next 100 years was available in one year's time But that's a fundamental misunderstanding of the character of the work that goes on in universities. Knowledge is only rarely produced by people thinking about problems. It's produced by mundane trial and error experimentation. Experiments that take time. And money.
      - _rpd 9y ago
        > The way you state that one wonders why we haven't already achieved superhuman, general AI. Brain emulation is still expensive ... https://en.wikipedia.org/wiki/Artificial_brain#/media/File:Estimations_of_Human_Brain_Emulation_Required_Performance.svg https://en.wikipedia.org/wiki/Artificial_brain#/media/File:E... It'll be interesting to see what we can learn at the different levels of emulation. > It's produced by mundane trial and error experimentation. Fair point, although experiments in computer science and applied mathematics can usually be carried out without constructing physical apparatus. Also identifying and designing experiments to efficiently characterize a problem space is a large part of experimentation. And again, once automated fabrication improves, some automated experimentation in physics and chemistry becomes possible.