6 ms·

> Assuming that a superintelligence will contain a program that includes all the programs that can be executed by a universal Turing machine on input potentiall

> Assuming that a superintelligence will contain a program that includes all the programs that can be executed by a universal Turing machine on input potentially as complex as the state of the world Rest easy folks. This is purely theoretical.

biolurker1 6y ago
Until it isn't and we are faced yet again with a much worse pandemic like situation.
- strofcon 6y ago
  Except that actual pandemics are demonstrable, predictable, and based in known science, yes?
  - rhn_mk1 6y ago
    The difference between pandemics and this is that pandemics have happened before. This is more like global warming in that respect.
  - vsareto 6y ago
    More like it's the politics that really decides what changes, regardless of what science proves
  - biolurker1 6y ago
    Superintelligence is a mathematical proof
- kryptiskt 6y ago
  Assuming that a superintelligence will contain a program that includes all the programs that can be executed by a universal Turing machine on input potentially as complex as the state of the world The visible universe is far too small to store that data. Like many exponentials too small. You can't even enumerate all the programs that work on a handful of inputs without running into the problem that the universe just isn't big enough for that.
  - biolurker1 6y ago
    Until there is a breakthrough. World genius were able to advanced humanity because they were open to possibilities. Lesser scientists were dogmatic that "this can't be because so and so"
ReadEvalPost 6y ago
"Purely theoretical" is too weak. "Physically impossible" is better. AI Safety guys really like using the physically impossible to advance their arguments. It's bizarre! Pure fantasy isn't something worth reacting to.
- jandrese 6y ago
  It's the classic case of seeing a curve pointing upwards and thinking it will continue doing that forever, even though the universe is like 0 for nearly-uncountable in cases where that has held true indefinitely. Every growth curve is an S curve. The AI Singularity is like the Grey Goo scenario. An oversimplified model projected out so far in the future that its flaws become apparent.
  - ben_w 6y ago
    Depends which version of “the singularity” is being discussed. IIRC, the original (almost certainly false) idea was a sufficiently powerful AI can start a sequence of ever power powerful AI with decreasing time between each step — reaching infinite power in finite time. I don’t need that version of the singularity to be worried. I think in terms of “the event horizon” rather than “the singularity”: all new tech changes the world, when the rate of change exceeds our capacity to keep up with the consequences, stuff will go wrong on a large scale for lots of people. As for grey goo? Self replicating nanomachines is just biology. It gets everywhere, and even single-celled forms can kill you by eating your flesh or suborning your cells, but it’s mostly no big deal because you evolved to deal with that threat.
    - jandrese 6y ago
      The Grey Goo scenario is that it starts eating the planet until the only thing left is a sea of nanomachines. However, thermodynamics says it becomes increasingly more difficult to distribute power (food) to a growing population and it hits a natural limit to growth, just like bacteria.
- postalrat 6y ago
  "Circle" and "exponential growth" are also physically impossible yet useful ideas.

biolurker1 6y ago

Until it isn't and we are faced yet again with a much worse pandemic like situation.

strofcon 6y ago
Except that actual pandemics are demonstrable, predictable, and based in known science, yes?
- rhn_mk1 6y ago
  The difference between pandemics and this is that pandemics have happened before. This is more like global warming in that respect.
- vsareto 6y ago
  More like it's the politics that really decides what changes, regardless of what science proves
- biolurker1 6y ago
  Superintelligence is a mathematical proof
kryptiskt 6y ago
Assuming that a superintelligence will contain a program that includes all the programs that can be executed by a universal Turing machine on input potentially as complex as the state of the world The visible universe is far too small to store that data. Like many exponentials too small. You can't even enumerate all the programs that work on a handful of inputs without running into the problem that the universe just isn't big enough for that.
- biolurker1 6y ago
  Until there is a breakthrough. World genius were able to advanced humanity because they were open to possibilities. Lesser scientists were dogmatic that "this can't be because so and so"

ReadEvalPost 6y ago

"Purely theoretical" is too weak. "Physically impossible" is better. AI Safety guys really like using the physically impossible to advance their arguments. It's bizarre! Pure fantasy isn't something worth reacting to.

jandrese 6y ago
It's the classic case of seeing a curve pointing upwards and thinking it will continue doing that forever, even though the universe is like 0 for nearly-uncountable in cases where that has held true indefinitely. Every growth curve is an S curve. The AI Singularity is like the Grey Goo scenario. An oversimplified model projected out so far in the future that its flaws become apparent.
- ben_w 6y ago
  Depends which version of “the singularity” is being discussed. IIRC, the original (almost certainly false) idea was a sufficiently powerful AI can start a sequence of ever power powerful AI with decreasing time between each step — reaching infinite power in finite time. I don’t need that version of the singularity to be worried. I think in terms of “the event horizon” rather than “the singularity”: all new tech changes the world, when the rate of change exceeds our capacity to keep up with the consequences, stuff will go wrong on a large scale for lots of people. As for grey goo? Self replicating nanomachines is just biology. It gets everywhere, and even single-celled forms can kill you by eating your flesh or suborning your cells, but it’s mostly no big deal because you evolved to deal with that threat.
  - jandrese 6y ago
    The Grey Goo scenario is that it starts eating the planet until the only thing left is a sea of nanomachines. However, thermodynamics says it becomes increasingly more difficult to distribute power (food) to a growing population and it hits a natural limit to growth, just like bacteria.
postalrat 6y ago
"Circle" and "exponential growth" are also physically impossible yet useful ideas.