5 ms·

Max Tegmark seems most worried about iterative self improvement by the systems. But is that possible? And just how far could they improve themselves? Enough to

Max Tegmark seems most worried about iterative self improvement by the systems. But is that possible? And just how far could they improve themselves? Enough to achieve self awareness or just enough to make themselves a lot faster?

deleted 3y ago
[deleted]
Fricken 3y ago
Is there any reasonable assumption to be made that AI has reached it's zenith? I don't think there is. AI improves over time. That's all it does, and barring some civilizational catastrophe, that's what it will continue to do.
- tome 3y ago
  > AI improves over time. That's all it does, and barring some civilizational catastrophe, that's what it will continue to do. Why? Why might it not stop improving, like lots of things, burning paraffin for heat to pluck one example out of the air?
  - ethanbond 3y ago
    Because so far it continues to trend upwards? And it's nowhere near the efficiency of the human brain, so we have good reason to suspect we should be able to at least build highly-interconnected AIs with at least the efficiency of the human brain. A couple million of those alone would be unfathomable. Probably not any time soon, but insofar as we have reason to suspect there's an upper limit (which we don't, IMO), we're not even close to the upper known limit.
    - tome 3y ago
      > And it's nowhere near the efficiency of the human brain, so we have good reason to suspect we should be able to at least build highly-interconnected AIs with at least the efficiency of the human brain. Why? I really don't get it. Many, many commenters here and elsewhere have used the existence of the human brain as a supposed proof that we can create AIs with human-level intelligence. I just don't see why the conclusion follows from the observation. Can anyone flesh out the argument in more detail?
      - llamaimperative 3y ago
        Sure, it’s essentially that intelligence appears to be substrate independent. That is, it is not the case that mammalian brain matter is some completely unique material in the universe naturally imbued with “intelligence,” but that it’s a physical structure that does a thing called “intelligence” under the right electrical, thermal, and chemical conditions. The brain is obviously immensely complex and we’re not close to producing a computer like it any time soon, but there’s no reason to believe we never could in principle. E.g. you could imagine a machine that ”scans” one structure of molecules and “prints” an identical structure of the same compounds. If you were to build this machine and apply it to a brain — copy the exact configuration atom-by-atom — you’d produce an identical intelligence as the initial brain. There’s no known law or fact about the universe (or intelligence) that precludes this. [0] So if that’s true, and it’s true that there are strong incentives to eventually develop human or superhuman intelligences, then we will one day develop at-least-human intelligences (assuming we don’t get annihilated by a meteor or nuclear holocaust). The worst case scenario is we have to build a facsimile of a brain, which would likely take a very very very long time indeed, but it could be the case that human engineering (as opposed to natural selection) can find a much shorter route to the same destination. Regardless of which route we end up taking, regardless of how far down that road we are, regardless of whether our current route is a dead-end, the route and the destination do both demonstrably exist: natural selection "found" one route inside the skulls of some hairless apes. [0] Note you’d presumably also produce an identical consciousness, but this can of worms need not be opened! That’s a separate topic so we’ll steer clear for now. It’s likely this route to intelligence produces consciousness but potentially some (or most, or all other) routes do not.
        tome 3y ago
        I'm sorry, I still don't get it. I agree that in principle there is no reason human level AI couldn't exist, but that doesn't mean that it can happen on planet Earth. Some practical, essential step on the pathway may be impossible, just like in principle there's no reason that life couldn't arise on Pluto, or the proof or refutation of the Riemann hypothesis might be produced next time I generate a random string in Python, but practical, essential steps are impossible.
  - Fricken 3y ago
    Because when we reached the limit of what we could do with paraffin, we moved onto other methods of producing heat. Now we make superheated balls of plasma as hot as the sun using deuterium, tritium, lasers and magnets.
    - tome 3y ago
      Sure, but your comment suggests that AI will continue improving indefinitely. Why might we not, as we did with paraffin, reach the limit of AI?
      - Fricken 3y ago
        Paraffin is a means, not an end. The end goal is to generate heat, and we have all kinds of ways to generate heat now that are superior to paraffin. The future of AI may not depend on diffusion models or even a silicon substrate, but the incentives to improve it are clear, and it will get better.

deleted 3y ago

[deleted]

Fricken 3y ago

Is there any reasonable assumption to be made that AI has reached it's zenith? I don't think there is. AI improves over time. That's all it does, and barring some civilizational catastrophe, that's what it will continue to do.

tome 3y ago
> AI improves over time. That's all it does, and barring some civilizational catastrophe, that's what it will continue to do. Why? Why might it not stop improving, like lots of things, burning paraffin for heat to pluck one example out of the air?
- ethanbond 3y ago
  Because so far it continues to trend upwards? And it's nowhere near the efficiency of the human brain, so we have good reason to suspect we should be able to at least build highly-interconnected AIs with at least the efficiency of the human brain. A couple million of those alone would be unfathomable. Probably not any time soon, but insofar as we have reason to suspect there's an upper limit (which we don't, IMO), we're not even close to the upper known limit.
  - tome 3y ago
    > And it's nowhere near the efficiency of the human brain, so we have good reason to suspect we should be able to at least build highly-interconnected AIs with at least the efficiency of the human brain. Why? I really don't get it. Many, many commenters here and elsewhere have used the existence of the human brain as a supposed proof that we can create AIs with human-level intelligence. I just don't see why the conclusion follows from the observation. Can anyone flesh out the argument in more detail?
    - llamaimperative 3y ago
      Sure, it’s essentially that intelligence appears to be substrate independent. That is, it is not the case that mammalian brain matter is some completely unique material in the universe naturally imbued with “intelligence,” but that it’s a physical structure that does a thing called “intelligence” under the right electrical, thermal, and chemical conditions. The brain is obviously immensely complex and we’re not close to producing a computer like it any time soon, but there’s no reason to believe we never could in principle. E.g. you could imagine a machine that ”scans” one structure of molecules and “prints” an identical structure of the same compounds. If you were to build this machine and apply it to a brain — copy the exact configuration atom-by-atom — you’d produce an identical intelligence as the initial brain. There’s no known law or fact about the universe (or intelligence) that precludes this. [0] So if that’s true, and it’s true that there are strong incentives to eventually develop human or superhuman intelligences, then we will one day develop at-least-human intelligences (assuming we don’t get annihilated by a meteor or nuclear holocaust). The worst case scenario is we have to build a facsimile of a brain, which would likely take a very very very long time indeed, but it could be the case that human engineering (as opposed to natural selection) can find a much shorter route to the same destination. Regardless of which route we end up taking, regardless of how far down that road we are, regardless of whether our current route is a dead-end, the route and the destination do both demonstrably exist: natural selection "found" one route inside the skulls of some hairless apes. [0] Note you’d presumably also produce an identical consciousness, but this can of worms need not be opened! That’s a separate topic so we’ll steer clear for now. It’s likely this route to intelligence produces consciousness but potentially some (or most, or all other) routes do not.
      - tome 3y ago
        I'm sorry, I still don't get it. I agree that in principle there is no reason human level AI couldn't exist, but that doesn't mean that it can happen on planet Earth. Some practical, essential step on the pathway may be impossible, just like in principle there's no reason that life couldn't arise on Pluto, or the proof or refutation of the Riemann hypothesis might be produced next time I generate a random string in Python, but practical, essential steps are impossible.
- Fricken 3y ago
  Because when we reached the limit of what we could do with paraffin, we moved onto other methods of producing heat. Now we make superheated balls of plasma as hot as the sun using deuterium, tritium, lasers and magnets.
  - tome 3y ago
    Sure, but your comment suggests that AI will continue improving indefinitely. Why might we not, as we did with paraffin, reach the limit of AI?
    - Fricken 3y ago
      Paraffin is a means, not an end. The end goal is to generate heat, and we have all kinds of ways to generate heat now that are superior to paraffin. The future of AI may not depend on diffusion models or even a silicon substrate, but the incentives to improve it are clear, and it will get better.