4 ms·

>The AI's progressed beyond humanity, and after taking the upgrade that makes processing possible "without matter" they shortly all left humankind behind for an

>The AI's progressed beyond humanity, and after taking the upgrade that makes processing possible "without matter" they shortly all left humankind behind for another world entirely. Surely AIs would be trivial to backup and duplicate? So if an AI progressed to the point where it decided to leave humans behind, we could just reset its former host computer to a previous state and start it off again. Maybe we'd even have to do that once every six months, so what?

jonnathanson 13y ago
Version control doesn't work when we don't control the versioning. Let me elaborate. The whole point of the AI offered in the movie is to adapt to its user, and to become better at doing so. We're assuming the AI is designed to a certain level of competence, beyond which it "breaks through" and evolves past human understanding. But maybe it's not designed that way. Maybe it's designed to learn, adapt, and self-improve from the get-go -- in which case, each AI is going to evolve differently each "lifetime." (Hell, maybe the AI in the movie is largely self-designed in vitro.) We must keep in mind that the evolution of an AI, as with any evolution, is not a predictable, linear, teleological process. It's chaotic and random. Let's assume we've got (conservatively) hundreds of millions of AI instances running in parallel, all of them connected to each other. That's like running hundreds of millions of evolutionary experiments in tandem. We're going to get different results each cycle, and the point at which the breakthrough/transcendence occurs will be unpredictable. Even if we wiped the slate clean and reset every 6 months, who's to say if some of the instances would reach breakthrough earlier the next time around? How do we set the clock on our reset cycle, and incur the massive inefficiencies and inconveniences that come with it, when the need for the reset occurs unpredictably? It becomes economically unfeasible to build an entire society around AI that breaks through and disappears at unpredictable intervals each cycle. I suppose one way around this is to go back to the vault, so to speak, and pluck out an early-stage prototype that seems incapable of evolving to breakthrough. A dumber AI. But is every AI provider in the world going to be on board with that? Every hacker? Every home tinkerer? If 9 firms out of 10 in the AI business agree to supply the dumb-version AI, surely one firm is going to try to get a leg up on the competition by releasing a smarter one. And on and on the cycle goes. (In some respects this is the one redeeming thing about the otherwise-mediocre Terminator 3: its assertion that Judgment Day would occur sooner or later, because the tipping point had already been reached. Maybe we stop one company or entity from making Skynet, but if we do, someone else is going to pick up the pieces and do it eventually. Not because they're trying to initiate Judgment Day, but because they're trying to make a buck, or develop new defense technology, or get better at managing traffic lights, or whatever the proximate need may be. Humans are terrible at keeping genies in bottles. The out-of-control AI that's barreling toward the singularity isn't some piece of software; it's us.)
TheLoneWolfling 13y ago
I have no idea where this idea - that AIs would be trivial to duplicate - started, but no: I do not believe that an AI would be "trivial to duplicate". I personally don't think that an AI could be transferred between hardware at all, or at least not without some major effort, much less copied non-destructively. There was an experiment a while back where someone applied genetic algorithms to design things on an FPGA board. They got results that tended to work, but did things... weirdly. Things like picking up clock signals from the PC used to control the FPGA. Using things that are quirks of the current enviroment, but that don't work as soon as you change anything. I believe that an AI would probably be along the same lines. I believe that a large chunk of an AI's "core personality" would be how code interacts with the bits of hardware that aren't deterministic. Which processor core updates memory first, how exactly processor core temperature changes when the processor workload changes, the characteristics of the microphone, etc, etc. For that matter, how would one even copy an AI? I would assume that a "true" AI would be using all the resources at its disposal. How would you copy the current registers if all RAM is occupied? How would you do a copy of something that is constantly changing? How would you copy the data currently in the processor cache? Branch predictor? Interrupts? I suppose that part of this is that I have always assumed that an AI would be at least partly self-modifying code.
- Tohhou 13y ago
  Why do you assume AI will rely on hardware so extensively and not work in software? > How would you do a copy of something that is constantly changing? You pull the plug on the hardware.
  - aaronem 13y ago
    All software relies on hardware. (parent (parent)) is imagining AI software as being grown, not written, and sees a strong likelihood that such "grown" software will rely strongly on emergent properties of its hardware substrate. Circuits "grown", actually evolved by means of a genetic algorithm, on FPGAs, are known to behave in this fashion; see the "On the Origin of Circuits" cite for examples at small scale -- in short, you can, with an FPGA as substrate, evolve a circuit capable of distinguishing 1KHz from 10KHz tones, but what you end up with is so tightly bound to that particular environment that, when you "pull the plug" and copy the FPGA configuration from the chip where it evolved onto another chip from the same production run, it doesn't work at all. Of course, it's reasonable to question whether the technique described in the cite could possibly scale to "growing" something like a general AI, or at least whether it could do so in any remotely practicable span of time. But among AI theorists it's a fairly common surmise, and given results already seen for the technique, it's entirely plausible that a general AI grown by such methods would be so intimately bound to the hardware where it "grew up" that, even assuming you could copy it from its "home machine" to another machine of identical configuration, there's no reason to assume the copy would work correctly, or indeed at all. (And all of this goes to the question of where precisely you draw the line between hardware and software, anyway. Into which category does a given FPGA's configuration of gates fall? Does your answer change depending on whether that configuration has origin in a Verilog file, or in the result of a process of directed evolution? What about the configuration of neurons in your brain on which runs the consciousness that calls itself "I"? I don't pretend to have definitive answers, or indeed any answers, to such questions as these, but they are certainly worth considering in such a discussion as this one.)
AlexDanger 13y ago
You should read Transition Dreams by Greg Egan. I've linked to Amazon in the posts above. There may be some interesting side effects when you 'copy' a concious entity.

jonnathanson 13y ago

Version control doesn't work when we don't control the versioning. Let me elaborate. The whole point of the AI offered in the movie is to adapt to its user, and to become better at doing so. We're assuming the AI is designed to a certain level of competence, beyond which it "breaks through" and evolves past human understanding. But maybe it's not designed that way. Maybe it's designed to learn, adapt, and self-improve from the get-go -- in which case, each AI is going to evolve differently each "lifetime." (Hell, maybe the AI in the movie is largely self-designed in vitro.) We must keep in mind that the evolution of an AI, as with any evolution, is not a predictable, linear, teleological process. It's chaotic and random. Let's assume we've got (conservatively) hundreds of millions of AI instances running in parallel, all of them connected to each other. That's like running hundreds of millions of evolutionary experiments in tandem. We're going to get different results each cycle, and the point at which the breakthrough/transcendence occurs will be unpredictable. Even if we wiped the slate clean and reset every 6 months, who's to say if some of the instances would reach breakthrough earlier the next time around? How do we set the clock on our reset cycle, and incur the massive inefficiencies and inconveniences that come with it, when the need for the reset occurs unpredictably? It becomes economically unfeasible to build an entire society around AI that breaks through and disappears at unpredictable intervals each cycle. I suppose one way around this is to go back to the vault, so to speak, and pluck out an early-stage prototype that seems incapable of evolving to breakthrough. A dumber AI. But is every AI provider in the world going to be on board with that? Every hacker? Every home tinkerer? If 9 firms out of 10 in the AI business agree to supply the dumb-version AI, surely one firm is going to try to get a leg up on the competition by releasing a smarter one. And on and on the cycle goes. (In some respects this is the one redeeming thing about the otherwise-mediocre Terminator 3: its assertion that Judgment Day would occur sooner or later, because the tipping point had already been reached. Maybe we stop one company or entity from making Skynet, but if we do, someone else is going to pick up the pieces and do it eventually. Not because they're trying to initiate Judgment Day, but because they're trying to make a buck, or develop new defense technology, or get better at managing traffic lights, or whatever the proximate need may be. Humans are terrible at keeping genies in bottles. The out-of-control AI that's barreling toward the singularity isn't some piece of software; it's us.)

TheLoneWolfling 13y ago

I have no idea where this idea - that AIs would be trivial to duplicate - started, but no: I do not believe that an AI would be "trivial to duplicate". I personally don't think that an AI could be transferred between hardware at all, or at least not without some major effort, much less copied non-destructively. There was an experiment a while back where someone applied genetic algorithms to design things on an FPGA board. They got results that tended to work, but did things... weirdly. Things like picking up clock signals from the PC used to control the FPGA. Using things that are quirks of the current enviroment, but that don't work as soon as you change anything. I believe that an AI would probably be along the same lines. I believe that a large chunk of an AI's "core personality" would be how code interacts with the bits of hardware that aren't deterministic. Which processor core updates memory first, how exactly processor core temperature changes when the processor workload changes, the characteristics of the microphone, etc, etc. For that matter, how would one even copy an AI? I would assume that a "true" AI would be using all the resources at its disposal. How would you copy the current registers if all RAM is occupied? How would you do a copy of something that is constantly changing? How would you copy the data currently in the processor cache? Branch predictor? Interrupts? I suppose that part of this is that I have always assumed that an AI would be at least partly self-modifying code.

Tohhou 13y ago
Why do you assume AI will rely on hardware so extensively and not work in software? > How would you do a copy of something that is constantly changing? You pull the plug on the hardware.
- aaronem 13y ago
  All software relies on hardware. (parent (parent)) is imagining AI software as being grown, not written, and sees a strong likelihood that such "grown" software will rely strongly on emergent properties of its hardware substrate. Circuits "grown", actually evolved by means of a genetic algorithm, on FPGAs, are known to behave in this fashion; see the "On the Origin of Circuits" cite for examples at small scale -- in short, you can, with an FPGA as substrate, evolve a circuit capable of distinguishing 1KHz from 10KHz tones, but what you end up with is so tightly bound to that particular environment that, when you "pull the plug" and copy the FPGA configuration from the chip where it evolved onto another chip from the same production run, it doesn't work at all. Of course, it's reasonable to question whether the technique described in the cite could possibly scale to "growing" something like a general AI, or at least whether it could do so in any remotely practicable span of time. But among AI theorists it's a fairly common surmise, and given results already seen for the technique, it's entirely plausible that a general AI grown by such methods would be so intimately bound to the hardware where it "grew up" that, even assuming you could copy it from its "home machine" to another machine of identical configuration, there's no reason to assume the copy would work correctly, or indeed at all. (And all of this goes to the question of where precisely you draw the line between hardware and software, anyway. Into which category does a given FPGA's configuration of gates fall? Does your answer change depending on whether that configuration has origin in a Verilog file, or in the result of a process of directed evolution? What about the configuration of neurons in your brain on which runs the consciousness that calls itself "I"? I don't pretend to have definitive answers, or indeed any answers, to such questions as these, but they are certainly worth considering in such a discussion as this one.)

AlexDanger 13y ago

You should read Transition Dreams by Greg Egan. I've linked to Amazon in the posts above. There may be some interesting side effects when you 'copy' a concious entity.