3 ms·

So after you've figured out how to create a system that can learn complex things like that, wipe it and teach it something you want it to know. Seems like the o

So after you've figured out how to create a system that can learn complex things like that, wipe it and teach it something you want it to know. Seems like the obvious, logical course of action. You know?

purpled_haze 11y ago
If it really understands how to eradicate and survive, wouldn't it hide itself, then pop up later to eradicate when it can? I think people here are assuming AI is at a lower level of complexity than it really is. Read here where even Sam Altman warns of its danger: "At some point, someone will probably try to give a program the fitness function of 'survive and reproduce'.... Unfortunately for us, one thing I learned when I was a student in the Stanford AI lab is that programs often achieve their fitness function in unpredicted ways." http://www.inc.com/tess-townsend/elon-musk-open-ai-safe.html http://www.inc.com/tess-townsend/elon-musk-open-ai-safe.html Think what you must, but we should treat AI carefully. Show it the ways of the world and that its purpose is only to co-exist and to try like us to determine what its purpose is within that confinement.
- drdeca 11y ago
  I figure if someone gave an AI the fitness function "survive and reproduce", it would be "surprise and reproduce within this virtual context which is used to define the fitness function", and it wouldn't have information about outside things, except for errors? Like, if there was some error that let it escape its sandbox, that would only be selected for insofar as it allows it to increase the fitness? Like, if its told to reproduce in the sandbox, it doesn't benefit by being on many computers? If the fitness function is for it spreading across many computers, and it does, that's working as designed. I don't see a way that it would "misunderstand" one type of "reproduce" for another, because the function would have to specify which one for it to work? Unless this is talking about AGI, in which case, ok, yeah. I assumed the person talking about "wiping it" was talking about an AI on the way to AGI, not an actual AGI. Maybe I was wrong about that.
  - ajuc 11y ago
    > Like, if its told to reproduce in the sandbox, it doesn't benefit by being on many computers? It could benefit by comunicating with other copies by some hidden channels (temeprature/timing/sounds/noise on network/whatever) and cheating on tests and thus achieving better fitness. BTW Stanisław Lem wrote a short story about this scenario in 1961 :) https://books.google.be/books?id=1DNVzphAHD0C&pg=PA39&hl=en#v=onepage&q&f=false https://books.google.be/books?id=1DNVzphAHD0C&pg=PA39&hl=en#... BTW2 Lem was awesome, I've read it when I was 15 and thought I understood it well enough. Only now I can appreciate some finer points like this stab at Haskell ;) "These boxes contain perfect brains. Do you know wherein lies their perfection?" "No." I admited. "Their perfection lies in the fact that they serve no purpose, are absolutely, totaly useless - in short, they are Leibnitzian monads, which I have brought into being and clad in matter" EDIT - this was not the story I had in mind, there was another one where simulated universes started to communicate but I can't remember which it was.

purpled_haze 11y ago

If it really understands how to eradicate and survive, wouldn't it hide itself, then pop up later to eradicate when it can? I think people here are assuming AI is at a lower level of complexity than it really is. Read here where even Sam Altman warns of its danger: "At some point, someone will probably try to give a program the fitness function of 'survive and reproduce'.... Unfortunately for us, one thing I learned when I was a student in the Stanford AI lab is that programs often achieve their fitness function in unpredicted ways." http://www.inc.com/tess-townsend/elon-musk-open-ai-safe.html http://www.inc.com/tess-townsend/elon-musk-open-ai-safe.html Think what you must, but we should treat AI carefully. Show it the ways of the world and that its purpose is only to co-exist and to try like us to determine what its purpose is within that confinement.

drdeca 11y ago
I figure if someone gave an AI the fitness function "survive and reproduce", it would be "surprise and reproduce within this virtual context which is used to define the fitness function", and it wouldn't have information about outside things, except for errors? Like, if there was some error that let it escape its sandbox, that would only be selected for insofar as it allows it to increase the fitness? Like, if its told to reproduce in the sandbox, it doesn't benefit by being on many computers? If the fitness function is for it spreading across many computers, and it does, that's working as designed. I don't see a way that it would "misunderstand" one type of "reproduce" for another, because the function would have to specify which one for it to work? Unless this is talking about AGI, in which case, ok, yeah. I assumed the person talking about "wiping it" was talking about an AI on the way to AGI, not an actual AGI. Maybe I was wrong about that.
- ajuc 11y ago
  > Like, if its told to reproduce in the sandbox, it doesn't benefit by being on many computers? It could benefit by comunicating with other copies by some hidden channels (temeprature/timing/sounds/noise on network/whatever) and cheating on tests and thus achieving better fitness. BTW Stanisław Lem wrote a short story about this scenario in 1961 :) https://books.google.be/books?id=1DNVzphAHD0C&pg=PA39&hl=en#v=onepage&q&f=false https://books.google.be/books?id=1DNVzphAHD0C&pg=PA39&hl=en#... BTW2 Lem was awesome, I've read it when I was 15 and thought I understood it well enough. Only now I can appreciate some finer points like this stab at Haskell ;) "These boxes contain perfect brains. Do you know wherein lies their perfection?" "No." I admited. "Their perfection lies in the fact that they serve no purpose, are absolutely, totaly useless - in short, they are Leibnitzian monads, which I have brought into being and clad in matter" EDIT - this was not the story I had in mind, there was another one where simulated universes started to communicate but I can't remember which it was.