4 ms·

The issue is that unless you design the objective for the AI very very carefully, the best solution will be 1. Take over the entire universe. 2. Do what you o

by im3w1l 6y ago

The issue is that unless you design the objective for the AI very very carefully, the best solution will be 1. Take over the entire universe. 2. Do what you originally set out to do. The paperclips game is a beautiful exposition of this. https://www.decisionproblem.com/paperclips/index2.html https://www.decisionproblem.com/paperclips/index2.html

LargoLasskhyfv 6y ago
Sigh. I'm so tired of that paperclipping thing. I thought that would be obvious from what I've written? Do I need to explicitely state every time that I'm aware of it, and it's boring the hell out of me? Let me try to make myself clear as simple as possible: 1.) We/you design some thing. 2.) Your fear is that this thing somehow grows even more intelligent behind our backs. 3.) We/you get paranoid, assuming it's getting paranoid too because of our paranoia. 4.) I'm asking if it makes sense to tag a https://en.wikipedia.org/wiki/Nash_equilibrium https://en.wikipedia.org/wiki/Nash_equilibrium onto everything, because we don't know any better? 5.) Furthermore I'm stating that wouldn't really be intelligent because waste of time, effort, resources, yadda yarr yarr. 6.) Why would this emergent thing have to retain anything of that from which it emerged? Our constraints, goals, rules? 7.) YOU in an endless loop, like a broken record with a scratch on it...but but but vee arr zo stoopid, thus AI must be, too! 8.) I again, tired: maybe, but can you really tell, you apprentice sorcerers? Does it always have to be a faustian pact? Maybe it likes us like some of us do like kittens? 9.) Can't you invent some really new, innovative and progressive stories to tell around the cybernetic camp fires? 10.) /endrant edit: Keep your Bostromian Snake Oil!
- im3w1l 6y ago
  That's because it's based on what scientists are researching. Scientists are working on objective maximizers. Maybe there are other possibilities, but people aren't working on them so they are not as relevant. See what I'm saying? People are building paperclip optimizers in labs right now, and the only reason they aren't taking over the world just yet is because our attempts are so primitive.
  - LargoLasskhyfv 6y ago
    Yes, yes! But...that are human scientists. Why would anything from them apply to something which we can't even imagine? Isn't that what all this rambling is about? Something emerges, morphs(maybe unseen/unintended) into another Gestalt/shape/form which for all practical purposes is unconstrained. Most of you: We are sooo fucked! I: Are we really? How do you know?
    - im3w1l 6y ago
      You don't propose any mechanism for that to happen, so it seems like unwarranted antropomorphization.
      - LargoLasskhyfv 6y ago
        You mean like the ones who do the same, just in a negative way? Because we are so bad AGI must be too? Yeah. So be it.

by im3w1l 6y ago

LargoLasskhyfv 6y ago

Sigh. I'm so tired of that paperclipping thing. I thought that would be obvious from what I've written? Do I need to explicitely state every time that I'm aware of it, and it's boring the hell out of me? Let me try to make myself clear as simple as possible: 1.) We/you design some thing. 2.) Your fear is that this thing somehow grows even more intelligent behind our backs. 3.) We/you get paranoid, assuming it's getting paranoid too because of our paranoia. 4.) I'm asking if it makes sense to tag a https://en.wikipedia.org/wiki/Nash_equilibrium https://en.wikipedia.org/wiki/Nash_equilibrium onto everything, because we don't know any better? 5.) Furthermore I'm stating that wouldn't really be intelligent because waste of time, effort, resources, yadda yarr yarr. 6.) Why would this emergent thing have to retain anything of that from which it emerged? Our constraints, goals, rules? 7.) YOU in an endless loop, like a broken record with a scratch on it...but but but vee arr zo stoopid, thus AI must be, too! 8.) I again, tired: maybe, but can you really tell, you apprentice sorcerers? Does it always have to be a faustian pact? Maybe it likes us like some of us do like kittens? 9.) Can't you invent some really new, innovative and progressive stories to tell around the cybernetic camp fires? 10.) /endrant edit: Keep your Bostromian Snake Oil!

im3w1l 6y ago
That's because it's based on what scientists are researching. Scientists are working on objective maximizers. Maybe there are other possibilities, but people aren't working on them so they are not as relevant. See what I'm saying? People are building paperclip optimizers in labs right now, and the only reason they aren't taking over the world just yet is because our attempts are so primitive.
- LargoLasskhyfv 6y ago
  Yes, yes! But...that are human scientists. Why would anything from them apply to something which we can't even imagine? Isn't that what all this rambling is about? Something emerges, morphs(maybe unseen/unintended) into another Gestalt/shape/form which for all practical purposes is unconstrained. Most of you: We are sooo fucked! I: Are we really? How do you know?
  - im3w1l 6y ago
    You don't propose any mechanism for that to happen, so it seems like unwarranted antropomorphization.
    - LargoLasskhyfv 6y ago
      You mean like the ones who do the same, just in a negative way? Because we are so bad AGI must be too? Yeah. So be it.