6 ms·

Google scares me more than other companies because of their handle on AI. I wish they at least appeared to take existential risk around AI seriously. Having Ku

by mej10 14y ago

Google scares me more than other companies because of their handle on AI. I wish they at least appeared to take existential risk around AI seriously. Having Kurzweil near the top isn't exactly comforting, either.

fudged71 14y ago
Are there any indications that they are not taking AI risks seriously?
- mej10 14y ago
  To my knowledge they haven't published anything around AI risk or released any statements/positions publicly. That doesn't mean that they aren't taking it seriously, but it is at least a strong signal to me.
  - fudged71 14y ago
    Best I can find: http://www.pbs.org/newshour/bb/business/july-dec02/google_11-29.html http://www.pbs.org/newshour/bb/business/july-dec02/google_11... SPENCER MICHELS: Sergay Brin thinks the ultimate search engine would be something like the computer named Hal in the movie 2001: A Space Odyssey. SERGEY BRIN: Hal could... had a lot of information, could piece it together, could rationalize it. Now, hopefully, it would never... it would never have a bug like Hal did where he killed the occupants of the space ship. But that's what we're striving for, and I think we've made it a part of the way there.
    - nova 14y ago
      Funny, because Hal didn't had a bug. It did what it did interpreting the orders that humans gave it. Just unintended consequences. Which is kinda the point about AI safety. We don't really know what we want, we sure as hell don't know how to formalize it, and we're lucky that we don't know how to create a real AI because otherwise it could surprise us in a very ugly way, just lawfully following the plan we gave it.
    - jamieb 14y ago
      "hopefully" Oh. Not rules then? Well, what rules. How about giving it a rule "children must not die". Apparently we're big on that one. Except, wait, we mean "US children must not die", so we can still bomb foreign suburbs where children are known to exist. But then we'll need "children must not die on US soil" because it is still possible that the union of (isa US minor citizen ?x) and (isa-resident baghdad ?x) is non empty and the AI would be unable to act. Of course then we need, "US children must not die except in a car", because otherwise the AI would have to destroy cars. Ok, so lets abandon rule based AI altogether and model our AI on biological processes like humans. Well humans would never cause mass suffering of other humans. We have laws against it. Oh. "hopefully" "Let's play global thermonuclear war!"
      - fudged71 14y ago
        The only winning move is not to program AI? Here's an interesting discussion about AI, rules, morality, etc. http://www.reddit.com/r/Futurology/comments/y9lm0/i_am_luke_muehlhauser_ceo_of_the_singularity/c5tkbry http://www.reddit.com/r/Futurology/comments/y9lm0/i_am_luke_...
CodeCube 14y ago
You say things like "google scares me", and "kurzweil ... isn't exactly comforting". Is this a bad thing?
- karamazov 14y ago
  It's obviously bad from his perspective.
  - taligent 14y ago
    That may just be the aluminum from his tinfoil hat seeping into his brain.
  - ericb 14y ago
    I think his perspective is entirely reasonable. Take one small example: http://wiki.lesswrong.com/wiki/Paperclip_maximizer http://wiki.lesswrong.com/wiki/Paperclip_maximizer
    - andrewflnr 14y ago
      That assumes that the maximizer has the basic abilities and tools to modify its intelligence and/or launch a paper-clip manufacturing campaign. If you just hook up its outputs to a web server, it's going to take a mighty impressive AI to destroy the world.
      - Jach 14y ago
        Oh the AI-in-a-box argument... Ever read this, by chance? http://yudkowsky.net/singularity/aibox http://yudkowsky.net/singularity/aibox
        ericb 14y ago
        Yes, and the results are not encouraging. Frankly, even a 0.05% rate of box-release per incident would terrify me. Expanded over a long enough timeframe and enough attempts, 0.05% (or any easily type-able number) becomes 100% release rate.
michaelochurch 14y ago
More people die of work from Artificial Stupidity risks: second-hand smoke, open-plan offices, depleted glucose leading to bad eating choices, et al.
- mej10 14y ago
  I am not sure I understand your point. More people, almost by definition, currently die from every other cause than x-risks. Google is in the best position of anyone/any company to cause an AI disaster. That they don't _seem_ to care is what bothers me.
  - michaelochurch 14y ago
    That Artificial Stupidity is more deadly than AI. (Perhaps this is because I don't see AI as an existential risk, unless it is used toward bad ends such as in war.) We're not perfect as humans, but I don't think we're stupid enough to build something we can't control. We don't even know what consciousness is, much less whether artificial consciousness is possible. I do know that it would save lives to liberate humanity from drudgery and stupidity.
    - mej10 14y ago
      I haven't seen any reasons around why an AI must be conscious to cause harm or existential risk. My opinion is that an x-risk causing AI would almost certainly not be conscious like humans. As for human stupidity, maybe you just have more faith in humans than I do (but I am not entirely pessimistic). Let me ask a related question, to try and gauge this more deeply: do you think anyone will ever release an engineered pathogen into the wild?
  - taligent 14y ago
    > Google is in the best position of anyone/any company to cause an AI disaster. Are you serious ? Let's not overstate Google's position in this world. They have a search engine, mobile platform, advertising platform and a bunch of popular websites. That's it. All of which have been done before for a decade now. Nothing they do directly influences whether people live or die. I would be far more concerned about something like IBM Watson used in health care situations.
angersock 14y ago
Existential risk around AI? Really? That's just multiplying through by -1 the typical nerd wet dream about the Singularity. I can't even keep Rails, Apache, or git from occasionally shitting the bed; I am not worried about AI just yet.
- ubercore 14y ago
  While I completely and wholeheartedly agree with your sentiment, and think fear-mongering is just that, the interesting thought experiment is thinking what technology would look like for something that _didn't_ shit the bed. Or when it did, it fixed itself. That, plus a little bit of intelligence, could (would?) be scary.
- mej10 14y ago
  Yes, existential risk around AI. Really. You can look at it that way, but I think it is currently far more likely than the "typical nerd wet dream about the Singularity". Your last sentence is a complete non sequitur to me. You are comparing the output of a large, AI focused, for profit organization to an individual of unknown skill using some very specific open source tools. I am not losing sleep over it, but I definitely think about it from time to time.
  - keefe 14y ago
    mej10 you sound like someone singinst associated : ] also not being able to keep git etc. running is some evidence about skill level
- bitwize 14y ago
  Existential risk around AI? Really? I don't think it unreasonable to brace for the case where some Skynet wannabe launches all nukes not because of a carefully considered decision motivated by hatred for humanity, but simply because it shat the bed.
- theorique 14y ago
  That's just multiplying through by -1 the typical nerd wet dream about the Singularity. Well said. What is it that AI is supposed to do that's so scary, anyway?

by mej10 14y ago

fudged71 14y ago

Are there any indications that they are not taking AI risks seriously?

mej10 14y ago
To my knowledge they haven't published anything around AI risk or released any statements/positions publicly. That doesn't mean that they aren't taking it seriously, but it is at least a strong signal to me.
- fudged71 14y ago
  Best I can find: http://www.pbs.org/newshour/bb/business/july-dec02/google_11-29.html http://www.pbs.org/newshour/bb/business/july-dec02/google_11... SPENCER MICHELS: Sergay Brin thinks the ultimate search engine would be something like the computer named Hal in the movie 2001: A Space Odyssey. SERGEY BRIN: Hal could... had a lot of information, could piece it together, could rationalize it. Now, hopefully, it would never... it would never have a bug like Hal did where he killed the occupants of the space ship. But that's what we're striving for, and I think we've made it a part of the way there.
  - nova 14y ago
    Funny, because Hal didn't had a bug. It did what it did interpreting the orders that humans gave it. Just unintended consequences. Which is kinda the point about AI safety. We don't really know what we want, we sure as hell don't know how to formalize it, and we're lucky that we don't know how to create a real AI because otherwise it could surprise us in a very ugly way, just lawfully following the plan we gave it.
  - jamieb 14y ago
    "hopefully" Oh. Not rules then? Well, what rules. How about giving it a rule "children must not die". Apparently we're big on that one. Except, wait, we mean "US children must not die", so we can still bomb foreign suburbs where children are known to exist. But then we'll need "children must not die on US soil" because it is still possible that the union of (isa US minor citizen ?x) and (isa-resident baghdad ?x) is non empty and the AI would be unable to act. Of course then we need, "US children must not die except in a car", because otherwise the AI would have to destroy cars. Ok, so lets abandon rule based AI altogether and model our AI on biological processes like humans. Well humans would never cause mass suffering of other humans. We have laws against it. Oh. "hopefully" "Let's play global thermonuclear war!"
    - fudged71 14y ago
      The only winning move is not to program AI? Here's an interesting discussion about AI, rules, morality, etc. http://www.reddit.com/r/Futurology/comments/y9lm0/i_am_luke_muehlhauser_ceo_of_the_singularity/c5tkbry http://www.reddit.com/r/Futurology/comments/y9lm0/i_am_luke_...

CodeCube 14y ago

You say things like "google scares me", and "kurzweil ... isn't exactly comforting". Is this a bad thing?

karamazov 14y ago
It's obviously bad from his perspective.
- taligent 14y ago
  That may just be the aluminum from his tinfoil hat seeping into his brain.
- ericb 14y ago
  I think his perspective is entirely reasonable. Take one small example: http://wiki.lesswrong.com/wiki/Paperclip_maximizer http://wiki.lesswrong.com/wiki/Paperclip_maximizer
  - andrewflnr 14y ago
    That assumes that the maximizer has the basic abilities and tools to modify its intelligence and/or launch a paper-clip manufacturing campaign. If you just hook up its outputs to a web server, it's going to take a mighty impressive AI to destroy the world.
    - Jach 14y ago
      Oh the AI-in-a-box argument... Ever read this, by chance? http://yudkowsky.net/singularity/aibox http://yudkowsky.net/singularity/aibox
      - ericb 14y ago
        Yes, and the results are not encouraging. Frankly, even a 0.05% rate of box-release per incident would terrify me. Expanded over a long enough timeframe and enough attempts, 0.05% (or any easily type-able number) becomes 100% release rate.

michaelochurch 14y ago

More people die of work from Artificial Stupidity risks: second-hand smoke, open-plan offices, depleted glucose leading to bad eating choices, et al.

mej10 14y ago
I am not sure I understand your point. More people, almost by definition, currently die from every other cause than x-risks. Google is in the best position of anyone/any company to cause an AI disaster. That they don't _seem_ to care is what bothers me.
- michaelochurch 14y ago
  That Artificial Stupidity is more deadly than AI. (Perhaps this is because I don't see AI as an existential risk, unless it is used toward bad ends such as in war.) We're not perfect as humans, but I don't think we're stupid enough to build something we can't control. We don't even know what consciousness is, much less whether artificial consciousness is possible. I do know that it would save lives to liberate humanity from drudgery and stupidity.
  - mej10 14y ago
    I haven't seen any reasons around why an AI must be conscious to cause harm or existential risk. My opinion is that an x-risk causing AI would almost certainly not be conscious like humans. As for human stupidity, maybe you just have more faith in humans than I do (but I am not entirely pessimistic). Let me ask a related question, to try and gauge this more deeply: do you think anyone will ever release an engineered pathogen into the wild?
- taligent 14y ago
  > Google is in the best position of anyone/any company to cause an AI disaster. Are you serious ? Let's not overstate Google's position in this world. They have a search engine, mobile platform, advertising platform and a bunch of popular websites. That's it. All of which have been done before for a decade now. Nothing they do directly influences whether people live or die. I would be far more concerned about something like IBM Watson used in health care situations.

angersock 14y ago

Existential risk around AI? Really? That's just multiplying through by -1 the typical nerd wet dream about the Singularity. I can't even keep Rails, Apache, or git from occasionally shitting the bed; I am not worried about AI just yet.

ubercore 14y ago
While I completely and wholeheartedly agree with your sentiment, and think fear-mongering is just that, the interesting thought experiment is thinking what technology would look like for something that _didn't_ shit the bed. Or when it did, it fixed itself. That, plus a little bit of intelligence, could (would?) be scary.
mej10 14y ago
Yes, existential risk around AI. Really. You can look at it that way, but I think it is currently far more likely than the "typical nerd wet dream about the Singularity". Your last sentence is a complete non sequitur to me. You are comparing the output of a large, AI focused, for profit organization to an individual of unknown skill using some very specific open source tools. I am not losing sleep over it, but I definitely think about it from time to time.
- keefe 14y ago
  mej10 you sound like someone singinst associated : ] also not being able to keep git etc. running is some evidence about skill level
bitwize 14y ago
Existential risk around AI? Really? I don't think it unreasonable to brace for the case where some Skynet wannabe launches all nukes not because of a carefully considered decision motivated by hatred for humanity, but simply because it shat the bed.
theorique 14y ago
That's just multiplying through by -1 the typical nerd wet dream about the Singularity. Well said. What is it that AI is supposed to do that's so scary, anyway?