7 ms·

My views align strongly with Andrew's stated views. >The reason I say that I don’t worry about AI turning evil is the same reason I don’t worry about overpopul

My views align strongly with Andrew's stated views. >The reason I say that I don’t worry about AI turning evil is the same reason I don’t worry about overpopulation on Mars. I think a technological singularity is a rational proposition and even consider it possible within this century. But I have no fear of a Terminator/Matrix/etc outcome. He seems to think the possibility is just too far away to even consider it. But I agree for a different reason. I don't worry because I don't think I (or any human) can accurately assess what a superintelligence would do. And to assume absolute destruction of all humans seems ridiculous. > In universe two you have one organization, maybe Carnegie Mellon or Google, that invents a self-driving car and bam! You have self-driving cars. It wasn’t available Tuesday but it’s on sale on Wednesday. >I’m in universe one. I am strongly in universe one. I have no faith in the basement dwelling loner making significant contributions to AI.

cgag 11y ago
It seems likely that one will eventually be built, and hard to assess what it'll do. How does that not seem like a problem worth working on. To assume it'd be safe seems at least as ridiculous as assuming it'd be a huge danger.
jimfleming 11y ago
I have to agree on a few points as well: We're much closer than we were a year ago but not as close as many think we are w.r.t self-driving cars, killer robots or even reliable speech recognition. Safe-guards are probably a good idea before unleashing an AGI onto the Internet but those working on it are not going to be surprised one morning that their AI project suddenly became sentient or dangerous. It will be a deliberate, massive project with lots of funding and lots of smart people working on it.
adrianm 11y ago
How do you feel about the possibility of a HAL-9000 like scenario where oversights in the design of the program lead to it exhibiting unpredictable and dangerous decision making behavior under certain conditions? The novel of '2001' goes into deeper detail than the film regarding the context in which HAL was programmed. To summarize, his mission was to deliberately withhold certain information about the mission from the crew due to the likelihood of xenophobic sentiment developing amongst them if they knew they were traveling to the site of extra terrestrial intelligence. In a sense, he was fulfilling the parameters of his mission in a way which was contrary to the purpose of it. I always interpreted this story as showing us that the more complex a system, the less accurate our predictions of its failure modes are going to be. I do not think absolute destruction of humanity is likely either, but I also don't think our first steps with AI on this level will be without the loss of life either (of both human and machine). Keep in mind, the same was and is true of space exploration and is essentially an inherent danger of working with the unknown and on the frontier of our knowledge.
- jjaredsimpson 11y ago
  I would say any intelligent entity must be unpredictable to actually be considered intelligent. Any person on the street could kill me for no reason whatsoever. But I don't live my life like that is a possibility. I also don't think about future superintelligences in that way either. I don't worry that they will be holocaust machines teetering on the brink of genocide.
Houshalter 11y ago
We don't need to come up with a convincing reason why it would kill humans, we need to come up with a convincing reason why it wouldn't. By default AI will optimize the universe to the greatest extent possible towards it's values. And if it's values aren't the same as ours, then we risk being optimized away. Our atoms are more efficiently used for other purposes. E.g. if the AI decides to use the mass of the Earth to build a dyson sphere around the sun. Or tries to build the biggest computer possible and turns the Earth into a giant computer. Or it values reproduction and tries to replicate as much as possible and outcompetes us. Or it values self preservation and kills us because we could make other AIs which could threaten it.
- jimfleming 11y ago
  > By default AI will optimize the universe to the greatest extent possible towards it's values. Hmm, I'm not sure I understand this line of thinking. Intelligence is complex, soft, and can even conflict with itself. Why do you envision artificial intelligence as if it were a pure optimization algorithm with high-level problem solving abilities? It seems to me that a fuzzy metaheuristic with lots of competing goals would be a better comparison and this does not lend itself to the "paperclip factory" quite as readily.
  - TheOtherHobbes 11y ago
    "Siri - make me a Dyson sphere." The irony about these conversations is that humans are already far more of a threat to humanity now than an AI might be, possibly, at some undetermined time in the future.
    - JoeAltmaier 11y ago
      Not true! AI's are thinking and acting at the speed of light. We put them in charge of our infrastructure all the time (internet, phone systems, stock trading, medical records and on and on). Already they have tremendous potential to do us harm, at a rate far outstripping any given humans. They only lack the will, so far.
Jach 11y ago
If you have no idea what the superintelligence will do, then you should set your expectation to be the outcome of any random one of possible superintelligent mind designs. Not all superintelligent mind designs are totally unpredictable -- if I know it will prefer winning in a game of chess to losing, I predict it will win every match against a human, even if I don't know exactly what moves it will make. If I know it will prefer existing to not existing, I predict it will seek the resources to continue existing. If the superintelligence has the hardwired goal of producing paperclips, one of the possible outcomes is tiling the solar system with paperclips. Another is that humans succeed in shutting it down but not until after it's killed people. Once you're done enumerating as many possibilities as you can for what a superintelligence might have as its goals, simple and complicated or even nonexistent, and what the various outcomes of a superintelligent agent with those goals broadly look like, find out how many of them are positive, negative, or neutral to the future of humanity, especially if these things the superintelligence does requires resources humans also use. I did this once, it convinced me that the likely case is something negative for humanity, even if it's unlikely to be a Terminator/Matrix/Hollywood scenario or really any specific enumeration. What further convinced me that the likely outcome is still negative for humanity, in the event of people making an honest but uninformed attempt at after having created an AGI making sure it won't be net-negative for humanity, is understanding the complexity of human value and how almost right is still very wrong. (http://wiki.lesswrong.com/wiki/Complexity_of_value http://wiki.lesswrong.com/wiki/Complexity_of_value)
- sampo 11y ago
  Kind of like a gorilla trying to predict what humans will do?
  - rndn 11y ago
    Exactly. However, a superintelligence might by vastly more superior to us than we are to chimps: http://intelligenceexplosion.com/wp-content/uploads/2011/12/scale_of_intelligence.png http://intelligenceexplosion.com/wp-content/uploads/2011/12/... I actually don't quite agree with the spaces on this chart because the scientific method should set us far apart from mice and chimps so that they would almost merge in a single point. Anyway, there are a handful of reasons to assume that an artificial general intelligence would likely turn out to be far superior to us: (1) Hardware is a much less noisy computation environment compared to the badly insulated neurons in our wetware (though this might be a feature rather than a bug), (2) our neurons operate at about 200 Hz, microchips at 10 million as much, (3) they would have extremely fast access to physical simulations and other computational resources, knowledge bases, other AIs, (4) they would never get tired, no body to maintain, no social obligations, and (5) they could replicate within seconds and easily modify their own sourcecode.
0xdeadbeefbabe 11y ago
What about a basement dwelling loner singularity? Gauss was a loner, and I'd have faith in him.
- jjaredsimpson 11y ago
  In what sense could gauss be considered a loner. He corresponded and collaborated extensively.
  - 0xdeadbeefbabe 11y ago
    At some point in his life he looked like a loner at least according to http://www.users.miamioh.edu/shermalw/honors_2001_fall/honors_papers_2001/pettita_2001.html http://www.users.miamioh.edu/shermalw/honors_2001_fall/honor... In another sense being a genius can make you feel alone even when you are collaborating.
heurist 11y ago
> I have no faith in the basement dwelling loner making significant contributions to AI. Why not? There's a strong history of individuals outside of academia making large contributions to science.
- astrocyte 11y ago
  He probably doesn't consider it due to group think... Probably the biggest thing limiting lots of individuals from ushering in a new paradigm. Truthfully stated and functionally so, the biggest contributions/changes come from someone thinking outside of the accepted (limited) view: the basement dweller not following the herd.. Many could be horrible wrong but all it takes is one to be right. Meanwhile, in the box, no one sees beyond it or its limits.
  - jjaredsimpson 11y ago
    This is a motivational poster, but it doesn't reflect reality. There are lots of people who go with the current and also against the current. The majority of people who go against the current of mainstream knowledge are quacks or misguided or simply not competent to make the discovery they want to make. The few who make novel contributions are not representative. People working in groups building on the success and knowledge of others make the vast majority of meaningful contributions in all fields.
- eli_gottlieb 11y ago
  Well, if the basement dweller can build software that performs unprecedentedly well on a known task with a known benchmark data-set, and explain how he did it, and he's read at least some previous academic literature... Sure. But most basement dwellers don't go to that much effort.
  - heurist 11y ago
    Most, not all... The real problem is vetting the "how" and dissemination of information. A non-academic might have no idea how to present their findings through journals or conferences, even if their ideas are well-formed and valid. I imagine there would also be strong push-back from academics averse to outside contributors (at least for big changes). What if known benchmarks are insufficient? How will the outsider convince the professionals? There's also a problem of access to literature... journals can be expensive if you aren't getting them for free at a university. There are a lot of free journals online nowadays but they don't always cover everything you might need. It's hard to tell a crank from a genius, but that doesn't mean there are no geniuses hidden away out there.

cgag 11y ago

It seems likely that one will eventually be built, and hard to assess what it'll do. How does that not seem like a problem worth working on. To assume it'd be safe seems at least as ridiculous as assuming it'd be a huge danger.

jimfleming 11y ago

I have to agree on a few points as well: We're much closer than we were a year ago but not as close as many think we are w.r.t self-driving cars, killer robots or even reliable speech recognition. Safe-guards are probably a good idea before unleashing an AGI onto the Internet but those working on it are not going to be surprised one morning that their AI project suddenly became sentient or dangerous. It will be a deliberate, massive project with lots of funding and lots of smart people working on it.

adrianm 11y ago

How do you feel about the possibility of a HAL-9000 like scenario where oversights in the design of the program lead to it exhibiting unpredictable and dangerous decision making behavior under certain conditions? The novel of '2001' goes into deeper detail than the film regarding the context in which HAL was programmed. To summarize, his mission was to deliberately withhold certain information about the mission from the crew due to the likelihood of xenophobic sentiment developing amongst them if they knew they were traveling to the site of extra terrestrial intelligence. In a sense, he was fulfilling the parameters of his mission in a way which was contrary to the purpose of it. I always interpreted this story as showing us that the more complex a system, the less accurate our predictions of its failure modes are going to be. I do not think absolute destruction of humanity is likely either, but I also don't think our first steps with AI on this level will be without the loss of life either (of both human and machine). Keep in mind, the same was and is true of space exploration and is essentially an inherent danger of working with the unknown and on the frontier of our knowledge.

jjaredsimpson 11y ago
I would say any intelligent entity must be unpredictable to actually be considered intelligent. Any person on the street could kill me for no reason whatsoever. But I don't live my life like that is a possibility. I also don't think about future superintelligences in that way either. I don't worry that they will be holocaust machines teetering on the brink of genocide.

Houshalter 11y ago

We don't need to come up with a convincing reason why it would kill humans, we need to come up with a convincing reason why it wouldn't. By default AI will optimize the universe to the greatest extent possible towards it's values. And if it's values aren't the same as ours, then we risk being optimized away. Our atoms are more efficiently used for other purposes. E.g. if the AI decides to use the mass of the Earth to build a dyson sphere around the sun. Or tries to build the biggest computer possible and turns the Earth into a giant computer. Or it values reproduction and tries to replicate as much as possible and outcompetes us. Or it values self preservation and kills us because we could make other AIs which could threaten it.

jimfleming 11y ago
> By default AI will optimize the universe to the greatest extent possible towards it's values. Hmm, I'm not sure I understand this line of thinking. Intelligence is complex, soft, and can even conflict with itself. Why do you envision artificial intelligence as if it were a pure optimization algorithm with high-level problem solving abilities? It seems to me that a fuzzy metaheuristic with lots of competing goals would be a better comparison and this does not lend itself to the "paperclip factory" quite as readily.
- TheOtherHobbes 11y ago
  "Siri - make me a Dyson sphere." The irony about these conversations is that humans are already far more of a threat to humanity now than an AI might be, possibly, at some undetermined time in the future.
  - JoeAltmaier 11y ago
    Not true! AI's are thinking and acting at the speed of light. We put them in charge of our infrastructure all the time (internet, phone systems, stock trading, medical records and on and on). Already they have tremendous potential to do us harm, at a rate far outstripping any given humans. They only lack the will, so far.

Jach 11y ago

If you have no idea what the superintelligence will do, then you should set your expectation to be the outcome of any random one of possible superintelligent mind designs. Not all superintelligent mind designs are totally unpredictable -- if I know it will prefer winning in a game of chess to losing, I predict it will win every match against a human, even if I don't know exactly what moves it will make. If I know it will prefer existing to not existing, I predict it will seek the resources to continue existing. If the superintelligence has the hardwired goal of producing paperclips, one of the possible outcomes is tiling the solar system with paperclips. Another is that humans succeed in shutting it down but not until after it's killed people. Once you're done enumerating as many possibilities as you can for what a superintelligence might have as its goals, simple and complicated or even nonexistent, and what the various outcomes of a superintelligent agent with those goals broadly look like, find out how many of them are positive, negative, or neutral to the future of humanity, especially if these things the superintelligence does requires resources humans also use. I did this once, it convinced me that the likely case is something negative for humanity, even if it's unlikely to be a Terminator/Matrix/Hollywood scenario or really any specific enumeration. What further convinced me that the likely outcome is still negative for humanity, in the event of people making an honest but uninformed attempt at after having created an AGI making sure it won't be net-negative for humanity, is understanding the complexity of human value and how almost right is still very wrong. (http://wiki.lesswrong.com/wiki/Complexity_of_value http://wiki.lesswrong.com/wiki/Complexity_of_value)

sampo 11y ago
Kind of like a gorilla trying to predict what humans will do?
- rndn 11y ago
  Exactly. However, a superintelligence might by vastly more superior to us than we are to chimps: http://intelligenceexplosion.com/wp-content/uploads/2011/12/scale_of_intelligence.png http://intelligenceexplosion.com/wp-content/uploads/2011/12/... I actually don't quite agree with the spaces on this chart because the scientific method should set us far apart from mice and chimps so that they would almost merge in a single point. Anyway, there are a handful of reasons to assume that an artificial general intelligence would likely turn out to be far superior to us: (1) Hardware is a much less noisy computation environment compared to the badly insulated neurons in our wetware (though this might be a feature rather than a bug), (2) our neurons operate at about 200 Hz, microchips at 10 million as much, (3) they would have extremely fast access to physical simulations and other computational resources, knowledge bases, other AIs, (4) they would never get tired, no body to maintain, no social obligations, and (5) they could replicate within seconds and easily modify their own sourcecode.

0xdeadbeefbabe 11y ago

What about a basement dwelling loner singularity? Gauss was a loner, and I'd have faith in him.

jjaredsimpson 11y ago
In what sense could gauss be considered a loner. He corresponded and collaborated extensively.
- 0xdeadbeefbabe 11y ago
  At some point in his life he looked like a loner at least according to http://www.users.miamioh.edu/shermalw/honors_2001_fall/honors_papers_2001/pettita_2001.html http://www.users.miamioh.edu/shermalw/honors_2001_fall/honor... In another sense being a genius can make you feel alone even when you are collaborating.

heurist 11y ago

> I have no faith in the basement dwelling loner making significant contributions to AI. Why not? There's a strong history of individuals outside of academia making large contributions to science.

astrocyte 11y ago
He probably doesn't consider it due to group think... Probably the biggest thing limiting lots of individuals from ushering in a new paradigm. Truthfully stated and functionally so, the biggest contributions/changes come from someone thinking outside of the accepted (limited) view: the basement dweller not following the herd.. Many could be horrible wrong but all it takes is one to be right. Meanwhile, in the box, no one sees beyond it or its limits.
- jjaredsimpson 11y ago
  This is a motivational poster, but it doesn't reflect reality. There are lots of people who go with the current and also against the current. The majority of people who go against the current of mainstream knowledge are quacks or misguided or simply not competent to make the discovery they want to make. The few who make novel contributions are not representative. People working in groups building on the success and knowledge of others make the vast majority of meaningful contributions in all fields.
eli_gottlieb 11y ago
Well, if the basement dweller can build software that performs unprecedentedly well on a known task with a known benchmark data-set, and explain how he did it, and he's read at least some previous academic literature... Sure. But most basement dwellers don't go to that much effort.
- heurist 11y ago
  Most, not all... The real problem is vetting the "how" and dissemination of information. A non-academic might have no idea how to present their findings through journals or conferences, even if their ideas are well-formed and valid. I imagine there would also be strong push-back from academics averse to outside contributors (at least for big changes). What if known benchmarks are insufficient? How will the outsider convince the professionals? There's also a problem of access to literature... journals can be expensive if you aren't getting them for free at a university. There are a lot of free journals online nowadays but they don't always cover everything you might need. It's hard to tell a crank from a genius, but that doesn't mean there are no geniuses hidden away out there.