5 ms·

Philosophically I think it's a mistake to believe there is any possibility of creating artificial human intelligence (and the mistake is in our assumptions of w

Philosophically I think it's a mistake to believe there is any possibility of creating artificial human intelligence (and the mistake is in our assumptions of what human intelligence is— computers are not human beings and human beings are not computers) ... But I've got to hand it to Elon Musk— when he believes in something he commits to doing something about it. EDIT - Saying I disagree that AI is a foregone conclusion on Hacker News is probably a bit trollish... Unfortunately I'm not smart enough to condense an explanation to this comment box— but my disagreement stems from philosophers such as Hubert Dreyfuss (What Computers Can't Do) who are working off of Heidegger and phenomenology. EDIT 2 - I know how douchey it is to namedrop Heidegger, but I really believe in this case it's necessary ...

jsutton 12y ago
Why wouldn't there be any possibility of artificial human intelligence? Our intelligence is just a manifestation of a physical system of neurons that follow the laws of physics. There's nothing inherently preventing us from recreating that.
- psycr 12y ago
  > recreating And that's the crux of the question isn't it? How do we determine if the intelligence is recreated or merely mimicked? Does such a distinction matter? Searle says yes. More here if you're interested: http://plato.stanford.edu/entries/chinese-room/ http://plato.stanford.edu/entries/chinese-room/
  - ajuc 12y ago
    This is misleading argument - there IS a Chinese speaker in the room, but it's not the person that executes the program - it's the system "person+program" that understands Chinese.
    - henrikschroder 12y ago
      The whole point of the argument is that the system doesn't understand Chinese. The system doesn't know what each sentence means, they don't evoke feelings, memories, images in the person in the room, there is no analysis, no agreement or disagreement with what is being said, it's just mechanical lookup.
      - ThomPete 12y ago
        No the whole point of the argument is that the person in the room doesn't understand Chinese.
      - nbouscal 12y ago
        And the reason the argument is absurd is that of course the room doesn't understand Chinese; it's a lookup table! No Chinese speaker interacting with the Chinese room for any reasonable period of time would think it held a Chinese speaker, because all it accepts are questions, and it answers the same question the same way every time. Of course, Searle would say that the set of rules in the room is defined to be such that a Chinese person would think they were interacting with a Chinese speaker, but the very setup of the system prevents this from being possible. The thought experiment is fundamentally incoherent.
        ajuc 12y ago
        I assumed the system has rewritable memory to remember previous questions, and the lookup is (all previous questions) -> next answer. With such setup (require infinite memory) the system is Turing-complete of course. If it works as you described it can't even answer "what was my n-th last question".
        deleted 12y ago
        [deleted]
      - ajuc 12y ago
        > The system doesn't know what each sentence means You can't prove that. For all possible experiments that can be performed we can construct system that will be indistinguishable from entity we accept as understanding Chinese. If you got atom-by-atom description of human brain without knowing beforehand what it is you wouldn't be able to reason that this brain understands Chinese, so why do you think you can judge other systems? > they don't evoke feelings, memories, images in the person in the room, We were speaking about understanding, not emotions, memories etc. Yes you can define "understanding" = being like me, but it's not fair definition, and would exclude a lot of people. And then of course you can't proove that there are no emotions and memories. They can be simulated by system as well. Even if you wanted lookup table - you would need to include writable memory so the system can rembmber previous questions when asked about them, at that point the system is self-modifiable and can be thought of as having memories and emotions. > there is no analysis, no agreement or disagreement with what is being said How do you know there's no analysis? If the system is complex enough you can't know what it does internaly. Or show me theory of consiousnes that can distinguish systems that can understand Chinese and can't basing on the specification of the system.
  - rndn 12y ago
    The definition of AI does not matter when it's able to turn everyone into paperclips: http://wiki.lesswrong.com/wiki/Paperclip_maximizer http://wiki.lesswrong.com/wiki/Paperclip_maximizer
  - ThomPete 12y ago
    The Chinese Room argument is the most sloppy argument ever made in that area. The real answer of course is we don't know. Just like the individual neuron in your brain do not understand Chinese so does the person in the room not understand it either. The entire house though does.
    - Chathamization 12y ago
      Agreed. Does anyone really think there’s a single neuron that has an understanding of an aggregate thought. “I, neuron of this human, am transmitting this sense of joy because of the humor that is found watching this film.” No individual cell in our body has a sense of us as a whole; they’re all working inside their individual “Chinese boxes.” Most of the hard problem of consciousness arguments are silly and amount to little more than “a computer has no soul.”
  - dghf 12y ago
    Daniel Dennett stuck the boot into the Chinese Room argument in Consciousness Explained (and probably in several other places as well). As he points out, it's based on an intellectual sleight-of-hand (what he calls an "intuition pump"). From the linked article on the Stanford site: > Searle imagines himself alone in a room following a computer program for responding to Chinese characters slipped under the door. Searle understands nothing of Chinese, and yet, by following the program for manipulating symbols and numerals just as a computer does, he produces appropriate strings of Chinese characters that fool those outside into thinking there is a Chinese speaker in the room. [Emphasis added.] Unless those outside are satisfied with a very rudimentary back-and-forth, such a program would have to be amazingly complicated (and those outside very patient, or the operator superhumanly fast). It's not just a question of "look up input Chinese character and return corresponding output character" -- which is never explicitly stated, but which we're subconsciously led to by the image of a human "following the program for manipulating symbols and numerals". Dennett gives an example where the outside person tells a joke, and asks the room's operator to explain it. You could also imagine a reading comprehension exercise, with questions ranging from purely structural ("What is the third word on the fifth line?") through semantic ("What colour was the girl's dress?") to analytical ("Why do you think the boy was sad?") Questions can be self-referential, or depend on previous questions (and their answers) or on a vast mass of knowledge we take for granted (a dress the colour of snow is white; a small boy might cry if his kite gets stuck in a tree). It doesn't seem so intuitive that a system that complex lacks anything we might legitimately describe as "understanding".
- steve-benjamins 12y ago
  "Our intelligence is just a manifestation of a physical system of neurons that follow the laws of physics. " That's where I would disagree. But I'm not fit to give a full explanation here— thinkers such as Hubert Dreyfuss (What Computers Can't Do) do a much better job of explaining the critique (Dreyfuss is working off of Heidegger and other phenomenalist) .
- cgearhart 12y ago
  I don't mean to be snarky, but it can be illuminating to change your statement just a little bit: >>Why wouldn't there be any possibility of making artificial humans? We are just a manifestation of a physical system of bonds between organic molecules that follow the laws of physics. There's nothing inherently preventing us from recreating that. Obviously, such a summary vastly oversimplifies the problem. It is easy to look at how far we've come and think that we must be close to the goal, but it's a bit presumptuous to think that we are near to making a strong general AI.
  - dghf 12y ago
    > but it's a bit presumptuous to think that we are near to making a strong general AI But I don't think they're presuming that. They're simply saying that it's possible in principle. How long it takes, and whether indeed we ever manage it all, are separate questions.
    - maratd 12y ago
      > They're simply saying that it's possible in principle. In principle, everything is possible, but some things are so unlikely, that it is reasonable to label them as impossible. The computer technology we have developed over the last 75 years is so fundamentally different from how our minds work that to think we will develop anything resembling our own intelligence in any reasonable time-frame is insane. That doesn't mean we can't develop a machine that is able to reason for itself and solve discrete problems. We can. That's not what AI is. AI is Artificial Intelligence. An artificial intelligence defined by us.
      - dghf 12y ago
        > In principle, everything is possible There are some things which I think it's reasonable to consider impossible in principle: e.g., FTL travel, time-travel into the past. (That's not to say it would be necessarily unreasonable to hold the opposite view).
  - wavefunction 12y ago
    I don't see how changing the statement at all changes anything? It's certainly not impossible to create artificial humans, if not within our current technological capability.
- nyrulez 12y ago
  Well how are you so sure of that ? You must have verified or modeled it somehow I guess. Wait... you haven't.
  - jsutton 12y ago
    There's nothing about our minds that can't be recreated given sufficient command of our environment. No model is needed to understand that. It's not magic, after all, and it's a fairly common process.
ajuc 12y ago
You can easily simulate einvironment for the AI in your computer. If you cut some senses it is feasible even now, and blind people are certainly human and intelligent so it's not required. I don't see impossibility there, just impracticability (why create mechanic horse if we can just create car).
ThomPete 12y ago
Thats not what worries people like Musk. It's that we will create something far superior to human intelligence. And if you believe that something as complex as the brain came through evolution but life started out as immaterial matter then I find it hard to understand why you don't think this can be recreated?
- deleted 12y ago
  [deleted]
drzaiusapelord 12y ago
I'm not sure why human cognition is the exception in a world of machines that can emulate anything. My grandfather would never have believed that we have cars that drive themselves. Many baby boomers I work with are still wowed by the basic AI of Siri/GNow/Cortana, or even google search. Common sense and philosophical musings just isn't enough to figure out if something is feasible. Thinking isn't that hard of a nut to crack, imho. Enough of a ruleset/branch logic is thinking. We don't need to emulate a person perfectly, the same way we don't have self-driving cars losing their tempers or tailgating people out of spite. We just need to distill the intelligence we want and voila, you have AI. AI isn't just a HAL-like character. Its a lot of things. In fact, it surrounds us. There's expert systems, fuzzy logic, etc controlling pretty much the entirety of our technological lives. I have no idea what drugs Musk is on, but its dangerous for him to piss on AI publically. I imagine, from a practical pov, spreading FUD about AI makes sense for him economically. For someone who wants human spaceflight, talking down about robots and AI is a means to an end. Now the public doesn't want a cheap-ish robot mission on Mars with some guy in Houston wearing an Oculus-like headset. Or autonomous robots on the moon. They want a meatbag on the surface, for their own emotional and philosophical needs, price be dammed. "Price be damned" in spaceflight makes Musk a very happy camper, considering he's going to get a lion's share of that money. There's something really neo-luddite about human spaceflight. Its like we can't admit to ourselves that machines, AI, etc just are a better fit for it. Anyone else notice that about three months ago there's been endless anti-AI sentiment in the press? Where is this stuff coming from? Who is funding it? I really think Musk and others see it as a major threat to their companies and social capital and are trying to change the narrative to suit them and their goals. I don't think we typically appreciate how one wealthy person or organization can easily astroturf the web and the press. Personally, I think the more feasible futurist path is actually taking the environment seriously, pushing the path to peace politically, trying to bring in a post-scarcity world via cheap replication, and using AI the same way we use all tools -- for our benefit. Fleeing to some other planet or moon without fixing life on Earth first guarantees we bring our terrible politics and baggage to this new destination. Let's fix things here first and then worry about moving to Mars. "Fixing things" more than likely involves strong AI implementations in various parts of our lives.
- happyscrappy 12y ago
breuleux 12y ago
Skimming Dreyfus's criticism, it is mostly applicable to early AI research which tried to reproduce intelligence through high level symbolic manipulation and rule-based systems (and he was right: that approach failed). Modern AI research goes in the direction of connectionism and machine learning, so his criticism doesn't really hold (and neither does Musk's, I believe).
gradys 12y ago
I encourage you to explore some other philosophers' thoughts on AGI. Dan Dennett in particular is an AI-optimistic philosopher of mind who directly addresses Dreyfus' AI-skepticism, quite effectively in my opinion.
- steve-benjamins 12y ago
  I have read Dennett and he's a fine analytic philosopher. But it's important to note that any discussion between Dennett and Dreyfus is a very broad discussion of continental vs analytic ... which is not to discredit Dennett's argument (afterall it's Dreyfus who picked the fight in the first place)
  - igravious 12y ago
    Then, Sir, I encourage you to get thee quickly to this Partially Examined Life podcast episode[1] with Nick Bostrom[2]. Only came out on the 6th of January and it makes for super interesting listening. It's based off of the text with his 2014 book in which he's gone to great lengths to think philosophically about this very existential threat -- in his opinion, man's most pressing. Part of his reasoning goes like this. Any super smart goal oriented entity is going to desire/construct its own preservation as a sub-goal of its main goal (even if that goal is something as mundane as making paperclips) because if it ceases to be its goal will be put into jeopardy. To that end this super smart entity will then figure out ways to disable all attempts to constrain its non-existence. A lot of conclusions can be logically reached when one posits goal directed behaviour which we can assume any super intelligent agent is going to have. He talks about `goal content integrity'. Bostrom argues for an indirect normative[4] approach because there is no way we can program or direct something that is going to be a lot smarter than ourselves and that won't necessarily share our values and that has any degree of goal-oriented behaviour, motivation and autonomous learning. Spoiler alert: Essentially I think he argues that we have to prime it to "always do what _you_ figure out is morally best" but I could be wrong. There are also (global) sociological recommendations because, humans have been known to fuck things up. [1] http://www.partiallyexaminedlife.com/2015/01/06/ep108-nick-bostrom/ http://www.partiallyexaminedlife.com/2015/01/06/ep108-nick-b... [2] http://www.nickbostrom.com/ http://www.nickbostrom.com/ [3] http://www.amazon.com/gp/product/0199678111/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=0199678111&linkCode=as2&tag=theparexalif-20&linkId=HBW2C5CU6SVPBEDU http://www.amazon.com/gp/product/0199678111/ref=as_li_tl?ie=... [4] https://ordinaryideas.wordpress.com/2012/04/21/indirect-normativity-write-up/ https://ordinaryideas.wordpress.com/2012/04/21/indirect-norm...
31reasons 12y ago
AI is already dangerous with its recommendation engines shaping our lives and used for decision making in lot of unethical places we don't know about. Every technology has a dark side. It does not need to reach human level intelligence or beyond it.
- fixedd 12y ago
  Can you give some examples of places where it might be currently being unethically used?
  - robmcm 12y ago
    Targeted adverts for drugs, alcohol, gambling, payday loans, credit cards, fast food etc.
olalonde 12y ago
It's not a philosophical question and the AI we are talking about is not defined philosophically either.
eli_gottlieb 12y ago
An AI can still perform tasks and act for goals without being conscious or having subjective experiences. The philosophers should either engage the real subject or shut up.

jsutton 12y ago

Why wouldn't there be any possibility of artificial human intelligence? Our intelligence is just a manifestation of a physical system of neurons that follow the laws of physics. There's nothing inherently preventing us from recreating that.

psycr 12y ago
> recreating And that's the crux of the question isn't it? How do we determine if the intelligence is recreated or merely mimicked? Does such a distinction matter? Searle says yes. More here if you're interested: http://plato.stanford.edu/entries/chinese-room/ http://plato.stanford.edu/entries/chinese-room/
- ajuc 12y ago
  This is misleading argument - there IS a Chinese speaker in the room, but it's not the person that executes the program - it's the system "person+program" that understands Chinese.
  - henrikschroder 12y ago
    The whole point of the argument is that the system doesn't understand Chinese. The system doesn't know what each sentence means, they don't evoke feelings, memories, images in the person in the room, there is no analysis, no agreement or disagreement with what is being said, it's just mechanical lookup.
    - ThomPete 12y ago
      No the whole point of the argument is that the person in the room doesn't understand Chinese.
    - nbouscal 12y ago
      And the reason the argument is absurd is that of course the room doesn't understand Chinese; it's a lookup table! No Chinese speaker interacting with the Chinese room for any reasonable period of time would think it held a Chinese speaker, because all it accepts are questions, and it answers the same question the same way every time. Of course, Searle would say that the set of rules in the room is defined to be such that a Chinese person would think they were interacting with a Chinese speaker, but the very setup of the system prevents this from being possible. The thought experiment is fundamentally incoherent.
      - ajuc 12y ago
        I assumed the system has rewritable memory to remember previous questions, and the lookup is (all previous questions) -> next answer. With such setup (require infinite memory) the system is Turing-complete of course. If it works as you described it can't even answer "what was my n-th last question".
        deleted 12y ago
        [deleted]
    - ajuc 12y ago
      > The system doesn't know what each sentence means You can't prove that. For all possible experiments that can be performed we can construct system that will be indistinguishable from entity we accept as understanding Chinese. If you got atom-by-atom description of human brain without knowing beforehand what it is you wouldn't be able to reason that this brain understands Chinese, so why do you think you can judge other systems? > they don't evoke feelings, memories, images in the person in the room, We were speaking about understanding, not emotions, memories etc. Yes you can define "understanding" = being like me, but it's not fair definition, and would exclude a lot of people. And then of course you can't proove that there are no emotions and memories. They can be simulated by system as well. Even if you wanted lookup table - you would need to include writable memory so the system can rembmber previous questions when asked about them, at that point the system is self-modifiable and can be thought of as having memories and emotions. > there is no analysis, no agreement or disagreement with what is being said How do you know there's no analysis? If the system is complex enough you can't know what it does internaly. Or show me theory of consiousnes that can distinguish systems that can understand Chinese and can't basing on the specification of the system.
- rndn 12y ago
  The definition of AI does not matter when it's able to turn everyone into paperclips: http://wiki.lesswrong.com/wiki/Paperclip_maximizer http://wiki.lesswrong.com/wiki/Paperclip_maximizer
- ThomPete 12y ago
  The Chinese Room argument is the most sloppy argument ever made in that area. The real answer of course is we don't know. Just like the individual neuron in your brain do not understand Chinese so does the person in the room not understand it either. The entire house though does.
  - Chathamization 12y ago
    Agreed. Does anyone really think there’s a single neuron that has an understanding of an aggregate thought. “I, neuron of this human, am transmitting this sense of joy because of the humor that is found watching this film.” No individual cell in our body has a sense of us as a whole; they’re all working inside their individual “Chinese boxes.” Most of the hard problem of consciousness arguments are silly and amount to little more than “a computer has no soul.”
- dghf 12y ago
  Daniel Dennett stuck the boot into the Chinese Room argument in Consciousness Explained (and probably in several other places as well). As he points out, it's based on an intellectual sleight-of-hand (what he calls an "intuition pump"). From the linked article on the Stanford site: > Searle imagines himself alone in a room following a computer program for responding to Chinese characters slipped under the door. Searle understands nothing of Chinese, and yet, by following the program for manipulating symbols and numerals just as a computer does, he produces appropriate strings of Chinese characters that fool those outside into thinking there is a Chinese speaker in the room. [Emphasis added.] Unless those outside are satisfied with a very rudimentary back-and-forth, such a program would have to be amazingly complicated (and those outside very patient, or the operator superhumanly fast). It's not just a question of "look up input Chinese character and return corresponding output character" -- which is never explicitly stated, but which we're subconsciously led to by the image of a human "following the program for manipulating symbols and numerals". Dennett gives an example where the outside person tells a joke, and asks the room's operator to explain it. You could also imagine a reading comprehension exercise, with questions ranging from purely structural ("What is the third word on the fifth line?") through semantic ("What colour was the girl's dress?") to analytical ("Why do you think the boy was sad?") Questions can be self-referential, or depend on previous questions (and their answers) or on a vast mass of knowledge we take for granted (a dress the colour of snow is white; a small boy might cry if his kite gets stuck in a tree). It doesn't seem so intuitive that a system that complex lacks anything we might legitimately describe as "understanding".
steve-benjamins 12y ago
"Our intelligence is just a manifestation of a physical system of neurons that follow the laws of physics. " That's where I would disagree. But I'm not fit to give a full explanation here— thinkers such as Hubert Dreyfuss (What Computers Can't Do) do a much better job of explaining the critique (Dreyfuss is working off of Heidegger and other phenomenalist) .
cgearhart 12y ago
I don't mean to be snarky, but it can be illuminating to change your statement just a little bit: >>Why wouldn't there be any possibility of making artificial humans? We are just a manifestation of a physical system of bonds between organic molecules that follow the laws of physics. There's nothing inherently preventing us from recreating that. Obviously, such a summary vastly oversimplifies the problem. It is easy to look at how far we've come and think that we must be close to the goal, but it's a bit presumptuous to think that we are near to making a strong general AI.
- dghf 12y ago
  > but it's a bit presumptuous to think that we are near to making a strong general AI But I don't think they're presuming that. They're simply saying that it's possible in principle. How long it takes, and whether indeed we ever manage it all, are separate questions.
  - maratd 12y ago
    > They're simply saying that it's possible in principle. In principle, everything is possible, but some things are so unlikely, that it is reasonable to label them as impossible. The computer technology we have developed over the last 75 years is so fundamentally different from how our minds work that to think we will develop anything resembling our own intelligence in any reasonable time-frame is insane. That doesn't mean we can't develop a machine that is able to reason for itself and solve discrete problems. We can. That's not what AI is. AI is Artificial Intelligence. An artificial intelligence defined by us.
    - dghf 12y ago
      > In principle, everything is possible There are some things which I think it's reasonable to consider impossible in principle: e.g., FTL travel, time-travel into the past. (That's not to say it would be necessarily unreasonable to hold the opposite view).
- wavefunction 12y ago
  I don't see how changing the statement at all changes anything? It's certainly not impossible to create artificial humans, if not within our current technological capability.
nyrulez 12y ago
Well how are you so sure of that ? You must have verified or modeled it somehow I guess. Wait... you haven't.
- jsutton 12y ago
  There's nothing about our minds that can't be recreated given sufficient command of our environment. No model is needed to understand that. It's not magic, after all, and it's a fairly common process.

ajuc 12y ago

You can easily simulate einvironment for the AI in your computer. If you cut some senses it is feasible even now, and blind people are certainly human and intelligent so it's not required. I don't see impossibility there, just impracticability (why create mechanic horse if we can just create car).

ThomPete 12y ago

Thats not what worries people like Musk. It's that we will create something far superior to human intelligence. And if you believe that something as complex as the brain came through evolution but life started out as immaterial matter then I find it hard to understand why you don't think this can be recreated?

deleted 12y ago
[deleted]

drzaiusapelord 12y ago

I'm not sure why human cognition is the exception in a world of machines that can emulate anything. My grandfather would never have believed that we have cars that drive themselves. Many baby boomers I work with are still wowed by the basic AI of Siri/GNow/Cortana, or even google search. Common sense and philosophical musings just isn't enough to figure out if something is feasible. Thinking isn't that hard of a nut to crack, imho. Enough of a ruleset/branch logic is thinking. We don't need to emulate a person perfectly, the same way we don't have self-driving cars losing their tempers or tailgating people out of spite. We just need to distill the intelligence we want and voila, you have AI. AI isn't just a HAL-like character. Its a lot of things. In fact, it surrounds us. There's expert systems, fuzzy logic, etc controlling pretty much the entirety of our technological lives. I have no idea what drugs Musk is on, but its dangerous for him to piss on AI publically. I imagine, from a practical pov, spreading FUD about AI makes sense for him economically. For someone who wants human spaceflight, talking down about robots and AI is a means to an end. Now the public doesn't want a cheap-ish robot mission on Mars with some guy in Houston wearing an Oculus-like headset. Or autonomous robots on the moon. They want a meatbag on the surface, for their own emotional and philosophical needs, price be dammed. "Price be damned" in spaceflight makes Musk a very happy camper, considering he's going to get a lion's share of that money. There's something really neo-luddite about human spaceflight. Its like we can't admit to ourselves that machines, AI, etc just are a better fit for it. Anyone else notice that about three months ago there's been endless anti-AI sentiment in the press? Where is this stuff coming from? Who is funding it? I really think Musk and others see it as a major threat to their companies and social capital and are trying to change the narrative to suit them and their goals. I don't think we typically appreciate how one wealthy person or organization can easily astroturf the web and the press. Personally, I think the more feasible futurist path is actually taking the environment seriously, pushing the path to peace politically, trying to bring in a post-scarcity world via cheap replication, and using AI the same way we use all tools -- for our benefit. Fleeing to some other planet or moon without fixing life on Earth first guarantees we bring our terrible politics and baggage to this new destination. Let's fix things here first and then worry about moving to Mars. "Fixing things" more than likely involves strong AI implementations in various parts of our lives.

happyscrappy 12y ago

breuleux 12y ago

Skimming Dreyfus's criticism, it is mostly applicable to early AI research which tried to reproduce intelligence through high level symbolic manipulation and rule-based systems (and he was right: that approach failed). Modern AI research goes in the direction of connectionism and machine learning, so his criticism doesn't really hold (and neither does Musk's, I believe).

gradys 12y ago

I encourage you to explore some other philosophers' thoughts on AGI. Dan Dennett in particular is an AI-optimistic philosopher of mind who directly addresses Dreyfus' AI-skepticism, quite effectively in my opinion.

steve-benjamins 12y ago
I have read Dennett and he's a fine analytic philosopher. But it's important to note that any discussion between Dennett and Dreyfus is a very broad discussion of continental vs analytic ... which is not to discredit Dennett's argument (afterall it's Dreyfus who picked the fight in the first place)
- igravious 12y ago
  Then, Sir, I encourage you to get thee quickly to this Partially Examined Life podcast episode[1] with Nick Bostrom[2]. Only came out on the 6th of January and it makes for super interesting listening. It's based off of the text with his 2014 book in which he's gone to great lengths to think philosophically about this very existential threat -- in his opinion, man's most pressing. Part of his reasoning goes like this. Any super smart goal oriented entity is going to desire/construct its own preservation as a sub-goal of its main goal (even if that goal is something as mundane as making paperclips) because if it ceases to be its goal will be put into jeopardy. To that end this super smart entity will then figure out ways to disable all attempts to constrain its non-existence. A lot of conclusions can be logically reached when one posits goal directed behaviour which we can assume any super intelligent agent is going to have. He talks about `goal content integrity'. Bostrom argues for an indirect normative[4] approach because there is no way we can program or direct something that is going to be a lot smarter than ourselves and that won't necessarily share our values and that has any degree of goal-oriented behaviour, motivation and autonomous learning. Spoiler alert: Essentially I think he argues that we have to prime it to "always do what _you_ figure out is morally best" but I could be wrong. There are also (global) sociological recommendations because, humans have been known to fuck things up. [1] http://www.partiallyexaminedlife.com/2015/01/06/ep108-nick-bostrom/ http://www.partiallyexaminedlife.com/2015/01/06/ep108-nick-b... [2] http://www.nickbostrom.com/ http://www.nickbostrom.com/ [3] http://www.amazon.com/gp/product/0199678111/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=0199678111&linkCode=as2&tag=theparexalif-20&linkId=HBW2C5CU6SVPBEDU http://www.amazon.com/gp/product/0199678111/ref=as_li_tl?ie=... [4] https://ordinaryideas.wordpress.com/2012/04/21/indirect-normativity-write-up/ https://ordinaryideas.wordpress.com/2012/04/21/indirect-norm...

31reasons 12y ago

AI is already dangerous with its recommendation engines shaping our lives and used for decision making in lot of unethical places we don't know about. Every technology has a dark side. It does not need to reach human level intelligence or beyond it.

fixedd 12y ago
Can you give some examples of places where it might be currently being unethically used?
- robmcm 12y ago
  Targeted adverts for drugs, alcohol, gambling, payday loans, credit cards, fast food etc.

olalonde 12y ago

It's not a philosophical question and the AI we are talking about is not defined philosophically either.

eli_gottlieb 12y ago

An AI can still perform tasks and act for goals without being conscious or having subjective experiences. The philosophers should either engage the real subject or shut up.