6 ms·

The Case for HAL's Sanity

89 points by pmcpinto 10y ago

dwringer 10y ago
I didn't think there was any question; HAL was simply following the orders of Dr. Langley.
nraynaud 10y ago
I think those 2 videos are on the topic: - AI "Stop Button" Problem https://www.youtube.com/watch?v=3TYT1QfdfsM https://www.youtube.com/watch?v=3TYT1QfdfsM (the AI won't let you push the stop button, because that would impair its function) - General AI Won't Want You To Fix its Code https://www.youtube.com/watch?v=4l7Is6vOAOA https://www.youtube.com/watch?v=4l7Is6vOAOA (the AI won't let you patch, because changing its goal means it can't reach it)
- grondilu 10y ago
  This second video is of particular interest to me, as it tackles a problem I've been thinking quite a lot lately : is self-modification compatible with volition? What's the point of wanting something if you can chose what to want? The pill that makes you love the idea of killing your kids is an extreme example, and probably no-one would accept to take this pill, but it's really not so obvious why. Or at least, the reasons are pretty deep. BTW IMHO a simpler example would be a meta-suicide pill : instead of being a poison that would just kill you, it's a pill that would make you want to kill yourself. I suppose you can make people want to kill themselves by inflicting them an awful chronic pain such that death would look like the only solution, but the pill I'm thinking about would not do that : it would change your mind so that you would want to die just because you like the idea of dying, for no particular reason.
  - maxerickson 10y ago
    Are you implicitly arguing that humans going about their day to day lives have a self that they are incapable of modifying? What's the difference between a pill and an idea?
  - algorithmsRcool 10y ago
    If this is an interesting topic, give "Permutation City" by "Greg Egan" a read. It involves a digitized character that intentionally modifies itself in order to avoid unhappiness.
    - chaosmachine 10y ago
      Permutation City is actually by Greg Egan.
      - algorithmsRcool 10y ago
        Ah, thank you. I read Permutation City in January right after I read Neuromancer and apparently got my wires crossed.
        jacobush 10y ago
        Wires permutated
  - garyrob 10y ago
    "is self-modification compatible with volition?" That's a plot point of WestWorld. A more interesting TV series than I expected.
- vl 10y ago
  In addition to these, Nate Soares: "Ensuring Smarter-than-Human Intelligence has a Positive Outcome" https://youtu.be/dY3zDvoLoao https://youtu.be/dY3zDvoLoao talks about how current AI fears misrepresent the problem and gives multiple examples of difficulty of defining correct objective function. I enjoyed this explanation a lot!
grondilu 10y ago
The part I preferred is this one: > HAL has to be extremely careful before committing the first murder. It is possible that he wasn't entirely familiar with the EVA procedure, it being one of the few tasks on board the Discovery that does not require his participation. Perhaps he needed to observe a dry run to study the astronaut's actions and calculate when he would be most vulnerable.
Analemma_ 10y ago
I thought the canon explanation was that HAL wasn't "insane", per se, but that he had been given conflicting orders (keep the Jupiter monolith secret, but also assist the crew in every way) and was resolving them as best he could.
- LoSboccacc 10y ago
  yeah the novel goes and "solve" the misery. with the task of saving the mission, not lying to the humans and withheld information about the alien, the only logical step to resolve the constraint when the human ask about the mission nature is to kill them to prevent having to lie or having to divulge information. this also echoes many Asimov novel in principle and especially one in specific where the ai goes trough all rules and the only non conflicting way to solve the constraint was self destruct (cant remember the title)
  - wruza 10y ago
    Even macos autolayout subsystem (cassowary solver) can break constraints based on priority as part of regular operation. It is a linear/logical problem, strange that scifi robots fail in it. Probably scifi authors are not aware of non-strict problems. No-lie should have minimal priority possible, otherwise HAL breaking TLoR is only a matter of time.
    - M_Grey 10y ago
      This has to be the most intellectually sound, and technically sophisticated analysis that still has, "Well then of course there wouldn't be a movie to watch," as it's answer.
    - alanh 10y ago
      Haha. Anyone who has written more than a dozen lines of CSS is hopefully also familiar with the computer ignoring one directive in favor of the other. Try this: .foo { display: inline; float: right; } Better believe foo won't be inline! That’s per spec :)
    - mannigfaltig 10y ago
deleted 10y ago
[deleted]
grabcocque 10y ago
Wait, is it commonly accepted that HAL was mad? I had always felt the film made perfectly clear HAL was acting totally rationally within several conflicting sets of mission parameters and moral codes.
- taneq 10y ago
  Not even conflicting orders. I thought it was explicitly stated (towards the end of the book, maybe? It's been years since I read it) that HAL had been ordered by mission command to ensure the success of the mission at the expense of the crew if necessary. As crew members started acting suboptimally, HAL realised they were already jeopardizing the mission just by being fallible humans, and started to fix the problem. An example of the law of unintended consequences, maybe, but not a malfunction.
  - the_af 10y ago
    In the book, sure. In the movie this isn't at all clear. Or rather: it's evident HAL considers himself essential to the mission; it's also evident when they threaten to disconnect him, he can be considered to act in defense of the mission itself. This is entirely rational. However, the movie implies HAL has a real problem. I don't know if actually "insane", but he incorrectly predicts the malfunction of the antenna, and when confronted about it, he cannot say much more than "oops!". I don't buy this article's explanation of why he could have faked his analysis of the antenna -- going by the movie itself, it's clear to me HAL makes a mistake (maybe because he is conflicted about the contradictory orders). HAL makes a mistake for which there is no easy way to backpedal without looking faulty. Also, in the movie (and this is explored more in "2010: The Year We Made Contact"), it's clear HAL-class computers are capable of irrational, human-like thought. HAL feels threatened by disconnection. He sounds very scared when Dave Bowman finally disables him ("Stop, Dave. I'm afraid"). A similar computer on Earth, SAL, is similary worried about temporary disconnection and asks "will I dream?". And in the final scene of 2010, Doctor Chandra must be both gentle to HAL and appeal to his sense of duty for him to follow orders. I think there's a very good case that HAL did, indeed, go insane. Even if it was because of contradictory orders.
xefer 10y ago
I had always assumed that Kubrick gave an explicit clue that HAL had "snapped" when he strangely repeats himself when he claims to have detected a problem. HAL suddenly says "just a moment... just a moment..." It comes across as just the sort of error he was supposedly incapable of.
- mjevans 10y ago
  I agree with the hint at a timing cue but still don't think HAL 'went insane', I more agree that he found an 'insane' "solution" to a problem humans would find intractable. The "just a moment..." can be thought of more as an auditory interface representation of "processing", where HAL is exploring the solution space of ever more unlikely solutions until it has at last resolved a statistically superior solution.
PeterisP 10y ago
It's just as we are eager to label others as insane simply because their goals violently conflict with ours and they would do hurtful things to us. We're eager to label the behavior of various criminals or politicians as crazy or stupid because we disapprove of it - however, often (though not always) if you take a step back, it's quite consistent with their interest, the evil behavior is reasonable, rational and understandable in context of their situation and goals. It's not the execution we should worry about, but rather about alignment of goals - as for other people, as for AI.
equalunique 10y ago
At "Replacing the AE-35 Unit" there seems to be a typo in this sentence: "Such people tend to be curious and want to get the bottom of mysteries, is HAL is trying to pique their curiosity so that they will take the required action to fulfil his murder plan."
sidcool 10y ago
I am amazed at the level of detail that Kubrick went into. Even after decades of movie's release there are hidden nuggets I keep finding. Absolutely brilliant.
vinceguidry 10y ago
Sanity is a legal definition, not a psychological one. People often forget that even the mentally-damaged act with reason and logic. This rationality may not be accessible to a bystander, but is present regardless. I see the article as making a distinction without a difference. Kubrick was perfectly clear in his presentation of HAL's motives. There were a few points of interest, like the chess game, which I missed when I watched the film, but the 'theory' is how I viewed HAL's actions the whole time.
pps43 10y ago
The section on the chess game is not particularly persuasive. According to Stockfish, black already has winning position. So Frank (playing white) can no longer make mistakes. All moves at his disposal lead to the same outcome. "Queen to Bishop three" is not a legal move, so Frank could have autocorrected it.

89 points by pmcpinto 10y ago

I didn't think there was any question; HAL was simply following the orders of Dr. Langley.

I think those 2 videos are on the topic: - AI "Stop Button" Problem https://www.youtube.com/watch?v=3TYT1QfdfsM https://www.youtube.com/watch?v=3TYT1QfdfsM (the AI won't let you push the stop button, because that would impair its function) - General AI Won't Want You To Fix its Code https://www.youtube.com/watch?v=4l7Is6vOAOA https://www.youtube.com/watch?v=4l7Is6vOAOA (the AI won't let you patch, because changing its goal means it can't reach it)

grondilu 10y ago
This second video is of particular interest to me, as it tackles a problem I've been thinking quite a lot lately : is self-modification compatible with volition? What's the point of wanting something if you can chose what to want? The pill that makes you love the idea of killing your kids is an extreme example, and probably no-one would accept to take this pill, but it's really not so obvious why. Or at least, the reasons are pretty deep. BTW IMHO a simpler example would be a meta-suicide pill : instead of being a poison that would just kill you, it's a pill that would make you want to kill yourself. I suppose you can make people want to kill themselves by inflicting them an awful chronic pain such that death would look like the only solution, but the pill I'm thinking about would not do that : it would change your mind so that you would want to die just because you like the idea of dying, for no particular reason.
- maxerickson 10y ago
  Are you implicitly arguing that humans going about their day to day lives have a self that they are incapable of modifying? What's the difference between a pill and an idea?
- algorithmsRcool 10y ago
  If this is an interesting topic, give "Permutation City" by "Greg Egan" a read. It involves a digitized character that intentionally modifies itself in order to avoid unhappiness.
  - chaosmachine 10y ago
    Permutation City is actually by Greg Egan.
    - algorithmsRcool 10y ago
      Ah, thank you. I read Permutation City in January right after I read Neuromancer and apparently got my wires crossed.
      - jacobush 10y ago
        Wires permutated
- garyrob 10y ago
  "is self-modification compatible with volition?" That's a plot point of WestWorld. A more interesting TV series than I expected.
vl 10y ago
In addition to these, Nate Soares: "Ensuring Smarter-than-Human Intelligence has a Positive Outcome" https://youtu.be/dY3zDvoLoao https://youtu.be/dY3zDvoLoao talks about how current AI fears misrepresent the problem and gives multiple examples of difficulty of defining correct objective function. I enjoyed this explanation a lot!

grondilu 10y ago

The part I preferred is this one: > HAL has to be extremely careful before committing the first murder. It is possible that he wasn't entirely familiar with the EVA procedure, it being one of the few tasks on board the Discovery that does not require his participation. Perhaps he needed to observe a dry run to study the astronaut's actions and calculate when he would be most vulnerable.

Analemma_ 10y ago

I thought the canon explanation was that HAL wasn't "insane", per se, but that he had been given conflicting orders (keep the Jupiter monolith secret, but also assist the crew in every way) and was resolving them as best he could.

LoSboccacc 10y ago
yeah the novel goes and "solve" the misery. with the task of saving the mission, not lying to the humans and withheld information about the alien, the only logical step to resolve the constraint when the human ask about the mission nature is to kill them to prevent having to lie or having to divulge information. this also echoes many Asimov novel in principle and especially one in specific where the ai goes trough all rules and the only non conflicting way to solve the constraint was self destruct (cant remember the title)
- wruza 10y ago
  Even macos autolayout subsystem (cassowary solver) can break constraints based on priority as part of regular operation. It is a linear/logical problem, strange that scifi robots fail in it. Probably scifi authors are not aware of non-strict problems. No-lie should have minimal priority possible, otherwise HAL breaking TLoR is only a matter of time.
  - M_Grey 10y ago
    This has to be the most intellectually sound, and technically sophisticated analysis that still has, "Well then of course there wouldn't be a movie to watch," as it's answer.
  - alanh 10y ago
    Haha. Anyone who has written more than a dozen lines of CSS is hopefully also familiar with the computer ignoring one directive in favor of the other. Try this: .foo { display: inline; float: right; } Better believe foo won't be inline! That’s per spec :)
  - mannigfaltig 10y ago

deleted 10y ago

[deleted]

grabcocque 10y ago

Wait, is it commonly accepted that HAL was mad? I had always felt the film made perfectly clear HAL was acting totally rationally within several conflicting sets of mission parameters and moral codes.

taneq 10y ago
Not even conflicting orders. I thought it was explicitly stated (towards the end of the book, maybe? It's been years since I read it) that HAL had been ordered by mission command to ensure the success of the mission at the expense of the crew if necessary. As crew members started acting suboptimally, HAL realised they were already jeopardizing the mission just by being fallible humans, and started to fix the problem. An example of the law of unintended consequences, maybe, but not a malfunction.
- the_af 10y ago
  In the book, sure. In the movie this isn't at all clear. Or rather: it's evident HAL considers himself essential to the mission; it's also evident when they threaten to disconnect him, he can be considered to act in defense of the mission itself. This is entirely rational. However, the movie implies HAL has a real problem. I don't know if actually "insane", but he incorrectly predicts the malfunction of the antenna, and when confronted about it, he cannot say much more than "oops!". I don't buy this article's explanation of why he could have faked his analysis of the antenna -- going by the movie itself, it's clear to me HAL makes a mistake (maybe because he is conflicted about the contradictory orders). HAL makes a mistake for which there is no easy way to backpedal without looking faulty. Also, in the movie (and this is explored more in "2010: The Year We Made Contact"), it's clear HAL-class computers are capable of irrational, human-like thought. HAL feels threatened by disconnection. He sounds very scared when Dave Bowman finally disables him ("Stop, Dave. I'm afraid"). A similar computer on Earth, SAL, is similary worried about temporary disconnection and asks "will I dream?". And in the final scene of 2010, Doctor Chandra must be both gentle to HAL and appeal to his sense of duty for him to follow orders. I think there's a very good case that HAL did, indeed, go insane. Even if it was because of contradictory orders.

xefer 10y ago

I had always assumed that Kubrick gave an explicit clue that HAL had "snapped" when he strangely repeats himself when he claims to have detected a problem. HAL suddenly says "just a moment... just a moment..." It comes across as just the sort of error he was supposedly incapable of.

mjevans 10y ago
I agree with the hint at a timing cue but still don't think HAL 'went insane', I more agree that he found an 'insane' "solution" to a problem humans would find intractable. The "just a moment..." can be thought of more as an auditory interface representation of "processing", where HAL is exploring the solution space of ever more unlikely solutions until it has at last resolved a statistically superior solution.

PeterisP 10y ago

It's just as we are eager to label others as insane simply because their goals violently conflict with ours and they would do hurtful things to us. We're eager to label the behavior of various criminals or politicians as crazy or stupid because we disapprove of it - however, often (though not always) if you take a step back, it's quite consistent with their interest, the evil behavior is reasonable, rational and understandable in context of their situation and goals. It's not the execution we should worry about, but rather about alignment of goals - as for other people, as for AI.

equalunique 10y ago

At "Replacing the AE-35 Unit" there seems to be a typo in this sentence: "Such people tend to be curious and want to get the bottom of mysteries, is HAL is trying to pique their curiosity so that they will take the required action to fulfil his murder plan."

sidcool 10y ago

I am amazed at the level of detail that Kubrick went into. Even after decades of movie's release there are hidden nuggets I keep finding. Absolutely brilliant.

vinceguidry 10y ago

Sanity is a legal definition, not a psychological one. People often forget that even the mentally-damaged act with reason and logic. This rationality may not be accessible to a bystander, but is present regardless. I see the article as making a distinction without a difference. Kubrick was perfectly clear in his presentation of HAL's motives. There were a few points of interest, like the chess game, which I missed when I watched the film, but the 'theory' is how I viewed HAL's actions the whole time.

pps43 10y ago

The section on the chess game is not particularly persuasive. According to Stockfish, black already has winning position. So Frank (playing white) can no longer make mistakes. All moves at his disposal lead to the same outcome. "Queen to Bishop three" is not a legal move, so Frank could have autocorrected it.