7 ms·

Did HAL Commit Murder?

59 points by ColinWright 5y ago

hirundo 5y ago
Can a gun commit murder? Same answer. HAL killed people in the course of following its human written program. It would make as much sense to charge autopilot software with homicide.
- intpete 5y ago
  Murder as a type of crime, maybe not. Did HAL kill a human being?
- Swizec 5y ago
  Can a smart gun that makes its own decisions commit murder? If not, how is it different than a smart pile of meat? Either AGI has free will and is responsible for its actions, or we don’t either.
- nckmi 5y ago
  Is a gun imbued with faculties approaching sentience and operator-independent decision-making?
- danboarder 5y ago
  While both a computer(HAL) and a gun share the attribute of being non-human, there is an argument to be made that HAL could make decisions and thus has "agency", while a gun cannot make decisions. This differentiates the question significantly. Edit - responding to your comment edit, I don't know if it's common knowledge that HAL was written by humans (or not) but it's perhaps less clear with machine learning and AI algorithms used for modern autopilot systems, the systems learn and further program their own behavior beyond what the human programmers wrote, so in that sense I do think self-agency is emerging and as such there is responsibility on the part of both the car and it's creator, perhaps similar to how a parent is responsible for the independent actions of a young child.
  - TheOtherHobbes 5y ago
    You can reduce that entire essay to "Does an AI have agency?" And the answer wouldn't change: "Not unless we know it has independent subjectivity and self-awareness." It always comes down to qualia. Our model of agency is based on a certain set of subjective experiences we take as working definitions of both self-awareness and free will. We assume that other humans operate the same way. We can't know for sure, but life gets very complicated - and difficult - if we assume we're sentient but everyone else is just a meat robot. So assuming other humans have similar experiences is a reasonable position. This breaks down with mechanical intelligence, because we can only understand it through visible behaviours. And of course you can have agency-like behaviours without human-like qualia and subjective awareness. I strongly suspect this is not a solvable problem. You can imagine some kind of hypothetical (magic) tool for measuring consciousness objectively. We can choose to believe in sentience, or we can choose not to. Neither position can be justified without objective measurement, and that isn't available without a magic device. But even if it was available, the device would be paradoxical. The output of any objective consciousness measuring device still has to be experienced subjectively, which means it may be unreliable. So - unsolvable.
    - jtsiskin 5y ago
      There’s a lot of discussion around this topic - https://en.wikipedia.org/wiki/Philosophical_zombie https://en.wikipedia.org/wiki/Philosophical_zombie
    - salawat 5y ago
      How ya figure? Agency is just a nominative dignpost for saying that we can't say for certain what somebody's motives are given dome physical measures. That the outcome of a probability function with them as an agent is not deterministic in nature. In that sense, HAL absolutely has agency. As certain as you to me.
- deleted 5y ago
  [deleted]
Thrymr 5y ago
Maybe not, but the crew's survivors could probably sue the H.A.L. plant in Urbana, Illinois, in civil court.
kens 5y ago
Seriously, I think elevators are a good model for this, as they are autonomous systems that occasionally kill people. Who's at fault? If the manufacturer messed up the design, they're liable. If the owner ignores maintenance, they're liable. If someone intentionally sabotages the elevator to kill someone, that would be murder. If someone is elevator surfing and dies, then it's their own fault. It's the same with other autonomous systems; the blame could lie anywhere, depending on the circumstances. (I've simplified the legal issues somewhat, of course.)
- deleted 5y ago
  [deleted]
- lifeisstillgood 5y ago
  I think factory floor robots will be a grey area before elevators. As long as things are deterministic then "fault" can be ascribed - it was the workers fault for climbing over the barriers, or it was the manufacturers fault for not testing the arm would stop moving in < 1.2 seconds. But as soon as AI / ML becomes involved, it stops being deterministic and becomes probability. "Judgement" is asked of the robot arm. Is that image a person or a car door. I think this is likely to hit before cars simply because so many of us drive cars but none of us operate giant robot arms.
  - blamazon 5y ago
    A common metric already used in industrial automation liability examination is Safety Integrity Level. [1] This is a probabilistic measure of how likely a failure is. [1] https://blog.msasafety.com/what-safety-integrity-level-means/ https://blog.msasafety.com/what-safety-integrity-level-means...
  - smcameron 5y ago
    But supposing the robot is "guilty". How (or why) would anyone punish it? Put it in jail? I don't think it will mind. So what would be the point of finding it guilty? Rather, it would make more sense to, in some way "fix" the robot. Same would be true of people, were they easily "fixed".
LeoPanthera 5y ago
At least from an in-universe perspective, this is discussed in some length in the sequel, "2010". (Moreso in the novel than in the (underrated) movie adaptation.) The explanation given is that HAL was given, in secret, orders not to reveal the true purpose of the mission to Poole and Bowman, the two crewmembers not in hibernation. Because even the designer[1] of HAL was not made aware of this secret instruction, they were not aware that HAL's fundamental design was incompatible with the concealment of information, and in attempting to obey the instruction, created what in humans would be called a mental health crisis. [1]: "Dr. Langley" in 2001 the movie, but in the book and in all adaptations of the sequel, he has the wonderful name of "Dr. Sivasubramanian Chandrasegarampillai".
- tiahura 5y ago
- thaumasiotes 5y ago
  > in the book and in all adaptations of the sequel, he has the wonderful name of "Dr. Sivasubramanian Chandrasegarampillai" "Wonderful"? It would appear to be a take-off on https://en.wikipedia.org/wiki/Subrahmanyan_Chandrasekhar https://en.wikipedia.org/wiki/Subrahmanyan_Chandrasekhar
  - LeoPanthera 5y ago
    I'm sure that is deliberate.
    - JNRowe 5y ago
      My favourite character naming is in the epilogue of 3001: So every word is mine: well, almost every word. I must confess that I found Professor Thirugnanasampanthamoorthy (Chapter 35) in the Colombo telephone directory; I hope the present owner of that name will not object to the loan. I actually think the epilogues are my favourite parts of the series, as Clarke wraps up thoughts and ties in real world developments in the thirty-odd years between the titles.
JKCalhoun 5y ago
Sadly, the astronauts were no different to HAL than the opposing chess pieces when he plays chess against Frank Poole. When they became a threat to the mission they were discarded. Cold logic. Murder is like "sin", something we ascribe to malice, against God. What HAL did is just as horrible, but perhaps not "murder"? The article introduced me to the idea of mens rea. I take it there may come a time when a machine moves beyond cold logic and we might have to call a similar act murder. Interesting Clarke suggesting it was perhaps a conflict within HAL to both conceal the mission from the two conscious astronauts and trying to be truthful.
- kadoban 5y ago
  > The article introduced me to the idea of mens rea. I take it there may come a time when a machine moves beyond cold logic and we might have to call a similar act murder. If a human uses cold logic to decide to kill someone, is that murder? Why not if a machine does it then?
marmalar 5y ago
I thought murder was the killing of a human by another human?
- giobox 5y ago
  In many Western legal systems, to prove a crime like Murder there is normally two things the prosecution must demonstrate: That you A. Carried out the act; typically known as the Actus Reus, and That you B. Had the guilty intent or mind, typically known as the Mens Rea. This is why the defense of "insanity" exists in so many legal systems; if you were insane you cannot satisfy condition B and therefore cannot have committed a crime such as murder, even if you performed the act of killing another human while insane. However, this is a very accademic distinction in practice in most places. There is usually a second slightly lesser crime known as something like "Manslaughter" or "Culpable Homicide", which is in essence a Murder charge without B - you killed someone but it was an accident for example - you didn't actually mean for them to die. Such cases satisfy the Actus Reus but not the Mens Rea of murder, and are therefore often known as a crime other than "murder" itself. The above is an absolute butchering of Western Criminal law practices, but the general distinction between the action and the intent to perform the action is found in a lot of places, and a Murder charge often requires prosecution to demonstrate both Actus Reus and Mens Rea or the charge cannot stand. The Manslaughter charge is sometimes the fallback position for when proving Mens Rea beyond reasonable doubt for Murder fails. The elevator pitch for this entire article is really, "Was Hal capable of mens rea - yes or no?" So far no machine has ever reached that bar, at least to my knowledge! No one really doubts a machine can do the actus reus - of course it can. > https://en.wikipedia.org/wiki/Actus_reus https://en.wikipedia.org/wiki/Actus_reus > https://en.wikipedia.org/wiki/Mens_rea https://en.wikipedia.org/wiki/Mens_rea > https://en.wikipedia.org/wiki/Manslaughter https://en.wikipedia.org/wiki/Manslaughter > https://en.wikipedia.org/wiki/Culpable_homicide https://en.wikipedia.org/wiki/Culpable_homicide
  - LexGray 5y ago
    Also coming into play would be sovereignty in space. As HAL is not technically a citizen so would not fall under a national jurisdiction. And good luck getting an investigator out to Jupiter to gather evidence for the prosecution.
alphabetting 5y ago
I don't consider HAL a form of AGI so probably would say no
adamrezich 5y ago
(2020) previously: https://news.ycombinator.com/item?id=22000494 https://news.ycombinator.com/item?id=22000494
softwarebeware 5y ago
Wasn’t the point of HAL to warn humans to think about potential unintended consequences of AI and avoid them?
deleted 5y ago
[deleted]
KETpXDDzR 5y ago
Related: https://www.youtube.com/watch?v=xC03hmS1Brk https://www.youtube.com/watch?v=xC03hmS1Brk
jedberg 5y ago
They didn't really murder HAL in the movie/story because they left him in a state that was restorable. So all they really did was hibernate him. Much like putting someone in a medical coma and then waking them up isn't killing them.
hammock 5y ago
Autonomous computers likely have already killed humans in Libya two years ago.[1] Four problems with this new reality: 1) Misidentification. When selecting a target, will autonomous weapons be able to distinguish between hostile soldiers and 12-year-olds playing with toy guns? 2) Low-end proliferation. Market pressures could result in the creation and widespread sale of what can be thought of as the autonomous weapon equivalent of the Kalashnikov assault rifle: killer robots that are cheap, effective and almost impossible to contain as they circulate around the globe. 3) High-end proliferation. High-end autonomous weapons are likely to lead to more frequent wars because they will decrease two of the primary forces that have historically prevented and shortened wars: concern for civilians abroad and concern for one's own soldiers. 4) Undermining the laws of war. How can autonomous weapons be held accountable? Who is to blame for a robot that commits war crimes? Who would be put on trial? The weapon? The soldier? The soldier's commanders? The corporation that made the weapon? (from https://www.commondreams.org/views/2021/12/30/humanitys-final-arms-race-un-fails-agree-killer-robot-ban https://www.commondreams.org/views/2021/12/30/humanitys-fina...) [1]https://www.npr.org/2021/06/01/1002196245/a-u-n-report-suggests-libya-saw-the-first-battlefield-killing-by-an-autonomous-d https://www.npr.org/2021/06/01/1002196245/a-u-n-report-sugge...
- karpierz 5y ago
  Autonomous weapons have existed for decades, we just call them landmines.
  - Digit-Al 5y ago
    A landmine is not an autonomous weapon. Autonomy suggests being able to select a target and harm only that target. A landmine just blows up when someone steps on it; there is no selection, thus no autonomy.
    - karpierz 5y ago
      A landmine is programmed to attack a target of a specific weight when it enters a specific area. The targeting model was programmed by engineers to try and target soldiers (or armoured vehicles), and the locations they were placed were also for that purpose. However, it's possible for the heuristics to fail, and for a mine to blow up a child, because the weights weren't tuned correctly, or to fail to explode when a soldier steps on it. By your same logic, any AI has no autonomy, because it does X when its code selects X. A guard dog robot attacks will act consistently given identical inputs.
tamaharbor 5y ago
HAL was a wimp compared to Colossus (Colossus: The Forbin Project, 1970).
mitchbob 5y ago
It's only a glitch: https://youtu.be/TstteJ1eIZg https://youtu.be/TstteJ1eIZg
ilamont 5y ago
Just watched this a few days ago (it's on HBO Max for those who subscribe, and it's a very high quality digitization). It's a fantastic movie that still holds its own as a science fiction tale and work of art. The setup (spoilers): - Impending malfunction that can't be duplicated by Hal's twin back on Earth - Poole and Bowman discuss what to do in the pod - Bowman goes out to fix the antenna and his pod-like vehicle sneaks up on him - something happens (not shown) that causes Bowman and the vehicle to go spinning off into space - Poole goes out in a second vehicle to retrieve the body - Hal [apparently] shuts down the hibernation pods of the three other astronauts - Hal refuses to let Poole (without helmet) back onto the ship, shuts down discussion - Poole removes the emergency hatch and blows himself back into the ship - Poole disconnects Hal's memory and logic controllers, as Hal pleads with him to talk things over. In this portrayal, Hal is a sentient machine, not a brain in a vat connected to life support systems. This is made clear in the final sequence in the list above. For audiences watching the movie (as opposed to reading the book), it's not clear if it acts with malice, detached self-interest, psychopathic delusions, or something else. Science Fiction at this time (mid-1960s) was already using crazed robots or corrupted/untrustworthy machine intelligences as a storytelling device, for instance in Herbert's Dune and Destination: Void, so these scenarios would not have been unfamiliar. This is quite different than say, a Tesla driving its trusting owner off a cliff or a military drone programmed to kill or directed by human operators to fire a missile.
- arunix 5y ago
  You've got Poole and Bowman reversed (Poole was killed, Bowman survived and disconnected Hal).
anotheraccount9 5y ago
Murder presupposes will. It requires an executive function, like cognitive flexibility. Computers can do that, but we're going back into a circular argument of free will(unimpeded choice) versus automation(determinism). As we create more complex systems and getting closer to AI, we are also discovering how much of a machine we are. We may never be able to have a clear distinction.
pontifier 5y ago
I found it interesting that, in some cultures, killing someone was not against the law of the land. My understanding is that if you killed someone in ancient Rome, for instance, it was the duty of those nearest the dispute to either avenge the cowardly deed or acknowledge that the dead person had it coming. The world is a brutal place sometimes.
- paxys 5y ago
  Well that's how the rest of the world works. Humans are the exception here. No one is putting a lion on trial for murdering a gazelle.
at_a_remove 5y ago
I had always interpreted HALs actions as he had come to a most curious solution: he had to "complete the mission" but he couldn't let those humans awake know of the mission. This can be accomplished by having an "empty set" of human beings. To arrive at the empty set involves humans dying, which then sets him up for his further actions (in inactions, in not opening the pod bay doors). This is the conflict which causes his breakdown. So, partially, it was a failure of the people giving the orders that an empty set solution might be the one HAL arrived at: no crew, no crew to keep secrets from.
tpoacher 5y ago
Google algorithm: I'm sorry Dave, but I can't let you publish that.

59 points by ColinWright 5y ago

Can a gun commit murder? Same answer. HAL killed people in the course of following its human written program. It would make as much sense to charge autopilot software with homicide.

intpete 5y ago
Murder as a type of crime, maybe not. Did HAL kill a human being?
Swizec 5y ago
Can a smart gun that makes its own decisions commit murder? If not, how is it different than a smart pile of meat? Either AGI has free will and is responsible for its actions, or we don’t either.
nckmi 5y ago
Is a gun imbued with faculties approaching sentience and operator-independent decision-making?
danboarder 5y ago
While both a computer(HAL) and a gun share the attribute of being non-human, there is an argument to be made that HAL could make decisions and thus has "agency", while a gun cannot make decisions. This differentiates the question significantly. Edit - responding to your comment edit, I don't know if it's common knowledge that HAL was written by humans (or not) but it's perhaps less clear with machine learning and AI algorithms used for modern autopilot systems, the systems learn and further program their own behavior beyond what the human programmers wrote, so in that sense I do think self-agency is emerging and as such there is responsibility on the part of both the car and it's creator, perhaps similar to how a parent is responsible for the independent actions of a young child.
- TheOtherHobbes 5y ago
  You can reduce that entire essay to "Does an AI have agency?" And the answer wouldn't change: "Not unless we know it has independent subjectivity and self-awareness." It always comes down to qualia. Our model of agency is based on a certain set of subjective experiences we take as working definitions of both self-awareness and free will. We assume that other humans operate the same way. We can't know for sure, but life gets very complicated - and difficult - if we assume we're sentient but everyone else is just a meat robot. So assuming other humans have similar experiences is a reasonable position. This breaks down with mechanical intelligence, because we can only understand it through visible behaviours. And of course you can have agency-like behaviours without human-like qualia and subjective awareness. I strongly suspect this is not a solvable problem. You can imagine some kind of hypothetical (magic) tool for measuring consciousness objectively. We can choose to believe in sentience, or we can choose not to. Neither position can be justified without objective measurement, and that isn't available without a magic device. But even if it was available, the device would be paradoxical. The output of any objective consciousness measuring device still has to be experienced subjectively, which means it may be unreliable. So - unsolvable.
  - jtsiskin 5y ago
    There’s a lot of discussion around this topic - https://en.wikipedia.org/wiki/Philosophical_zombie https://en.wikipedia.org/wiki/Philosophical_zombie
  - salawat 5y ago
    How ya figure? Agency is just a nominative dignpost for saying that we can't say for certain what somebody's motives are given dome physical measures. That the outcome of a probability function with them as an agent is not deterministic in nature. In that sense, HAL absolutely has agency. As certain as you to me.
deleted 5y ago
[deleted]

Thrymr 5y ago

Maybe not, but the crew's survivors could probably sue the H.A.L. plant in Urbana, Illinois, in civil court.

kens 5y ago

Seriously, I think elevators are a good model for this, as they are autonomous systems that occasionally kill people. Who's at fault? If the manufacturer messed up the design, they're liable. If the owner ignores maintenance, they're liable. If someone intentionally sabotages the elevator to kill someone, that would be murder. If someone is elevator surfing and dies, then it's their own fault. It's the same with other autonomous systems; the blame could lie anywhere, depending on the circumstances. (I've simplified the legal issues somewhat, of course.)

deleted 5y ago
[deleted]
lifeisstillgood 5y ago
I think factory floor robots will be a grey area before elevators. As long as things are deterministic then "fault" can be ascribed - it was the workers fault for climbing over the barriers, or it was the manufacturers fault for not testing the arm would stop moving in < 1.2 seconds. But as soon as AI / ML becomes involved, it stops being deterministic and becomes probability. "Judgement" is asked of the robot arm. Is that image a person or a car door. I think this is likely to hit before cars simply because so many of us drive cars but none of us operate giant robot arms.
- blamazon 5y ago
  A common metric already used in industrial automation liability examination is Safety Integrity Level. [1] This is a probabilistic measure of how likely a failure is. [1] https://blog.msasafety.com/what-safety-integrity-level-means/ https://blog.msasafety.com/what-safety-integrity-level-means...
- smcameron 5y ago
  But supposing the robot is "guilty". How (or why) would anyone punish it? Put it in jail? I don't think it will mind. So what would be the point of finding it guilty? Rather, it would make more sense to, in some way "fix" the robot. Same would be true of people, were they easily "fixed".

LeoPanthera 5y ago

At least from an in-universe perspective, this is discussed in some length in the sequel, "2010". (Moreso in the novel than in the (underrated) movie adaptation.) The explanation given is that HAL was given, in secret, orders not to reveal the true purpose of the mission to Poole and Bowman, the two crewmembers not in hibernation. Because even the designer[1] of HAL was not made aware of this secret instruction, they were not aware that HAL's fundamental design was incompatible with the concealment of information, and in attempting to obey the instruction, created what in humans would be called a mental health crisis. [1]: "Dr. Langley" in 2001 the movie, but in the book and in all adaptations of the sequel, he has the wonderful name of "Dr. Sivasubramanian Chandrasegarampillai".

tiahura 5y ago
thaumasiotes 5y ago
> in the book and in all adaptations of the sequel, he has the wonderful name of "Dr. Sivasubramanian Chandrasegarampillai" "Wonderful"? It would appear to be a take-off on https://en.wikipedia.org/wiki/Subrahmanyan_Chandrasekhar https://en.wikipedia.org/wiki/Subrahmanyan_Chandrasekhar
- LeoPanthera 5y ago
  I'm sure that is deliberate.
  - JNRowe 5y ago
    My favourite character naming is in the epilogue of 3001: So every word is mine: well, almost every word. I must confess that I found Professor Thirugnanasampanthamoorthy (Chapter 35) in the Colombo telephone directory; I hope the present owner of that name will not object to the loan. I actually think the epilogues are my favourite parts of the series, as Clarke wraps up thoughts and ties in real world developments in the thirty-odd years between the titles.

JKCalhoun 5y ago

Sadly, the astronauts were no different to HAL than the opposing chess pieces when he plays chess against Frank Poole. When they became a threat to the mission they were discarded. Cold logic. Murder is like "sin", something we ascribe to malice, against God. What HAL did is just as horrible, but perhaps not "murder"? The article introduced me to the idea of mens rea. I take it there may come a time when a machine moves beyond cold logic and we might have to call a similar act murder. Interesting Clarke suggesting it was perhaps a conflict within HAL to both conceal the mission from the two conscious astronauts and trying to be truthful.

kadoban 5y ago
> The article introduced me to the idea of mens rea. I take it there may come a time when a machine moves beyond cold logic and we might have to call a similar act murder. If a human uses cold logic to decide to kill someone, is that murder? Why not if a machine does it then?

marmalar 5y ago

I thought murder was the killing of a human by another human?

giobox 5y ago
In many Western legal systems, to prove a crime like Murder there is normally two things the prosecution must demonstrate: That you A. Carried out the act; typically known as the Actus Reus, and That you B. Had the guilty intent or mind, typically known as the Mens Rea. This is why the defense of "insanity" exists in so many legal systems; if you were insane you cannot satisfy condition B and therefore cannot have committed a crime such as murder, even if you performed the act of killing another human while insane. However, this is a very accademic distinction in practice in most places. There is usually a second slightly lesser crime known as something like "Manslaughter" or "Culpable Homicide", which is in essence a Murder charge without B - you killed someone but it was an accident for example - you didn't actually mean for them to die. Such cases satisfy the Actus Reus but not the Mens Rea of murder, and are therefore often known as a crime other than "murder" itself. The above is an absolute butchering of Western Criminal law practices, but the general distinction between the action and the intent to perform the action is found in a lot of places, and a Murder charge often requires prosecution to demonstrate both Actus Reus and Mens Rea or the charge cannot stand. The Manslaughter charge is sometimes the fallback position for when proving Mens Rea beyond reasonable doubt for Murder fails. The elevator pitch for this entire article is really, "Was Hal capable of mens rea - yes or no?" So far no machine has ever reached that bar, at least to my knowledge! No one really doubts a machine can do the actus reus - of course it can. > https://en.wikipedia.org/wiki/Actus_reus https://en.wikipedia.org/wiki/Actus_reus > https://en.wikipedia.org/wiki/Mens_rea https://en.wikipedia.org/wiki/Mens_rea > https://en.wikipedia.org/wiki/Manslaughter https://en.wikipedia.org/wiki/Manslaughter > https://en.wikipedia.org/wiki/Culpable_homicide https://en.wikipedia.org/wiki/Culpable_homicide
- LexGray 5y ago
  Also coming into play would be sovereignty in space. As HAL is not technically a citizen so would not fall under a national jurisdiction. And good luck getting an investigator out to Jupiter to gather evidence for the prosecution.

alphabetting 5y ago

I don't consider HAL a form of AGI so probably would say no

adamrezich 5y ago

(2020) previously: https://news.ycombinator.com/item?id=22000494 https://news.ycombinator.com/item?id=22000494

softwarebeware 5y ago

Wasn’t the point of HAL to warn humans to think about potential unintended consequences of AI and avoid them?

deleted 5y ago

[deleted]

KETpXDDzR 5y ago

Related: https://www.youtube.com/watch?v=xC03hmS1Brk https://www.youtube.com/watch?v=xC03hmS1Brk

jedberg 5y ago

They didn't really murder HAL in the movie/story because they left him in a state that was restorable. So all they really did was hibernate him. Much like putting someone in a medical coma and then waking them up isn't killing them.

hammock 5y ago

Autonomous computers likely have already killed humans in Libya two years ago.[1] Four problems with this new reality: 1) Misidentification. When selecting a target, will autonomous weapons be able to distinguish between hostile soldiers and 12-year-olds playing with toy guns? 2) Low-end proliferation. Market pressures could result in the creation and widespread sale of what can be thought of as the autonomous weapon equivalent of the Kalashnikov assault rifle: killer robots that are cheap, effective and almost impossible to contain as they circulate around the globe. 3) High-end proliferation. High-end autonomous weapons are likely to lead to more frequent wars because they will decrease two of the primary forces that have historically prevented and shortened wars: concern for civilians abroad and concern for one's own soldiers. 4) Undermining the laws of war. How can autonomous weapons be held accountable? Who is to blame for a robot that commits war crimes? Who would be put on trial? The weapon? The soldier? The soldier's commanders? The corporation that made the weapon? (from https://www.commondreams.org/views/2021/12/30/humanitys-final-arms-race-un-fails-agree-killer-robot-ban https://www.commondreams.org/views/2021/12/30/humanitys-fina...) [1]https://www.npr.org/2021/06/01/1002196245/a-u-n-report-suggests-libya-saw-the-first-battlefield-killing-by-an-autonomous-d https://www.npr.org/2021/06/01/1002196245/a-u-n-report-sugge...

karpierz 5y ago
Autonomous weapons have existed for decades, we just call them landmines.
- Digit-Al 5y ago
  A landmine is not an autonomous weapon. Autonomy suggests being able to select a target and harm only that target. A landmine just blows up when someone steps on it; there is no selection, thus no autonomy.
  - karpierz 5y ago
    A landmine is programmed to attack a target of a specific weight when it enters a specific area. The targeting model was programmed by engineers to try and target soldiers (or armoured vehicles), and the locations they were placed were also for that purpose. However, it's possible for the heuristics to fail, and for a mine to blow up a child, because the weights weren't tuned correctly, or to fail to explode when a soldier steps on it. By your same logic, any AI has no autonomy, because it does X when its code selects X. A guard dog robot attacks will act consistently given identical inputs.

tamaharbor 5y ago

HAL was a wimp compared to Colossus (Colossus: The Forbin Project, 1970).

mitchbob 5y ago

It's only a glitch: https://youtu.be/TstteJ1eIZg https://youtu.be/TstteJ1eIZg

ilamont 5y ago

Just watched this a few days ago (it's on HBO Max for those who subscribe, and it's a very high quality digitization). It's a fantastic movie that still holds its own as a science fiction tale and work of art. The setup (spoilers): - Impending malfunction that can't be duplicated by Hal's twin back on Earth - Poole and Bowman discuss what to do in the pod - Bowman goes out to fix the antenna and his pod-like vehicle sneaks up on him - something happens (not shown) that causes Bowman and the vehicle to go spinning off into space - Poole goes out in a second vehicle to retrieve the body - Hal [apparently] shuts down the hibernation pods of the three other astronauts - Hal refuses to let Poole (without helmet) back onto the ship, shuts down discussion - Poole removes the emergency hatch and blows himself back into the ship - Poole disconnects Hal's memory and logic controllers, as Hal pleads with him to talk things over. In this portrayal, Hal is a sentient machine, not a brain in a vat connected to life support systems. This is made clear in the final sequence in the list above. For audiences watching the movie (as opposed to reading the book), it's not clear if it acts with malice, detached self-interest, psychopathic delusions, or something else. Science Fiction at this time (mid-1960s) was already using crazed robots or corrupted/untrustworthy machine intelligences as a storytelling device, for instance in Herbert's Dune and Destination: Void, so these scenarios would not have been unfamiliar. This is quite different than say, a Tesla driving its trusting owner off a cliff or a military drone programmed to kill or directed by human operators to fire a missile.

arunix 5y ago
You've got Poole and Bowman reversed (Poole was killed, Bowman survived and disconnected Hal).

anotheraccount9 5y ago

Murder presupposes will. It requires an executive function, like cognitive flexibility. Computers can do that, but we're going back into a circular argument of free will(unimpeded choice) versus automation(determinism). As we create more complex systems and getting closer to AI, we are also discovering how much of a machine we are. We may never be able to have a clear distinction.

pontifier 5y ago

I found it interesting that, in some cultures, killing someone was not against the law of the land. My understanding is that if you killed someone in ancient Rome, for instance, it was the duty of those nearest the dispute to either avenge the cowardly deed or acknowledge that the dead person had it coming. The world is a brutal place sometimes.

paxys 5y ago
Well that's how the rest of the world works. Humans are the exception here. No one is putting a lion on trial for murdering a gazelle.

at_a_remove 5y ago

I had always interpreted HALs actions as he had come to a most curious solution: he had to "complete the mission" but he couldn't let those humans awake know of the mission. This can be accomplished by having an "empty set" of human beings. To arrive at the empty set involves humans dying, which then sets him up for his further actions (in inactions, in not opening the pod bay doors). This is the conflict which causes his breakdown. So, partially, it was a failure of the people giving the orders that an empty set solution might be the one HAL arrived at: no crew, no crew to keep secrets from.

tpoacher 5y ago

Google algorithm: I'm sorry Dave, but I can't let you publish that.