9 ms·

The article presents two sides - one end advocates humans taking an active role in the loop, like confirming a drone's target classification and authorising it

The article presents two sides - one end advocates humans taking an active role in the loop, like confirming a drone's target classification and authorising it to act lethally. The other side is where the human's role is to shape the operating envelope to what they think will have parameters appropriate for the ethics of the situation. The examples of the systems seems completely out of whack though. The active participant advocates are clearly taking about things like air to ground hunter killer missions on ambiguous targets. For example, a civilian vehicle converted to technical vs a plumber hauling some piping. A farmer with a weed killer in a ditch by the road vs an insurgent constructing an ambush. These are clearly beyond the realm of what the other side is talking about - easily identifying shit like russian airforce craft with no friendly transponders and armed active radar payloads under its wings. I don't understand the conversation that is trying to take place. Clearly they are too different to even consider. Is the conversation supposed to be hypothetical enough that we can pretend we can act in the first case, as we can in the second? Then the conversation seems pointless. Of course you can, if you accept the premise of the second in the first place. As an aside, the section that goes like: > exerting over the system when the system has been fine-tuned to make it easier for the human to do less, and for the operator to accept the machine’s judgment. Is just incredibly bad faith nonsense. Or extremely ignorant. It's hard to tell which.

bsenftner 4y ago
It is both incredibly bad faith nonsense and extremely ignorant. The ethics debates around AI are filled with military jargon dropping morons. It is extremely hard to find a rational anyone not pushing some corporate-wealth-scheme or crank-conspiracy nonsense. I work in AI, and the non-technically specific conversations, such as ethics, are extremely shallow.
- sicp-enjoyer 4y ago
  It's a form of marketing that appeals to people in tech.
- hutzlibu 4y ago
  The main problem with non technical people talking about AI, is that they mainly have no understanding about AI, but everyone has still a strong opinion (or thinks he has to have one)
  - Sakos 4y ago
    The main problem is that even in the field among experts, ethics has extremely low priority. It's shallow not because of laypersons, but because experts don't think ethics are worth thinking about.
    - visarga 4y ago
      Up until a few years ago AI systems were not sophisticated enough to have an impact. Papers only recently started having an ethics section, and even now it's mostly low effort, just write something and get over it.
    - jameshart 4y ago
      Right. If you are concerned that the only writing on ethics of AI is by people who lack the technical understanding of the field to meaningfully comment, then it is incumbent on the people who are technically competent in the field to meaningfully engage in discussing ethics. As it is I see AI researchers engaging in a curious doublethink: both assuming that someone else is thinking about the ethical questions, and at the same time whenever someone raises an ethical critique, dismissing it as technically unsophisticated.
  - astrange 4y ago
    Technical people, or tech-adjacent people at least, seem to be even worse. They seem to assume that since they know how computers work that AGI “obviously” works the same way and will obviously instantly self-reproduce and take over the world for… some reason. Of course, they were equally confident logic and expert systems could build an AI and despite none of that working out at all, it doesn’t seem to have stopped them.
    - jimrandomh 4y ago
- nonrandomstring 4y ago
  I am hoping to address that, as best I can. One must remember that ethics is philosophy that has only so much traction where it touches necessity and survival.
zmgsabst 4y ago
How would you classify a CROWS turret [1] or similar which could automatically fire based on image recognition? I think that’s the kind of case where you run into these questions about a “human in the loop” — eg, do we need a human to confirm target identification or can the robot automatically engage anyone who points a “firearm” at a soldier, even if that occasionally shoots farmers holding tools oddly? [1] https://en.wikipedia.org/wiki/CROWS https://en.wikipedia.org/wiki/CROWS
- theelous3 4y ago
  There is nothing on that page I could see in a brief scan that indicates this system ever acts on its own behalf. As far as I was aware, these are just to keep the gunner's head from poking out the top of the vehicle. It's operated by someone completely manually like 2 ft away.
  - zmgsabst 4y ago
    Yes — I was asking a hypothetical about a technology that’s almost viable. CROWS are remote turrets with cameras and automated aiming systems; all we’d need to do is add target recognition. But is that ethnical?
    - theelous3 4y ago
      You're asking me directly for the answer to all of these grand ethical questions, via the lens of a hypothetical technology you haven't defined the limits of. I really don't know what you expect in ways of an answer.
Frost1x 4y ago
>> exerting over the system when the system has been fine-tuned to make it easier for the human to do less, and for the operator to accept the machine’s judgment. >Is just incredibly bad faith nonsense. Or extremely ignorant. It's hard to tell which. When you talk to the military you have to remember that many levels of military are regarded as drones that go through motions and that much of what goes on is assumed to be defined without critical thinking. When things deviate, they ask for directions from higher levels where critical thinking occurs. This is different than assumptions you typically make with end users, you assume your system is being used by people who actively think, question, and have varied needs. In this light that argument makes a bit more sense, these are just processes that you're refining to make the process more fluid--"the human is a necessary evil because we lack the technology to do these and need meatsuits to execute" sort of thought process. Point there and shoot here, etc. When you think of much military action in this context, that people follow orders, execute predefined processes and don't think, this argument makes more sense... if I have biological robots why not replace them with faster non-biologicial robots? I completely disagree with this ideology but it's pretty deeply seated in the military from what I understand. The issue is people arguing for HITL (human-in-the-loop) don't want drones who don't think and they assume they're able to intervene and say "no" when something absurd, unethical, or ridiculous is thrown in the mix. They want them to consider when someone says, fire a missile here, what are the implications of that to some degree. That if on their radar they see a playground full of children they might reject that order or improper information, give feedback and even ask for more confirmation. I find the majority of the arguments presented in this article pretty weak, but don't feel like dissecting them all. The first does look at life through a Tayloristic perspective of efficiency above all else which I think is a fundamental problem humanity is facing everywhere, not just the military: the questioning of Taylorism and utility theory perspectives that dominant the world today, instead seeing a need for balancing the needs of humanity and considering things humanistically in a world increasingly ruled by the former.
- tialaramex 4y ago
  In the past few days HN linked something about changes in military organisation from the era when fixed formation set pieces were usual, and it emphasises that today we don't in fact really do what you've described. The soldiers aren't "drones" in this sense, they're responsible for some pretty sophisticated decision making which requires personal discretion. A British Army soldier, literate but probably with nothing beyond high school education, is expected to learn tactics needed to achieve strategic goals without babysitting. "Take that building" is a goal, working as a group, advancing under cover and flanking the enemy is tactics. Modern infantry work mostly in small units, a handful of people with a shared purpose, they have an NCO leader to make decisions but they are individually expected to understand the tactics. The officer commanding dozens or hundreds of troops remotely can't babysit them all even if they both wanted to and had the information to attempt it which they don't. The preference to not use humans is sentiment. Drones, robot guns, and so on don't leave grieving friends and relatives, they're just machines, and will be replaced. Sentiment is a massive political problem, the American people will pay for a $100Bn/ year war without flinching, hey it's full employment at the weapons factory - but when the planes full of coffins come home that's a voters' kid in each box.

bsenftner 4y ago

It is both incredibly bad faith nonsense and extremely ignorant. The ethics debates around AI are filled with military jargon dropping morons. It is extremely hard to find a rational anyone not pushing some corporate-wealth-scheme or crank-conspiracy nonsense. I work in AI, and the non-technically specific conversations, such as ethics, are extremely shallow.

sicp-enjoyer 4y ago
It's a form of marketing that appeals to people in tech.
hutzlibu 4y ago
The main problem with non technical people talking about AI, is that they mainly have no understanding about AI, but everyone has still a strong opinion (or thinks he has to have one)
- Sakos 4y ago
  The main problem is that even in the field among experts, ethics has extremely low priority. It's shallow not because of laypersons, but because experts don't think ethics are worth thinking about.
  - visarga 4y ago
    Up until a few years ago AI systems were not sophisticated enough to have an impact. Papers only recently started having an ethics section, and even now it's mostly low effort, just write something and get over it.
  - jameshart 4y ago
    Right. If you are concerned that the only writing on ethics of AI is by people who lack the technical understanding of the field to meaningfully comment, then it is incumbent on the people who are technically competent in the field to meaningfully engage in discussing ethics. As it is I see AI researchers engaging in a curious doublethink: both assuming that someone else is thinking about the ethical questions, and at the same time whenever someone raises an ethical critique, dismissing it as technically unsophisticated.
- astrange 4y ago
  Technical people, or tech-adjacent people at least, seem to be even worse. They seem to assume that since they know how computers work that AGI “obviously” works the same way and will obviously instantly self-reproduce and take over the world for… some reason. Of course, they were equally confident logic and expert systems could build an AI and despite none of that working out at all, it doesn’t seem to have stopped them.
  - jimrandomh 4y ago
nonrandomstring 4y ago
I am hoping to address that, as best I can. One must remember that ethics is philosophy that has only so much traction where it touches necessity and survival.

zmgsabst 4y ago

How would you classify a CROWS turret [1] or similar which could automatically fire based on image recognition? I think that’s the kind of case where you run into these questions about a “human in the loop” — eg, do we need a human to confirm target identification or can the robot automatically engage anyone who points a “firearm” at a soldier, even if that occasionally shoots farmers holding tools oddly? [1] https://en.wikipedia.org/wiki/CROWS https://en.wikipedia.org/wiki/CROWS

theelous3 4y ago
There is nothing on that page I could see in a brief scan that indicates this system ever acts on its own behalf. As far as I was aware, these are just to keep the gunner's head from poking out the top of the vehicle. It's operated by someone completely manually like 2 ft away.
- zmgsabst 4y ago
  Yes — I was asking a hypothetical about a technology that’s almost viable. CROWS are remote turrets with cameras and automated aiming systems; all we’d need to do is add target recognition. But is that ethnical?
  - theelous3 4y ago
    You're asking me directly for the answer to all of these grand ethical questions, via the lens of a hypothetical technology you haven't defined the limits of. I really don't know what you expect in ways of an answer.

Frost1x 4y ago

>> exerting over the system when the system has been fine-tuned to make it easier for the human to do less, and for the operator to accept the machine’s judgment. >Is just incredibly bad faith nonsense. Or extremely ignorant. It's hard to tell which. When you talk to the military you have to remember that many levels of military are regarded as drones that go through motions and that much of what goes on is assumed to be defined without critical thinking. When things deviate, they ask for directions from higher levels where critical thinking occurs. This is different than assumptions you typically make with end users, you assume your system is being used by people who actively think, question, and have varied needs. In this light that argument makes a bit more sense, these are just processes that you're refining to make the process more fluid--"the human is a necessary evil because we lack the technology to do these and need meatsuits to execute" sort of thought process. Point there and shoot here, etc. When you think of much military action in this context, that people follow orders, execute predefined processes and don't think, this argument makes more sense... if I have biological robots why not replace them with faster non-biologicial robots? I completely disagree with this ideology but it's pretty deeply seated in the military from what I understand. The issue is people arguing for HITL (human-in-the-loop) don't want drones who don't think and they assume they're able to intervene and say "no" when something absurd, unethical, or ridiculous is thrown in the mix. They want them to consider when someone says, fire a missile here, what are the implications of that to some degree. That if on their radar they see a playground full of children they might reject that order or improper information, give feedback and even ask for more confirmation. I find the majority of the arguments presented in this article pretty weak, but don't feel like dissecting them all. The first does look at life through a Tayloristic perspective of efficiency above all else which I think is a fundamental problem humanity is facing everywhere, not just the military: the questioning of Taylorism and utility theory perspectives that dominant the world today, instead seeing a need for balancing the needs of humanity and considering things humanistically in a world increasingly ruled by the former.

tialaramex 4y ago
In the past few days HN linked something about changes in military organisation from the era when fixed formation set pieces were usual, and it emphasises that today we don't in fact really do what you've described. The soldiers aren't "drones" in this sense, they're responsible for some pretty sophisticated decision making which requires personal discretion. A British Army soldier, literate but probably with nothing beyond high school education, is expected to learn tactics needed to achieve strategic goals without babysitting. "Take that building" is a goal, working as a group, advancing under cover and flanking the enemy is tactics. Modern infantry work mostly in small units, a handful of people with a shared purpose, they have an NCO leader to make decisions but they are individually expected to understand the tactics. The officer commanding dozens or hundreds of troops remotely can't babysit them all even if they both wanted to and had the information to attempt it which they don't. The preference to not use humans is sentiment. Drones, robot guns, and so on don't leave grieving friends and relatives, they're just machines, and will be replaced. Sentiment is a massive political problem, the American people will pay for a $100Bn/ year war without flinching, hey it's full employment at the weapons factory - but when the planes full of coffins come home that's a voters' kid in each box.