5 ms·

So the study[0] involved people making simulated drone strike decisions. These people were not qualified to make these decisions for real and also knew the asso

So the study[0] involved people making simulated drone strike decisions. These people were not qualified to make these decisions for real and also knew the associated outcomes were also not real. This sounds like a flawed study to me. Granted, the idea of someone playing video games to kill real people makes me angry and decision making around drone strikes is already questionable. > Our pre-registered target sample size was 100 undergraduates recruited in exchange for course credit. However, due to software development delays in preparation for a separate study, we had the opportunity to collect a raw sample of 145 participants. Data were prescreened for technical problems occurring in ten of the study sessions (e.g., the robot or video projection failing), yielding a final sample of 135 participants (78.5% female, Mage = 21.33 years, SD = 4.08). [0] https://www.nature.com/articles/s41598-024-69771-z https://www.nature.com/articles/s41598-024-69771-z

matusp 2y ago
How is this phenomenon of very weakly grounded research called? Science washing or something?
- exe34 2y ago
  [flagged]
- GuB-42 2y ago
  I don't know why exe34 has been flagged/downvoted to death, because pseudo-science is absolutely the right answer. But this isn't it, the paper is fine. It is peer-reviewed and published in a reputable journal. The reasoning is clearly described, there are experiments, statistics and a reasonable conclusion based on these results which is "The overall findings indicate a strong propensity to overtrust unreliable AI in life-or-death decisions made under uncertainty." And as always, nuance gets lost when you get to mainstream news, though this article is not that bad. First thing because it links to the paper, some supposedly reputable news websites don't do that. I just think the "alarming" part is too much. It is a bias that needs to be addressed. The point here is not that AIs kill, it is that we need to find a way to make the human in AI-assisted decision making less trusty in order to get more accurate results. It is not enough to simply make the AI better.
braza 2y ago
That a pet peeve that I have regarding corporate scientific communication nowadays: a clearly limited study with a conflated conclusion that dilutes the whole debate. Now what happens: people that already have their visions around “AI-bad” will cite and spread that headline along several talking points, and most of the great public even knows those methodological flaws.
nyrikki 2y ago
Automation bias is a well documented human problem. I am not defending this study, but establishing and maintaining distrust of automation is one of the few known methods to help combat automation bias. It is why missing data is often preferred to partial data in trend analysis to remove the ML portion from the concept. Boeing planes crashing is another.
- fallingknife 2y ago
  But why should we distrust automation? In almost every case it is better than humans at its task. It's why we built the machines. Pilots have to be specifically taught to trust their instruments over themselves.
  - burnished 2y ago
    Notice how you ask about automation then mention instrumentation?
  - tsimionescu 2y ago
    Instruments are not automation. Trusting your instruments over yourself is different from trusting the autopilot decisions over yourself, which I think pilots aren't also taught.
  - withinboredom 2y ago
    > Pilots have to be specifically taught to trust their instruments over themselves. There is a difference between metrics and automation. In this case, they are trusting metrics, not automation. > But why should we distrust automation? I can name one off the top of my head: exceptions. Automation sucks at exceptions. Either nobody thought of it or it is ignored and ends up screwing everyone. Take my credit history in the US as an example. In my late teens, I got into an accident and couldn't work. I couldn't pay my bills. Within a few months, I was back on my feet; but it took nearly 10 years to get my credit score to something somewhat reasonable. I had employers look at my credit history and go "nope" because the machine said I wasn't trust-worthy. Should you trust the automation? Probably never, unless it knows how to deal with exceptions.
deleted 2y ago
[deleted]
psunavy03 2y ago
> Granted, the idea of someone playing video games to kill real people makes me angry and decision making around drone strikes is already questionable. For that first part, though, what does that even mean? The military isn't gamifying things and giving folks freaking XBox achievements for racking up killstreaks or anything. It's just the same game people have been playing since putting an atlatl on a spear, a scope on a rifle, or a black powder cannon on a battlefield. How to attack the enemy without being at risk. Is it unethical for a general officer to be sitting in an operations center directing the fight by looking at real-time displays? Is that a "video game?" The drone strikes in the Global War on Terror were a direct product of political pressure to "do something, anything" to stop another September 11th attack while simultaneously freaking out about a so-called "quagmire" any time someone mentioned "boots on ground." Well, guess what? If you don't want SOF assaulters doing raids to capture people, if you don't want traditional military formations holding ground, and you don't want people trying to collect intelligence by actually going to these places, about the only option you have left is to fly a drone around and try to identify the terrorist and then go whack him when he goes out to take a leak. Or do nothing and hope you don't get hit again.
- batch12 2y ago
  >For that first part, though, what does that even mean? The military isn't gamifying things and giving folks freaking XBox achievements for racking up killstreaks or anything. It's just the same game people have been playing since putting an atlatl on a spear, a scope on a rifle, or a black powder cannon on a battlefield. How to attack the enemy without being at risk. Is it unethical for a general officer to be sitting in an operations center directing the fight by looking at real-time displays? Is that a "video game?" It's not the same thing. Not even close. Killing people is horrible enough. Sitting in a trailer, clicking a button and killing someone from behind a screen without any of the risk involved is cowardly and shitty. There is no justification you can provide that will change my mind. Before you disregard my ability to understand the situation, I say this as a combat veteran.
  - pintxo 2y ago
    Would you say the same about a drone operator in ukraine, attacking russian troops on ukraine territory? I feel like there is still a distinction here. Between "traditional" warfare and those "anti-terror" operations.
rustcleaner 2y ago
In total war, however you got there, it won't matter. More dead enemy faster means higher odds of victory, and if digitally turning combatants into fluffy Easter Bunnies on screen to reduce moral shock value, giving achievement badges, and automated mini-hits of MDMA make you a more effective killer, then it will happen in total war. I could even imagine a democratization of drone warfare with online gaming, where some small p percent of all games are actually reality-driven virtual reality, or gives real scenarios to players to wargame and the bot watches the strategies and outcomes for a few hours to decide. Something akin to k kill switches in an execution but only 1 (known only by the technician who set it up) actually does anything.
- deleted 2y ago
  [deleted]
tbrownaw 2y ago
> So the study[0] involved people making simulated drone strike decisions. These people were not qualified to make these decisions for real and also knew the associated outcomes were also not real. This sounds like a flawed study to me. Unless I missed something, they also don't check for differences between saying the random advice is from an AI vs saying it's from some other more traditional expert source.
xbmcuser 2y ago
Most people don't give a shit about killing from behind the computer screen they have become desensitized to it ai has nothing to do with it. US has killed 1000s of children in drone strikes in Afghanistan and Pakistan sometimes knowingly sometimes not. Most of the remote pilots do not give shit about what they did.

matusp 2y ago

How is this phenomenon of very weakly grounded research called? Science washing or something?

exe34 2y ago
[flagged]
GuB-42 2y ago
I don't know why exe34 has been flagged/downvoted to death, because pseudo-science is absolutely the right answer. But this isn't it, the paper is fine. It is peer-reviewed and published in a reputable journal. The reasoning is clearly described, there are experiments, statistics and a reasonable conclusion based on these results which is "The overall findings indicate a strong propensity to overtrust unreliable AI in life-or-death decisions made under uncertainty." And as always, nuance gets lost when you get to mainstream news, though this article is not that bad. First thing because it links to the paper, some supposedly reputable news websites don't do that. I just think the "alarming" part is too much. It is a bias that needs to be addressed. The point here is not that AIs kill, it is that we need to find a way to make the human in AI-assisted decision making less trusty in order to get more accurate results. It is not enough to simply make the AI better.

braza 2y ago

That a pet peeve that I have regarding corporate scientific communication nowadays: a clearly limited study with a conflated conclusion that dilutes the whole debate. Now what happens: people that already have their visions around “AI-bad” will cite and spread that headline along several talking points, and most of the great public even knows those methodological flaws.

nyrikki 2y ago

Automation bias is a well documented human problem. I am not defending this study, but establishing and maintaining distrust of automation is one of the few known methods to help combat automation bias. It is why missing data is often preferred to partial data in trend analysis to remove the ML portion from the concept. Boeing planes crashing is another.

fallingknife 2y ago
But why should we distrust automation? In almost every case it is better than humans at its task. It's why we built the machines. Pilots have to be specifically taught to trust their instruments over themselves.
- burnished 2y ago
  Notice how you ask about automation then mention instrumentation?
- tsimionescu 2y ago
  Instruments are not automation. Trusting your instruments over yourself is different from trusting the autopilot decisions over yourself, which I think pilots aren't also taught.
- withinboredom 2y ago
  > Pilots have to be specifically taught to trust their instruments over themselves. There is a difference between metrics and automation. In this case, they are trusting metrics, not automation. > But why should we distrust automation? I can name one off the top of my head: exceptions. Automation sucks at exceptions. Either nobody thought of it or it is ignored and ends up screwing everyone. Take my credit history in the US as an example. In my late teens, I got into an accident and couldn't work. I couldn't pay my bills. Within a few months, I was back on my feet; but it took nearly 10 years to get my credit score to something somewhat reasonable. I had employers look at my credit history and go "nope" because the machine said I wasn't trust-worthy. Should you trust the automation? Probably never, unless it knows how to deal with exceptions.

deleted 2y ago

[deleted]

psunavy03 2y ago

> Granted, the idea of someone playing video games to kill real people makes me angry and decision making around drone strikes is already questionable. For that first part, though, what does that even mean? The military isn't gamifying things and giving folks freaking XBox achievements for racking up killstreaks or anything. It's just the same game people have been playing since putting an atlatl on a spear, a scope on a rifle, or a black powder cannon on a battlefield. How to attack the enemy without being at risk. Is it unethical for a general officer to be sitting in an operations center directing the fight by looking at real-time displays? Is that a "video game?" The drone strikes in the Global War on Terror were a direct product of political pressure to "do something, anything" to stop another September 11th attack while simultaneously freaking out about a so-called "quagmire" any time someone mentioned "boots on ground." Well, guess what? If you don't want SOF assaulters doing raids to capture people, if you don't want traditional military formations holding ground, and you don't want people trying to collect intelligence by actually going to these places, about the only option you have left is to fly a drone around and try to identify the terrorist and then go whack him when he goes out to take a leak. Or do nothing and hope you don't get hit again.

batch12 2y ago
>For that first part, though, what does that even mean? The military isn't gamifying things and giving folks freaking XBox achievements for racking up killstreaks or anything. It's just the same game people have been playing since putting an atlatl on a spear, a scope on a rifle, or a black powder cannon on a battlefield. How to attack the enemy without being at risk. Is it unethical for a general officer to be sitting in an operations center directing the fight by looking at real-time displays? Is that a "video game?" It's not the same thing. Not even close. Killing people is horrible enough. Sitting in a trailer, clicking a button and killing someone from behind a screen without any of the risk involved is cowardly and shitty. There is no justification you can provide that will change my mind. Before you disregard my ability to understand the situation, I say this as a combat veteran.
- pintxo 2y ago
  Would you say the same about a drone operator in ukraine, attacking russian troops on ukraine territory? I feel like there is still a distinction here. Between "traditional" warfare and those "anti-terror" operations.

rustcleaner 2y ago

In total war, however you got there, it won't matter. More dead enemy faster means higher odds of victory, and if digitally turning combatants into fluffy Easter Bunnies on screen to reduce moral shock value, giving achievement badges, and automated mini-hits of MDMA make you a more effective killer, then it will happen in total war. I could even imagine a democratization of drone warfare with online gaming, where some small p percent of all games are actually reality-driven virtual reality, or gives real scenarios to players to wargame and the bot watches the strategies and outcomes for a few hours to decide. Something akin to k kill switches in an execution but only 1 (known only by the technician who set it up) actually does anything.

deleted 2y ago
[deleted]

tbrownaw 2y ago

> So the study[0] involved people making simulated drone strike decisions. These people were not qualified to make these decisions for real and also knew the associated outcomes were also not real. This sounds like a flawed study to me. Unless I missed something, they also don't check for differences between saying the random advice is from an AI vs saying it's from some other more traditional expert source.

xbmcuser 2y ago

Most people don't give a shit about killing from behind the computer screen they have become desensitized to it ai has nothing to do with it. US has killed 1000s of children in drone strikes in Afghanistan and Pakistan sometimes knowingly sometimes not. Most of the remote pilots do not give shit about what they did.