4 ms·

I've worked on building some AI for drones for the US, and I'm fairly certain that advents in new technologies will mitigate most of the concerns in the article

I've worked on building some AI for drones for the US, and I'm fairly certain that advents in new technologies will mitigate most of the concerns in the article. It will make for a seriously scary new world, where devices like this exist at massive scale. Dictatorships will be even more brutal, and oppressors, even more oppressive. The sociopath in control of these machines will no longer have to worry about someone not carrying out orders properly due to emotions or questioning their judgement. All that aside, the one aspect I don't see getting resolved is the psychological effects of someone relatively "normal" & smart watching the horrors and carnage of war and then heading to the grocery store for some ice cream after his/her shift. Unlike being in a miserable (i.e. battlefield) place where your body is filled with adrenaline and a good chunk of your mental resources are dedicated to keeping yourself alive and getting the hell out of there alive. Having the time to reflect on the serious inequities in the world that you've just been witness to can't easily be swept under the rug. Even if the systems are mostly automated and pictures have a higher resolution, provided that the person/analyst/commander behind the screen is a somewhat sane and intelligent member of society, the psychological tolls might be worse when you're experiencing carnage thru relative safety.

a_olt 13y ago
This article discussed one of the solutions proposed by psychologists to relieve drone pilots of the stress and guilt that comes with launching an attack: http://io9.com/psychologists-propose-horrifying-solution-to-ptsd-in-dr-1453349900 http://io9.com/psychologists-propose-horrifying-solution-to-...
- Killah911 13y ago
  Thx for posting the article. "Siri, kill those people", sounds like the makings of a new John Cleese bit. Had to check twice and make sure I wasn't reading an Onion article. I still wonder how much less existential strife an operator might experience by delegating the killing to an AI machine, especially if one can still see the faces (better resolution) and the carnage following "Siri" carrying out orders.
ryandrake 13y ago
Just curious, you said you worked on building some of these systems, so how did you personally square your morals/ethics with the fact that your employment was probably helping to bring about this "seriously scary new world"? I took a similar job once, sold on the awesome technical challenge (aeronautical mapping, gyro/sensor input, camera control, etc). But over the time I worked there, I became more and more ethically conflicted about some of the potential applications of the technology I was developing. Was it wrong to continue working on this, given the potential for it to be abused? Personally, I believe developers are a lot like doctors in that we have a power that most lay-people consider to amount to magic, and with that power comes an obligation to do no harm. On one hand, if I wasn't working on this technology, they'd just hire someone else to do it, so leaving would not stop the program. On the other hand, while one person can't have an effect, if we all made the ethical choice, it might ultimately become difficult for these types of programs to exist. In the end I decided I wanted to be in the group that avoided such ethically questionable projects, lucked into a great opportunity to move into mobile development, and the rest is history.
- Killah911 13y ago
  That's crazy, I ended up feeling the same way. At first just the cool factor of building something like skynet blinded me. I had friends deployed in wars and I often found myself being worried and told myself that what I'm working will someday make sure that my friends' probability of coming home in a bodybag will be reduced significantly. I also told myself that my code itself was so early that it would never see any real action in any meaningful way. But eventually what I was helping to build became too much to think about. I moved on to hacking car ecu's and making them go faster. Utterly meaningless compared to my previous job, but a good fun couple of years of what amounted in my mind to toy-building after what I'd previously worked on. You can say I joined your group as well. I can only imagine what the poor sobs in the article go through, they don't exactly have the option to join said group. Not to mention watching real people dying in near real time must have a crazy impact that I can't ever compare to my own experience.
fit2rule 13y ago
>Having the time to reflect on the serious inequities in the world that you've just been witness to can't easily be swept under the rug. If there is hope, it lies in the proles.
- marcosdumay 13y ago
  I'm sure there is something at the end of 1984 (the book) that applies here. But, anyway, not exatly for everybody but those things will become cheap someday, up to the point that a person can build one, and stay as lethal. If that threatens us, it also threatens the people controlling today's drones. (And, just to be clear, I'm making no moral judgment here.)

a_olt 13y ago

This article discussed one of the solutions proposed by psychologists to relieve drone pilots of the stress and guilt that comes with launching an attack: http://io9.com/psychologists-propose-horrifying-solution-to-ptsd-in-dr-1453349900 http://io9.com/psychologists-propose-horrifying-solution-to-...

Killah911 13y ago
Thx for posting the article. "Siri, kill those people", sounds like the makings of a new John Cleese bit. Had to check twice and make sure I wasn't reading an Onion article. I still wonder how much less existential strife an operator might experience by delegating the killing to an AI machine, especially if one can still see the faces (better resolution) and the carnage following "Siri" carrying out orders.

ryandrake 13y ago

Just curious, you said you worked on building some of these systems, so how did you personally square your morals/ethics with the fact that your employment was probably helping to bring about this "seriously scary new world"? I took a similar job once, sold on the awesome technical challenge (aeronautical mapping, gyro/sensor input, camera control, etc). But over the time I worked there, I became more and more ethically conflicted about some of the potential applications of the technology I was developing. Was it wrong to continue working on this, given the potential for it to be abused? Personally, I believe developers are a lot like doctors in that we have a power that most lay-people consider to amount to magic, and with that power comes an obligation to do no harm. On one hand, if I wasn't working on this technology, they'd just hire someone else to do it, so leaving would not stop the program. On the other hand, while one person can't have an effect, if we all made the ethical choice, it might ultimately become difficult for these types of programs to exist. In the end I decided I wanted to be in the group that avoided such ethically questionable projects, lucked into a great opportunity to move into mobile development, and the rest is history.

Killah911 13y ago
That's crazy, I ended up feeling the same way. At first just the cool factor of building something like skynet blinded me. I had friends deployed in wars and I often found myself being worried and told myself that what I'm working will someday make sure that my friends' probability of coming home in a bodybag will be reduced significantly. I also told myself that my code itself was so early that it would never see any real action in any meaningful way. But eventually what I was helping to build became too much to think about. I moved on to hacking car ecu's and making them go faster. Utterly meaningless compared to my previous job, but a good fun couple of years of what amounted in my mind to toy-building after what I'd previously worked on. You can say I joined your group as well. I can only imagine what the poor sobs in the article go through, they don't exactly have the option to join said group. Not to mention watching real people dying in near real time must have a crazy impact that I can't ever compare to my own experience.

fit2rule 13y ago

>Having the time to reflect on the serious inequities in the world that you've just been witness to can't easily be swept under the rug. If there is hope, it lies in the proles.

marcosdumay 13y ago
I'm sure there is something at the end of 1984 (the book) that applies here. But, anyway, not exatly for everybody but those things will become cheap someday, up to the point that a person can build one, and stay as lethal. If that threatens us, it also threatens the people controlling today's drones. (And, just to be clear, I'm making no moral judgment here.)