4 ms·

This is such a fascinating topic because the field of robotics outside of industrial applications is still so nascent. I'm not here to weigh in, just to drop

by dljsjr 13y ago

This is such a fascinating topic because the field of robotics outside of industrial applications is still so nascent. I'm not here to weigh in, just to drop off some useful tidbits for other people interested in this ethical gray area as well. This theme of "terminators" and "killer robots" has been really prevalent in the field lately because of the DARPA Robotics Challenge[1], the latest in DARPA's Grand Challenges, which I'm currently participating in on one of the Track B teams. Many people see the break away from bomb-squad bots and factory floor robotic arms in to humanoids as a really scary thing, and the DRC seems to amplify that (if a robot can hold a sawzall, it can hold a rifle). Just last month in Atlanta, the IEEE Humanoids conference took place. Dr. Ronald C. Arkin[2] gave a talk exactly on this topic from an ethics perspective, titled "How to NOT build a Terminator", and it was exceptional. It was a plenary talk and not a paper, so I can't really find any record of it to share. Unfortunate. Tangentially, the lab that I work in (the robotics group at the Florida Institute for Human and Machine Cognition[3][4]) has employed and continues to employ a principle that we call "co-active design"[5] where we actively work to keep a human in the loop at all times; we're definitely not looking to build a "killer robot". It's an interesting design problem that overlaps a lot with UI and UX, popular topics here on HN. And lastly, a shameless plug for the field itself; a lot of people don't realize just how software-oriented robotics research (especially humanoids, where the fun problems are) is. A lot of people are stuck on it being a hardware endeavor. While it's true that a chunk of robotics falls in the mechanical engineering domain, there's plenty of room for hackers from tons of different disciplines to get involved. There's interesting people solving interesting problems, from the Open Source Robotics Foundation (the Willow Garage spin-off) to private groups like Boston Dynamics, yet so many people still see the field as a black-box that only opens up for the hardware inclined. I could see a talented group with the right hacker mindset doing some really interesting stuff in robotics with the right impetus and execution. 1:http://theroboticschallenge.org http://theroboticschallenge.org 2:http://en.wikipedia.org/wiki/Ronald_C._Arkin http://en.wikipedia.org/wiki/Ronald_C._Arkin 3:http://ihmc.us http://ihmc.us 4:http://robots.ihmc.us http://robots.ihmc.us 5:http://www.jeffreymbradshaw.net/publications/20101008_CoactiveDesign.pdf http://www.jeffreymbradshaw.net/publications/20101008_Coacti...

jonnathanson 13y ago
Fascinating stuff, and a great comment overall. But one thing needs to be clarified here: the fear over "killer robots" is that the capacity to make war could be placed largely on autopilot. The danger isn't so much a Terminator-style robotic uprising, which is so farfetched in today's world as to be sensationalistic (if not laughable). Rather, the danger -- a very real one, at that -- lies in an accidental war as the result of malfunctions or data-interpretation errors. We have more to fear from dumb AI than from smart AI. As it turns out, the US and the Soviet Union came perilously close to nuclear war on multiple occasions over the result of computer errors. (For an interesting look into the subject, I recommend Eric Schlosser's "Command and Control," a recently released history of nuclear weapons policy over the last 50 years). Banning autonomous or semi-autonomous tactical systems ("Terminators") seems misguided and impractical. These things will get built, and some of them are being built already. The jury is still out on the ethics of building them (Do they save lives by taking humans off the front lines? Or do they take lives by turning war into a sort of game?). But they're being built, and that genie's not going back in the bottle. But banning fully autonomous strategic systems ("Skynet") seems more fruitful and worthwhile. Again, this is not because we expect "Skynet" to become self-aware and intentionally initiate a nuclear holocaust. It's because "Skynet" might misinterpret something and accidentally initiate a nuclear holocaust. (I don't mean to dismiss the very real role of human error in this same domain. But fail-safes, in the form of humans having the final decisionmaking authority over warfare at a strategic level, make a lot of sense.) I'm a layman here, and I'm way out of my depth in discussing the intricacies of war making AI. But the way I see it, we need to avoid a sort of dangerous middle ground here. Either we keep AI dumb and intentionally non-autonomous, or we make AI a heck of a lot smarter (therefore reducing the risk of fatal errors). In a weird way, building Skynet might be less risky than getting halfway to Skynet. A relatively dumb AI with full autonomy is a frightening thing. An extremely smart AI with the same degree of autonomy is a little unnerving, perhaps, but it's significantly less dangerous in practical terms.
- angersock 13y ago
  I come at it from the other end: A strategic AI system commanding human troops is reasonable; the deployment of tactical AI (more honestly dumb targeting and fire-control weapons systems) is not.

by dljsjr 13y ago

jonnathanson 13y ago

Fascinating stuff, and a great comment overall. But one thing needs to be clarified here: the fear over "killer robots" is that the capacity to make war could be placed largely on autopilot. The danger isn't so much a Terminator-style robotic uprising, which is so farfetched in today's world as to be sensationalistic (if not laughable). Rather, the danger -- a very real one, at that -- lies in an accidental war as the result of malfunctions or data-interpretation errors. We have more to fear from dumb AI than from smart AI. As it turns out, the US and the Soviet Union came perilously close to nuclear war on multiple occasions over the result of computer errors. (For an interesting look into the subject, I recommend Eric Schlosser's "Command and Control," a recently released history of nuclear weapons policy over the last 50 years). Banning autonomous or semi-autonomous tactical systems ("Terminators") seems misguided and impractical. These things will get built, and some of them are being built already. The jury is still out on the ethics of building them (Do they save lives by taking humans off the front lines? Or do they take lives by turning war into a sort of game?). But they're being built, and that genie's not going back in the bottle. But banning fully autonomous strategic systems ("Skynet") seems more fruitful and worthwhile. Again, this is not because we expect "Skynet" to become self-aware and intentionally initiate a nuclear holocaust. It's because "Skynet" might misinterpret something and accidentally initiate a nuclear holocaust. (I don't mean to dismiss the very real role of human error in this same domain. But fail-safes, in the form of humans having the final decisionmaking authority over warfare at a strategic level, make a lot of sense.) I'm a layman here, and I'm way out of my depth in discussing the intricacies of war making AI. But the way I see it, we need to avoid a sort of dangerous middle ground here. Either we keep AI dumb and intentionally non-autonomous, or we make AI a heck of a lot smarter (therefore reducing the risk of fatal errors). In a weird way, building Skynet might be less risky than getting halfway to Skynet. A relatively dumb AI with full autonomy is a frightening thing. An extremely smart AI with the same degree of autonomy is a little unnerving, perhaps, but it's significantly less dangerous in practical terms.

angersock 13y ago
I come at it from the other end: A strategic AI system commanding human troops is reasonable; the deployment of tactical AI (more honestly dumb targeting and fire-control weapons systems) is not.