4 ms·

Oh, yay. Given the magic of AI and ML, that's going to be a clusterfuck of epic proportions. If we're lucky, we'll even survive to tell the tale. It's going to

Oh, yay. Given the magic of AI and ML, that's going to be a clusterfuck of epic proportions. If we're lucky, we'll even survive to tell the tale. It's going to be vulnerable to (accidentally or intentionally) biased training data, and on top of that we're making rapid progress on adversarial patterns. On top of all that, the results will be relatively inscrutable. (Yes, you can somewhat explain the results you get. But the lure of ML is that it's right most of the time, and you start trusting it. Not to mention that explanations cost time and money)

Canada 9y ago
It's a reasonable strategy for processing images and other sensor data. People aren't perfect for the task either. We're expensive, get bored easily, and even at the best of times we're also vulnerable to plenty of adversarial patterns.
- tmalsburg2 9y ago
  Please define "reasonable" and explain how your definition accounts for the potential risks pointed out by OP.
  - Canada 9y ago
    Automated or not, the nature of the military mission guarantees it will take the lives and destroy the property of innocent people. (So don't take this as an endorsement of what the military is doing.) If all you care about is winning and you accept that human sentries sometimes shoot the wrong people, as all military forces do, then automation of sentry tasks makes perfect sense.
DyslexicAtheist 9y ago
poisoning ML training data is the new weapon against big-AI and the AI industrial complex :-D. Judging from your comment you probably have seen this already. Let me throw it out there for those who haven't: https://twitter.com/daniel_bilar/status/925541876484231168 https://twitter.com/daniel_bilar/status/925541876484231168
- groby_b 9y ago
  Or, for even more fun: http://www.cs.columbia.edu/~junfeng/papers/deepxplore-sosp17.pdf http://www.cs.columbia.edu/~junfeng/papers/deepxplore-sosp17... Deep learning exploits at a button press. Quoth: For all tested DL models, on average DeepXplore generated one test input demonstrating incorrect behavior within one second while running only on a commodity laptop. One second for triggering a failure, on any tested model. On a commodity laptop. That's just the technology you want to entrust with intelligence tasks.
  - tormeh 9y ago
    There has to be some kind of DNN research on using methods such as these to enhance accuracy of DNNs by using the antagonist examples as input. You could just take your existing labeled data and add light-medium noise to all your data (easiest with images?) and thereby get a lot more training data less prone to overfitting. It's so obvious to me so someone must have thought of it already. So my guess is that these kinds of easy antagonist examples won't be a thing for very long.
    - groby_b 9y ago
      Rule of thumb: If you find a blatantly obvious solution and you're not an expert in the field, assume people who are have thought of it and discarded it - or it would exist. (Note: I'm not an expert in the field, either. But the above rule works most of the time :)
  - abrichr 9y ago
    It's worth pointing out that their technique requires access to the internals of the network: First, we compute the gradient of the outputs of the neurons in both the output and hidden layers with the input value as a variable and the weight parameter as a constant. A well designed production system would not support this.
SN76477 9y ago
When working with humans use humans, when working with data use machines.
MR4D 9y ago
Wow, trying to stir things up, are we? Based on your comments, I don’t even think you read the article. Not sure how image processing will kill us all. That’s the conclusion that you drew. To address your pedantic comment further, I’d like to add that speeding up image processing or event correlation would reduce the grunt work and allow the intelligence folks to be more productive. Kind of like replacing typewriters with word processors. Definitely not the end of the world as you seem to think.

Canada 9y ago

It's a reasonable strategy for processing images and other sensor data. People aren't perfect for the task either. We're expensive, get bored easily, and even at the best of times we're also vulnerable to plenty of adversarial patterns.

tmalsburg2 9y ago
Please define "reasonable" and explain how your definition accounts for the potential risks pointed out by OP.
- Canada 9y ago
  Automated or not, the nature of the military mission guarantees it will take the lives and destroy the property of innocent people. (So don't take this as an endorsement of what the military is doing.) If all you care about is winning and you accept that human sentries sometimes shoot the wrong people, as all military forces do, then automation of sentry tasks makes perfect sense.

DyslexicAtheist 9y ago

poisoning ML training data is the new weapon against big-AI and the AI industrial complex :-D. Judging from your comment you probably have seen this already. Let me throw it out there for those who haven't: https://twitter.com/daniel_bilar/status/925541876484231168 https://twitter.com/daniel_bilar/status/925541876484231168

groby_b 9y ago
Or, for even more fun: http://www.cs.columbia.edu/~junfeng/papers/deepxplore-sosp17.pdf http://www.cs.columbia.edu/~junfeng/papers/deepxplore-sosp17... Deep learning exploits at a button press. Quoth: For all tested DL models, on average DeepXplore generated one test input demonstrating incorrect behavior within one second while running only on a commodity laptop. One second for triggering a failure, on any tested model. On a commodity laptop. That's just the technology you want to entrust with intelligence tasks.
- tormeh 9y ago
  There has to be some kind of DNN research on using methods such as these to enhance accuracy of DNNs by using the antagonist examples as input. You could just take your existing labeled data and add light-medium noise to all your data (easiest with images?) and thereby get a lot more training data less prone to overfitting. It's so obvious to me so someone must have thought of it already. So my guess is that these kinds of easy antagonist examples won't be a thing for very long.
  - groby_b 9y ago
    Rule of thumb: If you find a blatantly obvious solution and you're not an expert in the field, assume people who are have thought of it and discarded it - or it would exist. (Note: I'm not an expert in the field, either. But the above rule works most of the time :)
- abrichr 9y ago
  It's worth pointing out that their technique requires access to the internals of the network: First, we compute the gradient of the outputs of the neurons in both the output and hidden layers with the input value as a variable and the weight parameter as a constant. A well designed production system would not support this.

SN76477 9y ago

When working with humans use humans, when working with data use machines.

MR4D 9y ago

Wow, trying to stir things up, are we? Based on your comments, I don’t even think you read the article. Not sure how image processing will kill us all. That’s the conclusion that you drew. To address your pedantic comment further, I’d like to add that speeding up image processing or event correlation would reduce the grunt work and allow the intelligence folks to be more productive. Kind of like replacing typewriters with word processors. Definitely not the end of the world as you seem to think.