See relevant XKCD: https://xkcd.com/1958/ https://xkcd.com/1958/ I believe that most ML applications are fool-able by attacks of the same type as would fool a

See relevant XKCD: https://xkcd.com/1958/ https://xkcd.com/1958/ I believe that most ML applications are fool-able by attacks of the same type as would fool a human doing the same task as the ML. The really big difference is the scale of attack required to fool ML vs a human. For example, let's consider ML processing of camera+lidar data, similar to Google's self driving system, versus a cardboard cut-out of a puppy in the road. A human could be fooled by a really elaborate cardboard cut-out, or one viewed at a high speed. ML is likely to be fooled by a cut-out as well, but the elaborateness required of the cut-out would likely be lower (people would notice it doesn't look normal, while the ML has a much smaller dataset than the average person, and hasn't focussed on cute furry puppies as much). The ML would also likely be much better at dealing with high speeds than a human would be, because we don't have built in lidars and our brains work order of magnitudes slower; but taken to an extreme (ML can't complete processing in time to identify fake), the ML would not be able to tell the difference. There are certainly other ways to go about it, but I think this is the most straight forward and general 'attack', in that false positives are unavoidable in ML (and in humans).