3 ms·

this is a tough one, to make an exploitable attack you would need at a minimum to have a copy of the flight AI and be able to simulate possible encounters. Giv

by lumost 6y ago

this is a tough one, to make an exploitable attack you would need at a minimum to have a copy of the flight AI and be able to simulate possible encounters. Given the relatively finite number of side-channels available, it's probably far cheaper for the AI builder to run such simulations and adversarially test against the main exploitable variations such as blasting the aircraft's sensors with bad/confusing data. There just aren't that many methods of interacting with a plane >1km away, and even fewer at the 50-100KM ranges that modern fighters are built to fight at.

godelski 6y ago
> to make an exploitable attack you would need at a minimum to have a copy of the flight AI and be able to simulate possible encounters. I'd disagree. Data is more important than the model. But also you can do black box attacks on AI pretty effectively. What you do is train your AI to perform a similar feat and then figure out its weaknesses. Those weaknesses are likely to translate to the other AI. Robustness is a difficult task. As for blasting confusing data, well that already happens to human pilots. There are radar jammers, targeting jammers, visually occluding devices, etc. But hey, if an AI can operate under 90% of typical dog fight situations, then that's a big reduction in needed pilots. Then those pilots can be all trained to be aces.
- johbjo 6y ago
  If weaknesses can be known in advance, then they can "just" be incorporated into the training environment. Unless you can feed the sensors "false" data, there aren't necessarily that many possible exploits.
  - godelski 6y ago
    You should read up more on adversarial machine learning. If you could solve things so simply you'd get a lot of papers really quick (and lots of money).
imtringued 6y ago
When you shoot down an aircraft it leaves a wreck on the ground. You can extract the AI and find weaknesses that way.

by lumost 6y ago

godelski 6y ago

> to make an exploitable attack you would need at a minimum to have a copy of the flight AI and be able to simulate possible encounters. I'd disagree. Data is more important than the model. But also you can do black box attacks on AI pretty effectively. What you do is train your AI to perform a similar feat and then figure out its weaknesses. Those weaknesses are likely to translate to the other AI. Robustness is a difficult task. As for blasting confusing data, well that already happens to human pilots. There are radar jammers, targeting jammers, visually occluding devices, etc. But hey, if an AI can operate under 90% of typical dog fight situations, then that's a big reduction in needed pilots. Then those pilots can be all trained to be aces.

johbjo 6y ago
If weaknesses can be known in advance, then they can "just" be incorporated into the training environment. Unless you can feed the sensors "false" data, there aren't necessarily that many possible exploits.
- godelski 6y ago
  You should read up more on adversarial machine learning. If you could solve things so simply you'd get a lot of papers really quick (and lots of money).

imtringued 6y ago

When you shoot down an aircraft it leaves a wreck on the ground. You can extract the AI and find weaknesses that way.