3 ms·

I am reasonably well versed in the whole adversarial-input research. I am not sure whether this is really the problem. It's a tradeoff. This essentially introdu

I am reasonably well versed in the whole adversarial-input research. I am not sure whether this is really the problem. It's a tradeoff. This essentially introduces a random chance that you classification will fail (since adversarial input is not constrained to hard-to-classify situations), but while less accurate methods may be more robust to random errors (which is hard to verify for many approaches) but they have a higher likelihood to fail with hard situations. Doesn't every component in an aircraft have a random chance of failure? There's even a name for one critical component, the Jesus nut. I don't really get the interpretable argument. I think what you want is to verify it to a reasonable degree.

areoform 8y ago
> Doesn't every component in an aircraft have a random chance of failure? There's even a name for one critical component, the Jesus nut. Two things; 1) This might be theoretically true, but Boeing and other aircraft manufacturers monitor data on every component at a very fine-grained level and they use inspections to predict what will go bad and when. They're even adding real-time capabilities to it so that parts are replaced immediately after a flight if imminent failure is predicted; https://www.boeing.com/commercial/aeromagazine/articles/qtr_3_07/AERO_Q307_article4.pdf https://www.boeing.com/commercial/aeromagazine/articles/qtr_... 2) The core issue over here isn't that a component can fail. The issue is that when it fails can we trace the error and forestall it in the future? If the error is something that's unreplicable and unpredictable then that's quite literally impossible and it makes the machine by its very definition untenably unsafe as you don't know when or why it won't work. Someday neural networks will become a part of safety-critical systems, but this generation of neural networks probably won't be it.
- LeanderK 8y ago
  I convinced that neural network will never be interpretable like code. They will just get even more complex, so to employ them in critical applications (my motivation is more healthcare, where similiar constraints are present. People can die if there's a wrong prediction) we have to build a notion why we allow something and what we don't allow without some human in the loop getting convined that the reasoning itself is sound. I think this boils down to assumptions about robustness (like the l2-norm adversarial examples in image classification) and statistics. to 1). Neural networks are code, they are always 100% the same if you ship it. So you can test them way more thoughly than some equipment, which (i hope) would allow you to minimze the chance of failure to a reasonable degree. to 2) "you don't know when or why it won't work." I think a fundamental disagreement lies here. I am convinced that this is not possible with these complex function approximators, where we just optimize them until we are happy. If we adopt a thinking like above, we can retrain the networks with these additional inputs, adjust our notion of robustness and think whether there's a flaw in our statistical assumptions. But these are just random thoughts. I am in touch with people working in healthcare with this stuff, so there's some exposure to these kinds of problems. But I have never read any work discussing these issues or really reflection whether my reasoning is actually sound. Maybe I'm wrong and we can somehow make them interpretable in a sense that is actually relevant to verifying them.

areoform 8y ago

> Doesn't every component in an aircraft have a random chance of failure? There's even a name for one critical component, the Jesus nut. Two things; 1) This might be theoretically true, but Boeing and other aircraft manufacturers monitor data on every component at a very fine-grained level and they use inspections to predict what will go bad and when. They're even adding real-time capabilities to it so that parts are replaced immediately after a flight if imminent failure is predicted; https://www.boeing.com/commercial/aeromagazine/articles/qtr_3_07/AERO_Q307_article4.pdf https://www.boeing.com/commercial/aeromagazine/articles/qtr_... 2) The core issue over here isn't that a component can fail. The issue is that when it fails can we trace the error and forestall it in the future? If the error is something that's unreplicable and unpredictable then that's quite literally impossible and it makes the machine by its very definition untenably unsafe as you don't know when or why it won't work. Someday neural networks will become a part of safety-critical systems, but this generation of neural networks probably won't be it.

LeanderK 8y ago
I convinced that neural network will never be interpretable like code. They will just get even more complex, so to employ them in critical applications (my motivation is more healthcare, where similiar constraints are present. People can die if there's a wrong prediction) we have to build a notion why we allow something and what we don't allow without some human in the loop getting convined that the reasoning itself is sound. I think this boils down to assumptions about robustness (like the l2-norm adversarial examples in image classification) and statistics. to 1). Neural networks are code, they are always 100% the same if you ship it. So you can test them way more thoughly than some equipment, which (i hope) would allow you to minimze the chance of failure to a reasonable degree. to 2) "you don't know when or why it won't work." I think a fundamental disagreement lies here. I am convinced that this is not possible with these complex function approximators, where we just optimize them until we are happy. If we adopt a thinking like above, we can retrain the networks with these additional inputs, adjust our notion of robustness and think whether there's a flaw in our statistical assumptions. But these are just random thoughts. I am in touch with people working in healthcare with this stuff, so there's some exposure to these kinds of problems. But I have never read any work discussing these issues or really reflection whether my reasoning is actually sound. Maybe I'm wrong and we can somehow make them interpretable in a sense that is actually relevant to verifying them.