3 ms·
I am beginning to realize that neural networks have their own class of “vulnerabilities” that are not the same as other software bugs (implementation errors, et
by TaylorAlexander 9y ago
I am beginning to realize that neural networks have their own class of “vulnerabilities” that are not the same as other software bugs (implementation errors, etc) but are at the same time serious functional flaws. Like “oh I found the bug in your program! Here you import an older CNN, which last year was found to silently fail under this specific set of lighting and lens conditions. You need to update to the latest version and the problem will go away.”
- dodobirdlord 9y agoOne of the major structural problems with using neural networks in production is that all failures are silent failures. You could create an ensemble of models that base their conclusions on reliably different features and report on disagreements with the ensemble decision, but that doesn't actually tell you which models were in error, just that some models were in error. Also, reliably determining that models are using different features is difficult.