3 ms·
That's the thing about neural networks: any QA is going to be superficial due to their statistical black box nature.
by varjag 4y ago
That's the thing about neural networks: any QA is going to be superficial due to their statistical black box nature.
- dreamcompiler 4y agoExactly. It's the same reason that no amount of unit tests can replace formal methods for safety-critical software, and we cannot apply formal methods to neural nets [yet].
- IshKebab 4y agoThat's not really true. Most safety critical software is tested without formal verification. They are just really really thorough and rigorous. Formal verification is obviously better if you can do it. But it's still really really difficult, and plenty of software simply can't be formally verified. Even in hardware where the problem is a lot easier we've only recently got the technology to formally verify a lot of things, and plenty of things are still out of reach. And even if you do formally verify some software it doesn't guarantee it is free of bugs.
- ChickenNugger 4y agoWhat? Black box testing has plenty of techniques: https://en.wikipedia.org/wiki/Black-box_testing https://en.wikipedia.org/wiki/Black-box_testing Whether it's a neural network inside or not is completely irrelevant. That's why it's called "black box".
- erikerikson 4y agoThis seems to ignore that if you look inside the box at code you could understand it whereas looking at the activation values is unlikely to illuminate.
- varjag 4y agoPractical neural networks operate in enormous parameter spaces that are impossible to meaningfully test for all possible adversarial inputs and degraded outputs. Your FSD could well recognize stop signs in your battery of tests but not when someone drew a squirrel on it with green sharpie.
- dtheodor 4y agoYour run of the mill computer program also "operates in enormous parameter spaces that are impossible to meaningfully test for all possible adversarial inputs and degraded outputs".
- varjag 4y agoThis is hardly similar as the state of a typical computer program can be meaningfully inspected, allowing both useful insights for adversarial test setups and designing comprehensive formal tests.
- dtheodor 4y agoRight, if you consider the internal state, it is hardly similar. You talked about black box and QA though. Black box by definition holds the internal state as irrelevant, and QA mostly treats the software it tests as a black box, or in other words the tests are "superficial" as you call it.
- heleninboodler 4y agoBlack box testing in typical software is, however, less superficial, because the tester can make inferences and predictions about what inputs will affect the results. When you're testing a drawing program, for example, you may not know how the rectangle tool actually works internally, but you can make some pretty educated guesses about what types of inputs and program state affect its behavior. If the whole thing is controlled by a neural network with a staggering amount of internal state, the connections you can draw are much, much more tenuous, and consequently the confidence in your test methodology is significantly harder to come by.
- deleted 4y ago[deleted]
- JPLeRouzic 4y agoSomething a bit similar is clinical trial and it is accepted without problem. You make a black box test on several thousands (sometimes only hundreds) patients, and if patients who received the drug perform better the patients who received the placebo, then the drug is usually accepted for commercialization. Yet one isolated patient may be subject to several comorbidities, her environment could be weird, she could ingest other drugs (or coffee, OTC vitamins or even pomelo) without having declared it. In a recent past women were not part of clinical trials because being pregnant makes them very "non-standard'.