2 ms·
That's how safety critical devices are already built, so yes. We have standardized probabilities of failure (e.g. SIL [0]) from the unexpected, because mitigati
by AlotOfReading 4y ago
That's how safety critical devices are already built, so yes. We have standardized probabilities of failure (e.g. SIL [0]) from the unexpected, because mitigating 100% of risk is somewhere between impractical and impossible.
[0] https://en.wikipedia.org/wiki/Safety_integrity_level https://en.wikipedia.org/wiki/Safety_integrity_level
- davedx 4y agoIndeed. Some of the commentators on this article should really examine their assumptions about how reliable and correct “Software 1.0” is.
- ArchD 4y agoFrom a quick reading of the wiki, the associated methodology seems rather limited: "System complexity, particularly in software systems, making SIL estimation difficult to impossible" "The requirements of these schemes can be met either by establishing a rigorous development process, or by establishing that the device has sufficient operating history to argue that it has been proven in use." You could prove that normal code satisfies some specs, but you can't do that with neural nets unless the number of possible inputs is tiny. So, the only way to establish that the black box neural net meets some SIL target is through "sufficient operating history".
- AlotOfReading 4y agoTo clarify, I wasn't offering SIL up as an example of how we should validate ML systems, but instead to demonstrate that "software 1.0" systems are already designed the way GP is questioning. Best practices for applying integrity level concepts to ML is still a topic of active debate right now.