3 ms·

Software isn't all that different. There are plenty of things that the government could mandate. For example, there could be coverage requirements. Want to wor

Software isn't all that different. There are plenty of things that the government could mandate. For example, there could be coverage requirements. Want to work on software that determines if a bridge will be safe or not? Then you need 100% branch coverage. You gave as an example something from a building code. That's just a test case. There's no reason why for common types of software (CAD, fluid simulators) we could not have repositories of test cases that you must pass to within some tolerance. You could even make it adverserial and let people suggest new test cases. Or we could decide on some reference implementation that you must either match, or exceed. So if you design CAD software, on any input that both your CAD software can handle that this reference implementation can too, you must match. That's more of a programmatic open ended test case. If we're worried about discrimination, there's no reason why the the government couldn't publish a set of typical profiles (fake names, locations, credit scores, etc.) that your algorithm must treat equally to within some tolerance. etc. We can totally regulate software like we do other engineering disciplines. If we wanted to and thought that the resulting increase in cost and slowdown in development is important enough.

Gigachad 4y ago
Code coverage is not a very useful metric. It’s very easy to have 100% coverage without really testing anything. It works as a vague measure of test quality as long as none of the developers care about the value. As soon as they start making the coverage stat a goal itself, it becomes useless.
kortilla 4y ago
Code coverage tells you code was executed, not tested. What a worthless metric.
pbourke 4y ago
I think the regulation should happen upstream of the software. Code that goes in a medical device should be subject to different levels of rigor than code that goes into a free to play mobile game. To use your example of CAD software, certainly the system used to design a bridge should be built differently than the one used to make inputs to a consumer level 3D printer. There’s no such thing as a “casual” bridge or a “good enough” pacemaker, but that analogy holds in the software industry. Software’s infinite flexibility means that it should be regulated within its context of use.

Gigachad 4y ago

Code coverage is not a very useful metric. It’s very easy to have 100% coverage without really testing anything. It works as a vague measure of test quality as long as none of the developers care about the value. As soon as they start making the coverage stat a goal itself, it becomes useless.

kortilla 4y ago

Code coverage tells you code was executed, not tested. What a worthless metric.

pbourke 4y ago

I think the regulation should happen upstream of the software. Code that goes in a medical device should be subject to different levels of rigor than code that goes into a free to play mobile game. To use your example of CAD software, certainly the system used to design a bridge should be built differently than the one used to make inputs to a consumer level 3D printer. There’s no such thing as a “casual” bridge or a “good enough” pacemaker, but that analogy holds in the software industry. Software’s infinite flexibility means that it should be regulated within its context of use.