3 ms·

>> You can't unit test your way to secure, correct code in the kernel either. Only a community of testers and verifiers can do that. Unit tests cannot show the

>> You can't unit test your way to secure, correct code in the kernel either. Only a community of testers and verifiers can do that. Unit tests cannot show the presence of bugs that you are not teating for, but they can show the absence of bugs you've fixed and do not want to re-introduce / regressions. Not having unit tests strikes me as either hubris or lack of resources. Are you really going to have people run repeated tests even if some of the tests could be automated? If a test cannot be automated, then that's understandable, but why wouldn't you automate what you can with unit tests and integration tests? I would further argue that even if unit tests, integration tests, and a community of testers and verifiers are working together there will still be bugs, some of which will be critical. Why wouldn't you use every tool at your disposal in addition to community / people to find and fix the bugs?

Xelynega 4y ago
If interface A is verified to be correct code(not tested) and it's replaced with interface B that's also verified to be correct, why would there be any regressions? The reason I see not to do it is it takes time away from writing correct code. I could spend half my time writing unit tests or I can spend twice the time writing the code in a verifiable language.
- thesuperbigfrog 4y ago
  >> If interface A is verified to be correct code (not tested) and it's replaced with interface B that's also verified to be correct, why would there be any regressions? How are you verifying that interface A and interface B are correct? You would either need to test manually, test automatically (i.e. test automation, unit tests, integration tests, fuzzing, and perhaps even mutation testing or design by contract), or maybe you are using formal methods (proving correctness with mathematical rigor). If you are not testing, how are you verifying that the code is correct? "It works on my machine" or "my teammate peer reviewed it and did not see anything wrong" are pretty low standards and how a lot of bugs get shipped and discovered in production. >> The reason I see not to do it is it takes time away from writing correct code. Again, how do you know the code is correct? Spending more time writing code is pointless if the code has bugs and exploits.

Xelynega 4y ago

If interface A is verified to be correct code(not tested) and it's replaced with interface B that's also verified to be correct, why would there be any regressions? The reason I see not to do it is it takes time away from writing correct code. I could spend half my time writing unit tests or I can spend twice the time writing the code in a verifiable language.

thesuperbigfrog 4y ago
>> If interface A is verified to be correct code (not tested) and it's replaced with interface B that's also verified to be correct, why would there be any regressions? How are you verifying that interface A and interface B are correct? You would either need to test manually, test automatically (i.e. test automation, unit tests, integration tests, fuzzing, and perhaps even mutation testing or design by contract), or maybe you are using formal methods (proving correctness with mathematical rigor). If you are not testing, how are you verifying that the code is correct? "It works on my machine" or "my teammate peer reviewed it and did not see anything wrong" are pretty low standards and how a lot of bugs get shipped and discovered in production. >> The reason I see not to do it is it takes time away from writing correct code. Again, how do you know the code is correct? Spending more time writing code is pointless if the code has bugs and exploits.