3 ms·
It's impossible to prove that code works. But tests are a strong indicator and at least put bounds on where the program does work. If you're paid to write code
by godelski 3y ago
It's impossible to prove that code works. But tests are a strong indicator and at least put bounds on where the program does work.
If you're paid to write code that works, you're paid to write tests. This is fairly standard in every other engineering field.
- makeitdouble 3y agoThis is the kind of shortcut that gets easily forgotten after a while IMHO. Why you write tests is important, and for instance coverage numbers are not that. Most automated coverage assessments still won't guarantee you're testing all the critical patterns (you just need enough to touch all the paths) and a low number doesn't always mean it's not enough. I understand the use as an heuristic's, but as it gets widely adopted it also becomes more and more useless. I mean, today we see people eyeing at LLMs to boost their coverage numbers automatically, and that trend of writing low effort tests has been going all for a while IMO.
- godelski 3y ago> I understand the use as an heuristic's, but as it gets widely adopted it also becomes more and more useless. I too am a big fan of Goodhart's Law. It seems many took this as advice and not a warning.
- ffsm8 3y agoIt's like that common misconception about the testing pyramid. The reason it's smaller at the top isn't because you should have numerically more tests at the bottom then at the top. It just shows that if you're doing a higher level test, you're also testing the layers below this. As an unrealistic example: if you'd have 1 IT and 1 UT, you'd still have double coverage at the bottom. You're probably still gonna create more UT then ITs though, as they're easier to write... so this is probably more academic pedantry then anything insightful
- saurik 3y agoYou can at least try to prove the code works, and you can get a lot further than people seem to bother. Hell: even just using a language with types--which many people don't do--is a form of invariant that proves away a lot of potential bugs you would otherwise have to test. And like, we absolutely have proof assistants and model checkers and more advanced languages with dependent types (even C++ can do a lot more than many languages due to how it can template over values, which includes stuff like the sizes of buffers on which it can do math at compile time, but we should be spending more time coding in the like of Coq/Idris/Lean)... let's not normalize a world in which people entirely give up on formal correctness. The problem with tests is that people use them as a crutch to not have to even just prove to themselves in their head -- much less to someone else or in a way that can be checked by a machine -- why their code does or doesn't work... they just throw together a ton of tests and if they hammer the code and when the tests stop failing they proudly announce "I guess it works now" and move on. My challenge: try to code for a while with the mentality that every every time you stop typing to test it or run and it doesn't work (including "my tests failed"), that is a serious problem that should be avoided. Instead, try your best to make it so the first time you get around to testing your code it works because you are that confident in how it works.