3 ms·

Our customers pay us and don't want to pay for a broken experience. I respect our customers too much to ship untested code. Heck, even if we were free, I'd exp

Our customers pay us and don't want to pay for a broken experience. I respect our customers too much to ship untested code. Heck, even if we were free, I'd expect far more users to experience bugs than report them. Why would those users and customers continue to trust the reliability of your software? Thus reducing future conversions and giving reason for churn.

whatever1 3y ago
Your tests make you feel like your code is correct. The fact is you have no clue if your tests cover 99, 9 or 0.9% of all potential cases. If your code is critical you need proof of correctness, not feelings.
- sethammons 3y ago
  Nonsense. The argument isn't 100% proof based tests vs yolo shipping directly to prod. Our tests are based on user actions and expected error conditions (from bad input to network errors). We don't cover the combinatorial explosion of potential inputs but we don't need 100% mathematical proof based confidence. Tests boost the confidence from "i think the code works" with no tests to "i know this code works in these N cases." It is risk management. Could we miss an unknown percentage of potential errors? Of course. But the errors that happen become fewer and fewer as the deployment safety net grows. And we monitor bug reports, errors, and incidents over time to gauge the effectiveness. This has allowed us to confidently ship code that has earned our company, literally, hundreds of millions of dollars and has evolved over time. I know my tests mean the software works as expected because my tests are my expectations. Maybe there is a cache bug or a memory leak or a logic bug or a misunderstood business requirement, but I _know_ that when my code deploys, standard usage of it will absolutely work. If it didn't, then a test should have failed. I've been developing software for a couple of decades now and I have a good feel on what to test at what layer. If the argument is only "let my valued customers experience bugs and report them to me" vs tests before code release, I'm siding on tests before releasing.

whatever1 3y ago

Your tests make you feel like your code is correct. The fact is you have no clue if your tests cover 99, 9 or 0.9% of all potential cases. If your code is critical you need proof of correctness, not feelings.

sethammons 3y ago
Nonsense. The argument isn't 100% proof based tests vs yolo shipping directly to prod. Our tests are based on user actions and expected error conditions (from bad input to network errors). We don't cover the combinatorial explosion of potential inputs but we don't need 100% mathematical proof based confidence. Tests boost the confidence from "i think the code works" with no tests to "i know this code works in these N cases." It is risk management. Could we miss an unknown percentage of potential errors? Of course. But the errors that happen become fewer and fewer as the deployment safety net grows. And we monitor bug reports, errors, and incidents over time to gauge the effectiveness. This has allowed us to confidently ship code that has earned our company, literally, hundreds of millions of dollars and has evolved over time. I know my tests mean the software works as expected because my tests are my expectations. Maybe there is a cache bug or a memory leak or a logic bug or a misunderstood business requirement, but I _know_ that when my code deploys, standard usage of it will absolutely work. If it didn't, then a test should have failed. I've been developing software for a couple of decades now and I have a good feel on what to test at what layer. If the argument is only "let my valued customers experience bugs and report them to me" vs tests before code release, I'm siding on tests before releasing.