3 ms·

>> Problems cost exponentially more to fix the later you catch them, and the damage they can cause only grows with time. > Citation needed. Really? Isn't this

by feoren 3y ago

>> Problems cost exponentially more to fix the later you catch them, and the damage they can cause only grows with time. > Citation needed. Really? Isn't this one of the most clear and repeatedly demonstrated facts in all of software engineering? We don't have a whole lot of clear-cut demonstrable facts in this industry, but this is absolutely one of them. How about this 2002 NIST report, Table 1-5 on page 1-13 is "Relative Costs to Repair Defects when Found at Different Stages of the Life-Cycle" and shows 1X at "Requirements", 90X at "System Testing", 440X at "acceptance testing", and up to 880X at "operation and maintenance". (Warning: 300-page PDF) https://www.nist.gov/system/files/documents/director/planning/report02-3.pdf https://www.nist.gov/system/files/documents/director/plannin... You can find thousands of people saying the same thing from their personal experience with a quick Google. On the list of uncontroversial statements about software engineering, this is right at the top. Is our discipline so immature that we can't even agree on this extremely basic fact? Another (okay, they don't say "exponential" here, or try to quantify it at all): > The cost of finding and fixing bugs or defects is the largest single expense element in the software lifecycle. ... The earlier in the development lifecycle defects are found, the more economical the overall delivery will be. The Cost of Poor Software Quality in the US: A 2020 Report, Consortium for Information & Software Quality https://www.it-cisq.org/cisq-files/pdf/CPSQ-2020-report.pdf https://www.it-cisq.org/cisq-files/pdf/CPSQ-2020-report.pdf > My day job is writing code that runs on a server I control. Mistakes there are super cheap to fix. That's good to hear, but mistakes caught before they get to your server are even cheaper still. Perhaps your exponential factors are smaller than NASA's, but it still clearly costs you much more to fix a problem that made it to your production server (and possibly impacted your users) than to fix it immediately after typing it because your IDE showed you a red squiggly line.

boxed 3y ago
> Really? Isn't this one of the most clear and repeatedly demonstrated facts in all of software engineering? It's a repeated saying. > You can find thousands of people saying the same thing from their personal experience with a quick Google. I know. > That's good to hear, but mistakes caught before they get to your server are even cheaper still That's a nonsense statement. If they are caught AT NO COST they are of course cheaper yea. But that's the thing. It's all tradeoffs. Everything is about tradeoffs. What is the marginal cost of finding bugs? > than to fix it immediately after typing it because your IDE showed you a red squiggly line. Yea, that's nice. In that case the marginal cost is approaching zero. But how many percentage of all bugs is that? And what is the cost? How many bugs will my IDE find if I switch from Python to Haskell or Rust? And how will that impact my development speed? I write my frontend in Elm because that's code I ship to users (via the browser) and THERE the costs are enormously worse, often because the problems can be seen by customers and not internal users (again: CONTEXT). I'm willing to tolerate a big productivity cut for being sure the code is correct there. But on my server? Nope. I am not willing to leave an easy order of magnitude of development time on the table for that type of safety. And why stop at static typing? Why don't you run ALL your code through Property Based Testing AND Mutation Testing 100% before shipping? What's the marginal cost of catching a bug then? Why not run ALL your code through a theory prover like Coq? Bugs are NOT always "cheaper" when found early. It's about the COST of finding the bug early. Of course it is! If the cost is a trillion dollars in programmer time to find a bug that takes 1 second to fix in production at a cost of a few dollars of programmer time, by definition it was NOT worth it. It's ALWAYS a tradeoff. Of course it is. Everything is. It's maddening to me that people refuse to see this when it's so blatantly obvious.
- feoren 3y ago
  There's a reason I used the word "problem" and not "bug" in my original post. In fact every time I found myself writing "bug", I deleted it and wrote "problem" instead. You're thinking very specifically about "bugs": clear spots a codebase that are clearly causing some clear error. That's actually a small percentage of the scope of "problems" that I'm talking about. Problems can be: your idea is bad, your architecture doesn't fit together, your performance is bad, your calculations are wrong, etc. Static typing, when done well, can help prevent many more of these issues than that narrow definition of "bug". Static typing helps you think clearly about your code. > the marginal cost [of fixing a red squiggly] is approaching zero. But how many percentage of all bugs is that? And what is the cost? The marginal cost of fixing a red squiggly underline is indeed basically zero for typos, and typos are a class of bugs I'd like my static type system to help me catch; and it does, and dynamic type systems generally don't. So the percentage of typos that become bugs is basically zero, but that's partly because of static typing! When you say "the percentage of all bugs", your definition of "bug" is a-priori excluding things that were caught before they became bugs! But you can also get a red squiggly underline because you thought some variable was in scope when it's actually not -- that bug could have been caught earlier. The cost of fixing that bug is not near zero: the structure of your program is different between your brain and your code. The easiest bug to fix is the one you never write in the first place. Or you get a red squiggly because you thought you could pass an object of type X into a method you're relying on, but you actually can't. Dynamic typing will not help you with this, and the cost of fixing that bug is not zero: again, your assumptions about how your program is fitting together are incorrect. I'd much rather realize that I can't use a method in exactly the way I thought right at the moment of typing it than later during test time. The latter requires writing and maintaining a test in order to catch a problem late that I could have caught early with no test. > And why stop at static typing? Why don't you run ALL your code through Property Based Testing AND Mutation Testing 100% before shipping? I directly addressed the subject of testing in my original post. Static typing allows me to write many fewer tests than dynamic typing. > If the cost is a trillion dollars in programmer time to find a bug that takes 1 second to fix in production at a cost of a few dollars of programmer time, by definition it was NOT worth it. I just don't understand this hypothetical. Why would it ever take a trillion dollars to prevent a bug that takes 1 second to fix in production? This just doesn't jive with any developer experience I've ever heard about, any example I've ever read about, or anything I can even imagine. You seem to be operating under the assumption that introducing static typing makes you slower at programming, but we're claiming it's worth it. No, that's not my claim. My claim is that above some level of program complexity, static typing makes you strictly faster at programming, full stop. That level of program complexity may not be your internal server or whatever, but it definitely is most production-grade software out there. > It's ALWAYS a tradeoff. Of course it is. Everything is. It's maddening to me that people refuse to see this when it's so blatantly obvious. This is generally true in engineering, but not always. It is actually possible to simply advance the discipline. When you're building a suspension bridge, there's a tradeoff between Stainless Steel and High-Carbon Steel, but there's not a tradeoff between Steel and Popsicle Sticks. Unless, of course, you're building a toy or a proof of concept. Dynamic typing is fine for toys and proofs of concepts; nobody is disputing that. Professional engineers don't spend most of their time discussing the best way to build toys.

by feoren 3y ago

boxed 3y ago

> Really? Isn't this one of the most clear and repeatedly demonstrated facts in all of software engineering? It's a repeated saying. > You can find thousands of people saying the same thing from their personal experience with a quick Google. I know. > That's good to hear, but mistakes caught before they get to your server are even cheaper still That's a nonsense statement. If they are caught AT NO COST they are of course cheaper yea. But that's the thing. It's all tradeoffs. Everything is about tradeoffs. What is the marginal cost of finding bugs? > than to fix it immediately after typing it because your IDE showed you a red squiggly line. Yea, that's nice. In that case the marginal cost is approaching zero. But how many percentage of all bugs is that? And what is the cost? How many bugs will my IDE find if I switch from Python to Haskell or Rust? And how will that impact my development speed? I write my frontend in Elm because that's code I ship to users (via the browser) and THERE the costs are enormously worse, often because the problems can be seen by customers and not internal users (again: CONTEXT). I'm willing to tolerate a big productivity cut for being sure the code is correct there. But on my server? Nope. I am not willing to leave an easy order of magnitude of development time on the table for that type of safety. And why stop at static typing? Why don't you run ALL your code through Property Based Testing AND Mutation Testing 100% before shipping? What's the marginal cost of catching a bug then? Why not run ALL your code through a theory prover like Coq? Bugs are NOT always "cheaper" when found early. It's about the COST of finding the bug early. Of course it is! If the cost is a trillion dollars in programmer time to find a bug that takes 1 second to fix in production at a cost of a few dollars of programmer time, by definition it was NOT worth it. It's ALWAYS a tradeoff. Of course it is. Everything is. It's maddening to me that people refuse to see this when it's so blatantly obvious.

feoren 3y ago
There's a reason I used the word "problem" and not "bug" in my original post. In fact every time I found myself writing "bug", I deleted it and wrote "problem" instead. You're thinking very specifically about "bugs": clear spots a codebase that are clearly causing some clear error. That's actually a small percentage of the scope of "problems" that I'm talking about. Problems can be: your idea is bad, your architecture doesn't fit together, your performance is bad, your calculations are wrong, etc. Static typing, when done well, can help prevent many more of these issues than that narrow definition of "bug". Static typing helps you think clearly about your code. > the marginal cost [of fixing a red squiggly] is approaching zero. But how many percentage of all bugs is that? And what is the cost? The marginal cost of fixing a red squiggly underline is indeed basically zero for typos, and typos are a class of bugs I'd like my static type system to help me catch; and it does, and dynamic type systems generally don't. So the percentage of typos that become bugs is basically zero, but that's partly because of static typing! When you say "the percentage of all bugs", your definition of "bug" is a-priori excluding things that were caught before they became bugs! But you can also get a red squiggly underline because you thought some variable was in scope when it's actually not -- that bug could have been caught earlier. The cost of fixing that bug is not near zero: the structure of your program is different between your brain and your code. The easiest bug to fix is the one you never write in the first place. Or you get a red squiggly because you thought you could pass an object of type X into a method you're relying on, but you actually can't. Dynamic typing will not help you with this, and the cost of fixing that bug is not zero: again, your assumptions about how your program is fitting together are incorrect. I'd much rather realize that I can't use a method in exactly the way I thought right at the moment of typing it than later during test time. The latter requires writing and maintaining a test in order to catch a problem late that I could have caught early with no test. > And why stop at static typing? Why don't you run ALL your code through Property Based Testing AND Mutation Testing 100% before shipping? I directly addressed the subject of testing in my original post. Static typing allows me to write many fewer tests than dynamic typing. > If the cost is a trillion dollars in programmer time to find a bug that takes 1 second to fix in production at a cost of a few dollars of programmer time, by definition it was NOT worth it. I just don't understand this hypothetical. Why would it ever take a trillion dollars to prevent a bug that takes 1 second to fix in production? This just doesn't jive with any developer experience I've ever heard about, any example I've ever read about, or anything I can even imagine. You seem to be operating under the assumption that introducing static typing makes you slower at programming, but we're claiming it's worth it. No, that's not my claim. My claim is that above some level of program complexity, static typing makes you strictly faster at programming, full stop. That level of program complexity may not be your internal server or whatever, but it definitely is most production-grade software out there. > It's ALWAYS a tradeoff. Of course it is. Everything is. It's maddening to me that people refuse to see this when it's so blatantly obvious. This is generally true in engineering, but not always. It is actually possible to simply advance the discipline. When you're building a suspension bridge, there's a tradeoff between Stainless Steel and High-Carbon Steel, but there's not a tradeoff between Steel and Popsicle Sticks. Unless, of course, you're building a toy or a proof of concept. Dynamic typing is fine for toys and proofs of concepts; nobody is disputing that. Professional engineers don't spend most of their time discussing the best way to build toys.