5 ms·

This whole thread is a response to: https://news.ycombinator.com/item?id=15591190 https://news.ycombinator.com/item?id=15591190 Your points are directly addres

This whole thread is a response to: https://news.ycombinator.com/item?id=15591190 https://news.ycombinator.com/item?id=15591190 Your points are directly addressed in the pdf. One of his general points being, in practice, the tests become the legacy system instead. And I'd add to that. Given that you've at minimum doubled the code (and doubled the bugs), it seems like a really bad long-term trade off. Also DI does not reduce coupling. I've seen plenty of code with DI that's just injecting like 30 things, which is obviously therefore coupled. It just makes it really obvious, but DI itself has massive downsides. If you've ever worked with bad programmers and seen it in the wild now, I'm sure we can agree DI and TDD doesn't stop bad programmers writing bad code. In fact, all it seems to do is make even more of a mess. Not only do you have to pick apart the bad code, you have to start dealing with carefully moving methods to the right places because DI can make it hard to figure out what's being used where, and then on top of that tests break all over the place because they're entirely dependant on the implementation instead of the functionality.

hacker_9 9y ago
What exactly is your experience working in a commercial environment? When the updates you ship can affect tens of thousands of customers? In these situations, 'Sods Law' often comes to mind - "what can happen, will happen". If you have a defect in your untested code, you can absolutely bet it will come out at the most painful time possible in front of all your clients. Being blamed for that kind of stuff is a stressful way to live your life, much nicer to have tests shout at you instead. * and to reply to your edit: of course tests break when you change the code they are testing! But after reviewing the broken tests, you see the intent and re-adjust the test. But what often happens, is you realise you didn't fully understand the code previously, and actually after reading the test you need to undo your refactoring as it didn't make sense in the first place.
- ozim 9y ago
  Pick the tool for the job, if your experience is with things that last 5+ years then ok. I work on a system which is 5 years old and all things from 5 years ago are not relevant. Actually I am working on it only for 3 years now but we pretty much each year rebuild whole thing. With minor things going in and out. Of course we spend a lot of money on automated tests but those tests had to be thrown away because of amount of changes in the system.
  - hacker_9 9y ago
    So you disagree with TDD, but you need to rebuild your system from scratch every 5 years? I feel like this is an argument for TDD. As for throwing away tests; Unit tests are meant to be pretty simple - rule of thumb is you can run a thousand tests in ten seconds. Arrange, Act, Assert - they don't need to be complex, they just need to imprint the intent into the codebase. If the intent changes, by all means remove the test.
    - nilkn 9y ago
      > So you disagree with TDD, but you need to rebuild your system from scratch every 5 years? I feel like this is an argument for TDD. Not necessarily. If the rewrite is only addressing issues that would have been prevented with tests, then sure, this is clearly an argument for TDD. However, if the rewrite is going beyond what could have been provided by tests, then having a large testing system could actually make the rewrites harder, which means it's an argument against TDD. It's hard to say which is the case without knowing the details, and I'd certainly err on the side of saying a yearly rewrite is not a good sign. But if a company is undergoing rapid growth, it's not that unusual for certain systems to be rewritten frequently as fundamental new insights are gathered about how to tackle problems that are hard to scale. And if the system isn't that large to begin with, periodic rewrites could be easier than writing a single version that's supposed to last 10+ years as business requirements dramatically change and expand.
      - Fifer82 9y ago
        I agree. My experience is: 1. Wow it made 10k! Lets rewrite 2. Wow it made 100k! Lets rewrite 3. Wow it made 1M! Lets rewrite but let's properly think hard about the future of this product (Introduce tests and more people). ALWAYS DO TESTING should always be given the context (if budget allows).
    - ozim 9y ago
      We rebuild because business needs change, because compliance and stuff. It is not like we rewrite because code is crap. We have like 30 percent of test coverage because that is part that changed only a bit.
- Fifer82 9y ago
  I think that everyone should have a 5 year stint with zero tests. I genuinely believe it offered me an insight into the deepest parts of depression. In saying that, it was raw and I think I enjoyed my job more. There is no safety harness. You write decent working code and sign your name by it or you run home to your mothers nipple. I definitely learned a lot about programming during that time. To take a terrible system and try to research ways to make it better for your own mental health, not the companies is a personal drive and golden age of discovery which plays a part of my programming to this day. I get none of this from TTD. It is intensely dull and unsatisfying but gets the job done.
- mattmanser 9y ago
  Had over a million people use my code last year, when I checked for vanity. 12 years commercial experience. I'm effectively the tech lead for two startups, one getting about 750k users per year, the other 250k. Freelancer, but I do a lot of time for 2 clients at the moment. Both were projects started by other people, but a significant chunk of the code is mine now (one of them I've virtually totally rewritten from VB.Net to C#) and the other I've done huge amounts of refactoring to fix big performance problems, reducing the "main" pages from 10 sec load times for complicated orders to 250ms. I've also refactored a lot of javascript for both of them without any tests, significantly improving client-side compile times and page load times. I played with unit tests a few years ago, one of my clients has some, but we mainly don't add any new ones and they never catch anything[1]. [1]That's a slight fib, one caught a bug last month for the first time in the 1.5 years I've worked for this client. It would almost certainly have been caught in testing though.
dkarl 9y ago
Also DI does not reduce coupling. I've seen plenty of code with DI that's just injecting like 30 things, which is obviously therefore coupled. It just makes it really obvious, but DI itself has massive downsides. This matches much of my experience with DI on real projects. Dependency injection is, unfortunately, used to reduce the labor required to achieve massive coupling. Of course the intended and potentially useful application of DI is being able to rewire an application with different components for different purposes or different environments, and the tradeoff for this flexibility is that you lose explicitness. You can no longer see the explicit wiring in code, which is a big downside. Yet I've worked on several DI-heavy projects with seasoned engineers who get this completely backwards. They see DI as a labor-saving device that allows them to write heavily coupled systems without having to explicitly work out the dependencies between things. In fact, they even see DI as eliminating the cost of complex interdependencies because it reduces the work and cleverness required to create them. Partly I think this reflects a desire to work on "real" "enterprise" systems. Instead of fighting to keep complexity below the level where DI actually helps, they embrace the flattering thought of, hey, we're doing big boy work here — it's going to get complex, so we'd better use DI from the outset. People who take a lot of pride in working on big systems can't help creating them.
gilbetron 9y ago
When TDD first started coming out years ago, in an email thread I asked one of the proponents how they handle the fact that tests can easily increase the complexity of a system by adding more code. I was expecting a rational discussion of how to balance tests making your code simpler with tests be a source of added complexity. Instead it was met with anger because I should "count" tests as part of the codebase, and how stupid I was for even considering such a thing. I still stand by my point. 100 lines of code with 1000 lines of testing code means your codebase is 10x what it would be without the tests. That extra 10x had better result in a hell of a lot of benefit. Additionally, over the years I've found that easily-tested code sacrifices readability, and that tiny functions isn't always the best way to go. I've gone back to liking larger, more monolithic functions as often I don't want to create a bunch of generic functions for hypothetical use for other projects, I just want some code that fits my needs to a T.
- hacker_9 9y ago
  > I asked one of the proponents how they handle the fact that tests can easily increase the complexity of a system by adding more code. Adding more code != more complexity. Complexity comes from high coupling between code, that doesn't separate concerns, which make it difficult to untangle, aka. spaghetti. Unit tests are mean to be simple, and anything they test will be no more complicated that how the system will actually be used. In this way they clarify the code, by encoding the intent, available for all to see for years to come. > I've gone back to liking larger, more monolithic functions as often I don't want to create a bunch of generic functions for hypothetical use for other projects, I just want some code that fits my needs to a T. This isn't what TDD causes, it just makes you break down things into small testable units. It doesn't make you create super generic wrappers, which I also agree are useless because of their un-readability.
  - gilbetron 9y ago
    Again, this is the argument made to me over 10 years ago, and again, they are wrong. While it is possible to add more code and not get more complexity, it is very difficult. Generally speaking, adding code adds complexity. Full stop. Excellently executed TDD will reduce overall complexity, but I almost never see that level. Rather, TDD's ability to catch bugs is mostly enough to reduce bugs (and other benefits) overall. But the costs are high.
lowbloodsugar 9y ago
That article is garbage. For example, the claim that an object has trillions of states ignores our ability to classify ranges of values. Sure, an int may contain "four billion states", but for the requirements (which he makes such a big deal about) its highly likely that we can classify the integer into three states: less than zero, zero, greater than zero. As a bank, I might not care how much money you have, only that you have more than zero. In a transaction, I don't care how much money changes hands, as long as no money is lost. The article is a tantrum.
- falcolas 9y ago
  > we can classify the integer into three states: less than zero, zero, greater than zero Err, not quite. We're running on computers, remember. A comprehensive unit test that doesn't go through every possible integer input should still include: maximum and minimum integer values for that size maximum and minimum expected inputs 0, 1, -1 So, there's 7 test cases for a single integral input. And since you bring up banking, let's imagine you're working with a function with two inputs. Since most bugs come about because of the interaction between two variables [0], you'd want to check out each combination. So, 7 possible inputs for the first integer, 7 for the second: 49 test cases. Why bother? You're working for a bank... imagine facing a client whose balance shot negative because of an overflow error and started accruing "overdraft coverage" fees. Back to the original point: when was the last time anybody but the AFL tool wrote the "proper" 49+ unit tests for an "add_to_balance(int, int)" style function, when one test would give you 100% coverage for that function? [0] https://csrc.nist.gov/Projects/Automated-Combinatorial-Testing-for-Software https://csrc.nist.gov/Projects/Automated-Combinatorial-Testi...
  - lowbloodsugar 9y ago
    If you mean "nobody would write 49 tests for a function when they could get away with writing one test and have it show up as 100% code coverage because people are lazy bastards" then I agree, and that's the problem: cargo culting the metrics. Look we have 100% code coverage! We are awesome! If you mean, "no look seriously, you'd need 49 tests for every method because you're a bank", well then, no, you actually need less than 49 (since some are subsets of the others), and in reality even this would get very tiring so you'd write a currency class that can't overflow, and yes, being a bank we would test every case and certainly use automated test generation while we're at it. But if you mean that in general, for non-banks, the classes of values is too large to manually write unit tests for, then we disagree. That means that your method is too complex.
tieTYT 9y ago
> Given that you've at minimum doubled the code (and doubled the bugs), it seems like a really bad long-term trade off. I'd say you've at maximum doubled the code. The test ensures you write only what you need to get the test pass. Without them, devs get distracted and wander until the feature works. Usually distracting themselves with tons of YAGNI violations along the way. In my experience, untested code bases have a ton of unnecessary code. I don't understand how it would double the bugs. The article has references saying it reduces them. But, even thinking about it, I don't see why you'd say that.

hacker_9 9y ago

What exactly is your experience working in a commercial environment? When the updates you ship can affect tens of thousands of customers? In these situations, 'Sods Law' often comes to mind - "what can happen, will happen". If you have a defect in your untested code, you can absolutely bet it will come out at the most painful time possible in front of all your clients. Being blamed for that kind of stuff is a stressful way to live your life, much nicer to have tests shout at you instead. * and to reply to your edit: of course tests break when you change the code they are testing! But after reviewing the broken tests, you see the intent and re-adjust the test. But what often happens, is you realise you didn't fully understand the code previously, and actually after reading the test you need to undo your refactoring as it didn't make sense in the first place.

ozim 9y ago
Pick the tool for the job, if your experience is with things that last 5+ years then ok. I work on a system which is 5 years old and all things from 5 years ago are not relevant. Actually I am working on it only for 3 years now but we pretty much each year rebuild whole thing. With minor things going in and out. Of course we spend a lot of money on automated tests but those tests had to be thrown away because of amount of changes in the system.
- hacker_9 9y ago
  So you disagree with TDD, but you need to rebuild your system from scratch every 5 years? I feel like this is an argument for TDD. As for throwing away tests; Unit tests are meant to be pretty simple - rule of thumb is you can run a thousand tests in ten seconds. Arrange, Act, Assert - they don't need to be complex, they just need to imprint the intent into the codebase. If the intent changes, by all means remove the test.
  - nilkn 9y ago
    > So you disagree with TDD, but you need to rebuild your system from scratch every 5 years? I feel like this is an argument for TDD. Not necessarily. If the rewrite is only addressing issues that would have been prevented with tests, then sure, this is clearly an argument for TDD. However, if the rewrite is going beyond what could have been provided by tests, then having a large testing system could actually make the rewrites harder, which means it's an argument against TDD. It's hard to say which is the case without knowing the details, and I'd certainly err on the side of saying a yearly rewrite is not a good sign. But if a company is undergoing rapid growth, it's not that unusual for certain systems to be rewritten frequently as fundamental new insights are gathered about how to tackle problems that are hard to scale. And if the system isn't that large to begin with, periodic rewrites could be easier than writing a single version that's supposed to last 10+ years as business requirements dramatically change and expand.
    - Fifer82 9y ago
      I agree. My experience is: 1. Wow it made 10k! Lets rewrite 2. Wow it made 100k! Lets rewrite 3. Wow it made 1M! Lets rewrite but let's properly think hard about the future of this product (Introduce tests and more people). ALWAYS DO TESTING should always be given the context (if budget allows).
  - ozim 9y ago
    We rebuild because business needs change, because compliance and stuff. It is not like we rewrite because code is crap. We have like 30 percent of test coverage because that is part that changed only a bit.
Fifer82 9y ago
I think that everyone should have a 5 year stint with zero tests. I genuinely believe it offered me an insight into the deepest parts of depression. In saying that, it was raw and I think I enjoyed my job more. There is no safety harness. You write decent working code and sign your name by it or you run home to your mothers nipple. I definitely learned a lot about programming during that time. To take a terrible system and try to research ways to make it better for your own mental health, not the companies is a personal drive and golden age of discovery which plays a part of my programming to this day. I get none of this from TTD. It is intensely dull and unsatisfying but gets the job done.
mattmanser 9y ago
Had over a million people use my code last year, when I checked for vanity. 12 years commercial experience. I'm effectively the tech lead for two startups, one getting about 750k users per year, the other 250k. Freelancer, but I do a lot of time for 2 clients at the moment. Both were projects started by other people, but a significant chunk of the code is mine now (one of them I've virtually totally rewritten from VB.Net to C#) and the other I've done huge amounts of refactoring to fix big performance problems, reducing the "main" pages from 10 sec load times for complicated orders to 250ms. I've also refactored a lot of javascript for both of them without any tests, significantly improving client-side compile times and page load times. I played with unit tests a few years ago, one of my clients has some, but we mainly don't add any new ones and they never catch anything[1]. [1]That's a slight fib, one caught a bug last month for the first time in the 1.5 years I've worked for this client. It would almost certainly have been caught in testing though.

dkarl 9y ago

Also DI does not reduce coupling. I've seen plenty of code with DI that's just injecting like 30 things, which is obviously therefore coupled. It just makes it really obvious, but DI itself has massive downsides. This matches much of my experience with DI on real projects. Dependency injection is, unfortunately, used to reduce the labor required to achieve massive coupling. Of course the intended and potentially useful application of DI is being able to rewire an application with different components for different purposes or different environments, and the tradeoff for this flexibility is that you lose explicitness. You can no longer see the explicit wiring in code, which is a big downside. Yet I've worked on several DI-heavy projects with seasoned engineers who get this completely backwards. They see DI as a labor-saving device that allows them to write heavily coupled systems without having to explicitly work out the dependencies between things. In fact, they even see DI as eliminating the cost of complex interdependencies because it reduces the work and cleverness required to create them. Partly I think this reflects a desire to work on "real" "enterprise" systems. Instead of fighting to keep complexity below the level where DI actually helps, they embrace the flattering thought of, hey, we're doing big boy work here — it's going to get complex, so we'd better use DI from the outset. People who take a lot of pride in working on big systems can't help creating them.

gilbetron 9y ago

When TDD first started coming out years ago, in an email thread I asked one of the proponents how they handle the fact that tests can easily increase the complexity of a system by adding more code. I was expecting a rational discussion of how to balance tests making your code simpler with tests be a source of added complexity. Instead it was met with anger because I should "count" tests as part of the codebase, and how stupid I was for even considering such a thing. I still stand by my point. 100 lines of code with 1000 lines of testing code means your codebase is 10x what it would be without the tests. That extra 10x had better result in a hell of a lot of benefit. Additionally, over the years I've found that easily-tested code sacrifices readability, and that tiny functions isn't always the best way to go. I've gone back to liking larger, more monolithic functions as often I don't want to create a bunch of generic functions for hypothetical use for other projects, I just want some code that fits my needs to a T.

hacker_9 9y ago
> I asked one of the proponents how they handle the fact that tests can easily increase the complexity of a system by adding more code. Adding more code != more complexity. Complexity comes from high coupling between code, that doesn't separate concerns, which make it difficult to untangle, aka. spaghetti. Unit tests are mean to be simple, and anything they test will be no more complicated that how the system will actually be used. In this way they clarify the code, by encoding the intent, available for all to see for years to come. > I've gone back to liking larger, more monolithic functions as often I don't want to create a bunch of generic functions for hypothetical use for other projects, I just want some code that fits my needs to a T. This isn't what TDD causes, it just makes you break down things into small testable units. It doesn't make you create super generic wrappers, which I also agree are useless because of their un-readability.
- gilbetron 9y ago
  Again, this is the argument made to me over 10 years ago, and again, they are wrong. While it is possible to add more code and not get more complexity, it is very difficult. Generally speaking, adding code adds complexity. Full stop. Excellently executed TDD will reduce overall complexity, but I almost never see that level. Rather, TDD's ability to catch bugs is mostly enough to reduce bugs (and other benefits) overall. But the costs are high.

lowbloodsugar 9y ago

That article is garbage. For example, the claim that an object has trillions of states ignores our ability to classify ranges of values. Sure, an int may contain "four billion states", but for the requirements (which he makes such a big deal about) its highly likely that we can classify the integer into three states: less than zero, zero, greater than zero. As a bank, I might not care how much money you have, only that you have more than zero. In a transaction, I don't care how much money changes hands, as long as no money is lost. The article is a tantrum.

falcolas 9y ago
> we can classify the integer into three states: less than zero, zero, greater than zero Err, not quite. We're running on computers, remember. A comprehensive unit test that doesn't go through every possible integer input should still include: maximum and minimum integer values for that size maximum and minimum expected inputs 0, 1, -1 So, there's 7 test cases for a single integral input. And since you bring up banking, let's imagine you're working with a function with two inputs. Since most bugs come about because of the interaction between two variables [0], you'd want to check out each combination. So, 7 possible inputs for the first integer, 7 for the second: 49 test cases. Why bother? You're working for a bank... imagine facing a client whose balance shot negative because of an overflow error and started accruing "overdraft coverage" fees. Back to the original point: when was the last time anybody but the AFL tool wrote the "proper" 49+ unit tests for an "add_to_balance(int, int)" style function, when one test would give you 100% coverage for that function? [0] https://csrc.nist.gov/Projects/Automated-Combinatorial-Testing-for-Software https://csrc.nist.gov/Projects/Automated-Combinatorial-Testi...
- lowbloodsugar 9y ago
  If you mean "nobody would write 49 tests for a function when they could get away with writing one test and have it show up as 100% code coverage because people are lazy bastards" then I agree, and that's the problem: cargo culting the metrics. Look we have 100% code coverage! We are awesome! If you mean, "no look seriously, you'd need 49 tests for every method because you're a bank", well then, no, you actually need less than 49 (since some are subsets of the others), and in reality even this would get very tiring so you'd write a currency class that can't overflow, and yes, being a bank we would test every case and certainly use automated test generation while we're at it. But if you mean that in general, for non-banks, the classes of values is too large to manually write unit tests for, then we disagree. That means that your method is too complex.

tieTYT 9y ago

> Given that you've at minimum doubled the code (and doubled the bugs), it seems like a really bad long-term trade off. I'd say you've at maximum doubled the code. The test ensures you write only what you need to get the test pass. Without them, devs get distracted and wander until the feature works. Usually distracting themselves with tons of YAGNI violations along the way. In my experience, untested code bases have a ton of unnecessary code. I don't understand how it would double the bugs. The article has references saying it reduces them. But, even thinking about it, I don't see why you'd say that.