6 ms·

What DHH describes is called the test pyramid (http://martinfowler.com/bliki/TestPyramid.html http://martinfowler.com/bliki/TestPyramid.html), which despite the

What DHH describes is called the test pyramid (http://martinfowler.com/bliki/TestPyramid.html http://martinfowler.com/bliki/TestPyramid.html), which despite the fact that DHH seems to dislike Uncle Bob, it is something Uncle Bob has demonstrated and advocated multiple times in very sensible ways. The bigger thing DHH doesn't mention or explain is that Basecamp is designed to be intentionally simple. Much of the complexity that you get in client work where you don't control the requirements Basecamp is able to simply avoid by not building. The idea that if you own your own product and control the requirements that you can have a MUCH cleaner codebase without needing as many tests is 100% true. But, David didn't say that and it's a shame. It's a much more powerful point because there are cases where clean architectures are valuable and the standard Rails MVC pattern isn't enough. At the same time, there are plenty of times where the basics will get you a very long ways. If you control the complexity, simple MVC might be all you ever need. Everything in software development involves managing requirements and tradeoffs. If you simplify requirements, you reduce complexity. You reduce complexity, you reduce surface area required to test. It's like weight reduction in cars, you reduce overall weight a few pounds, you can also reduce an extra 5-10 lbs. in required components to support the added weight/complexity. I just wish that people could discuss these ideas with nuance and empathy towards other project's requirements. Some projects require a lot of tests and have significant inherent complexity, some don't. A one size fits all philosophy might sound appealing, but it doesn't work. Don't throw the TDD baby out with the bathwater.

edwinnathaniel 12y ago
I wish this post stays at the top of _ANY_ discussion about automation testing. Couple V.E.R.Y important points: 1) Testing Pyramid People, who agree or disagree, talk a lot about unit-testing yet they never talk about the other types of automation testing; as if the rest were forgotten. To make matter worse, people often haven't had the experience of managing full-blown System Test. Hint: it's very brittle, slow, and expensive (even with the improved infrastructure automation tools such as Chef, Packer, Docker, Vagrant). Don't forget the magic word "tightly-coupled" that'll demotivate you hard and quick whenever you want to make some changes. 2) Control the requirement, control the codebase This is something that NO one ever mentioned in ANY "clean architecture/codebase" discussion. People often focus on the tools (Rails, Haskell, Clojure) and never mentioned that they actually control the requirements (or in another words: opinionated) or the requirements take a back seat against cleaner code. For example: if it didn't fit with Rails, don't do it <= controlling the requirement.
- graffic 12y ago
  I would prefer this post stays as an example of how not to do a post about the testing pyramid. Martin Fowler post/article is better IMHO, and I'm sure there are much better posts. There are some good ideas mixed with fallacies and disdain/contempt. These are not good in any discussion.
wpietri 12y ago
Another thing I think DHH has missed here is that his experience is not particularly broad. His achievements are very impressive, but if I recall rightly, he started Rails before finishing his CS degree, and that's basically all he's done since. One of the core assumptions of Rails is having a database. A lot of it is focused on turning HTTP requests into SQL requests, and turning SQL responses into HTML. Which is fine; that's what a lot of web stuff is, and it's fine for that. But it's not the only thing that happens in the world. I've TDDed systems that are not locked in intimate embraces with databases, and TDD is much easier and more pleasant there. On one project we did like that, we ended up with 40k lines of test (and a similar amount of production code) that ran in a few seconds. It was the most pleasant development experience of my career, and TDD was no problem there.
- IgorPartola 12y ago
  It's a lot easier to write unit tests for, you know, units. For example, if you have a date-to-readable-string conversion library, you can test the crap out of it since it's input is a date object and its output is a string. It's a unit of code, so all the testing that's required is unit testing. On the other hand, when the "unit" you are testing relies on six external API's and your own database to work, best of luck to ya. It's not a unit at this point, it's the glue that holds other units together. It is important to get it right, but given how many different combinations of failure modes there are you are not going to test it all and you are not going to get it right. Mocks/fakes do not make this stuff much better as timing and latency are important too. What if the UPS address service you are using doesn't return an error but times out? Now your production code has to include enough of a framework for simulating network errors. I do agree with TFA: TDD is a niche thing that has been so widely embraced for all the wrong reasons that it's no longer useful even in the niche. Also, system-wide testing is very important as well and the tooling for it sucks. My rule of thumb is to be pragmatic about automated testing: test the things that will give you the greatest ROI in terms of time/effort/time-to-market/etc. but don't be religious about it. The tests are not the end product and should not be treated as such.
  - wpietri 12y ago

edwinnathaniel 12y ago

I wish this post stays at the top of _ANY_ discussion about automation testing. Couple V.E.R.Y important points: 1) Testing Pyramid People, who agree or disagree, talk a lot about unit-testing yet they never talk about the other types of automation testing; as if the rest were forgotten. To make matter worse, people often haven't had the experience of managing full-blown System Test. Hint: it's very brittle, slow, and expensive (even with the improved infrastructure automation tools such as Chef, Packer, Docker, Vagrant). Don't forget the magic word "tightly-coupled" that'll demotivate you hard and quick whenever you want to make some changes. 2) Control the requirement, control the codebase This is something that NO one ever mentioned in ANY "clean architecture/codebase" discussion. People often focus on the tools (Rails, Haskell, Clojure) and never mentioned that they actually control the requirements (or in another words: opinionated) or the requirements take a back seat against cleaner code. For example: if it didn't fit with Rails, don't do it <= controlling the requirement.

graffic 12y ago
I would prefer this post stays as an example of how not to do a post about the testing pyramid. Martin Fowler post/article is better IMHO, and I'm sure there are much better posts. There are some good ideas mixed with fallacies and disdain/contempt. These are not good in any discussion.

wpietri 12y ago

Another thing I think DHH has missed here is that his experience is not particularly broad. His achievements are very impressive, but if I recall rightly, he started Rails before finishing his CS degree, and that's basically all he's done since. One of the core assumptions of Rails is having a database. A lot of it is focused on turning HTTP requests into SQL requests, and turning SQL responses into HTML. Which is fine; that's what a lot of web stuff is, and it's fine for that. But it's not the only thing that happens in the world. I've TDDed systems that are not locked in intimate embraces with databases, and TDD is much easier and more pleasant there. On one project we did like that, we ended up with 40k lines of test (and a similar amount of production code) that ran in a few seconds. It was the most pleasant development experience of my career, and TDD was no problem there.

IgorPartola 12y ago
It's a lot easier to write unit tests for, you know, units. For example, if you have a date-to-readable-string conversion library, you can test the crap out of it since it's input is a date object and its output is a string. It's a unit of code, so all the testing that's required is unit testing. On the other hand, when the "unit" you are testing relies on six external API's and your own database to work, best of luck to ya. It's not a unit at this point, it's the glue that holds other units together. It is important to get it right, but given how many different combinations of failure modes there are you are not going to test it all and you are not going to get it right. Mocks/fakes do not make this stuff much better as timing and latency are important too. What if the UPS address service you are using doesn't return an error but times out? Now your production code has to include enough of a framework for simulating network errors. I do agree with TFA: TDD is a niche thing that has been so widely embraced for all the wrong reasons that it's no longer useful even in the niche. Also, system-wide testing is very important as well and the tooling for it sucks. My rule of thumb is to be pragmatic about automated testing: test the things that will give you the greatest ROI in terms of time/effort/time-to-market/etc. but don't be religious about it. The tests are not the end product and should not be treated as such.
- wpietri 12y ago