7 ms·

Here's a transcript of the podcast I believe Kent is referring to. Judge for yourself whether Joel "makes comments that make clear his lack of knowledge". http

by lacker 18y ago

Here's a transcript of the podcast I believe Kent is referring to. Judge for yourself whether Joel "makes comments that make clear his lack of knowledge". http://www.joelonsoftware.com/items/2009/01/31.html http://www.joelonsoftware.com/items/2009/01/31.html I'd like to see Kent respond to Joel's specific points. E.g: The real problem with unit tests as I've discovered is that the type of changes that you tend to make as code evolves tend to break a constant percentage of your unit tests.... So the end result is that, as your project gets bigger and bigger, if you really have a lot of unit tests, the amount of investment you'll have to make in maintaining those unit tests, keeping them up-to-date and keeping them passing, starts to become disproportional to the amount of benefit that you get out of them.

akkartik 18y ago
In this sort of situation test coverage is actively harmful. You want to write lots of tests at a fine grain, and only test 'happy paths' above that level of abstraction. When I layer my tests I rarely end up with thrashing tests. Multiple tests often fail, but it's easy to figure out which test to focus on. And fixing the right test case fixes them all. Even if it makes sense to write a test for menu placement, it rarely makes sense to write more than one.
joe_the_user 18y ago
I've generally had some antipathy to Joel Spolsky, seeing him as more management oriented than programmer oriented. But I have to say that he seems both right and honest here. TDD is great if you use it appropriate along with good design - but "tests for everything" is just ideology. As he says, it's like the old OO schema, where you created these ridiculous inheritance trees and unnecessarily complex diagrams and such.
- dws 18y ago
  If Spolsky is going after "tests for everything", his target is Robert Martin, not Kent Beck. Beck freely admits that he doesn't test everything. Pulling Beck into the argument feels like a cheap debate trick.
- jacobscott 18y ago
  If you are: * selling software people pay money for * writing software people install -- that is, /not/ web stuff Then under what circumstances would it make sense to ship untested code? When is there a difference between code that has unit tests, versus that with only end-to-end tests?
  - silentbicycle 18y ago
    I don't think anybody is arguing that shipping completely untested code is a good idea.
nostrademons 18y ago
In this case, I think Kent's right. Joel's doing it wrong. You should never have 10% of your unit tests depend upon the location of a menu. You should have one unit test that depends upon the location of the menu, and everything else should be isolated by stubs, shunts, mocks, whatever. Otherwise, they aren't really unit tests, because they're testing more than a unit. I didn't realize how wrong I was doing it until I got to Google. (Then again, a lot of Google's best programmers are doing it wrong too.) Perhaps this is because doing the right thing - mocking out your dependencies and testing only one feature per test - is often harder in the short term than just running all your code when you test the topmost layer. But I think the point needs to be hammered home: most programmers who're doing "unit" testing are not doing unit testing, and their code would be much less brittle if they took the time to stub out lower layers.
- zaius 18y ago
  As someone who is relatively new to the world of testing, I think I finally understand the point of stubs/mocks/etc. Thanks!
- axod 18y ago
  I agree, in theory. However, when you're quickly iterating, redesigning, changing how things work, having to completely rework the unit tests as well is pretty much wasted effort. I think it depends on the project - is it a 'design up front, go away code once' project, or an iterative constantly changing thing.
  - swombat 18y ago
    'Fraid not, axod. It's when you're quickly iterating and changing lots of stuff that quality unit testing helps you the most. Let's look at 2 possible scenarios: 1) You don't bother testing much. You make lots of changes, iterate quickly. Before the changes, you have code that, let's say, you know works. After the changes, you know there's probably bugs, but you don't know where. 2) You test fairly thoroughly. You make lots of changes, iterate quickly. Before the changes you have code that you know works. After the changes, you know where most of the bugs are because your tests tell you where they are. It's the difference between walking with your eyes open or closed... it may be more work to process all that information about obstacles, but it sure is helpful if you want to avoid those obstacles.
firdaus 18y ago
I've always thought that that was a benefit of unit testing and high code coverage - knowing when a change has broken some other piece of code.
paul 18y ago
I think I agree with everything Joel said in that post. These crazy rules people come up with for "proper object oriented" programming remind me of extreme religious rituals. You have to observe all these stupid little rules or else your code will become impure you and will spend an eternity in code maintenance hell. I'll make a somewhat heretical claim even: Unit test are 20% useful engineering, and 80% fad. They are great for tricky code with well defined behavior (like a parser), but wasteful for most code.
- sayrer 18y ago
  I think it depends on whether your users can write programs for your program. I work on a web browser, and every test is worth something. To some degree, we don't care what the test is testing. We want to be sure we don't change it unintentionally, thus breaking the web.
- dnene 18y ago
  I have the deepest respect for you and your writings, but exactly how much maximum code coverage have you achieved and sustained for at least a month using automated unit tests ? Just wondering how long you've visited the region you're writing a travel review about. Unit tests are about confidence in the software. Automation is the mechanism for sustained confidence. Engineering (and / or fad as you put it) is a vehicle to get there.
  - axod 18y ago
    You can have 100% code coverage, and still not be testing anything. Unless you're testing thoroughly the right parts, they're a false sense of security.
    - dnene 18y ago
      I agree with that. Its just an (imperfect) proxy measure of the depth / extent of unit testing. Asking how much code coverage is likely to give a better reflection of the effort behind unit testing rather than just a - "have you done unit testing".
- 18y ago
andreyf 18y ago
I'd like to see Kent respond to Joel's specific points Psht, why do that when you can just ad hominem for 3 paragraphs?
anuraggoel 18y ago
I think what Martin and Kent are really objecting to is this (from the same podcast): "They've just gone off the deep end, and I don't think these people write very much code if they're coming up with these principles, to be honest, it doesn't even make sense." It probably wasn't meant that way, but it was taken as an unnecessary personal attack (despite Joel's frequent 'don't listen to me' implorations)
- plinkplonk 18y ago
  ".. and I don't think these people write very much code" and why is this objectionable? Joel Spolsky thinks some agile "gurus" don't write much code. Fwiw, I know at least a couple of these "guru" (note the quotes) folks who don't write production code and haven't done so in years and make their living telling other people how to write their code ! The correct way to counter this is to point at (open source) code they wrote (Bob Martin does this by pointing at the Fitnesse code base, so he got this part right imo), or if it is not possible, just calmly assert that Spolsky's thinking is wrong and leave it at that.
gaius 18y ago
In most cases people who think they want unit tests really wanted to stay awake in CS101 where they'd have learnt about referential transparency and composability.
rbanffy 18y ago
If your iterations break too many tests, then you are writing too specific tests for things that can't really (or shouldn't) be tested. And, BTW, I never found Joel Spolsky a particularly insightful person.

by lacker 18y ago

akkartik 18y ago

In this sort of situation test coverage is actively harmful. You want to write lots of tests at a fine grain, and only test 'happy paths' above that level of abstraction. When I layer my tests I rarely end up with thrashing tests. Multiple tests often fail, but it's easy to figure out which test to focus on. And fixing the right test case fixes them all. Even if it makes sense to write a test for menu placement, it rarely makes sense to write more than one.

joe_the_user 18y ago

I've generally had some antipathy to Joel Spolsky, seeing him as more management oriented than programmer oriented. But I have to say that he seems both right and honest here. TDD is great if you use it appropriate along with good design - but "tests for everything" is just ideology. As he says, it's like the old OO schema, where you created these ridiculous inheritance trees and unnecessarily complex diagrams and such.

dws 18y ago
If Spolsky is going after "tests for everything", his target is Robert Martin, not Kent Beck. Beck freely admits that he doesn't test everything. Pulling Beck into the argument feels like a cheap debate trick.
jacobscott 18y ago
If you are: * selling software people pay money for * writing software people install -- that is, /not/ web stuff Then under what circumstances would it make sense to ship untested code? When is there a difference between code that has unit tests, versus that with only end-to-end tests?
- silentbicycle 18y ago
  I don't think anybody is arguing that shipping completely untested code is a good idea.

nostrademons 18y ago

In this case, I think Kent's right. Joel's doing it wrong. You should never have 10% of your unit tests depend upon the location of a menu. You should have one unit test that depends upon the location of the menu, and everything else should be isolated by stubs, shunts, mocks, whatever. Otherwise, they aren't really unit tests, because they're testing more than a unit. I didn't realize how wrong I was doing it until I got to Google. (Then again, a lot of Google's best programmers are doing it wrong too.) Perhaps this is because doing the right thing - mocking out your dependencies and testing only one feature per test - is often harder in the short term than just running all your code when you test the topmost layer. But I think the point needs to be hammered home: most programmers who're doing "unit" testing are not doing unit testing, and their code would be much less brittle if they took the time to stub out lower layers.

zaius 18y ago
As someone who is relatively new to the world of testing, I think I finally understand the point of stubs/mocks/etc. Thanks!
axod 18y ago
I agree, in theory. However, when you're quickly iterating, redesigning, changing how things work, having to completely rework the unit tests as well is pretty much wasted effort. I think it depends on the project - is it a 'design up front, go away code once' project, or an iterative constantly changing thing.
- swombat 18y ago
  'Fraid not, axod. It's when you're quickly iterating and changing lots of stuff that quality unit testing helps you the most. Let's look at 2 possible scenarios: 1) You don't bother testing much. You make lots of changes, iterate quickly. Before the changes, you have code that, let's say, you know works. After the changes, you know there's probably bugs, but you don't know where. 2) You test fairly thoroughly. You make lots of changes, iterate quickly. Before the changes you have code that you know works. After the changes, you know where most of the bugs are because your tests tell you where they are. It's the difference between walking with your eyes open or closed... it may be more work to process all that information about obstacles, but it sure is helpful if you want to avoid those obstacles.

firdaus 18y ago

I've always thought that that was a benefit of unit testing and high code coverage - knowing when a change has broken some other piece of code.

paul 18y ago

I think I agree with everything Joel said in that post. These crazy rules people come up with for "proper object oriented" programming remind me of extreme religious rituals. You have to observe all these stupid little rules or else your code will become impure you and will spend an eternity in code maintenance hell. I'll make a somewhat heretical claim even: Unit test are 20% useful engineering, and 80% fad. They are great for tricky code with well defined behavior (like a parser), but wasteful for most code.

sayrer 18y ago
I think it depends on whether your users can write programs for your program. I work on a web browser, and every test is worth something. To some degree, we don't care what the test is testing. We want to be sure we don't change it unintentionally, thus breaking the web.
dnene 18y ago
I have the deepest respect for you and your writings, but exactly how much maximum code coverage have you achieved and sustained for at least a month using automated unit tests ? Just wondering how long you've visited the region you're writing a travel review about. Unit tests are about confidence in the software. Automation is the mechanism for sustained confidence. Engineering (and / or fad as you put it) is a vehicle to get there.
- axod 18y ago
  You can have 100% code coverage, and still not be testing anything. Unless you're testing thoroughly the right parts, they're a false sense of security.
  - dnene 18y ago
    I agree with that. Its just an (imperfect) proxy measure of the depth / extent of unit testing. Asking how much code coverage is likely to give a better reflection of the effort behind unit testing rather than just a - "have you done unit testing".
18y ago

andreyf 18y ago

I'd like to see Kent respond to Joel's specific points Psht, why do that when you can just ad hominem for 3 paragraphs?

anuraggoel 18y ago

I think what Martin and Kent are really objecting to is this (from the same podcast): "They've just gone off the deep end, and I don't think these people write very much code if they're coming up with these principles, to be honest, it doesn't even make sense." It probably wasn't meant that way, but it was taken as an unnecessary personal attack (despite Joel's frequent 'don't listen to me' implorations)

plinkplonk 18y ago
".. and I don't think these people write very much code" and why is this objectionable? Joel Spolsky thinks some agile "gurus" don't write much code. Fwiw, I know at least a couple of these "guru" (note the quotes) folks who don't write production code and haven't done so in years and make their living telling other people how to write their code ! The correct way to counter this is to point at (open source) code they wrote (Bob Martin does this by pointing at the Fitnesse code base, so he got this part right imo), or if it is not possible, just calmly assert that Spolsky's thinking is wrong and leave it at that.

gaius 18y ago

In most cases people who think they want unit tests really wanted to stay awake in CS101 where they'd have learnt about referential transparency and composability.

rbanffy 18y ago

If your iterations break too many tests, then you are writing too specific tests for things that can't really (or shouldn't) be tested. And, BTW, I never found Joel Spolsky a particularly insightful person.