4 ms·

"(semi-)automated testing for any complex code: essential" Where did you see this idea in Coders At Work? Most of them say testing is important . Most of the

"(semi-)automated testing for any complex code: essential" Where did you see this idea in Coders At Work? Most of them say testing is important . Most of them don't say "semi automated" testing is "essential". So where is your perceived "consensus" of programming legends coming from? or this "TDD (writing the tests before the code): good in theory but alien to most folks" for that matter? Sibel mentions 5 developers in his blog post - Zawinski, Knuth, Norvig, Bloch, Armstrong. I didn't see anyone saying "good in theory but alien to most folks". Only Armstrong says he regularly does (soemthing like) "test driven", but if you read his interview he also says he spends a lot of time doodling and getting his abstractions right before coding, hardly the "YAGNI" style of coding and theory of "emergent design" propounded by most agile consultants. Who in CaW do you think concludes either of the two statements you put forward as "general conclusions from programming legends". Or are you saying Robert Martin and co are programming legends? Bit hard to justify I'd think ;-) You say, "There still seems to be a general reaction here" .. "I'm too smart and/or busy to write tests" and the corollary "Anyone who writes tests, or advocates testing, isn't a good programmer"." The former " I am too busy to write (TDD style) tests" is a justifiable argument in certain circumstances (Zawinski says as much) and the latter "Anyone who advocates testing is a poor programmer" is something I don't see anyone here saying. Who said this? Links please? And if someone did, why do you think it is a "general reaction"? fwiw I see the evolving consensus here ) as "writing unit tests is valuable in certain circumstances (and not so in others) as long as you aren't fanatical about them" and "what is important is not the methodology but the end result". You seem to be putting forward your beliefs about the value/validity of TDD/testing etc as the "general conclusion, from various programming legends". I am curious as to how you came up with this apparent consensus. Don't get me wrong, you certainly have a right to your conclusions. I am just saying the CaW interviews and the posts here don't seem to converge to a conclusion along the lines of you say they did.

ZeroGravitas 17y ago
I got all this directly from the linked article: Zawkinski: created automated tests for a complicated corpus of quasi-standard email headers that he triggered manually. When he rewrote the mailer in Java he unit-tested because it was easier in a Java OO environment. He says there's specific areas where unit-testing would have sped them up, but that they were under so much time-pressure, with short-term ROI horizons that they didn't bother, they just went with the gamble of "getting it right first time". If they had more time then unit-testing was "certainly the way to go". He specifically states that they traded quality for a ship date. Fitzpatrick: Does "a lot of testing". Tests anything half-way "clever" because others will break it. Forces people who work for him to write tests. "At a certain point, people realize, “Holy crap, it does pay off,” especially maintenance costs later." Joshua Bloch: automated testing of transactional shared-memory with a "basher". When it failed at high concurrency he had to use unit tests to locate the bug, which wasn't in his code but the underlying Solaris assembly code. This was early 90s and he implies that finding a bug in such code today would probably result in the engineer being shouted at for inadequate testing, though perhaps not in concurrency areas as testing those is still an "art". Argues that writing an API before the code, which he does, isn't technically TDD though I'd say it's in the same ballpark. Knuth: is "a fan of after-the-fact torture tests". The blog author interprets the other comments as anti-TDD, though they seem unrelated to me. Joe Armstrong: Actually does TDD. Norvig: "I think test-driven design is great. I do that a lot more than I used to do." I don't understand how you could have come up with a different reading. The originally post seems to be half-heartedly questioning TDD (without much support from the quoted sections), but that is on the far end of a continuum that has automated tests, then unit tests, on the way towards it. Which I thought I made clear with my list where the near end of that spectrum (automating tests for complicated areas) is better supported than the far end (TDD according to the book). Many responses I see here seem to reject testing entirely. (edit: just to be clear when I say "automated" I mean they wrote a program to test their program. It doesn't necessarily mean that they run it on every change, or check-in or whatever. I just mean it runs and says: pass or fail without a human being having to examine the output, though that output may be helpful when it does fail.)
- plinkplonk 17y ago
  51/2 for 6? (EDIT: you first claimed that of the 6 developers from CaW 5-1/2 of 6 supported your claims. Knuth being the "1/2" You've since edited your post) 51/2 for 6? You must be kidding. You are cherry picking statements that support your viewpoint and ignoring those that go in an opposite direction. these were what you said "programming legends" had a consensus on "• (semi-)automated testing for any complex code: essential • unit tests for any long lived code: great • TDD (writing the tests before the code): good in theory but alien to most folks" Let's see now. "after the fact torture tests" (Knuth) is hardly support for "TDD' or "semi automated testing is essential". SO that's one down (or 1/2 down :-P) So that yo are down to 5 of 6 right there. BTW Knuth also said (http://www.informit.com/articles/article.aspx?p=1193856 http://www.informit.com/articles/article.aspx?p=1193856) "As to your real question, the idea of immediate compilation and "unit tests" appeals to me only rarely, when I’m feeling my way in a totally unknown environment and need feedback about what works and what doesn’t. Otherwise, lots of time is wasted on activities that I simply never need to perform or even think about. Nothing needs to be "mocked up." Zawinski, Knuth, Norvig etc don't say anything like "semi automated testing is essential" (one of your conclusions) or "TDD is good in theory". It is one technique among many they sometimes use,certainly. Norvig (for example) specifically says "Seibel: What about the idea of using tests to drive design? Norvig: I see tests more as a way of correcting errors rather than as a way of design. This extreme approach of saying, “Well, the first thing you do is write a test that says I get the right answer at the end,” and then you run it and see that it fails, and then you say, “What do I need next?”—that doesn’t seem like the right way to design something to me" So that is 4 of 6. Zawinski says "Seibel: In retrospect, do you think you suffered at all because of that? Would development have been easier or faster if you guys had been more disciplined about testing? Zawinski: I don’t think so. I think it would have just slowed us down. There’s a lot to be said for just getting it right the first time. In the early days we were so focused on speed. We had to ship the thing even if it wasn’t perfect. We can ship it later and it would be higher quality but someone else might have eaten our lunch by then." In other words in retrospect, If speed is a concern, "being disciplined about testing" is NOT important which is hardly "consensus" about the centrality of testing. So that is 3 of 6. About Bloch, you say "Argues that writing an API before the code, which he does, isn't technically TDD " and you concluded "it is in the same ballpark" ?! You conclude something 180 degrees opposite to what the interviewer understood? Josh said " I do disagree with Martin (Fowler) on one point: I don’t think tests are even remotely an acceptable substitute for documentation. Once you’re trying to write something that other people can code to, you need precise specs, and the tests should test that the code conforms to those specs. " So write precise specs, write code and then add tests that the code confirms to written specs is "in the same ballpark" as TDD? This is how sw is being developed for ages. By this logic, any thinking about code before coding is "in the same ballpark" as TDD. A strange argument. Besides Josh Bloch wrote a test when everything else failed. And when specifically asked if an automated test would have been useful, he says "I think a good automated unit test of the mutex facility could have saved me from this particular agony, but keep in mind that this was in the early ‘90s. It never even occurred to me to blame the engineer involved for not writing good enough unit tests. Even today, writing unit tests for concurrency utilities is an art form." More nuanced than your claim that Josh was all for automated testing don't you think? So 2 of 6. So how did you come to your conclusions "TDD is great in theory but too strange for most folks" as something they concluded? You are concluding a few things and cherry picking statements to support this. Every developer in the world would love an extensive (semi) automated test suite if they didn't have a cost to write and maintain. You can't take a statement from someone that "tests are great" and ignore "tests are for correcting errors" or "when speed is critical, tests are not important". My point was that you say "programming legends" conclude various things. As I see it they don't conclude anything beyond "tests are good in certain circumstances for specific purposes". They make some generic statements in support of "testing" but are very cautious of "test driven development" or "you must test every bit of code" por even "you must have an automated test suite". There is no consensus for the stronger statements you put forward. Nobody says "TDD is good in theory but most people don't get it". Almost everyone says it is a bad idea. The second part of your original post , that the consensus evolving here on HN is "anyone who advocates testing is abda programmer" is still unsupported. Please, show me links. EDIT: I see you've edited your post and are now proposing a weaker claim "testing" vs "tdd". Sure if you are saying "programs that test other programs" are a useful technique to have in your arsenal, sure thing. Programmers have written tests drivers (in other words programs that tests programs) from the beginning of programming. This is hardly an argument for "there is a consensus among programming legends different from teh consensus here on HN" .

ZeroGravitas 17y ago

I got all this directly from the linked article: Zawkinski: created automated tests for a complicated corpus of quasi-standard email headers that he triggered manually. When he rewrote the mailer in Java he unit-tested because it was easier in a Java OO environment. He says there's specific areas where unit-testing would have sped them up, but that they were under so much time-pressure, with short-term ROI horizons that they didn't bother, they just went with the gamble of "getting it right first time". If they had more time then unit-testing was "certainly the way to go". He specifically states that they traded quality for a ship date. Fitzpatrick: Does "a lot of testing". Tests anything half-way "clever" because others will break it. Forces people who work for him to write tests. "At a certain point, people realize, “Holy crap, it does pay off,” especially maintenance costs later." Joshua Bloch: automated testing of transactional shared-memory with a "basher". When it failed at high concurrency he had to use unit tests to locate the bug, which wasn't in his code but the underlying Solaris assembly code. This was early 90s and he implies that finding a bug in such code today would probably result in the engineer being shouted at for inadequate testing, though perhaps not in concurrency areas as testing those is still an "art". Argues that writing an API before the code, which he does, isn't technically TDD though I'd say it's in the same ballpark. Knuth: is "a fan of after-the-fact torture tests". The blog author interprets the other comments as anti-TDD, though they seem unrelated to me. Joe Armstrong: Actually does TDD. Norvig: "I think test-driven design is great. I do that a lot more than I used to do." I don't understand how you could have come up with a different reading. The originally post seems to be half-heartedly questioning TDD (without much support from the quoted sections), but that is on the far end of a continuum that has automated tests, then unit tests, on the way towards it. Which I thought I made clear with my list where the near end of that spectrum (automating tests for complicated areas) is better supported than the far end (TDD according to the book). Many responses I see here seem to reject testing entirely. (edit: just to be clear when I say "automated" I mean they wrote a program to test their program. It doesn't necessarily mean that they run it on every change, or check-in or whatever. I just mean it runs and says: pass or fail without a human being having to examine the output, though that output may be helpful when it does fail.)

plinkplonk 17y ago
51/2 for 6? (EDIT: you first claimed that of the 6 developers from CaW 5-1/2 of 6 supported your claims. Knuth being the "1/2" You've since edited your post) 51/2 for 6? You must be kidding. You are cherry picking statements that support your viewpoint and ignoring those that go in an opposite direction. these were what you said "programming legends" had a consensus on "• (semi-)automated testing for any complex code: essential • unit tests for any long lived code: great • TDD (writing the tests before the code): good in theory but alien to most folks" Let's see now. "after the fact torture tests" (Knuth) is hardly support for "TDD' or "semi automated testing is essential". SO that's one down (or 1/2 down :-P) So that yo are down to 5 of 6 right there. BTW Knuth also said (http://www.informit.com/articles/article.aspx?p=1193856 http://www.informit.com/articles/article.aspx?p=1193856) "As to your real question, the idea of immediate compilation and "unit tests" appeals to me only rarely, when I’m feeling my way in a totally unknown environment and need feedback about what works and what doesn’t. Otherwise, lots of time is wasted on activities that I simply never need to perform or even think about. Nothing needs to be "mocked up." Zawinski, Knuth, Norvig etc don't say anything like "semi automated testing is essential" (one of your conclusions) or "TDD is good in theory". It is one technique among many they sometimes use,certainly. Norvig (for example) specifically says "Seibel: What about the idea of using tests to drive design? Norvig: I see tests more as a way of correcting errors rather than as a way of design. This extreme approach of saying, “Well, the first thing you do is write a test that says I get the right answer at the end,” and then you run it and see that it fails, and then you say, “What do I need next?”—that doesn’t seem like the right way to design something to me" So that is 4 of 6. Zawinski says "Seibel: In retrospect, do you think you suffered at all because of that? Would development have been easier or faster if you guys had been more disciplined about testing? Zawinski: I don’t think so. I think it would have just slowed us down. There’s a lot to be said for just getting it right the first time. In the early days we were so focused on speed. We had to ship the thing even if it wasn’t perfect. We can ship it later and it would be higher quality but someone else might have eaten our lunch by then." In other words in retrospect, If speed is a concern, "being disciplined about testing" is NOT important which is hardly "consensus" about the centrality of testing. So that is 3 of 6. About Bloch, you say "Argues that writing an API before the code, which he does, isn't technically TDD " and you concluded "it is in the same ballpark" ?! You conclude something 180 degrees opposite to what the interviewer understood? Josh said " I do disagree with Martin (Fowler) on one point: I don’t think tests are even remotely an acceptable substitute for documentation. Once you’re trying to write something that other people can code to, you need precise specs, and the tests should test that the code conforms to those specs. " So write precise specs, write code and then add tests that the code confirms to written specs is "in the same ballpark" as TDD? This is how sw is being developed for ages. By this logic, any thinking about code before coding is "in the same ballpark" as TDD. A strange argument. Besides Josh Bloch wrote a test when everything else failed. And when specifically asked if an automated test would have been useful, he says "I think a good automated unit test of the mutex facility could have saved me from this particular agony, but keep in mind that this was in the early ‘90s. It never even occurred to me to blame the engineer involved for not writing good enough unit tests. Even today, writing unit tests for concurrency utilities is an art form." More nuanced than your claim that Josh was all for automated testing don't you think? So 2 of 6. So how did you come to your conclusions "TDD is great in theory but too strange for most folks" as something they concluded? You are concluding a few things and cherry picking statements to support this. Every developer in the world would love an extensive (semi) automated test suite if they didn't have a cost to write and maintain. You can't take a statement from someone that "tests are great" and ignore "tests are for correcting errors" or "when speed is critical, tests are not important". My point was that you say "programming legends" conclude various things. As I see it they don't conclude anything beyond "tests are good in certain circumstances for specific purposes". They make some generic statements in support of "testing" but are very cautious of "test driven development" or "you must test every bit of code" por even "you must have an automated test suite". There is no consensus for the stronger statements you put forward. Nobody says "TDD is good in theory but most people don't get it". Almost everyone says it is a bad idea. The second part of your original post , that the consensus evolving here on HN is "anyone who advocates testing is abda programmer" is still unsupported. Please, show me links. EDIT: I see you've edited your post and are now proposing a weaker claim "testing" vs "tdd". Sure if you are saying "programs that test other programs" are a useful technique to have in your arsenal, sure thing. Programmers have written tests drivers (in other words programs that tests programs) from the beginning of programming. This is hardly an argument for "there is a consensus among programming legends different from teh consensus here on HN" .