3 ms·

Does it seem weird to anybody else that about half the list of prominent projects using TDD he listed are testing tools? That's like saying type systems are go

Does it seem weird to anybody else that about half the list of prominent projects using TDD he listed are testing tools? That's like saying type systems are good because they help you write sound proofs about type systems. I know they can be really useful, but it's an incredibly circular argument.

idlewords 17y ago
It seems really weird. I've been trying to get an answer to the question "what popular apps have actually been built using TDD?" for a while, without success. All of the entries on his list are either libraries, programming languages, or testing tools and frameworks. Can anyone name actual apps that TDD has blessed us with?
- silentbicycle 17y ago
  More importantly, that were done with TDD as opposed to unit testing, testing with a repl, etc.? A lot of claims about TDD's benefits seem to take credit for testing in general. (Automated testing is a clear win, IMHO.) Insisting on writing tests upfront does mean that you test consistently, but then it's hardly the primary cause. I'm sure it's a highly useful tool in some cases, but the people pushing it uncritically make it harder to tell when it's actually a good fit. In my experience, it's very useful for safely making changes in a legacy code, but I suspect it can encourage over-engineering surface details or getting caught in local maxima when used for initial design. (I usually just design with a REPL or notepad/whiteboard.)
- chadaustin 17y ago
  Much of IMVU's client and website (tens of millions of customers) were written with TDD. I've long been a proponent of TDD once you pass the prototyping stage and you need to write a production-ready component with [most of] the edge cases figured out. The "red green refactor" loop is more like: 1) What's the next problem this software needs to solve? 2) Write a failing test that demonstrates the missing need. 3) Implement said behavior. 4) Refactor until you don't see anything else to refactor. I think step #1 is one of the most important and yet I don't see much discussion of it. Pick the most "key" tests to write first, whether that be your happy path or the functionality that has the most risk. On refactoring... I work with people who TDD all of their work and other who write unit tests afterwards. In my experience, those who test-drive are more comfortable refactoring otherwise. When I was test-driving imvu.task ( http://thespeedbump.livejournal.com/63798.html http://thespeedbump.livejournal.com/63798.html ), I spent a great deal of time just staring at the code after it was written... it's very continuation-heavy code... and suddenly I saw a refactoring that collapsed it by 1/3. We've hardly touched the code since. It solves the right problem with the right amount of code and I think it's only had a couple very obscure race conditions. In summary, TDD is one of my favorite tools for focusing my efforts so we build what we need, but only what we need.
- joshuab 17y ago
  http://www.thoughtworks.com/our-clients/ http://www.thoughtworks.com/our-clients/ http://pivotallabs.com/clients http://pivotallabs.com/clients http://www.hashrocket.com/projects http://www.hashrocket.com/projects http://www.thoughtbot.com/work http://www.thoughtbot.com/work
  - silentbicycle 17y ago
    Could you flesh that out a bit, so we know you're not just spamming?
  - joe_the_user 17y ago
    It's hard not to see that as a list of clients of consulting firms rather than a list of serious open source software built with TDD. I know the Rubinious project was built aiming to use TDD and it doesn't seem to have gone anywhere YET. As far as I can tell, a compiler is one beast that needs a strong abstract design first rather than a set of tests first - but I'm prepared to be surprised. Pivotallabs lists Twitter first but I would be rather surprised if someone could verify that a substantial part of Twitter's app has been built using TDD (go ahead, surprise me).
    - ZeroGravitas 17y ago
      One of the comments on the original post claims open source apps don't count because they "are not subject to many of the challenges the average developer faces in a corporate environment" so it appears TDD fans can't win either way. But since you're looking for open source success stories there was some very positive noises about TDD from a Twisted dev recently: "I would strongly recommend at least practicing some TDD, too. Learning how to write tests was useful for me, but practicing TDD really expanded my design skills in a bunch of surprising directions that were almost unrelated to testing. I originally felt as you do, that I wasn't disciplined enough to do it; now I feel like I'm too undisciplined not to do it ;-)." From: http://glyph.twistedmatrix.com/2009/09/diesel-case-study-in-that-thing-i-just_24.html http://glyph.twistedmatrix.com/2009/09/diesel-case-study-in-... Though he specifically states that Twisted didn't start out TDD, which he regrets, but maybe that disqualifies it. Regarding Rubinius (and JRuby that was listed in the original post) I'm not sure trying to create a spec for an existing language implementation and then try to meet it would count as "true" TDD if people care about such differences. But maybe they do TDD for other bits lower in the stack, I'm not sure.
- gwern 17y ago
  > All of the entries on his list are either libraries, programming languages, or testing tools and frameworks. I've never used or looked much into TDD, but it seems to me that that's where TDD would be best: areas where input is easily done programmatically & where output can be easily checked for whatever property. How does one test a big GUI app - write some sort of binding to xlib to move the mouse around and go through routines? I'm sure it's possible, but I suspect it's pretty difficult to come up with useful but reliable tests.
  - silentbicycle 17y ago
    That's true of automated testing in general, really. It helps to design the GUI as a thin layer around a bunch of code that can be tested, scripted, etc. in isolation, rather than tangling the major functionality with the GUI code itself.
algorias 17y ago
Sounds equivalent to getting rich selling the secret of success.
- Psyonic 17y ago
  Was just about to right a similar comment before I saw yours. Also, productivity "gurus" who use their god-like productive capacity to... talk about productivity. Good times.
jlouis 17y ago
The circularity-argument for type systems is actually correct, but that is an aside. Type systems on programming languages are proof systems, so they can be used as tools for building better type systems. There is a big lack of TDD in highly succesful Open Source endeavors. Part of this is perhaps because a failed project never get attention. But also note that defects in Open Source projects are good. They instill new programmers to come to the project and work on it because there are some easy low-hanging fruit. Furthermore, as exemplified by Keith Packard recently (see http://lwn.net/Articles/354408/ http://lwn.net/Articles/354408/ -- which unfortunately is behind the Iron Curtain for 24 hours more). Keith mentions that he sometimes deliberately breaks some old code by introducing obvious bugs into it. Then, when nobody have complained for a couple of years, it can be removed on the grounds that it was not interesting. With rigorous testing this is unachievable.
- silentbicycle 17y ago
  Interesting point - The social factors of Open Source have numerous subtle consequences. It reminds me of how someone noted that it can be easier to get help with (some) Open Source software by going on IRC and spouting misinformation (and getting corrected by a mob), rather than asking for help and getting treated like a stupid newbie. My note about type systems doesn't have anything to do with whether it's correct or not - just, if the first example people give for the usefulness of type systems is more type systems, it makes me wonder how much consideration the speaker gives practicality. (Then again, I'm not a grad student...or an agile methodology coach.)

idlewords 17y ago

It seems really weird. I've been trying to get an answer to the question "what popular apps have actually been built using TDD?" for a while, without success. All of the entries on his list are either libraries, programming languages, or testing tools and frameworks. Can anyone name actual apps that TDD has blessed us with?

silentbicycle 17y ago
More importantly, that were done with TDD as opposed to unit testing, testing with a repl, etc.? A lot of claims about TDD's benefits seem to take credit for testing in general. (Automated testing is a clear win, IMHO.) Insisting on writing tests upfront does mean that you test consistently, but then it's hardly the primary cause. I'm sure it's a highly useful tool in some cases, but the people pushing it uncritically make it harder to tell when it's actually a good fit. In my experience, it's very useful for safely making changes in a legacy code, but I suspect it can encourage over-engineering surface details or getting caught in local maxima when used for initial design. (I usually just design with a REPL or notepad/whiteboard.)
chadaustin 17y ago
Much of IMVU's client and website (tens of millions of customers) were written with TDD. I've long been a proponent of TDD once you pass the prototyping stage and you need to write a production-ready component with [most of] the edge cases figured out. The "red green refactor" loop is more like: 1) What's the next problem this software needs to solve? 2) Write a failing test that demonstrates the missing need. 3) Implement said behavior. 4) Refactor until you don't see anything else to refactor. I think step #1 is one of the most important and yet I don't see much discussion of it. Pick the most "key" tests to write first, whether that be your happy path or the functionality that has the most risk. On refactoring... I work with people who TDD all of their work and other who write unit tests afterwards. In my experience, those who test-drive are more comfortable refactoring otherwise. When I was test-driving imvu.task ( http://thespeedbump.livejournal.com/63798.html http://thespeedbump.livejournal.com/63798.html ), I spent a great deal of time just staring at the code after it was written... it's very continuation-heavy code... and suddenly I saw a refactoring that collapsed it by 1/3. We've hardly touched the code since. It solves the right problem with the right amount of code and I think it's only had a couple very obscure race conditions. In summary, TDD is one of my favorite tools for focusing my efforts so we build what we need, but only what we need.
joshuab 17y ago
http://www.thoughtworks.com/our-clients/ http://www.thoughtworks.com/our-clients/ http://pivotallabs.com/clients http://pivotallabs.com/clients http://www.hashrocket.com/projects http://www.hashrocket.com/projects http://www.thoughtbot.com/work http://www.thoughtbot.com/work
- silentbicycle 17y ago
  Could you flesh that out a bit, so we know you're not just spamming?
- joe_the_user 17y ago
  It's hard not to see that as a list of clients of consulting firms rather than a list of serious open source software built with TDD. I know the Rubinious project was built aiming to use TDD and it doesn't seem to have gone anywhere YET. As far as I can tell, a compiler is one beast that needs a strong abstract design first rather than a set of tests first - but I'm prepared to be surprised. Pivotallabs lists Twitter first but I would be rather surprised if someone could verify that a substantial part of Twitter's app has been built using TDD (go ahead, surprise me).
  - ZeroGravitas 17y ago
    One of the comments on the original post claims open source apps don't count because they "are not subject to many of the challenges the average developer faces in a corporate environment" so it appears TDD fans can't win either way. But since you're looking for open source success stories there was some very positive noises about TDD from a Twisted dev recently: "I would strongly recommend at least practicing some TDD, too. Learning how to write tests was useful for me, but practicing TDD really expanded my design skills in a bunch of surprising directions that were almost unrelated to testing. I originally felt as you do, that I wasn't disciplined enough to do it; now I feel like I'm too undisciplined not to do it ;-)." From: http://glyph.twistedmatrix.com/2009/09/diesel-case-study-in-that-thing-i-just_24.html http://glyph.twistedmatrix.com/2009/09/diesel-case-study-in-... Though he specifically states that Twisted didn't start out TDD, which he regrets, but maybe that disqualifies it. Regarding Rubinius (and JRuby that was listed in the original post) I'm not sure trying to create a spec for an existing language implementation and then try to meet it would count as "true" TDD if people care about such differences. But maybe they do TDD for other bits lower in the stack, I'm not sure.
gwern 17y ago
> All of the entries on his list are either libraries, programming languages, or testing tools and frameworks. I've never used or looked much into TDD, but it seems to me that that's where TDD would be best: areas where input is easily done programmatically & where output can be easily checked for whatever property. How does one test a big GUI app - write some sort of binding to xlib to move the mouse around and go through routines? I'm sure it's possible, but I suspect it's pretty difficult to come up with useful but reliable tests.
- silentbicycle 17y ago
  That's true of automated testing in general, really. It helps to design the GUI as a thin layer around a bunch of code that can be tested, scripted, etc. in isolation, rather than tangling the major functionality with the GUI code itself.

algorias 17y ago

Sounds equivalent to getting rich selling the secret of success.

Psyonic 17y ago
Was just about to right a similar comment before I saw yours. Also, productivity "gurus" who use their god-like productive capacity to... talk about productivity. Good times.

jlouis 17y ago

The circularity-argument for type systems is actually correct, but that is an aside. Type systems on programming languages are proof systems, so they can be used as tools for building better type systems. There is a big lack of TDD in highly succesful Open Source endeavors. Part of this is perhaps because a failed project never get attention. But also note that defects in Open Source projects are good. They instill new programmers to come to the project and work on it because there are some easy low-hanging fruit. Furthermore, as exemplified by Keith Packard recently (see http://lwn.net/Articles/354408/ http://lwn.net/Articles/354408/ -- which unfortunately is behind the Iron Curtain for 24 hours more). Keith mentions that he sometimes deliberately breaks some old code by introducing obvious bugs into it. Then, when nobody have complained for a couple of years, it can be removed on the grounds that it was not interesting. With rigorous testing this is unachievable.

silentbicycle 17y ago
Interesting point - The social factors of Open Source have numerous subtle consequences. It reminds me of how someone noted that it can be easier to get help with (some) Open Source software by going on IRC and spouting misinformation (and getting corrected by a mob), rather than asking for help and getting treated like a stupid newbie. My note about type systems doesn't have anything to do with whether it's correct or not - just, if the first example people give for the usefulness of type systems is more type systems, it makes me wonder how much consideration the speaker gives practicality. (Then again, I'm not a grad student...or an agile methodology coach.)