4 ms·

We don't write tests. There just isn't time for luxuries.

24 points by byosko 19y ago

imsteve 19y ago
Sometimes it just does slow you down overall. Remind me again, how many web startups have failed because of a bug? How many? Hm.
- bct 19y ago
  Are you implying that the number is very small or very large?
  - imsteve 19y ago
    SMALL. Are you implying that it's not? Or maybe you're with the aaron crowd in believing that it has to be absolutely perfect or it's useless. Heck, even that only could really apply to the design and architecture of an app. Perfection is rarely necessary. edit: And what's with so many downvotes? I thought yc was more mature.
    - sbraford 19y ago
      How many teams fail to execute and lose precious time to market debugging things that they could've caught earlier? I am not a 100% TDD fanboy either, but it's not something to dismiss wholeheartedly.
      - imsteve 19y ago
        That's why I said sometimes. If you read the article, it claims that this is never the case. I presented my disagreement with this.
        sbraford 19y ago
        Sorry, I think you're right. There are cases when it can slow you down overall. Adding tests up front, from my experience, adds about 40% overhead to a project. (time spent actually writing the tests) You may get some of this time back later, as the author claims, as the automated test suite is run. I've also been on projects where the code & entire test suite was re-written & scrapped after 6 months of initial development, pretty much making the whole test up front thing of little value.
        imsteve 19y ago
        To be fair, it seems that everyone other than I took this in the context of more traditional software project, not an ill-specified one man startup project. I was referring to the later.
        ardit33 19y ago
        ok, so now I get it. TTD is better suited for those large companies where people don't know what the f are doing, so they need to test everything (just making sure), and when speed doesn't matter, as big companies everything goes slow. The same enviroment and thinking that brought us java, and made it a clusterfuck of "abstractions", and "design principles". I'd rather just code ad design at will, and be able to refactor, change, or throw away code without being held hostage of these tests which will break at some point, not because your code is not right, but because they were design to test something according to a thinking, and when that something changes, all those tests are obsolete.
        jamesgolick 19y ago
        I think the problem that you might be facing is more related to the quality of your tests than testing itself. If your tests are written correctly, you should never have to throw them away when you refactor. In fact, that's one of the major benefits of testing: a refactoring safety net!
        pius 19y ago
    - bct 19y ago
      > Are you implying that it's not? No, it just wasn't clear from your original comment.
brintoul 19y ago
I dislike writing more code for my tests than the "actual" code. I do not eat the "test driven development" drivel hook-line-and-sinker. Sorry, call me an unenlightened developer.
- davidw 19y ago
  Yes, TDD implies that there is a more or less exact specification. Otherwise, if you're just experimenting, you would have to write the test and your code, and that's going to make you less inclined to throw it away and test out something else (see "Planning is highly overrated"). My strategy has generally been to throw some tests that cover most things in a general way, and when I find bugs, add lots of detailed tests that make sure those bugs will never come back again. That makes it feel less pedantic.
  - pius 19y ago
    "Yes, TDD implies that there is a more or less exact specification." That's actually the opposite of the truth. The second D is for Design (or Development). The idea is that you organically develop the spec and the code in tandem.
    - davidw 19y ago
      How can you test something if you don't even know how or if it works? You need to hack on it and see if you can get things going before you nail it down, no?
      - pius 19y ago
        Let's say you know you've gotta design a bowling pin from scratch. TDD and "standard" coding practices both require the same initial thought. You'd say, ok, what's this thing actually have to do? Maybe you start with, "it has to fall when hit by a bowling ball hard enough." If you weren't using TDD, you might go off and create a BowlingPin class, a BowlingBall class (if it's not extant), and hack together a method in some arbitrary order and method. That's cool, but the TDD approach would have you write a mini "test." In pseudoRuby: [http://pastie.caboo.se/103470 http://pastie.caboo.se/103470] This, of course, will fail because there is no BowlingPin class. So you write the class skeleton and re-run the tests. Fails again. Why? There is no got_hit method on BowlingPin? Shit. Now you've got to go and write the simplest got_hit method that'll work. So you write something like: [http://pastie.caboo.se/103471 http://pastie.caboo.se/103471] Rerun the tests. Dammit! There's no "fell?" method -- gotta implement it! And so on and so on. There are a few things to note here. First, it's pretty critical to use a decent test harness. Running and writing tests has to be crazy fast otherwise this approach won't work well. Next, in running through the TDD cycle you're actually (1) specifying the interfaces and behavior of your code and (2) implementing as narrowly as you can. The actual "tests" you get out of it are almost secondary! If implemented properly, it's really a design methodology. That's why the new trend is to change the language from Test Driven to Behavior Driven. The use of the word "test" seems to (understandably) evoke the same response you had of needing an iron-clad spec to work from initially. It's just not the case.
breck 19y ago
I'm a chronic manual tester. But I want to start getting into automated tests. What are the most important things to test? What are some good intros to pick up the basics on writing automated tests?
- pius 19y ago
  I personally think that test/behavior driven development is the way to go, so my advice will be geared in that direction. With that in mind, the "reference" text on it is probably "Test Driven Development: By Example" by Kent Beck. I've only skimmed it, myself, but it seems solid and Kent Beck is a smart guy. The cool thing about testing (and TDD in particular) is that even introducing a bare minimum of that sort of discipline into your work consistent will quickly pay pretty big dividends. A quick way to get started is to read this Wikipedia article: http://en.wikipedia.org/wiki/Test-driven_development http://en.wikipedia.org/wiki/Test-driven_development. Focus on understanding the TDD cycle. Mocks and stubs are useful to know, but you can safely consider them "advanced" and skip them if they confuse you initially. Then, pick a test harness for your platform, say RSpec (http://rspec.rubyforge.org http://rspec.rubyforge.org), if you're a Ruby person. Read the documentation and search for blogs and tutorial articles on the framework you've chosen. Then just start designing a simple module using it! As you practice, you'll find it useful to read up on Mocks and Stubs and dig deeper into the philosophy and best practices. Dave Astels has some great writings out there.
  - breck 19y ago
    All great advice. Thanks!
  - plinkplonk 19y ago
    I personally found Kent Beck's book very tedious and boring. In it , simple classes are bludgeoned into submission through the over use of tests. From Ganssle's article at http://www.ganssle.com/articles/deconstructing%20xp.htm http://www.ganssle.com/articles/deconstructing%20xp.htm "XP's motivating guru, Kent Beck, also has a new book out. Test Driven Development (Addison Wesley, 2003, ISBN 0-321-14653-0) focuses on XP's testing practice. I bought the book because I'm fascinated with testing. It's usually done as an afterthought, and rarely hits the hard conditions. Unit tests are notoriously poor. The programmer runs his new function through the simplest of situations, totally ignoring timing issues or boundary conditions. So of course the function passes, only to inflict the team with agony when integrated into the entire system. XP's idea of writing the tests in parallel with the code is quite brilliant; only then do we see all possible conditions that can occur. Beck argues for building tests first and deriving the code from these tests. It's sort of like building a comprehensive final exam and then designing a class from that. An intriguing idea. But X what if the test is wrong? Test driven development (TDD) then guarantees the code will be wrong as well. Worse, TDD calls for building even the smallest project by implementing the minimal functionality needed to do anything. Need a function to compute the Fibonacci series? Create the simplest possible test first X in this case, check to see that fibonnaci(0) is 0. Then write a function that passes that test. Try fibonnaci(1); that breaks the test, so recode both the test and the function. Iterate till correct. The book shows how to build a factorial program, which results in 91 lines of code and 89 of test... after 125 compilations! Klaxons sound, common sense alarms trip. Agile proponents love the interactive and fast action of this sort of method. But programming isn't about playing Doom and Quake. If you're looking for an adrenaline rush try bungee jumping. 125 compilations for a trivial bit of code is not fast. Dynamic? You bet. But not fast. " Test Driven Development is not quite the same as having unit tests or automated tests, which I believe the poster asked for. The jury is still very much out on how useful TDD (and XP for that matter) really is. Many talented programmers believe XP/TDD is essentially hype. . This is not to say Pius is wrong, just to add a note of caution, be skeptical of the XP hype machine. "Try it and decide for yourself" is probably the best advice possible
niels 19y ago
I've recently begun using automated tests of basically all my code. Both frontend and backend, when I'm doing web stuff. I usually write the code to a minimum first, and then implement tests. It's a huge relief to use these tests when I refactor and refine code. I definitely think automated testing is the way to go in most cases.
bsaunder 19y ago
"the break even point for an automated tester is when they've run their tests twenty times"... I find that (at least in early development), when I'm in the zone I refactor several times a day (differnent parts of the system of course). If my break even point is twenty times, I'd never get there. Every refactoring would virtually obliterate all of the previously automated tests.
geebee 19y ago
"Everybody Tests" is a good point. Some devs throw away their tests, and other devs keep the tests around and run them periodically, but devs who write code iteratively (which seems like pretty much all of them) write something that could be easily turned into a unit test. If you don't like TDD (and I have to admit I'm not a fan), you can still produce tests while you write code (you probably already are). Whenever you write a bit of code, you probably do something to check whether it is producing the expected outcome - even if it is as simple as writing a little script to generate some output that you visually inspect as it flashes by on the screen. If it looks good, you keep developing. If it fails, you figure out why. Just store your these mini-testing scripts in a unit test, and verify the expectation/output in a short statement. If you use a framework, you'll have some methods for this available to you - if you don't, no biggie, just write something that honks loud and gives you a message diagnosing the failure. It's not going to satisfy the TDD crowd, and honestly your coverage won't be as good as if you had done TDD, but there you go - you just got yourself some unit tests, and you didn't have to disrupt the mental flow that you prefer to TDD.
euccastro 19y ago
I wrote some opinions on this in other thread: http://news.ycombinator.com/item?id=46526 http://news.ycombinator.com/item?id=46526 Unless your software will kill people if it fails or something, my advice is: observe what proportion of your development time you spend debugging. If it's one half or more, unit testing would probably benefit you. If it's significantly less than 50%, then you're probably better off riding bareback, because writing and updating those tests will take at least half of your time. The problem I see with the "you'll test anyway, you may just as well automate it" argument is that unit tests are no substitute for testing with real users.
edw519 19y ago
I have always believed that there is an inverse correlation between the amount of precise attention devoted in design/development and the amount of time spent in any debugger. Nothing here changes that thinking.

24 points by byosko 19y ago

Sometimes it just does slow you down overall. Remind me again, how many web startups have failed because of a bug? How many? Hm.

bct 19y ago
Are you implying that the number is very small or very large?
- imsteve 19y ago
  SMALL. Are you implying that it's not? Or maybe you're with the aaron crowd in believing that it has to be absolutely perfect or it's useless. Heck, even that only could really apply to the design and architecture of an app. Perfection is rarely necessary. edit: And what's with so many downvotes? I thought yc was more mature.
  - sbraford 19y ago
    How many teams fail to execute and lose precious time to market debugging things that they could've caught earlier? I am not a 100% TDD fanboy either, but it's not something to dismiss wholeheartedly.
    - imsteve 19y ago
      That's why I said sometimes. If you read the article, it claims that this is never the case. I presented my disagreement with this.
      - sbraford 19y ago
        Sorry, I think you're right. There are cases when it can slow you down overall. Adding tests up front, from my experience, adds about 40% overhead to a project. (time spent actually writing the tests) You may get some of this time back later, as the author claims, as the automated test suite is run. I've also been on projects where the code & entire test suite was re-written & scrapped after 6 months of initial development, pretty much making the whole test up front thing of little value.
        imsteve 19y ago
        To be fair, it seems that everyone other than I took this in the context of more traditional software project, not an ill-specified one man startup project. I was referring to the later.
        ardit33 19y ago
        ok, so now I get it. TTD is better suited for those large companies where people don't know what the f are doing, so they need to test everything (just making sure), and when speed doesn't matter, as big companies everything goes slow. The same enviroment and thinking that brought us java, and made it a clusterfuck of "abstractions", and "design principles". I'd rather just code ad design at will, and be able to refactor, change, or throw away code without being held hostage of these tests which will break at some point, not because your code is not right, but because they were design to test something according to a thinking, and when that something changes, all those tests are obsolete.
        jamesgolick 19y ago
        I think the problem that you might be facing is more related to the quality of your tests than testing itself. If your tests are written correctly, you should never have to throw them away when you refactor. In fact, that's one of the major benefits of testing: a refactoring safety net!
        pius 19y ago
  - bct 19y ago
    > Are you implying that it's not? No, it just wasn't clear from your original comment.

brintoul 19y ago

I dislike writing more code for my tests than the "actual" code. I do not eat the "test driven development" drivel hook-line-and-sinker. Sorry, call me an unenlightened developer.

davidw 19y ago
Yes, TDD implies that there is a more or less exact specification. Otherwise, if you're just experimenting, you would have to write the test and your code, and that's going to make you less inclined to throw it away and test out something else (see "Planning is highly overrated"). My strategy has generally been to throw some tests that cover most things in a general way, and when I find bugs, add lots of detailed tests that make sure those bugs will never come back again. That makes it feel less pedantic.
- pius 19y ago
  "Yes, TDD implies that there is a more or less exact specification." That's actually the opposite of the truth. The second D is for Design (or Development). The idea is that you organically develop the spec and the code in tandem.
  - davidw 19y ago
    How can you test something if you don't even know how or if it works? You need to hack on it and see if you can get things going before you nail it down, no?
    - pius 19y ago
      Let's say you know you've gotta design a bowling pin from scratch. TDD and "standard" coding practices both require the same initial thought. You'd say, ok, what's this thing actually have to do? Maybe you start with, "it has to fall when hit by a bowling ball hard enough." If you weren't using TDD, you might go off and create a BowlingPin class, a BowlingBall class (if it's not extant), and hack together a method in some arbitrary order and method. That's cool, but the TDD approach would have you write a mini "test." In pseudoRuby: [http://pastie.caboo.se/103470 http://pastie.caboo.se/103470] This, of course, will fail because there is no BowlingPin class. So you write the class skeleton and re-run the tests. Fails again. Why? There is no got_hit method on BowlingPin? Shit. Now you've got to go and write the simplest got_hit method that'll work. So you write something like: [http://pastie.caboo.se/103471 http://pastie.caboo.se/103471] Rerun the tests. Dammit! There's no "fell?" method -- gotta implement it! And so on and so on. There are a few things to note here. First, it's pretty critical to use a decent test harness. Running and writing tests has to be crazy fast otherwise this approach won't work well. Next, in running through the TDD cycle you're actually (1) specifying the interfaces and behavior of your code and (2) implementing as narrowly as you can. The actual "tests" you get out of it are almost secondary! If implemented properly, it's really a design methodology. That's why the new trend is to change the language from Test Driven to Behavior Driven. The use of the word "test" seems to (understandably) evoke the same response you had of needing an iron-clad spec to work from initially. It's just not the case.

breck 19y ago

I'm a chronic manual tester. But I want to start getting into automated tests. What are the most important things to test? What are some good intros to pick up the basics on writing automated tests?

pius 19y ago
I personally think that test/behavior driven development is the way to go, so my advice will be geared in that direction. With that in mind, the "reference" text on it is probably "Test Driven Development: By Example" by Kent Beck. I've only skimmed it, myself, but it seems solid and Kent Beck is a smart guy. The cool thing about testing (and TDD in particular) is that even introducing a bare minimum of that sort of discipline into your work consistent will quickly pay pretty big dividends. A quick way to get started is to read this Wikipedia article: http://en.wikipedia.org/wiki/Test-driven_development http://en.wikipedia.org/wiki/Test-driven_development. Focus on understanding the TDD cycle. Mocks and stubs are useful to know, but you can safely consider them "advanced" and skip them if they confuse you initially. Then, pick a test harness for your platform, say RSpec (http://rspec.rubyforge.org http://rspec.rubyforge.org), if you're a Ruby person. Read the documentation and search for blogs and tutorial articles on the framework you've chosen. Then just start designing a simple module using it! As you practice, you'll find it useful to read up on Mocks and Stubs and dig deeper into the philosophy and best practices. Dave Astels has some great writings out there.
- breck 19y ago
  All great advice. Thanks!
- plinkplonk 19y ago
  I personally found Kent Beck's book very tedious and boring. In it , simple classes are bludgeoned into submission through the over use of tests. From Ganssle's article at http://www.ganssle.com/articles/deconstructing%20xp.htm http://www.ganssle.com/articles/deconstructing%20xp.htm "XP's motivating guru, Kent Beck, also has a new book out. Test Driven Development (Addison Wesley, 2003, ISBN 0-321-14653-0) focuses on XP's testing practice. I bought the book because I'm fascinated with testing. It's usually done as an afterthought, and rarely hits the hard conditions. Unit tests are notoriously poor. The programmer runs his new function through the simplest of situations, totally ignoring timing issues or boundary conditions. So of course the function passes, only to inflict the team with agony when integrated into the entire system. XP's idea of writing the tests in parallel with the code is quite brilliant; only then do we see all possible conditions that can occur. Beck argues for building tests first and deriving the code from these tests. It's sort of like building a comprehensive final exam and then designing a class from that. An intriguing idea. But X what if the test is wrong? Test driven development (TDD) then guarantees the code will be wrong as well. Worse, TDD calls for building even the smallest project by implementing the minimal functionality needed to do anything. Need a function to compute the Fibonacci series? Create the simplest possible test first X in this case, check to see that fibonnaci(0) is 0. Then write a function that passes that test. Try fibonnaci(1); that breaks the test, so recode both the test and the function. Iterate till correct. The book shows how to build a factorial program, which results in 91 lines of code and 89 of test... after 125 compilations! Klaxons sound, common sense alarms trip. Agile proponents love the interactive and fast action of this sort of method. But programming isn't about playing Doom and Quake. If you're looking for an adrenaline rush try bungee jumping. 125 compilations for a trivial bit of code is not fast. Dynamic? You bet. But not fast. " Test Driven Development is not quite the same as having unit tests or automated tests, which I believe the poster asked for. The jury is still very much out on how useful TDD (and XP for that matter) really is. Many talented programmers believe XP/TDD is essentially hype. . This is not to say Pius is wrong, just to add a note of caution, be skeptical of the XP hype machine. "Try it and decide for yourself" is probably the best advice possible

niels 19y ago

I've recently begun using automated tests of basically all my code. Both frontend and backend, when I'm doing web stuff. I usually write the code to a minimum first, and then implement tests. It's a huge relief to use these tests when I refactor and refine code. I definitely think automated testing is the way to go in most cases.

bsaunder 19y ago

"the break even point for an automated tester is when they've run their tests twenty times"... I find that (at least in early development), when I'm in the zone I refactor several times a day (differnent parts of the system of course). If my break even point is twenty times, I'd never get there. Every refactoring would virtually obliterate all of the previously automated tests.

geebee 19y ago

"Everybody Tests" is a good point. Some devs throw away their tests, and other devs keep the tests around and run them periodically, but devs who write code iteratively (which seems like pretty much all of them) write something that could be easily turned into a unit test. If you don't like TDD (and I have to admit I'm not a fan), you can still produce tests while you write code (you probably already are). Whenever you write a bit of code, you probably do something to check whether it is producing the expected outcome - even if it is as simple as writing a little script to generate some output that you visually inspect as it flashes by on the screen. If it looks good, you keep developing. If it fails, you figure out why. Just store your these mini-testing scripts in a unit test, and verify the expectation/output in a short statement. If you use a framework, you'll have some methods for this available to you - if you don't, no biggie, just write something that honks loud and gives you a message diagnosing the failure. It's not going to satisfy the TDD crowd, and honestly your coverage won't be as good as if you had done TDD, but there you go - you just got yourself some unit tests, and you didn't have to disrupt the mental flow that you prefer to TDD.

euccastro 19y ago

I wrote some opinions on this in other thread: http://news.ycombinator.com/item?id=46526 http://news.ycombinator.com/item?id=46526 Unless your software will kill people if it fails or something, my advice is: observe what proportion of your development time you spend debugging. If it's one half or more, unit testing would probably benefit you. If it's significantly less than 50%, then you're probably better off riding bareback, because writing and updating those tests will take at least half of your time. The problem I see with the "you'll test anyway, you may just as well automate it" argument is that unit tests are no substitute for testing with real users.

edw519 19y ago

I have always believed that there is an inverse correlation between the amount of precise attention devoted in design/development and the amount of time spent in any debugger. Nothing here changes that thinking.