5 ms·

I'm an agile coach and startup junkie. TDD/BDD doesn't fit the mold of startups. Here's why: TDD/BDD assumes you know the problem and are coding to create a s

I'm an agile coach and startup junkie. TDD/BDD doesn't fit the mold of startups. Here's why: TDD/BDD assumes you know the problem and are coding to create a solution. In startups, however, you do not know the problem. Sure, you can imagine some kind of customer that might want some kind of code to do something or another. But that's all pipe dreams. You have no idea if method X is useful or not. Sure, you might know if method X meets the requirements of your imagination, or your architecture dreamed up in your imagination to deal with your imaginary customers with their imaginary problems, but you really don't know. So it's a waste of time. Startups are all about providing value -- not flexibility, not bug-free code, not code you can hang on a wall and take pictures of. Value. If you want to be the guy who can make business solutions happen, the guy that customers can come to with any problem and you can make it all happen, you need to bone up on this stuff. But in the business world, you've already got the money, your job is to make the solution happen. In the startup world, you don't have the money yet. Big difference. Big, big difference. Look at it this way: 19 out of 20 startups fail. That means that odds are that you will never see this code again. You'd be a fool to spend any more time on it than absolutely necessary. But the math works out completely differently in the commercial world, where most things you write stay around forever. What I found over and over again with Agile is teams and individuals buying into the marketing/herd mentality of agile and forgetting about the adaptive/iterative nature. Everybody wants to either use a recipe book or just repeat their last project that they thought was really cool. "True" agile means ditching whatever isn't working. Pronto. There are no sacred cows. Everything is on the table.

bdclimber14 16y ago
I've always thought it ironic how inflexible agile was, at least based on the perception from agile purists I'm exposed to.
- DanielBMarkham 16y ago
  What I find is that I teach agile as being results and value driven first, everything else doesn't matter. All the stuff like TDD and BDD are just canned versions of patterns other teams have found useful -- each team can take them or leave them (assuming that they understand them first -- many teams are all too willing to ditch things that they have no idea what they are talking about. Helping teams make these decisions is a big part of coaching) The fascinating thing is that once I teach that it's all flexible, guess what? People come in and make it all rigid and orthodox anyway. It's not Agile that's the problem, it's the people that, over and over again, just beat the living hell out of agile trying to make it into some rigid 1-2-3 formula for success. No matter how I beg and plead with them not to do this, still they persist. Strange. I'm left with the conclusion that the basic problem in software development is people's fear of having a truly unpredictable, flexible, and open way of working -- even the agile adherents (especially the agile adherents, it seems). That's simply too much for some folks. They'd rather have a fixed list of things to do each day, even if they know the list is full of crap, they are taking ten times longer than they should, and they are doing busywork things that provide no value. As an example, I've had teams be very successful keeping their list of things to do on post-its on a wall. Then I leave for a year and come back and they've got some kind of spreadsheet monstrosity that has hundreds of items nobody can remember. Nobody likes the new system (except for perhaps the person who runs it, who will also admit that they don't like it, but only in private), it doesn't fit their needs, it's a huge time sink, yet they don't know how to make it stop. They are prisoners of their own need for fixed systems and over-engineering everything. This is because somehow they feel by over-engineering things that they make them "safer". Even though at some level they realize that this is not the case at all. Very strange.
  - deleted 16y ago
    [deleted]
    - abalashov 16y ago
      Accidentally upvoted, meant to downvote (reading on an iPhone, go figure). Who the hell posts such a retarded comment? Truly evocative of the vicious anti-intellectualism of our age.
      - andybak 16y ago
        It's [deleted] now. What was it was that drew such well-worded ire?
- wccrawford 16y ago
  @pragdave recently gave a keynote speech at the Magic Ruby conference and made a big point of saying "Agile is not a noun." Agile isn't about subscribing to practices and then writing everything in stone. It's about doing that works for you situation. So don't blame 'Agile' for being inflexible, blame the people who are using it for not actually being it.
  - swombat 16y ago
    If that's what Agile is, then Agile is just common sense. So, as someone else commented, it apparently has been diluted down to nothing.
  - plastics 16y ago
    If Agile is about doing what works for your situation, how is it different from magic pixie dust solving all your problems? :)
  - kaylarose 16y ago
    I keep hoping somebody recorded a video of that talk! It was great, and I always want to share it when the concept of 'Agile' comes up, but can never find a way to sum it adequately.
- gry 16y ago
  The rub with agile is people conflate process with communication. Things agile (hopefully) expedite commuincation, however, are only as good as those who guide it. So perhaps, more precarious. It's not a blue pill.
Lewisham 16y ago
TDD/BDD assumes you know the problem and are coding to create a solution. In startups, however, you do not know the problem. I think this is where these posts on HN always end up talking at cross-purposes. "Startup" can mean many different stages. I definitely agree with this. It's one of those things where when I think "startup" I think "they've got a problem, they're solving it" whereas some people read it at that very early stage instead. TDD doesn't suit that stage, because, like you said, you don't even have an idea of what success looks like.
jamesjyu 16y ago
A big +1 to that. Most of the successful startups I know have literally 0 tests for the first few years while they iterate.
petercooper 16y ago
TDD/BDD assumes you know the problem and are coding to create a solution. In startups, however, you do not know the problem. This seems like two ideas conflating into one. The business 'problems' (1) and the problem you're solving in a particular piece of code (2). These aren't usually directly related. For example, if I wanted to build a YouTube killer, the TDD process isn't relevant to the high level "problems" I want to solve (which, as you say, one might not know of yet). TDD comes to play (from a developer POV) when I actually start to write some code that, say, transcodes a video or provides an authentication system. In those areas, the problem is obvious and contained and TDD can work well. That means that odds are that you will never see this code again. You'd be a fool to spend any more time on it than absolutely necessary. Which does not necessarily mean TDD is "a waste of time". If practicing TDD in a particular situation will result in fewer hours spent developing a feature, the upfront "cost" of those hours is not a waste of time if you spend more hours debugging your way through building a non-tested equivalent. A developer should have a feel for which way works for them. In my case, I know that the time I spend is ultimately lowered through using some TDD principles (though not all) vs tiresome debugging of untested code. The reward cycle of TDD is commonly overlooked. "Code->Yay!->Code->Yay!" beats "Code->Code->Code->Code->2 hours of debugging->FFFFUU!!" any day. "True" agile means ditching whatever isn't working. Pronto. There are no sacred cows. Everything is on the table. Agreed, but that runs counter to making absolute statements like "TDD doesn't fit the mold of startups" and that "it's a waste of time." It can fit and it can save time (just not always, sure, or if you're 'doing it wrong' for your situation).
- ashr 16y ago
  Ditto. Couldn't agree more, in fact, I was going to write something very similar. At high level also, Start-ups do need a TDD philosophy. For Start-up business ideas, the market is the test suite (to take the software analogy further) that is already there and changing the execution plan to make sure you pass those tests is vital.
  - wisty 16y ago
    I'm not sure I agree that the market makes a great test suite. It kinda hurts when you fail. But yeah, there's other things you can use, like - A/B testing (for web apps). You can automate "code smell" detectors, and get a continuous graph on how good your code seems to be. Of course, things like converters and math functions (anything non-GUI) can benefit from unit tests.
ashr 16y ago
Note: This response is about software TDD and not some high level TDD If you don't know the problem then you are just winging it. Writing software is always with a goal (can it be without?) and thus you should be able to write a test (first) that will tell you if it does what you expect. If you expect it to record click stream, then it records click stream or whatever. When I was new to TDD, I too found it to be very cumbersome. It took me some time to realize that I don't have to figure out the entire design before I write any test code. I can start with a test, then write the functionality that it is testing and when I realize that this is not what I want but something else then I change my test to test that new thing. Iterate. That is what agile is about. Discovery that you want something else, happens whether or not you are doing TDD.
deleted 16y ago
[deleted]
dasil003 16y ago
Agile is meaningless at the point. It started to die the minute the first methodology guy with no actual coding experience thought it would be a good marketable product for his portfolio. It's been diluted and redefined so many times, and so many people are so quick to swear by their one true definition, that it's an impossible term to even discuss rationally anymore. My feeling on automated testing is that it's far more valuable and saves more time than people who are inexperienced with it can understand, and yet it has shortcomings and limitations that go far beyond what any zealot is willing to admit. I do encourage every developer to spend 3 months doing pure TDD, just to learn how to test. It's not that testing first inherently makes code better (it doesn't; test code is meta), but it's the fact that to realize the value of testing you have to be good at it, and to be good at it you have to practice. Once you are good at testing you realize that in many circumstances writing an automated test takes exactly the same amount of time as manual testing, plus then you have a regression test. As far as test coverage is concerned, if you are coding a medium to large application in Rails, you should have base coverage on every controller, period. This is a good basic sanity check that is necessary for a language as dynamic as Ruby (if you're using Java it's an order of magnitude less important, if you're using Haskell, two orders of magnitude). The only excuse is if you're really sure that this code is going to either live on in exactly the same state or be erased. The fact that you're a startup doesn't mean that tests have less value. To the contrary, if you are iterating quickly, you need a safety net when you are refactoring. Once the application reaches a certain size, you will never have time to add test coverage, and there will be certain classes of changes that become impossible because they are just too risky.
- rapind 16y ago
  "if you are coding a medium to large application in Rails, you should have base coverage on every controller, period" I actually find this to be a huge time sink for most projects. Usually in a rails app for a client's application there's only a few fancy controller methods that do anything outside of typical REST scaffolding (there's no way I'm testing all of the typical admin CRUD). Add in something like inherited_resources using thin controllers and you're wasting even more time. IMO only write tests for functionality that's non-standard, which is probably going to be a very small percentage of your controller code. For client work I find that most of the complicated or non-standard functionality happens on the client side, and rarely on the rails side. Which means a huge portion of my tests would just be fluff (like commenting method names and params manually) if I were to test all of my controllers.
shareme 16y ago
TDD/BDD assumes you know the problem and are coding to create a solution. Actually, no.. TDD is almost never used when you fully know the problem! Look at this way.. would you trust that free source clone of facebook with your data knowing the site security holes i the first release simply because they did not test using something like TDD/BDD? I would not.. That being said TDD/BDD is not agile! Neither is waterfall! WHAT?! Agile is critical analysis of processes and tools used in the dev process to reach a set of dev and management objectives and goals by modifying those processes and tools. You got some of it right and some of it wrong
rue 16y ago
I've never quite understood the anti-TDD attitude around here, but an experienced developer and coach such as Mr. Markham being so confused about the scope to which TDD/BDD is applied is eye-opening. The kinder gentlemen above have explained why the parent is fundamentally incorrect.
tmfkmoney 16y ago
I'm not sure what your suggestion here is instead of TDD. Just wing it? Your whole premise is based on the the idea that your startup is going to fail. In that case why write any code at all? If your startup doesn't fail then you've really made things extra hard on yourself. One of my favorite things about working on my startup is getting to learn and apply good practices. It's just as easy to write good code as it is to write bad code. I find that TDD speeds me up not slows me down. It's not as much overhead as people seem to suggest. You can write new code and then keep running the program to see if it works. Or you could write the code and keep running a test until it works. The test is typically faster for this because you don't have to deal with the rest of the program which is unrelated to what you're working on. Once you have working unit tests the implementation is typically trivial. In practice, I don't write tests for everything but I code in a way that means I could test everything if I wanted to.
chris123 16y ago
This whole thread should be required reading for startups, especially bootstrapped ones, but you made the key point in a sentence: "19 out of 20 startups fail. That means that odds are that you will never see this code again. You'd be a fool to spend any more time on it than absolutely necessary."

bdclimber14 16y ago

I've always thought it ironic how inflexible agile was, at least based on the perception from agile purists I'm exposed to.

DanielBMarkham 16y ago
What I find is that I teach agile as being results and value driven first, everything else doesn't matter. All the stuff like TDD and BDD are just canned versions of patterns other teams have found useful -- each team can take them or leave them (assuming that they understand them first -- many teams are all too willing to ditch things that they have no idea what they are talking about. Helping teams make these decisions is a big part of coaching) The fascinating thing is that once I teach that it's all flexible, guess what? People come in and make it all rigid and orthodox anyway. It's not Agile that's the problem, it's the people that, over and over again, just beat the living hell out of agile trying to make it into some rigid 1-2-3 formula for success. No matter how I beg and plead with them not to do this, still they persist. Strange. I'm left with the conclusion that the basic problem in software development is people's fear of having a truly unpredictable, flexible, and open way of working -- even the agile adherents (especially the agile adherents, it seems). That's simply too much for some folks. They'd rather have a fixed list of things to do each day, even if they know the list is full of crap, they are taking ten times longer than they should, and they are doing busywork things that provide no value. As an example, I've had teams be very successful keeping their list of things to do on post-its on a wall. Then I leave for a year and come back and they've got some kind of spreadsheet monstrosity that has hundreds of items nobody can remember. Nobody likes the new system (except for perhaps the person who runs it, who will also admit that they don't like it, but only in private), it doesn't fit their needs, it's a huge time sink, yet they don't know how to make it stop. They are prisoners of their own need for fixed systems and over-engineering everything. This is because somehow they feel by over-engineering things that they make them "safer". Even though at some level they realize that this is not the case at all. Very strange.
- deleted 16y ago
  [deleted]
  - abalashov 16y ago
    Accidentally upvoted, meant to downvote (reading on an iPhone, go figure). Who the hell posts such a retarded comment? Truly evocative of the vicious anti-intellectualism of our age.
    - andybak 16y ago
      It's [deleted] now. What was it was that drew such well-worded ire?
wccrawford 16y ago
@pragdave recently gave a keynote speech at the Magic Ruby conference and made a big point of saying "Agile is not a noun." Agile isn't about subscribing to practices and then writing everything in stone. It's about doing that works for you situation. So don't blame 'Agile' for being inflexible, blame the people who are using it for not actually being it.
- swombat 16y ago
  If that's what Agile is, then Agile is just common sense. So, as someone else commented, it apparently has been diluted down to nothing.
- plastics 16y ago
  If Agile is about doing what works for your situation, how is it different from magic pixie dust solving all your problems? :)
- kaylarose 16y ago
  I keep hoping somebody recorded a video of that talk! It was great, and I always want to share it when the concept of 'Agile' comes up, but can never find a way to sum it adequately.
gry 16y ago
The rub with agile is people conflate process with communication. Things agile (hopefully) expedite commuincation, however, are only as good as those who guide it. So perhaps, more precarious. It's not a blue pill.

Lewisham 16y ago

TDD/BDD assumes you know the problem and are coding to create a solution. In startups, however, you do not know the problem. I think this is where these posts on HN always end up talking at cross-purposes. "Startup" can mean many different stages. I definitely agree with this. It's one of those things where when I think "startup" I think "they've got a problem, they're solving it" whereas some people read it at that very early stage instead. TDD doesn't suit that stage, because, like you said, you don't even have an idea of what success looks like.

jamesjyu 16y ago

A big +1 to that. Most of the successful startups I know have literally 0 tests for the first few years while they iterate.

petercooper 16y ago

TDD/BDD assumes you know the problem and are coding to create a solution. In startups, however, you do not know the problem. This seems like two ideas conflating into one. The business 'problems' (1) and the problem you're solving in a particular piece of code (2). These aren't usually directly related. For example, if I wanted to build a YouTube killer, the TDD process isn't relevant to the high level "problems" I want to solve (which, as you say, one might not know of yet). TDD comes to play (from a developer POV) when I actually start to write some code that, say, transcodes a video or provides an authentication system. In those areas, the problem is obvious and contained and TDD can work well. That means that odds are that you will never see this code again. You'd be a fool to spend any more time on it than absolutely necessary. Which does not necessarily mean TDD is "a waste of time". If practicing TDD in a particular situation will result in fewer hours spent developing a feature, the upfront "cost" of those hours is not a waste of time if you spend more hours debugging your way through building a non-tested equivalent. A developer should have a feel for which way works for them. In my case, I know that the time I spend is ultimately lowered through using some TDD principles (though not all) vs tiresome debugging of untested code. The reward cycle of TDD is commonly overlooked. "Code->Yay!->Code->Yay!" beats "Code->Code->Code->Code->2 hours of debugging->FFFFUU!!" any day. "True" agile means ditching whatever isn't working. Pronto. There are no sacred cows. Everything is on the table. Agreed, but that runs counter to making absolute statements like "TDD doesn't fit the mold of startups" and that "it's a waste of time." It can fit and it can save time (just not always, sure, or if you're 'doing it wrong' for your situation).

ashr 16y ago
Ditto. Couldn't agree more, in fact, I was going to write something very similar. At high level also, Start-ups do need a TDD philosophy. For Start-up business ideas, the market is the test suite (to take the software analogy further) that is already there and changing the execution plan to make sure you pass those tests is vital.
- wisty 16y ago
  I'm not sure I agree that the market makes a great test suite. It kinda hurts when you fail. But yeah, there's other things you can use, like - A/B testing (for web apps). You can automate "code smell" detectors, and get a continuous graph on how good your code seems to be. Of course, things like converters and math functions (anything non-GUI) can benefit from unit tests.

ashr 16y ago

Note: This response is about software TDD and not some high level TDD If you don't know the problem then you are just winging it. Writing software is always with a goal (can it be without?) and thus you should be able to write a test (first) that will tell you if it does what you expect. If you expect it to record click stream, then it records click stream or whatever. When I was new to TDD, I too found it to be very cumbersome. It took me some time to realize that I don't have to figure out the entire design before I write any test code. I can start with a test, then write the functionality that it is testing and when I realize that this is not what I want but something else then I change my test to test that new thing. Iterate. That is what agile is about. Discovery that you want something else, happens whether or not you are doing TDD.

deleted 16y ago

[deleted]

dasil003 16y ago

Agile is meaningless at the point. It started to die the minute the first methodology guy with no actual coding experience thought it would be a good marketable product for his portfolio. It's been diluted and redefined so many times, and so many people are so quick to swear by their one true definition, that it's an impossible term to even discuss rationally anymore. My feeling on automated testing is that it's far more valuable and saves more time than people who are inexperienced with it can understand, and yet it has shortcomings and limitations that go far beyond what any zealot is willing to admit. I do encourage every developer to spend 3 months doing pure TDD, just to learn how to test. It's not that testing first inherently makes code better (it doesn't; test code is meta), but it's the fact that to realize the value of testing you have to be good at it, and to be good at it you have to practice. Once you are good at testing you realize that in many circumstances writing an automated test takes exactly the same amount of time as manual testing, plus then you have a regression test. As far as test coverage is concerned, if you are coding a medium to large application in Rails, you should have base coverage on every controller, period. This is a good basic sanity check that is necessary for a language as dynamic as Ruby (if you're using Java it's an order of magnitude less important, if you're using Haskell, two orders of magnitude). The only excuse is if you're really sure that this code is going to either live on in exactly the same state or be erased. The fact that you're a startup doesn't mean that tests have less value. To the contrary, if you are iterating quickly, you need a safety net when you are refactoring. Once the application reaches a certain size, you will never have time to add test coverage, and there will be certain classes of changes that become impossible because they are just too risky.

rapind 16y ago
"if you are coding a medium to large application in Rails, you should have base coverage on every controller, period" I actually find this to be a huge time sink for most projects. Usually in a rails app for a client's application there's only a few fancy controller methods that do anything outside of typical REST scaffolding (there's no way I'm testing all of the typical admin CRUD). Add in something like inherited_resources using thin controllers and you're wasting even more time. IMO only write tests for functionality that's non-standard, which is probably going to be a very small percentage of your controller code. For client work I find that most of the complicated or non-standard functionality happens on the client side, and rarely on the rails side. Which means a huge portion of my tests would just be fluff (like commenting method names and params manually) if I were to test all of my controllers.

shareme 16y ago

TDD/BDD assumes you know the problem and are coding to create a solution. Actually, no.. TDD is almost never used when you fully know the problem! Look at this way.. would you trust that free source clone of facebook with your data knowing the site security holes i the first release simply because they did not test using something like TDD/BDD? I would not.. That being said TDD/BDD is not agile! Neither is waterfall! WHAT?! Agile is critical analysis of processes and tools used in the dev process to reach a set of dev and management objectives and goals by modifying those processes and tools. You got some of it right and some of it wrong

rue 16y ago

I've never quite understood the anti-TDD attitude around here, but an experienced developer and coach such as Mr. Markham being so confused about the scope to which TDD/BDD is applied is eye-opening. The kinder gentlemen above have explained why the parent is fundamentally incorrect.

tmfkmoney 16y ago

I'm not sure what your suggestion here is instead of TDD. Just wing it? Your whole premise is based on the the idea that your startup is going to fail. In that case why write any code at all? If your startup doesn't fail then you've really made things extra hard on yourself. One of my favorite things about working on my startup is getting to learn and apply good practices. It's just as easy to write good code as it is to write bad code. I find that TDD speeds me up not slows me down. It's not as much overhead as people seem to suggest. You can write new code and then keep running the program to see if it works. Or you could write the code and keep running a test until it works. The test is typically faster for this because you don't have to deal with the rest of the program which is unrelated to what you're working on. Once you have working unit tests the implementation is typically trivial. In practice, I don't write tests for everything but I code in a way that means I could test everything if I wanted to.

chris123 16y ago

This whole thread should be required reading for startups, especially bootstrapped ones, but you made the key point in a sentence: "19 out of 20 startups fail. That means that odds are that you will never see this code again. You'd be a fool to spend any more time on it than absolutely necessary."