8 ms·

Test driven development at Transloadit

133 points by felixge 16y ago

sreuter 16y ago
nice one!
felixge 16y ago
A few facts that I didn't get to fit into the article: - We have ~1.6x as much test code as we have code being tested. - Our unit test suite takes < 5 seconds to run. Our system test suite takes < 60 seconds. - We use Hudson for continuous integration - The hardest part with TDD for us was reaching 1.0 as you always feel like it would be "faster" to stop testing. - We usually don't go around refactoring stuff just because we can. In fact we usually feel more confident building upon existing stuff since we have tests that are saying it works. And it goes without saying, feel free to ask me anything : ).
- singular 16y ago
  Thanks for this, very interesting. You didn't mention mocking in the article - to what degree do you use mocking, and where does it fit in the testing pipeline?
  - felixge 16y ago
    I have written a library called gently. This library lets you define a series of expected function calls spawning multiple objects (since overall order is important to us). Whenever you define such an expected call, gently returns a closure that you can use to inject this expectation in the right place. From my understanding this is hybrid between Mocking (where you have an object with an expected call / state sequence), and Stubbing (where you have various pre-recorded answers to function calls). But generally I found the semantics of the various TDD methodologies very difficult to translate into actual code. That's why I'm using as little abstraction as possible.
    - onewland 16y ago
      Is gently open source or could it be?
      - felixge 16y ago
        Yes, http://github.com/felixge/node-gently http://github.com/felixge/node-gently . However, it's still rather minimal and I'll add more stuff to it now that I actually know what I need to do with it frequently.
ericb 16y ago
Test driven development, to me, starts as the harness you use to write your code in the first place. I'm not sure how people write solid code with any kind of speed without a harness testing/driving it. Why not make your harness your tests? Everyone keeps talking about how it is more work, which confuses me. I don't make a religion out of test coverage, so maybe that is the difference? As in, I don't write a test for every basic thing, like an accessor, for example, because I feel like there's a point of diminishing returns. edit: To those downvoting, I'm interested in a dialogue and hearing your thoughts.
- deleted 16y ago
  [deleted]
- felixge 16y ago
  The costs for TDD greatly vary. Trying to do TDD for GUI / frontend programming is much more expensive. I also find TDD very hard when working within a framework that was not written using TDD. So of course, if your environment makes TDD as cheap as ours does, I'm with you and think you have to be a little crazy to not at least try it. But it's not for everybody / everything by default. About skipping tests for accessors and stuff: That's a slippery slope, I'm afraid it would lead me to skip more "obvious" tests which in turn make it harder to write the right test when this stuff actually is part of a bug in the future.
  - ericb 16y ago
    Agreed, my comment above was not GUI/frontend-centric and that is a different case. To me there is another slippery slope for TDD in getting bogged down. I'm usually interested in TDD in as much as it can be a competitive advantage. The point where I feel like the returns drop off and I'm burning my man-hours and paying too much for the TDD insurance on very predictable parts is where my tests get sparser. Keep in mind, the degree of TDD insurance I'd pay for is different for every project--for a Mars Rover you bet I'll test my accessors.
    - felixge 16y ago
DanielBMarkham 16y ago
I'm extremely happy to upvote first-hand TDD articles whenever I see them on here. There have been quite a few nice stories, both pro and con. I'd like to see more folks talk about competitive advantage. After all, code just doesn't exist by itself -- it's supposed to do something. Does TDD help make technology make people's lives better? Or is it just a big PITA that ends up with more maintainable code? There's nothing wrong with that, but it's the kind of perspective question can be a critical success factor for startups.
- felixge 16y ago
  Regarding competitive advantages: We are working on a new feature that will allow us to encode video uploads while they are still uploading. This change will affect the most critical aspects of our code base and it's very easy to introduce subtle bugs. TDD is helping us to make fundamental changes like this without being too afraid of breaking things. I don't think we would undertake this feature if we didn't have the tests we do. (And TDD is the only way to get tests you really trust)
  - ericwaller 16y ago
    (And TDD is the only way to get tests you really trust) Can you expand on this point? This is one aspect of TDD, which seems to focus most intensely on unit testing, I've never understood. Why wouldn't full-stack, acceptance testing be just as effective?
    - felixge 16y ago
      You can get value out of writing a test after the fact. But the value is higher for a test that you have written before the code because you have seen the test go from failing to passing. This way you know it will also fail if you break the code in the future. I also think writing the test before the feature will cause you to write better assert statements. If you write the test afterwards you'll tend to either assert everything (which leads to every test breaking on a small change), or to be horribly confused about what to assert and what not.
    - 16y ago
jsdalton 16y ago
Do you find it hard to change directions with a feature or component, because your tests have sort of "locked you in" to the structure you were working on? How religiously do you follow TDD? Do you literally not write a single line of code unless it's in furtherance of passing a failing unit test, or do you just worry about the system test you put in place at the start and unit test aspects as you need to? Also, you mentioned changes to the API of third-party libraries. How do go about catching how those changes impact your code? (I would assume you are mocking these in your unit tests.) Wow I just asked you a lot of questions. Seriously though, thanks for posting this article. It's so rare to be able to read an article about unit testing that isn't just some guy demonstrating how he'd use TDD to determine if a number is in the Fibonacci sequence or something.
- jcoby 16y ago
  I can give you my experience with TDD: You generally don't need to change directions very much. TDD will expose bad interfaces or interfaces that aren't abstract enough early on because you can't test them easily. And if you do decide to change directions you sit down and write the tests and then make the changes until your code works and the tests pass. Generally you want to write tests before code and make your code pass the tests. How many tests and how much coverage is up to you. It's important that the tests are small and brittle so you can pick out as many breakages as early as possible. You use the API in your tests. When the tests start to fail you know your API has changed. It gets a bit more difficult with web APIs if they don't provide a test environment.
- felixge 16y ago
  > Do you find it hard to change directions with a feature or component, because your tests have sort of "locked you in" to the structure you were working on? Do you mean changing the direction of the implementation, or the outcome of the feature? If it's just the implementation we usually throw away the unit tests along with the code we don't like and restart. The system test is ultimately responsible for making sure the feature itself is working. The unit tests just allow you to test different parameters without having to write a system test for each of those. I have never felt "locked in" so far. Generally I feel very good about the stability of the Software and my ability to change it as needed. > How religiously do you follow TDD? Do you literally not write a single line of code unless it's in furtherance of passing a failing unit test, or do you just worry about the system test you put in place at the start and unit test aspects as you need to Very religiously with almost no exceptions. When I make an exception it is usually a sign of problems in the code, and I'll revisit it later on / improve my tools. I think it is difficult to do anything but full TDD, it's just to easy to drop the ball once you make compromises. > Also, you mentioned changes to the API of third-party libraries. How do go about catching how those changes impact your code? (I would assume you are mocking these in your unit tests.) System tests are the only line of defense here. That's why we make sure that all mission critical code paths are covered by them. There are no 100% guarantees, but 95% certainty beats 20% certainty after a big library upgrade by a lot : ). > Seriously though, thanks for posting this article. It's so rare to be able to read an article about unit testing that isn't just some guy demonstrating how he'd use TDD to determine if a number is in the Fibonacci sequence or something. I never saw the benefits from these articles either. It took the node.js API to break our app in about 100 different parts for me to see the value in TDD. I guess it's just like backups ... ; )
steilpass 16y ago
BTW Felix was interviewed about Node.js at http://chaosradio.ccc.de/cre167.html http://chaosradio.ccc.de/cre167.html (German)
ericHosick 16y ago
I think it would also behoove you to start using BDD in your development process. With BDD, you get a one-to-many correlation between behavior and code. This also greatly improves on quality. I say this because quality can only be based against requirements. BDD forces the developer to have the behavioral requirements upfront. BDD allows you to take a given behavioral requirement and see all the code that supports that requirement and visa-versa. If a customer ever asks you, you can point to exactly where you are fulfilling that behavioral requirement in the source code. And BDD languages such as Gherkin are easy to read for most non-technical people. Of course, this isn't a replacement for TDD.
- swombat 16y ago
  I take the definition (proposed, usually, by BDD people like David Chelimsky and the like) that BDD is not fundamentally different from TDD. BDD is just TDD done right. As such, it makes little sense to propose using BDD "instead of" TDD. If you're doing TDD right, there's no benefit to "switching to BDD", other than a monstrous task of rewriting your tests using another set of sub-frameworks...
  - ericHosick 16y ago
    I think there is quite a big difference, fundamentally, between BDD and TDD (though they are not mutually exclusive and do overlap). Let's try a quick example of a google maps type app: locating where you are and where you want to be. From BDD perspective, you get a mockup of the screen (could even be some non visual process) and this represents the behavior you want of the product: the behavioral requirements. From this mockup we can pull out all the behavior on that screen (or within that system). Feature: In order to see if I am where I want to be As a traveler I need to know my location. Scenario: At My Location Given I want to be at Lon XXXX Lat YYYY And I am at Log XXXX Lat YYYY Then I should see "You are There" Scenario: Not there Yet Given I want to be at Lon XXXX Lat YYYY And I am at Log XXXX Lat YYYY2 Then I should see "You have 4 meters to go". From this behavior we can then build out the system which inevitably leads to fine grained software specific behavior which should be tested using TDD such as: it should "calculate the distance between two points correct" In the case of BDD, we don't worry about how the two points are calculated nor if it is even done correctly (we don't need full fringe test coverage in BDD). We are able to assume that the underlying calculations will be correct. We do need to make sure that the behavior of the system as a whole is working correctly. In this case, we are assuming if the calculation is incorrect we will not see "You are There" on the screen. Why was it incorrect? Doesn't matter. That specific behavior, the calculation, was driven by Tests. Personally, and for efficiency, it is really important to use both in a project and not look at BDD as "TDD done right." Just my two cents.
ollysb 16y ago
One of the most pleasurable benefits of TDD is that I can refactor code and not worry that I'm going to break something. On larger projects this is really liberating, I can develop with the same freedom that I did on day 1. It means that a code base can evolve smoothly and naturally, assimilating new requirements. Without that safety net you end up with code that gradually becomes a poorer fit for the requirements, changes becoming huge snaps as they are delayed over the fear of regressions.
- felixge 16y ago
  Yes, plus you'll be doing less rewriting because you spend more time and thoughts on getting it right in the first place.
- ddlatham 16y ago
  Is that specific to TDD or just having a comprehensive suite of tests?
  - ollysb 16y ago
    There's 2 aspects to it. First, the coverage which gives you the safety net. Second, specifying behaviour before implementation which allows you to maintain tight alignment between your requirements and code as they evolve. According to the BDD guys it's the second aspect that is more important.
    - ollysb 16y ago
      I'm not sure why my comment has been downvoted, it answers the question and to my knowledge is accurate. Perhaps I can expand on something for you?
dalore 16y ago
Is it really a unit test if you make a call to a REST service? Is that not now an integration test?
- felixge 16y ago
  Yes, that is an integration test. You need both.
olliesaunders 16y ago
John Hughes, (author of Why FP Matter) is mitigating some of the problems of TDD with QuickCheck (http://en.wikipedia.org/wiki/QuickCheck http://en.wikipedia.org/wiki/QuickCheck). Video: http://www.infoq.com/presentations/The-Joy-of-Testing http://www.infoq.com/presentations/The-Joy-of-Testing I've been using it for the last couple of days. You'll have to excuse not being able to see the demo properly. InfoQ fail. Stick with it, there's some really cool stuff at the end. Another video: http://video.google.com/videoplay?docid=4655369445141008672# http://video.google.com/videoplay?docid=4655369445141008672#
Tomek_ 16y ago
Until recently I was looking at TDD as a necessary evil - yes, it saves your ass a lot of times, but writing tests doesn't feel fun nor very productive - I don't know if I can speak for others, but for myself there are two things that I like most in programming, 1st: creating cool, new features; 2nd: polishing/refactoring the code until it becomes a state of art. And what I've found out not a long time ago is that TDD helps a lot with the 2nd point, but only (again, at least for me) if you aim for the 100% code coverage. Yes, I know it sounds silly, and pointless, and that code coverage doesn't mean that much, but anything lower than those 100% makes you too relaxed about quality of the code and leads to skipping the parts that are harder to test, and most often they are hard to test because they are badly written (bad architecture, bad API decisions, etc.). When you go for a 100% CC you have to examine all those parts, and change the code so it actually can become testable, and it's often hard and requires making big changes in many places. It's challenging, but challenging means fun. And you learn a lot, but learning a lot is fun too. So, double win (triple, if you count easier bugspotting)!
johnwatson11218 16y ago
I really like the author's assertion that writing the test feels like proving mathematics. I have always been fascinated by this connection. When you look at the history of mathematics w/r/t the discovery of calculus there was a 150+ year period in which people could use it to do work but nobody could prove calculus. It wasn't until the early 1800s and the technique involved bounding errors. I'm referring to the delta-epsilon proofs from analysis. To me this is exactly like what is happening in computer programming right now. In my opinion automated testing is some of the most cutting edge stuff going on right now.
Ixiaus 16y ago
I'm building a startup with Python and Pylons - Pylons comes with paster (python web server) and a small unit testing hook for your app; pylons makes it so easy to test your application, I love it. You can write two types of tests: functional tests and unit tests. I split functional tests into the tests that actually make a web request (by spawning paster, loading the application, and pretending to be a user) - my lowest "unit" there are my controllers. I have to have some special trickery to handle sessions, but it isn't bad, beats having no tests. My "unit" tests are a bit less fine-grained than yours are. I've had difficulty narrowing tests down below the module level with anything but libraries; primarily because models (the culprits here) have a few interdependencies. I do my best to keep my models orthogonal but not all of them are, so most of my unit tests are basically testing models as they operate on the database tables and some of their assistant methods. If I write a library, I generally try to determine whether it can be its own "module" or should be a module within the application namespace. For example, I wrote an authorize.net payment library and ultimately decided that it should have its own tests - separate from the application, which lead me to splitting it off as its own project (PayPy now, with adapter support for some other gateways). Writing some of the library modules in their own namespace with their own tests keeps it clean orthogonal. Overall, I would say this: I'm not as OCD about writing tests as you are because I'm the sole developer and there are too many features to build to be writing functional tests every time I build a controller. My philosophy is this, though: the models should all be covered, no matter what; since they are the messiest bit of logic (usually) in a web application - translation between relational data into object oriented data has a lot of side effects. Having all models covered ensures that the majority of my logic that interacts with the database is solid. It also keeps any schema/model changes in check that may affect other models I've forgotten about. After that, I try to make sure any critical controllers/pages/functional pieces are covered - the really trivial stuff I worry less about and just make sure it "works right" by doing some manual testing myself before pushing it. I love functional/unit testing - I will never go back. My old way of development feels like I was lost in the dark ages of my career or something...
omouse 16y ago
Writing those tests feels like proofing mathematical theorems Uh, not really. Have you ever written tests for functions in math? It's a pain in the ass once you get past 5 variables. This is why we have proofs in the first place, to prove that something is true for each and every case without having to run through every test...

133 points by felixge 16y ago

nice one!

A few facts that I didn't get to fit into the article: - We have ~1.6x as much test code as we have code being tested. - Our unit test suite takes < 5 seconds to run. Our system test suite takes < 60 seconds. - We use Hudson for continuous integration - The hardest part with TDD for us was reaching 1.0 as you always feel like it would be "faster" to stop testing. - We usually don't go around refactoring stuff just because we can. In fact we usually feel more confident building upon existing stuff since we have tests that are saying it works. And it goes without saying, feel free to ask me anything : ).

singular 16y ago
Thanks for this, very interesting. You didn't mention mocking in the article - to what degree do you use mocking, and where does it fit in the testing pipeline?
- felixge 16y ago
  I have written a library called gently. This library lets you define a series of expected function calls spawning multiple objects (since overall order is important to us). Whenever you define such an expected call, gently returns a closure that you can use to inject this expectation in the right place. From my understanding this is hybrid between Mocking (where you have an object with an expected call / state sequence), and Stubbing (where you have various pre-recorded answers to function calls). But generally I found the semantics of the various TDD methodologies very difficult to translate into actual code. That's why I'm using as little abstraction as possible.
  - onewland 16y ago
    Is gently open source or could it be?
    - felixge 16y ago
      Yes, http://github.com/felixge/node-gently http://github.com/felixge/node-gently . However, it's still rather minimal and I'll add more stuff to it now that I actually know what I need to do with it frequently.

ericb 16y ago

Test driven development, to me, starts as the harness you use to write your code in the first place. I'm not sure how people write solid code with any kind of speed without a harness testing/driving it. Why not make your harness your tests? Everyone keeps talking about how it is more work, which confuses me. I don't make a religion out of test coverage, so maybe that is the difference? As in, I don't write a test for every basic thing, like an accessor, for example, because I feel like there's a point of diminishing returns. edit: To those downvoting, I'm interested in a dialogue and hearing your thoughts.

deleted 16y ago
[deleted]
felixge 16y ago
The costs for TDD greatly vary. Trying to do TDD for GUI / frontend programming is much more expensive. I also find TDD very hard when working within a framework that was not written using TDD. So of course, if your environment makes TDD as cheap as ours does, I'm with you and think you have to be a little crazy to not at least try it. But it's not for everybody / everything by default. About skipping tests for accessors and stuff: That's a slippery slope, I'm afraid it would lead me to skip more "obvious" tests which in turn make it harder to write the right test when this stuff actually is part of a bug in the future.
- ericb 16y ago
  Agreed, my comment above was not GUI/frontend-centric and that is a different case. To me there is another slippery slope for TDD in getting bogged down. I'm usually interested in TDD in as much as it can be a competitive advantage. The point where I feel like the returns drop off and I'm burning my man-hours and paying too much for the TDD insurance on very predictable parts is where my tests get sparser. Keep in mind, the degree of TDD insurance I'd pay for is different for every project--for a Mars Rover you bet I'll test my accessors.
  - felixge 16y ago

DanielBMarkham 16y ago

I'm extremely happy to upvote first-hand TDD articles whenever I see them on here. There have been quite a few nice stories, both pro and con. I'd like to see more folks talk about competitive advantage. After all, code just doesn't exist by itself -- it's supposed to do something. Does TDD help make technology make people's lives better? Or is it just a big PITA that ends up with more maintainable code? There's nothing wrong with that, but it's the kind of perspective question can be a critical success factor for startups.

felixge 16y ago
Regarding competitive advantages: We are working on a new feature that will allow us to encode video uploads while they are still uploading. This change will affect the most critical aspects of our code base and it's very easy to introduce subtle bugs. TDD is helping us to make fundamental changes like this without being too afraid of breaking things. I don't think we would undertake this feature if we didn't have the tests we do. (And TDD is the only way to get tests you really trust)
- ericwaller 16y ago
  (And TDD is the only way to get tests you really trust) Can you expand on this point? This is one aspect of TDD, which seems to focus most intensely on unit testing, I've never understood. Why wouldn't full-stack, acceptance testing be just as effective?
  - felixge 16y ago
    You can get value out of writing a test after the fact. But the value is higher for a test that you have written before the code because you have seen the test go from failing to passing. This way you know it will also fail if you break the code in the future. I also think writing the test before the feature will cause you to write better assert statements. If you write the test afterwards you'll tend to either assert everything (which leads to every test breaking on a small change), or to be horribly confused about what to assert and what not.
  - 16y ago

jsdalton 16y ago

Do you find it hard to change directions with a feature or component, because your tests have sort of "locked you in" to the structure you were working on? How religiously do you follow TDD? Do you literally not write a single line of code unless it's in furtherance of passing a failing unit test, or do you just worry about the system test you put in place at the start and unit test aspects as you need to? Also, you mentioned changes to the API of third-party libraries. How do go about catching how those changes impact your code? (I would assume you are mocking these in your unit tests.) Wow I just asked you a lot of questions. Seriously though, thanks for posting this article. It's so rare to be able to read an article about unit testing that isn't just some guy demonstrating how he'd use TDD to determine if a number is in the Fibonacci sequence or something.

jcoby 16y ago
I can give you my experience with TDD: You generally don't need to change directions very much. TDD will expose bad interfaces or interfaces that aren't abstract enough early on because you can't test them easily. And if you do decide to change directions you sit down and write the tests and then make the changes until your code works and the tests pass. Generally you want to write tests before code and make your code pass the tests. How many tests and how much coverage is up to you. It's important that the tests are small and brittle so you can pick out as many breakages as early as possible. You use the API in your tests. When the tests start to fail you know your API has changed. It gets a bit more difficult with web APIs if they don't provide a test environment.
felixge 16y ago
> Do you find it hard to change directions with a feature or component, because your tests have sort of "locked you in" to the structure you were working on? Do you mean changing the direction of the implementation, or the outcome of the feature? If it's just the implementation we usually throw away the unit tests along with the code we don't like and restart. The system test is ultimately responsible for making sure the feature itself is working. The unit tests just allow you to test different parameters without having to write a system test for each of those. I have never felt "locked in" so far. Generally I feel very good about the stability of the Software and my ability to change it as needed. > How religiously do you follow TDD? Do you literally not write a single line of code unless it's in furtherance of passing a failing unit test, or do you just worry about the system test you put in place at the start and unit test aspects as you need to Very religiously with almost no exceptions. When I make an exception it is usually a sign of problems in the code, and I'll revisit it later on / improve my tools. I think it is difficult to do anything but full TDD, it's just to easy to drop the ball once you make compromises. > Also, you mentioned changes to the API of third-party libraries. How do go about catching how those changes impact your code? (I would assume you are mocking these in your unit tests.) System tests are the only line of defense here. That's why we make sure that all mission critical code paths are covered by them. There are no 100% guarantees, but 95% certainty beats 20% certainty after a big library upgrade by a lot : ). > Seriously though, thanks for posting this article. It's so rare to be able to read an article about unit testing that isn't just some guy demonstrating how he'd use TDD to determine if a number is in the Fibonacci sequence or something. I never saw the benefits from these articles either. It took the node.js API to break our app in about 100 different parts for me to see the value in TDD. I guess it's just like backups ... ; )

steilpass 16y ago

BTW Felix was interviewed about Node.js at http://chaosradio.ccc.de/cre167.html http://chaosradio.ccc.de/cre167.html (German)

ericHosick 16y ago

I think it would also behoove you to start using BDD in your development process. With BDD, you get a one-to-many correlation between behavior and code. This also greatly improves on quality. I say this because quality can only be based against requirements. BDD forces the developer to have the behavioral requirements upfront. BDD allows you to take a given behavioral requirement and see all the code that supports that requirement and visa-versa. If a customer ever asks you, you can point to exactly where you are fulfilling that behavioral requirement in the source code. And BDD languages such as Gherkin are easy to read for most non-technical people. Of course, this isn't a replacement for TDD.

swombat 16y ago
I take the definition (proposed, usually, by BDD people like David Chelimsky and the like) that BDD is not fundamentally different from TDD. BDD is just TDD done right. As such, it makes little sense to propose using BDD "instead of" TDD. If you're doing TDD right, there's no benefit to "switching to BDD", other than a monstrous task of rewriting your tests using another set of sub-frameworks...
- ericHosick 16y ago
  I think there is quite a big difference, fundamentally, between BDD and TDD (though they are not mutually exclusive and do overlap). Let's try a quick example of a google maps type app: locating where you are and where you want to be. From BDD perspective, you get a mockup of the screen (could even be some non visual process) and this represents the behavior you want of the product: the behavioral requirements. From this mockup we can pull out all the behavior on that screen (or within that system). Feature: In order to see if I am where I want to be As a traveler I need to know my location. Scenario: At My Location Given I want to be at Lon XXXX Lat YYYY And I am at Log XXXX Lat YYYY Then I should see "You are There" Scenario: Not there Yet Given I want to be at Lon XXXX Lat YYYY And I am at Log XXXX Lat YYYY2 Then I should see "You have 4 meters to go". From this behavior we can then build out the system which inevitably leads to fine grained software specific behavior which should be tested using TDD such as: it should "calculate the distance between two points correct" In the case of BDD, we don't worry about how the two points are calculated nor if it is even done correctly (we don't need full fringe test coverage in BDD). We are able to assume that the underlying calculations will be correct. We do need to make sure that the behavior of the system as a whole is working correctly. In this case, we are assuming if the calculation is incorrect we will not see "You are There" on the screen. Why was it incorrect? Doesn't matter. That specific behavior, the calculation, was driven by Tests. Personally, and for efficiency, it is really important to use both in a project and not look at BDD as "TDD done right." Just my two cents.

ollysb 16y ago

One of the most pleasurable benefits of TDD is that I can refactor code and not worry that I'm going to break something. On larger projects this is really liberating, I can develop with the same freedom that I did on day 1. It means that a code base can evolve smoothly and naturally, assimilating new requirements. Without that safety net you end up with code that gradually becomes a poorer fit for the requirements, changes becoming huge snaps as they are delayed over the fear of regressions.

felixge 16y ago
Yes, plus you'll be doing less rewriting because you spend more time and thoughts on getting it right in the first place.
ddlatham 16y ago
Is that specific to TDD or just having a comprehensive suite of tests?
- ollysb 16y ago
  There's 2 aspects to it. First, the coverage which gives you the safety net. Second, specifying behaviour before implementation which allows you to maintain tight alignment between your requirements and code as they evolve. According to the BDD guys it's the second aspect that is more important.
  - ollysb 16y ago
    I'm not sure why my comment has been downvoted, it answers the question and to my knowledge is accurate. Perhaps I can expand on something for you?

dalore 16y ago

Is it really a unit test if you make a call to a REST service? Is that not now an integration test?

felixge 16y ago
Yes, that is an integration test. You need both.

olliesaunders 16y ago

John Hughes, (author of Why FP Matter) is mitigating some of the problems of TDD with QuickCheck (http://en.wikipedia.org/wiki/QuickCheck http://en.wikipedia.org/wiki/QuickCheck). Video: http://www.infoq.com/presentations/The-Joy-of-Testing http://www.infoq.com/presentations/The-Joy-of-Testing I've been using it for the last couple of days. You'll have to excuse not being able to see the demo properly. InfoQ fail. Stick with it, there's some really cool stuff at the end. Another video: http://video.google.com/videoplay?docid=4655369445141008672# http://video.google.com/videoplay?docid=4655369445141008672#

Tomek_ 16y ago

Until recently I was looking at TDD as a necessary evil - yes, it saves your ass a lot of times, but writing tests doesn't feel fun nor very productive - I don't know if I can speak for others, but for myself there are two things that I like most in programming, 1st: creating cool, new features; 2nd: polishing/refactoring the code until it becomes a state of art. And what I've found out not a long time ago is that TDD helps a lot with the 2nd point, but only (again, at least for me) if you aim for the 100% code coverage. Yes, I know it sounds silly, and pointless, and that code coverage doesn't mean that much, but anything lower than those 100% makes you too relaxed about quality of the code and leads to skipping the parts that are harder to test, and most often they are hard to test because they are badly written (bad architecture, bad API decisions, etc.). When you go for a 100% CC you have to examine all those parts, and change the code so it actually can become testable, and it's often hard and requires making big changes in many places. It's challenging, but challenging means fun. And you learn a lot, but learning a lot is fun too. So, double win (triple, if you count easier bugspotting)!

johnwatson11218 16y ago

I really like the author's assertion that writing the test feels like proving mathematics. I have always been fascinated by this connection. When you look at the history of mathematics w/r/t the discovery of calculus there was a 150+ year period in which people could use it to do work but nobody could prove calculus. It wasn't until the early 1800s and the technique involved bounding errors. I'm referring to the delta-epsilon proofs from analysis. To me this is exactly like what is happening in computer programming right now. In my opinion automated testing is some of the most cutting edge stuff going on right now.

Ixiaus 16y ago

I'm building a startup with Python and Pylons - Pylons comes with paster (python web server) and a small unit testing hook for your app; pylons makes it so easy to test your application, I love it. You can write two types of tests: functional tests and unit tests. I split functional tests into the tests that actually make a web request (by spawning paster, loading the application, and pretending to be a user) - my lowest "unit" there are my controllers. I have to have some special trickery to handle sessions, but it isn't bad, beats having no tests. My "unit" tests are a bit less fine-grained than yours are. I've had difficulty narrowing tests down below the module level with anything but libraries; primarily because models (the culprits here) have a few interdependencies. I do my best to keep my models orthogonal but not all of them are, so most of my unit tests are basically testing models as they operate on the database tables and some of their assistant methods. If I write a library, I generally try to determine whether it can be its own "module" or should be a module within the application namespace. For example, I wrote an authorize.net payment library and ultimately decided that it should have its own tests - separate from the application, which lead me to splitting it off as its own project (PayPy now, with adapter support for some other gateways). Writing some of the library modules in their own namespace with their own tests keeps it clean orthogonal. Overall, I would say this: I'm not as OCD about writing tests as you are because I'm the sole developer and there are too many features to build to be writing functional tests every time I build a controller. My philosophy is this, though: the models should all be covered, no matter what; since they are the messiest bit of logic (usually) in a web application - translation between relational data into object oriented data has a lot of side effects. Having all models covered ensures that the majority of my logic that interacts with the database is solid. It also keeps any schema/model changes in check that may affect other models I've forgotten about. After that, I try to make sure any critical controllers/pages/functional pieces are covered - the really trivial stuff I worry less about and just make sure it "works right" by doing some manual testing myself before pushing it. I love functional/unit testing - I will never go back. My old way of development feels like I was lost in the dark ages of my career or something...

omouse 16y ago

Writing those tests feels like proofing mathematical theorems Uh, not really. Have you ever written tests for functions in math? It's a pain in the ass once you get past 5 variables. This is why we have proofs in the first place, to prove that something is true for each and every case without having to run through every test...