5 ms·

Thanks for your excellent replies. I realize that part of the difficulties I experience with using TDD in real life is that testing feels "expensive" to me, e.g

Thanks for your excellent replies. I realize that part of the difficulties I experience with using TDD in real life is that testing feels "expensive" to me, e.g. I frequently spend more time futzing around with test code than I do production code. Your responses illustrate a different perspective, that testing is cheap (this probably comes with experience, I would imagine). Anyhow, I can see that if I thought of tests as cheap, I wouldn't get so hung up on my tests breaking when I change direction on a feature. I also write a fair amount of code that consumes web APIs, so I have lost a lot of hair figuring out to test against those properly without mocking out everything to the point of meaninglessness.

felixge 16y ago
I hope I'm not giving you the wrong picture. We did have to cut features when rewriting our software (after node.js changed all APIs) using TDD. It does take more time to write tested code, and you do not get an advantage until you have the code in production. It is only at that point that you are seeing the returns on that investment. So generally you need to try to find a small enough product so you can build it using TDD. Also stay as simple as you can with your testing - even if it feels very repetitive. I'm only now thinking of adding more tools to my testing toolchain after I have a better understanding of the actual problems I run into often. About testing web APIs. You need both unit and system tests. The unit tests make sure your internal processing is solid. The system tests will directly make calls to the service. That won't always feel pretty, but it's the best way to approach it IMO.
- boundlessdreamz 16y ago
  How do you test something which requires browser interaction like oauth authentication? Edit: I know this may not be applicable in transloadit's case. I'm asking for a general strategy
  - felixge 16y ago
    The core of our service is a REST API, so we don't have to deal with that much. For browser interaction we would probably use a combination of Selenium (for system tests), and maybe qunit for the frontend testing. But anyway, I am no expert on this so I can't give you solid advice.
    - johnwatson11218 16y ago
      I have used canoo webtest to test browser interaction that involved a log in to a deployed server.
  - knuckle_cake 16y ago
    One method I've used (with Ruby) is use of FakeWeb, and collecting the various responses from oauth providers to test against. It's the closest I've seen to hitting the real services. If you're testing javascript, you'll need to use Selenium or something similar.
  - ollysb 16y ago
    Webmock is an excellent ruby framework for solving this problem. My current app intereracts with both github and heroku and it's come in very handing for filling in what would otherwise be gaps. It's newer than fakeweb and is much better at letting you specify interactions on a per test basis rather than the entire suite.
steveklabnik 16y ago
> testing feels "expensive" to me This is actually one of the bigger 'problems' with {T,B}DD; it's much harder to figure out how much time it takes to complete things. BDD: Wrote the test, wrote the feature, checked it in, wall clock time, 1 hour. Next week, something unrelated changes, my tests show right where the problem is, 10 minute fix. Total time: 1:10. No testing: wrote the feature, half an hour. Next week, something unrelated changes, breaks the feature, spend an hour fixing it. Total time: 1:30. And keep adding time when other things break later, too... In projects with a high test coverage, I almost never spend any time debugging. That doesn't mean it's easy to recognize the saved time, though.
- nickelplate 16y ago
  What about POUT (plain old unit testing)? Ie writing he test after the feature?
  - steveklabnik 16y ago
    This might be my own personal failing, but those tests will never get written. Once the feature works, the temptation to move onto the next thing is just too great. Also, by writing the test first, you ensure that the code is actually easy to test. This gets easier as you get more experienced with testing, but still.
    - demallien 16y ago
      Also, writing the test first helps in identifying a sane API, I personally consider this to be the single greatest advantage of TDD. I don't write as many tests as you do though, Steve. I generally stop once I have the nominal test cases in place, I don't test edge cases. They seem to tend to get hit by upstream unit tests anyway (I don't mock except where it is absolutely necessary). If I do hit a difficult-to-find bug, I use tests as one of my main debugging tools, writing tests for any code for which I have some doubts. By the end of a project, I generally have a fairly high code coverage for my tests, but I never feel like I'm just writing them for the methodology - they are either testing the nominal case, or testing to verify behaviour when tracking down a bug

felixge 16y ago

I hope I'm not giving you the wrong picture. We did have to cut features when rewriting our software (after node.js changed all APIs) using TDD. It does take more time to write tested code, and you do not get an advantage until you have the code in production. It is only at that point that you are seeing the returns on that investment. So generally you need to try to find a small enough product so you can build it using TDD. Also stay as simple as you can with your testing - even if it feels very repetitive. I'm only now thinking of adding more tools to my testing toolchain after I have a better understanding of the actual problems I run into often. About testing web APIs. You need both unit and system tests. The unit tests make sure your internal processing is solid. The system tests will directly make calls to the service. That won't always feel pretty, but it's the best way to approach it IMO.

boundlessdreamz 16y ago
How do you test something which requires browser interaction like oauth authentication? Edit: I know this may not be applicable in transloadit's case. I'm asking for a general strategy
- felixge 16y ago
  The core of our service is a REST API, so we don't have to deal with that much. For browser interaction we would probably use a combination of Selenium (for system tests), and maybe qunit for the frontend testing. But anyway, I am no expert on this so I can't give you solid advice.
  - johnwatson11218 16y ago
    I have used canoo webtest to test browser interaction that involved a log in to a deployed server.
- knuckle_cake 16y ago
  One method I've used (with Ruby) is use of FakeWeb, and collecting the various responses from oauth providers to test against. It's the closest I've seen to hitting the real services. If you're testing javascript, you'll need to use Selenium or something similar.
- ollysb 16y ago
  Webmock is an excellent ruby framework for solving this problem. My current app intereracts with both github and heroku and it's come in very handing for filling in what would otherwise be gaps. It's newer than fakeweb and is much better at letting you specify interactions on a per test basis rather than the entire suite.

steveklabnik 16y ago

> testing feels "expensive" to me This is actually one of the bigger 'problems' with {T,B}DD; it's much harder to figure out how much time it takes to complete things. BDD: Wrote the test, wrote the feature, checked it in, wall clock time, 1 hour. Next week, something unrelated changes, my tests show right where the problem is, 10 minute fix. Total time: 1:10. No testing: wrote the feature, half an hour. Next week, something unrelated changes, breaks the feature, spend an hour fixing it. Total time: 1:30. And keep adding time when other things break later, too... In projects with a high test coverage, I almost never spend any time debugging. That doesn't mean it's easy to recognize the saved time, though.

nickelplate 16y ago
What about POUT (plain old unit testing)? Ie writing he test after the feature?
- steveklabnik 16y ago
  This might be my own personal failing, but those tests will never get written. Once the feature works, the temptation to move onto the next thing is just too great. Also, by writing the test first, you ensure that the code is actually easy to test. This gets easier as you get more experienced with testing, but still.
  - demallien 16y ago
    Also, writing the test first helps in identifying a sane API, I personally consider this to be the single greatest advantage of TDD. I don't write as many tests as you do though, Steve. I generally stop once I have the nominal test cases in place, I don't test edge cases. They seem to tend to get hit by upstream unit tests anyway (I don't mock except where it is absolutely necessary). If I do hit a difficult-to-find bug, I use tests as one of my main debugging tools, writing tests for any code for which I have some doubts. By the end of a project, I generally have a fairly high code coverage for my tests, but I never feel like I'm just writing them for the methodology - they are either testing the nominal case, or testing to verify behaviour when tracking down a bug