4 ms·

I find TDD impossible for all but the most trivial of coding tasks. As the article asks, what should the resulting API look like? The act of solving a problem

I find TDD impossible for all but the most trivial of coding tasks. As the article asks, what should the resulting API look like? The act of solving a problem gives me insight into the actual requirements, and the API results from that knowledge. Another issue I have with unit testing is that the cases which, for me at least, identify the most bugs pre-production are more integration-ish. Does the timeout setting for the HTTP API work as expected? Does this transaction really roll-back against an Oracle DB with the "thin" driver on JBoss? These are cases which are really hard to catch with traditional, use-the-UI testing, either manual or automated. I should read more about what others are doing for these semi-integration cases. A couple of years ago, I committed a test case called TestHowDatabasesWork because of a team argument about how Oracle handled concurrency and locking. It got a good laugh, but it also documented a few behaviors specific to the problem at hand.

btilly 17y ago
A couple of years ago, I committed a test case called TestHowDatabasesWork because of a team argument about how Oracle handled concurrency and locking. It got a good laugh, but it also documented a few behaviors specific to the problem at hand. There is a good possibility that in documenting what you saw working you failed to notice some important details of its optimistic locking policy. If you're ever again faced with such questions about Oracle I highly recommend picking up an appropriate book by Tom Kyte. For a developer in particular I would recommend http://www.amazon.com/exec/obidos/tg/detail/-/0072230657 http://www.amazon.com/exec/obidos/tg/detail/-/0072230657.
steveklabnik 17y ago
See, I really enjoy freezing the API early. The API should be very, very simple. I tend to code by designing top down, then coding from the bottom up. APIs should _not_ be from the perspective of "this is how I solve this problem" and should be from the perspective of "this solves my problem." For example, TempCalc tc = new TempCalc(); tc.f = 50; System.out.println("50 degrees farenheit is " + tc.convertToCelcius() + " degress celcius"); is bad design. Yes, you may need to get a farenheit value, and change it to celcius, but I don't care about setting internal state, then calling functions to change it. I just want an "fToC(int f)" function. This is an admittedly contrived example, but I see this principle in API design all the time. Everything should be from the perspective of the user, not the coder. "I'd love to use this api if it looked like this." Anyway... as far as that integration stuff goes, have you used Selenium? I'm not super up on Java testing stuff, as I'm a Rubyist, but I use it, and I know it has Java bindings of some kind...
- ShabbyDoo 17y ago
  Selenium 2 (the Google "Webdriver" stuff) works quite well and isn't specific to Java in any significant way. If you can bring up your UI with a bunch of mocks/stubs under it, Selenium is a great way to do automated regression testing of your web UI. On your API design comments: My issue with TDD is that it's implicitly big-design-up-front. I used to think I understood a problem before I stated coding, but I don't anymore. So, anything I design in the beginning ends up being modified (if not scrapped entirely). There's no better way for me to understand a problem than by trying to solve it. Also, for important enough APIs (public stuff, widely used in a codebase, or whatever), the design of the API is dependent on the problem being solved. Let's say you are building an API for a yet-to-be-developed photo storage SaaS. For any solution of significant sophistication, can you really think through all the issues that have API design implications before attempting to create the service itself? Is it better to stream back the bits of a photo found via a tag search or should a "ticket" in the form of a GUID be provided instead? I don't know, but I'll have a better idea once I spend some time solving the problem.
  - steveklabnik 17y ago
    Thanks for confirming on the Selenium thing. I guess it just comes down to personal taste, then. I feel that I can't understand how the coding should be laid out until after I've designed how it will be used.
CWuestefeld 17y ago
Agree. TDD in the middle tiers of the app is easy, but that's not where the difficulties lie. Testing the UI layer is quite difficult (at least in the absence of something like MVC, which too many developers can't handle and only want VB-like or WebForms-like coding). And in any paradigm I've ever heard of, real testing of the bottom layers of DB interaction is darned near impossible. As the OP asserts, imposing a well-defined testing regimen is where most of the bang for the buck derives.
- btilly 17y ago
  Testing all of the ins and outs of possible DB behavior is out of scope for most software projects. You need to assume your driver works until proven otherwise. As for testing interactions with the database, one approach that I've seen work is to have data in your test cases with which you build up database objects that you then interact with. If your classes all have appropriate helpers and appropriate ways of inserting reasonable defaults on fields not explicitly set in the testing code, this makes testing the expected database interactions fairly straightforward.
- steveklabnik 17y ago
  > And in any paradigm I've ever heard of, real testing of the bottom layers of DB interaction is darned near impossible. Is it that because the code is written in an untestable way? A pretty standard mocking approach should be able to test this.
  - mbrubeck 17y ago
    In my experience, problems "at the bottom" usually occur because underlying third-party components like the database (or the filesystem or the network stack or the remote server) don't behave the way you thought they did. They're buggy or undocumented, or you just misunderstood their specs. When you mock them away for testing, the mock objects act like you expect and don't have the same quirks as their real-life counterparts. If you don't mock them away, then you have hard-to-run or hard-to-reproduce tests, or tests that give different results with different instances of the external dependencies.

btilly 17y ago

A couple of years ago, I committed a test case called TestHowDatabasesWork because of a team argument about how Oracle handled concurrency and locking. It got a good laugh, but it also documented a few behaviors specific to the problem at hand. There is a good possibility that in documenting what you saw working you failed to notice some important details of its optimistic locking policy. If you're ever again faced with such questions about Oracle I highly recommend picking up an appropriate book by Tom Kyte. For a developer in particular I would recommend http://www.amazon.com/exec/obidos/tg/detail/-/0072230657 http://www.amazon.com/exec/obidos/tg/detail/-/0072230657.

steveklabnik 17y ago

See, I really enjoy freezing the API early. The API should be very, very simple. I tend to code by designing top down, then coding from the bottom up. APIs should _not_ be from the perspective of "this is how I solve this problem" and should be from the perspective of "this solves my problem." For example, TempCalc tc = new TempCalc(); tc.f = 50; System.out.println("50 degrees farenheit is " + tc.convertToCelcius() + " degress celcius"); is bad design. Yes, you may need to get a farenheit value, and change it to celcius, but I don't care about setting internal state, then calling functions to change it. I just want an "fToC(int f)" function. This is an admittedly contrived example, but I see this principle in API design all the time. Everything should be from the perspective of the user, not the coder. "I'd love to use this api if it looked like this." Anyway... as far as that integration stuff goes, have you used Selenium? I'm not super up on Java testing stuff, as I'm a Rubyist, but I use it, and I know it has Java bindings of some kind...

ShabbyDoo 17y ago
Selenium 2 (the Google "Webdriver" stuff) works quite well and isn't specific to Java in any significant way. If you can bring up your UI with a bunch of mocks/stubs under it, Selenium is a great way to do automated regression testing of your web UI. On your API design comments: My issue with TDD is that it's implicitly big-design-up-front. I used to think I understood a problem before I stated coding, but I don't anymore. So, anything I design in the beginning ends up being modified (if not scrapped entirely). There's no better way for me to understand a problem than by trying to solve it. Also, for important enough APIs (public stuff, widely used in a codebase, or whatever), the design of the API is dependent on the problem being solved. Let's say you are building an API for a yet-to-be-developed photo storage SaaS. For any solution of significant sophistication, can you really think through all the issues that have API design implications before attempting to create the service itself? Is it better to stream back the bits of a photo found via a tag search or should a "ticket" in the form of a GUID be provided instead? I don't know, but I'll have a better idea once I spend some time solving the problem.
- steveklabnik 17y ago
  Thanks for confirming on the Selenium thing. I guess it just comes down to personal taste, then. I feel that I can't understand how the coding should be laid out until after I've designed how it will be used.

CWuestefeld 17y ago

Agree. TDD in the middle tiers of the app is easy, but that's not where the difficulties lie. Testing the UI layer is quite difficult (at least in the absence of something like MVC, which too many developers can't handle and only want VB-like or WebForms-like coding). And in any paradigm I've ever heard of, real testing of the bottom layers of DB interaction is darned near impossible. As the OP asserts, imposing a well-defined testing regimen is where most of the bang for the buck derives.

btilly 17y ago
Testing all of the ins and outs of possible DB behavior is out of scope for most software projects. You need to assume your driver works until proven otherwise. As for testing interactions with the database, one approach that I've seen work is to have data in your test cases with which you build up database objects that you then interact with. If your classes all have appropriate helpers and appropriate ways of inserting reasonable defaults on fields not explicitly set in the testing code, this makes testing the expected database interactions fairly straightforward.
steveklabnik 17y ago
> And in any paradigm I've ever heard of, real testing of the bottom layers of DB interaction is darned near impossible. Is it that because the code is written in an untestable way? A pretty standard mocking approach should be able to test this.
- mbrubeck 17y ago
  In my experience, problems "at the bottom" usually occur because underlying third-party components like the database (or the filesystem or the network stack or the remote server) don't behave the way you thought they did. They're buggy or undocumented, or you just misunderstood their specs. When you mock them away for testing, the mock objects act like you expect and don't have the same quirks as their real-life counterparts. If you don't mock them away, then you have hard-to-run or hard-to-reproduce tests, or tests that give different results with different instances of the external dependencies.