4 ms·

Agree. TDD in the middle tiers of the app is easy, but that's not where the difficulties lie. Testing the UI layer is quite difficult (at least in the absence

Agree. TDD in the middle tiers of the app is easy, but that's not where the difficulties lie. Testing the UI layer is quite difficult (at least in the absence of something like MVC, which too many developers can't handle and only want VB-like or WebForms-like coding). And in any paradigm I've ever heard of, real testing of the bottom layers of DB interaction is darned near impossible. As the OP asserts, imposing a well-defined testing regimen is where most of the bang for the buck derives.

btilly 17y ago
Testing all of the ins and outs of possible DB behavior is out of scope for most software projects. You need to assume your driver works until proven otherwise. As for testing interactions with the database, one approach that I've seen work is to have data in your test cases with which you build up database objects that you then interact with. If your classes all have appropriate helpers and appropriate ways of inserting reasonable defaults on fields not explicitly set in the testing code, this makes testing the expected database interactions fairly straightforward.
steveklabnik 17y ago
> And in any paradigm I've ever heard of, real testing of the bottom layers of DB interaction is darned near impossible. Is it that because the code is written in an untestable way? A pretty standard mocking approach should be able to test this.
- mbrubeck 17y ago
  In my experience, problems "at the bottom" usually occur because underlying third-party components like the database (or the filesystem or the network stack or the remote server) don't behave the way you thought they did. They're buggy or undocumented, or you just misunderstood their specs. When you mock them away for testing, the mock objects act like you expect and don't have the same quirks as their real-life counterparts. If you don't mock them away, then you have hard-to-run or hard-to-reproduce tests, or tests that give different results with different instances of the external dependencies.
  - steveklabnik 17y ago
    Ah. That makes sense. I guess I've had the luxury of only working with MySQL, Postres, and sqlite, and have never run into bugs with them.
    - ShabbyDoo 17y ago
      I think the issue is that a mock of a DB doesn't imitate all the nuanced behaviors of the real thing. Transactionality, concurrency, isolation, error generation, etc. So, while you can prove functional correctness with a DBUnit/etc., you can't prove that the difficult stuff works.
      - steveklabnik 17y ago
        Isn't that the job of the DB package, and not of you, the user?
        ShabbyDoo 17y ago
        It's the job of the DB package to perform as specified. The problem is that one can write code that works correctly under the wrong assumptions of DB behavior. Here's an example I've experienced: We were building (to be purposefully vague) an auction system where an auction could end at any point in time. Bids flowed in via a queue. It didn't really matter within reason which order the bids were processed, but we had to make sure that a bid which caused an auction to end was processed in isolation from other bids that depended on the auction still being open. Our code relied upon the DB's locking behaviors (SELECT FOR UPDATE) to ensure that the code processing a particular bid laid claim to the state (auction status, etc) used for its work. All fine and good except that we didn't quite understand how Oracle implemented locking at various isolation levels. So, some code which would have worked against, say, DBUnit, failed under Oracle. This wasn't a failure of the DB, but we wouldn't have caught this problem unless we ran an integration test.
        steveklabnik 17y ago
- btilly 17y ago
  Suppose your code needs to interact with stored procedures in the database. How does a standard mocking approach in your code help you test the stored procedures?

btilly 17y ago

Testing all of the ins and outs of possible DB behavior is out of scope for most software projects. You need to assume your driver works until proven otherwise. As for testing interactions with the database, one approach that I've seen work is to have data in your test cases with which you build up database objects that you then interact with. If your classes all have appropriate helpers and appropriate ways of inserting reasonable defaults on fields not explicitly set in the testing code, this makes testing the expected database interactions fairly straightforward.

steveklabnik 17y ago

> And in any paradigm I've ever heard of, real testing of the bottom layers of DB interaction is darned near impossible. Is it that because the code is written in an untestable way? A pretty standard mocking approach should be able to test this.

mbrubeck 17y ago
In my experience, problems "at the bottom" usually occur because underlying third-party components like the database (or the filesystem or the network stack or the remote server) don't behave the way you thought they did. They're buggy or undocumented, or you just misunderstood their specs. When you mock them away for testing, the mock objects act like you expect and don't have the same quirks as their real-life counterparts. If you don't mock them away, then you have hard-to-run or hard-to-reproduce tests, or tests that give different results with different instances of the external dependencies.
- steveklabnik 17y ago
  Ah. That makes sense. I guess I've had the luxury of only working with MySQL, Postres, and sqlite, and have never run into bugs with them.
  - ShabbyDoo 17y ago
    I think the issue is that a mock of a DB doesn't imitate all the nuanced behaviors of the real thing. Transactionality, concurrency, isolation, error generation, etc. So, while you can prove functional correctness with a DBUnit/etc., you can't prove that the difficult stuff works.
    - steveklabnik 17y ago
      Isn't that the job of the DB package, and not of you, the user?
      - ShabbyDoo 17y ago
        It's the job of the DB package to perform as specified. The problem is that one can write code that works correctly under the wrong assumptions of DB behavior. Here's an example I've experienced: We were building (to be purposefully vague) an auction system where an auction could end at any point in time. Bids flowed in via a queue. It didn't really matter within reason which order the bids were processed, but we had to make sure that a bid which caused an auction to end was processed in isolation from other bids that depended on the auction still being open. Our code relied upon the DB's locking behaviors (SELECT FOR UPDATE) to ensure that the code processing a particular bid laid claim to the state (auction status, etc) used for its work. All fine and good except that we didn't quite understand how Oracle implemented locking at various isolation levels. So, some code which would have worked against, say, DBUnit, failed under Oracle. This wasn't a failure of the DB, but we wouldn't have caught this problem unless we ran an integration test.
        steveklabnik 17y ago
btilly 17y ago
Suppose your code needs to interact with stored procedures in the database. How does a standard mocking approach in your code help you test the stored procedures?