5 ms·

In my experience, problems "at the bottom" usually occur because underlying third-party components like the database (or the filesystem or the network stack or

In my experience, problems "at the bottom" usually occur because underlying third-party components like the database (or the filesystem or the network stack or the remote server) don't behave the way you thought they did. They're buggy or undocumented, or you just misunderstood their specs. When you mock them away for testing, the mock objects act like you expect and don't have the same quirks as their real-life counterparts. If you don't mock them away, then you have hard-to-run or hard-to-reproduce tests, or tests that give different results with different instances of the external dependencies.

steveklabnik 17y ago
Ah. That makes sense. I guess I've had the luxury of only working with MySQL, Postres, and sqlite, and have never run into bugs with them.
- ShabbyDoo 17y ago
  I think the issue is that a mock of a DB doesn't imitate all the nuanced behaviors of the real thing. Transactionality, concurrency, isolation, error generation, etc. So, while you can prove functional correctness with a DBUnit/etc., you can't prove that the difficult stuff works.
  - steveklabnik 17y ago
    Isn't that the job of the DB package, and not of you, the user?
    - ShabbyDoo 17y ago
      It's the job of the DB package to perform as specified. The problem is that one can write code that works correctly under the wrong assumptions of DB behavior. Here's an example I've experienced: We were building (to be purposefully vague) an auction system where an auction could end at any point in time. Bids flowed in via a queue. It didn't really matter within reason which order the bids were processed, but we had to make sure that a bid which caused an auction to end was processed in isolation from other bids that depended on the auction still being open. Our code relied upon the DB's locking behaviors (SELECT FOR UPDATE) to ensure that the code processing a particular bid laid claim to the state (auction status, etc) used for its work. All fine and good except that we didn't quite understand how Oracle implemented locking at various isolation levels. So, some code which would have worked against, say, DBUnit, failed under Oracle. This wasn't a failure of the DB, but we wouldn't have caught this problem unless we ran an integration test.
      - steveklabnik 17y ago
        Ah. Interesting. Databases have always been one of my weaker points, and lately, ORMs have allowed me to be pretty lazy on that front. Thanks for walking me through that.
    - logicalmind 17y ago
      No, and I'll give you an example. Take a simple relational table with some kind of constraint. Begin a transaction and insert some data into this table that violates the constraint. What happens? Guess what, it depends on the db. Some databases/drivers will throw an exception immediately upon insert. Some will wait until the commit has occurred to throw the exception. Subtle differences like these exist all across the various relational db world. Not to mention that they all do locking and concurrency differently, which is probably the biggest problem.
zb 17y ago
Interesting - in the embedded software world we have the exact same problem with hardware. Mocking the hardware can confirm the internal consistency of your ideas about how it works, but most problems stem from your ideas being inconsistent with how it actually works. Until now I always assumed that the environment from which TDD emerged (presumably database-driven "Enterprise" apps) was different in a way that made it seem like a better idea than it does to me, but your comment is causing me to seriously rethink that assumption.

steveklabnik 17y ago

Ah. That makes sense. I guess I've had the luxury of only working with MySQL, Postres, and sqlite, and have never run into bugs with them.

ShabbyDoo 17y ago
I think the issue is that a mock of a DB doesn't imitate all the nuanced behaviors of the real thing. Transactionality, concurrency, isolation, error generation, etc. So, while you can prove functional correctness with a DBUnit/etc., you can't prove that the difficult stuff works.
- steveklabnik 17y ago
  Isn't that the job of the DB package, and not of you, the user?
  - ShabbyDoo 17y ago
    It's the job of the DB package to perform as specified. The problem is that one can write code that works correctly under the wrong assumptions of DB behavior. Here's an example I've experienced: We were building (to be purposefully vague) an auction system where an auction could end at any point in time. Bids flowed in via a queue. It didn't really matter within reason which order the bids were processed, but we had to make sure that a bid which caused an auction to end was processed in isolation from other bids that depended on the auction still being open. Our code relied upon the DB's locking behaviors (SELECT FOR UPDATE) to ensure that the code processing a particular bid laid claim to the state (auction status, etc) used for its work. All fine and good except that we didn't quite understand how Oracle implemented locking at various isolation levels. So, some code which would have worked against, say, DBUnit, failed under Oracle. This wasn't a failure of the DB, but we wouldn't have caught this problem unless we ran an integration test.
    - steveklabnik 17y ago
      Ah. Interesting. Databases have always been one of my weaker points, and lately, ORMs have allowed me to be pretty lazy on that front. Thanks for walking me through that.
  - logicalmind 17y ago
    No, and I'll give you an example. Take a simple relational table with some kind of constraint. Begin a transaction and insert some data into this table that violates the constraint. What happens? Guess what, it depends on the db. Some databases/drivers will throw an exception immediately upon insert. Some will wait until the commit has occurred to throw the exception. Subtle differences like these exist all across the various relational db world. Not to mention that they all do locking and concurrency differently, which is probably the biggest problem.

zb 17y ago

Interesting - in the embedded software world we have the exact same problem with hardware. Mocking the hardware can confirm the internal consistency of your ideas about how it works, but most problems stem from your ideas being inconsistent with how it actually works. Until now I always assumed that the environment from which TDD emerged (presumably database-driven "Enterprise" apps) was different in a way that made it seem like a better idea than it does to me, but your comment is causing me to seriously rethink that assumption.