3 ms·

So you stop the program immediately and allow no more transactions while you debug? Unlikely. Keep the data. Log that it's weird.

by xapata 9y ago

stouset 9y ago
What is your software development background? I ask because it would not have once occurred to me in the past fifteen years of software development that “enforcing database constraints” in any way implies shutting everything down and stopping transactions until the cause of a violation is rooted out. If you’re approaching it from the position of collecting raw data from sensors or something, that might make sense. But from the perspective of a webapp, allowing duplicate (or empty) usernames or failing to keep foreign key relations valid often violate basic assumptions of soundness that the application is based upon. In these cases, the right answer is to drop the data on the floor (loudly) so a developer can look at it. Inconveniencing a customer once here is infinitely better than inconveniencing them constantly due to the site being buggy or broken as a result of the invalid data, and prevents needing to migrate the data in the future which can actually be impossible to do correctly in some cases.
- xapata 9y ago
  > background Web dev for some years, then data science. > implies shutting everything down That's what "loudly" implied to me. Loud means crash, no? > basic assumptions of soundness I like to enforce that at the application layer, so that I can quickly swap storage without worrying. Whether I'm writing to database or a mock object, I'd hope the exceptions that arise in the application are the same. Things like unique index are easy to implement with a mock object, but more complex constraints I'd rather enforce in code that I find easier to read. > buggy or broken All I'm saying is not to be an absolutist. Some things are good to have as database constraints, others are good as application logic. I'm arguing against the idea that "validations are [never] enough." In contrast, database constraints are almost never enough to manage the complexity of data integrity.
  - mixmastamyk 9y ago
    App should not crash but return/log the database error, ideally translated to layman terms on the customer end. Logs sent high priority to dev team for fixing.
    - xapata 9y ago
      Indeed, I agree.
Slansitartop 9y ago
> So you stop the program immediately and allow no more transactions while you debug? Unlikely. You stop the transaction on a constraint violation, not usually the entire system. > Keep the data. Log that it's weird. I've worked on systems that do that, and they're nightmares. The problem with that style can be summed up in "if it doesn't look broke, people don't bother fixing it." No one actually watches those logs, and users don't try to fix data that was "accepted" with no errors. The result is bad data piles up, and often causes problems in other parts of the system. Maybe you (as a developer) love working on support tickets to manually fix bad data the users entered or a buggy process created, but I sure as hell don't. The solution to this is to fail fast: https://en.wikipedia.org/wiki/Fail-fast https://en.wikipedia.org/wiki/Fail-fast.
- deleted 9y ago
  [deleted]
- xapata 9y ago
  I think our practices are closer than you realize. Maybe I didn't explain myself well. The idea that troubles me is including more strict constraints in the database than in the application itself, because that indicates the application isn't fully tested. If the application is well tested, then it doesn't matter where the logic resides, db or app, so there's no standing to the claim that application logic is never enough. Also, there are many kinds of data. Transactions, sure, reject if it doesn't match. Telemetry data, I'd like to hang on to it regardless of how weird it seems.

So you stop the program immediately and allow no more transactions while you debug? Unlikely. Keep the data. Log that it's weird.

by xapata 9y ago

So you stop the program immediately and allow no more transactions while you debug? Unlikely. Keep the data. Log that it's weird.

stouset 9y ago

What is your software development background? I ask because it would not have once occurred to me in the past fifteen years of software development that “enforcing database constraints” in any way implies shutting everything down and stopping transactions until the cause of a violation is rooted out. If you’re approaching it from the position of collecting raw data from sensors or something, that might make sense. But from the perspective of a webapp, allowing duplicate (or empty) usernames or failing to keep foreign key relations valid often violate basic assumptions of soundness that the application is based upon. In these cases, the right answer is to drop the data on the floor (loudly) so a developer can look at it. Inconveniencing a customer once here is infinitely better than inconveniencing them constantly due to the site being buggy or broken as a result of the invalid data, and prevents needing to migrate the data in the future which can actually be impossible to do correctly in some cases.

xapata 9y ago
> background Web dev for some years, then data science. > implies shutting everything down That's what "loudly" implied to me. Loud means crash, no? > basic assumptions of soundness I like to enforce that at the application layer, so that I can quickly swap storage without worrying. Whether I'm writing to database or a mock object, I'd hope the exceptions that arise in the application are the same. Things like unique index are easy to implement with a mock object, but more complex constraints I'd rather enforce in code that I find easier to read. > buggy or broken All I'm saying is not to be an absolutist. Some things are good to have as database constraints, others are good as application logic. I'm arguing against the idea that "validations are [never] enough." In contrast, database constraints are almost never enough to manage the complexity of data integrity.
- mixmastamyk 9y ago
  App should not crash but return/log the database error, ideally translated to layman terms on the customer end. Logs sent high priority to dev team for fixing.
  - xapata 9y ago
    Indeed, I agree.

Slansitartop 9y ago

> So you stop the program immediately and allow no more transactions while you debug? Unlikely. You stop the transaction on a constraint violation, not usually the entire system. > Keep the data. Log that it's weird. I've worked on systems that do that, and they're nightmares. The problem with that style can be summed up in "if it doesn't look broke, people don't bother fixing it." No one actually watches those logs, and users don't try to fix data that was "accepted" with no errors. The result is bad data piles up, and often causes problems in other parts of the system. Maybe you (as a developer) love working on support tickets to manually fix bad data the users entered or a buggy process created, but I sure as hell don't. The solution to this is to fail fast: https://en.wikipedia.org/wiki/Fail-fast https://en.wikipedia.org/wiki/Fail-fast.

deleted 9y ago
[deleted]
xapata 9y ago
I think our practices are closer than you realize. Maybe I didn't explain myself well. The idea that troubles me is including more strict constraints in the database than in the application itself, because that indicates the application isn't fully tested. If the application is well tested, then it doesn't matter where the logic resides, db or app, so there's no standing to the claim that application logic is never enough. Also, there are many kinds of data. Transactions, sure, reject if it doesn't match. Telemetry data, I'd like to hang on to it regardless of how weird it seems.