4 ms·

To be honest, most people don’t need high levels of isolation. Most people aren’t doing payments. I am not sure I would use an open source db for payments, tho

To be honest, most people don’t need high levels of isolation. Most people aren’t doing payments. I am not sure I would use an open source db for payments, though I have not needed to do the research. Even in eCommerce, the most contention will be for inventory. Most SaaS businesses have low contention, for example. And errors are typically non serious. When I’ve worked in non-transaction based data stores, I’ve learned to find and fix errors in data. That said, I HATE WHEN TOOLS LIE to developers. They should definitely do what they say. And different databases should have the same meanings for the same isolation levels. BTW: this is how I got involved in Firebug back in the day… I noticed that it was not telling the truth when debugging my code and that upset me to no end. I sunk time and effort so others wouldn’t distrust their debugging tools.

deleted 3y ago
[deleted]
et1337 3y ago
Proper isolation is required to do anything. Every system has invariants that need to be enforced. The consequences of those breaking may vary in severity, but they will break, and someone will have to go in and fix it.
- deleted 3y ago
  [deleted]
- sroussey 3y ago
  Going in and fixing may be the cheapest and most scalable option. And having that muscle built will have the benefit that when stuff gets in a bad state from poor logic or bugs in app code anyhow, you can deal. Certainly aim for perfect, but in most cases perfection itself is too high a cost.
dathinab 3y ago
The problem is that in my experience a lot of devs thing SQL transactions with the read committed transaction level given them guarantees they do not. Then the DB gets into a bad state. And then they switch to nosql because SQL "doesn't work anyway". :facepalm: (I have seen that story more then once.) Most applications can get away with read committed + a sprinkle of row level locks (mainly `for update`). If not I prefer (in postgres) to use full serializable snapshot isolation with scaffolds which automatically setup/comitts/aborts and retries transactions, setting read only and deferrable as needed etc. Sadly it's often not viable.
hinkley 3y ago
Telecom and credit card companies have understood levels of concurrent systems that most of us have only just begun to appreciate. The paper, “Your coffee shop doesn’t use two-phase commit” was making the rounds sometime around 2006-7, and it really helped crystallize things for a lot of people. Motivational papers and speeches often find one quadrant of the “known” continuum and do well in them. They can explain a problem that nobody knows about in a way that makes them care. They can draw attention to something that you have felt but could never express. The frisson here can be palpable. Or they can explain something you’ve always known in a way that you see it again, perhaps giving you new ways to bring others deeper into the Known Knowns quadrant. This one bridged several. It worked for people who hadn’t heard of it, and it facilitated conversations with people who were going blue in the face from discussing it and getting little traction. https://ieeexplore.ieee.org/document/1407829 https://ieeexplore.ieee.org/document/1407829
- leksak 3y ago
  This was great. Any other good short reads like this on technical topics? Doesn't just have to be DB related
  - hinkley 3y ago
    I’m not great at this kind of question, but off the top of my head, https://www.joelonsoftware.com/2002/01/06/fire-and-motion/ https://www.joelonsoftware.com/2002/01/06/fire-and-motion/ http://www.norvig.com/sudoku.html http://www.norvig.com/sudoku.html
- GauntletWizard 3y ago
  Telecom and Credit Card companies are building with the same shitty tech stacks that everyone else is. There's not institutional knowledge in these places; There's a lot of smart people who used to work there, who did understand those things, but it's long been forgotten and replaced.
  - hinkley 3y ago
    You might be right. It’s not always the innovators who have the best stuff anyway. They have Dancing Bear problems. It’s not that the bear dances well, but that it dances at all. Someone else has to come up with better dancers and better dances.
nilamo 3y ago
> I am not sure I would use an open source db for payments This suggests a level of mistrust in postgres or a dozen other high quality, open source, production grade software. ...why?
- giovannibonetti 3y ago
  I worked in a million customers Fintech where most of the data was in Postgres and it worked great. Wouldn't change anything in that regard if I had to do it again.
otabdeveloper4 3y ago
> I would use an open source db for payments The idea that you're using something for mission-critical things and can't even look at the source code seems terrifying and absolutely nuts to me.
- tatersolid 3y ago
  UEFI, CPU microcode, SSD firmware, TPM/HSM code… you lost the “everything critical must be open source” battle when you bought/rented your server.
  - comex 3y ago
    (Not the OP.) “Terrifying and absolutely nuts” may be over the top, but in my experience it can be extremely helpful to have the source code to as many components as possible when something goes wrong. Sometimes you find a bug in one of those components. Sometimes the bug turns out to be in your own code, but attaching a debugger to one of those components is still the quickest way to track it down (e.g. by letting you determine what exactly is triggering some vague error message). So all else being equal, I find it a lot easier to trust open-source code.
  - otabdeveloper4 3y ago
    Yes, and I still stand by my "terrifying and absolutely nuts" opinion. Also, there is a huge difference of category here - the things you listed are interchangeable commodities. You can always use some other storage mechanism instead of a misbehaving SSD. Software like Oracle or Db2 aren't, once you're locked into them, you can't leave.

deleted 3y ago

[deleted]

et1337 3y ago

Proper isolation is required to do anything. Every system has invariants that need to be enforced. The consequences of those breaking may vary in severity, but they will break, and someone will have to go in and fix it.

deleted 3y ago
[deleted]
sroussey 3y ago
Going in and fixing may be the cheapest and most scalable option. And having that muscle built will have the benefit that when stuff gets in a bad state from poor logic or bugs in app code anyhow, you can deal. Certainly aim for perfect, but in most cases perfection itself is too high a cost.

dathinab 3y ago

The problem is that in my experience a lot of devs thing SQL transactions with the read committed transaction level given them guarantees they do not. Then the DB gets into a bad state. And then they switch to nosql because SQL "doesn't work anyway". :facepalm: (I have seen that story more then once.) Most applications can get away with read committed + a sprinkle of row level locks (mainly `for update`). If not I prefer (in postgres) to use full serializable snapshot isolation with scaffolds which automatically setup/comitts/aborts and retries transactions, setting read only and deferrable as needed etc. Sadly it's often not viable.

hinkley 3y ago

Telecom and credit card companies have understood levels of concurrent systems that most of us have only just begun to appreciate. The paper, “Your coffee shop doesn’t use two-phase commit” was making the rounds sometime around 2006-7, and it really helped crystallize things for a lot of people. Motivational papers and speeches often find one quadrant of the “known” continuum and do well in them. They can explain a problem that nobody knows about in a way that makes them care. They can draw attention to something that you have felt but could never express. The frisson here can be palpable. Or they can explain something you’ve always known in a way that you see it again, perhaps giving you new ways to bring others deeper into the Known Knowns quadrant. This one bridged several. It worked for people who hadn’t heard of it, and it facilitated conversations with people who were going blue in the face from discussing it and getting little traction. https://ieeexplore.ieee.org/document/1407829 https://ieeexplore.ieee.org/document/1407829

leksak 3y ago
This was great. Any other good short reads like this on technical topics? Doesn't just have to be DB related
- hinkley 3y ago
  I’m not great at this kind of question, but off the top of my head, https://www.joelonsoftware.com/2002/01/06/fire-and-motion/ https://www.joelonsoftware.com/2002/01/06/fire-and-motion/ http://www.norvig.com/sudoku.html http://www.norvig.com/sudoku.html
GauntletWizard 3y ago
Telecom and Credit Card companies are building with the same shitty tech stacks that everyone else is. There's not institutional knowledge in these places; There's a lot of smart people who used to work there, who did understand those things, but it's long been forgotten and replaced.
- hinkley 3y ago
  You might be right. It’s not always the innovators who have the best stuff anyway. They have Dancing Bear problems. It’s not that the bear dances well, but that it dances at all. Someone else has to come up with better dancers and better dances.

nilamo 3y ago

> I am not sure I would use an open source db for payments This suggests a level of mistrust in postgres or a dozen other high quality, open source, production grade software. ...why?

giovannibonetti 3y ago
I worked in a million customers Fintech where most of the data was in Postgres and it worked great. Wouldn't change anything in that regard if I had to do it again.

otabdeveloper4 3y ago

> I would use an open source db for payments The idea that you're using something for mission-critical things and can't even look at the source code seems terrifying and absolutely nuts to me.

tatersolid 3y ago
UEFI, CPU microcode, SSD firmware, TPM/HSM code… you lost the “everything critical must be open source” battle when you bought/rented your server.
- comex 3y ago
  (Not the OP.) “Terrifying and absolutely nuts” may be over the top, but in my experience it can be extremely helpful to have the source code to as many components as possible when something goes wrong. Sometimes you find a bug in one of those components. Sometimes the bug turns out to be in your own code, but attaching a debugger to one of those components is still the quickest way to track it down (e.g. by letting you determine what exactly is triggering some vague error message). So all else being equal, I find it a lot easier to trust open-source code.
- otabdeveloper4 3y ago
  Yes, and I still stand by my "terrifying and absolutely nuts" opinion. Also, there is a huge difference of category here - the things you listed are interchangeable commodities. You can always use some other storage mechanism instead of a misbehaving SSD. Software like Oracle or Db2 aren't, once you're locked into them, you can't leave.