4 ms·

Data consistency is one of those things that sounds like it matters but often doesn't. There's not much in Netflix's platform that screams data consistency is

Data consistency is one of those things that sounds like it matters but often doesn't. There's not much in Netflix's platform that screams data consistency is an ultra high priority. Any application that deploys active/active multi-region is by definition going to encounter scenarios where data loss is possible. There's just no way around CAP. I'd venture a guess that most applications have all sorts of race conditions that could cause data corruption. The fact of the matter is that almost nobody notices or even cares.

stickfigure 8y ago
A little birdie who worked at Grab once told me that they pretty much don't use transactions anywhere. So... maybe there are problem domains where you can get away without transactions, but I'm quite sure that a marketplace that arranges transportation for fee is not one of them. The next time you're standing in a monsoon waiting for the "coming" car that never comes, remember this post :-) Aside from that, I've found that even in non-mission-critical scenarios ("it's just porn!") it's incredibly convenient to have a limited number of states the system can be in. It makes debugging easier and reduces the number of edge cases ("why is this null??") you have to handle.
- flukus 8y ago
  > maybe there are problem domains where you can get away without transactions, but I'm quite sure that a marketplace that arranges transportation for fee is not one of them. I think you'd be surprised/alarmed at how little transactions actually get used in the software world. Not just on small systems where it doesn't matter but I've seen a complete absence of them in big financial ones handling billions of dollars worth transactions (the real world kind) a day. Some senior, highly paid people even defend this practice for performance reasons because they don't realize the performance cost of implicit transactions. And this is just the in process stuff where transactions are totally feasible, it get's even worse when you look at how much is moved around via csv files to FTP and excel sheets attached to emails. I've spent the last 2 weeks being paid to fix data consistency issues that should never have been issues in the first place. Maybe when we're teaching database theory we shouldn't start at select/join but at begin transaction/commit/rollback?
- davewritescode 8y ago
  Having spent a good amount of time fixing bugs related to code that fetches data with a consistency level of READ_UNCOMMITTED for "performance" reasons I can appreciate what transactions give you as an application developer. However, I would argue that transactions are overkill. What's the worst case scenario if I book a ride for Grab and my request gets corrupted? I'm guessing I'll see an error message and I'll have to re-request my ride.
  - thermodynthrway 8y ago
    I just set everything to level SERIAL and have never seen data corruption because of it. I guess that makes me old school, lol.

stickfigure 8y ago

A little birdie who worked at Grab once told me that they pretty much don't use transactions anywhere. So... maybe there are problem domains where you can get away without transactions, but I'm quite sure that a marketplace that arranges transportation for fee is not one of them. The next time you're standing in a monsoon waiting for the "coming" car that never comes, remember this post :-) Aside from that, I've found that even in non-mission-critical scenarios ("it's just porn!") it's incredibly convenient to have a limited number of states the system can be in. It makes debugging easier and reduces the number of edge cases ("why is this null??") you have to handle.

flukus 8y ago
> maybe there are problem domains where you can get away without transactions, but I'm quite sure that a marketplace that arranges transportation for fee is not one of them. I think you'd be surprised/alarmed at how little transactions actually get used in the software world. Not just on small systems where it doesn't matter but I've seen a complete absence of them in big financial ones handling billions of dollars worth transactions (the real world kind) a day. Some senior, highly paid people even defend this practice for performance reasons because they don't realize the performance cost of implicit transactions. And this is just the in process stuff where transactions are totally feasible, it get's even worse when you look at how much is moved around via csv files to FTP and excel sheets attached to emails. I've spent the last 2 weeks being paid to fix data consistency issues that should never have been issues in the first place. Maybe when we're teaching database theory we shouldn't start at select/join but at begin transaction/commit/rollback?
davewritescode 8y ago
Having spent a good amount of time fixing bugs related to code that fetches data with a consistency level of READ_UNCOMMITTED for "performance" reasons I can appreciate what transactions give you as an application developer. However, I would argue that transactions are overkill. What's the worst case scenario if I book a ride for Grab and my request gets corrupted? I'm guessing I'll see an error message and I'll have to re-request my ride.
- thermodynthrway 8y ago
  I just set everything to level SERIAL and have never seen data corruption because of it. I guess that makes me old school, lol.