4 ms·

At Uber scale (or Facebook scale, or Google scale, or any non-trivial scale) you simply cannot say "Atomicity or bust", because there isn't a distributed databa

by malux85 10y ago

At Uber scale (or Facebook scale, or Google scale, or any non-trivial scale) you simply cannot say "Atomicity or bust", because there isn't a distributed database that can give you atomicity at 1M writes a second : That technology simply doesn't exist at this price point, or it would be 10x increase in server costs. You DONT need atomicity all the time, sometimes you do (Financial Transactions), but a lot of the time you dont: Facebook likes. Do they need to be atomic? Does it matter if a web request gets a stale "like count" and the number of likes for an item is 12,544 instead of the 12,545 on someones page view because the database isn't atomic? No, it doesn't matter. Uber GPS data, Uber writes the GPS locations as your trip progresses - does it matter that the last 1 or 2 locations might not be visible to the app or external reports for a few seconds? No, it doesn't matter. Ad-click data, Does it matter that the last 1 or 2 hundred impressions are not visible in a report who's absolute values are in the millions or 10s of millions? No, it doesn't matter - even though this can represent money, the value of a couple of hundred impressions missing on the report is chump change, not even worth thinking about. With the exception of financial transactions, some safety critical machine data, and a few other cases, atomicity is not required, and that's where these distributed databases shine.

Annatar 10y ago
You DONT need atomicity all the time, sometimes you do (Financial Transactions) Bingo. No atomicity, bad money or no money. I'd have a problem. I also have this extreme dislike of the "garbage in, garbage out" principle, so either what I'm delivering is correct, or I should be fired, because using computers at that point is pointless. If we don't care about correctness or completeness at all, then we just discounted two of the three reasons why computers exist at all. That leaves us with speed, and at that point, what use is it?
- zimpenfish 10y ago
  But if Uber went the "correct and complete" route, which would cost them either a lot more or drastically reduce the number of writes, that would actively hamper their business. At what point do you consider it a valid tradeoff to keep the business alive?
  - Annatar 10y ago
    I would consider correctness to be critical to the survival of any business; therefore, it is my unshaken conviction that this is a thing were no compromises can be made. I'm deeply convicted about that.
    - OskarS 10y ago
      Then you're simply wrong. Spending 10x the money on making sure all data is atomically written and freshly read is idiotic if there's no problem with reading slightly stale data.
      - Annatar 10y ago
        You are assuming that ten times more money needs to be spent, but without any real basis, that's nothing but a meaningless assertion. It doesn't have to be expensive to be fast, reliable and correct.
- wpietri 10y ago
  > the three reasons why computers exist at all In a commercial context, computers exist to serve human needs. If increasing correctness or completeness serves neither a user need nor a business need, then the work to do that is just developer self-indulgence. And honestly, I don't have a problem with a little developer self-indulgence now and again. Sometimes I'll just clean up something because it's ugly and I don't like it. But it's important to recognize when we're doing that, because it's decidedly not what we're paid for.
- snowwrestler 10y ago
  It's easy to sit back on principle and point out all the ways that other people's work fails to live up to your ideals. It's a lot harder to go out and build something that actually delivers your ideals. If you think a distributed database should provide full correct completeness from all nodes while hitting 1 million writes per second, go build one that does it. If you succeed, not only will you make yourself happy, you will make yourself a shitload of money.
  - Annatar 10y ago
    If you think a distributed database should provide full correct completeness from all nodes while hitting 1 million writes per second, go build one that does it. I do that for a living. If I didn't, I would not have commented.
morgante 10y ago
> You DONT need atomicity all the time, sometimes you do (Financial Transactions), but a lot of the time you dont: In fact, financial transactions rather famously do not implement strict consistency.
ddorian43 10y ago
Actually most of those companies run consistent systems compared to cassandra stuff. They may have a kafka-log at the beginning but mostly consistent dbs at the other end.
deleted 10y ago
[deleted]
evanelias 10y ago
> Facebook likes. Do they need to be atomic? Does it matter if a web request gets a stale "like count" and the number of likes for an item is 12,544 > instead of the 12,545 on someones page view because the database isn't atomic? > No, it doesn't matter. Actually, it does matter. Atomicity isn't the specific issue there, but rather it's read-after-write consistency guarantees that are important. That is, product-data writes need to be immediately consistent (synchronous) in the context of whichever data store is serving subsequent read requests for the same user/session. Read-after-write consistency is essential for social networks. If you take an action and then click a new link -- or even just refresh the current page -- the effect of your action definitely needs to still be visible. Facebook doesn't use Cassandra, fwiw.

by malux85 10y ago

Annatar 10y ago

You DONT need atomicity all the time, sometimes you do (Financial Transactions) Bingo. No atomicity, bad money or no money. I'd have a problem. I also have this extreme dislike of the "garbage in, garbage out" principle, so either what I'm delivering is correct, or I should be fired, because using computers at that point is pointless. If we don't care about correctness or completeness at all, then we just discounted two of the three reasons why computers exist at all. That leaves us with speed, and at that point, what use is it?

zimpenfish 10y ago
But if Uber went the "correct and complete" route, which would cost them either a lot more or drastically reduce the number of writes, that would actively hamper their business. At what point do you consider it a valid tradeoff to keep the business alive?
- Annatar 10y ago
  I would consider correctness to be critical to the survival of any business; therefore, it is my unshaken conviction that this is a thing were no compromises can be made. I'm deeply convicted about that.
  - OskarS 10y ago
    Then you're simply wrong. Spending 10x the money on making sure all data is atomically written and freshly read is idiotic if there's no problem with reading slightly stale data.
    - Annatar 10y ago
      You are assuming that ten times more money needs to be spent, but without any real basis, that's nothing but a meaningless assertion. It doesn't have to be expensive to be fast, reliable and correct.
wpietri 10y ago
> the three reasons why computers exist at all In a commercial context, computers exist to serve human needs. If increasing correctness or completeness serves neither a user need nor a business need, then the work to do that is just developer self-indulgence. And honestly, I don't have a problem with a little developer self-indulgence now and again. Sometimes I'll just clean up something because it's ugly and I don't like it. But it's important to recognize when we're doing that, because it's decidedly not what we're paid for.
snowwrestler 10y ago
It's easy to sit back on principle and point out all the ways that other people's work fails to live up to your ideals. It's a lot harder to go out and build something that actually delivers your ideals. If you think a distributed database should provide full correct completeness from all nodes while hitting 1 million writes per second, go build one that does it. If you succeed, not only will you make yourself happy, you will make yourself a shitload of money.
- Annatar 10y ago
  If you think a distributed database should provide full correct completeness from all nodes while hitting 1 million writes per second, go build one that does it. I do that for a living. If I didn't, I would not have commented.

morgante 10y ago

> You DONT need atomicity all the time, sometimes you do (Financial Transactions), but a lot of the time you dont: In fact, financial transactions rather famously do not implement strict consistency.

ddorian43 10y ago

Actually most of those companies run consistent systems compared to cassandra stuff. They may have a kafka-log at the beginning but mostly consistent dbs at the other end.

deleted 10y ago

[deleted]

evanelias 10y ago

> Facebook likes. Do they need to be atomic? Does it matter if a web request gets a stale "like count" and the number of likes for an item is 12,544 > instead of the 12,545 on someones page view because the database isn't atomic? > No, it doesn't matter. Actually, it does matter. Atomicity isn't the specific issue there, but rather it's read-after-write consistency guarantees that are important. That is, product-data writes need to be immediately consistent (synchronous) in the context of whichever data store is serving subsequent read requests for the same user/session. Read-after-write consistency is essential for social networks. If you take an action and then click a new link -- or even just refresh the current page -- the effect of your action definitely needs to still be visible. Facebook doesn't use Cassandra, fwiw.