4 ms·

Cassandra is designed to be eventually consistent; that means that if the node didn't yet get the update(s), a client will get stale data. This is basically the

by Annatar 10y ago

Cassandra is designed to be eventually consistent; that means that if the node didn't yet get the update(s), a client will get stale data. This is basically the same issue as with DNS. If I define that incomplete data equals to corrupt data, which I do, I am incapable of conceiving a scenario where corrupt data would be acceptable. Why would one define it so? Because I'm convicted that computers should always deliver correct data, and in my conviction, data can never be correct unless it is also complete. Therefore, atomicity or bust; therefore, Cassandra = false, and in such a case, milion writes per second is of no consolation.

amelius 10y ago
Sometimes there are no good (efficient) alternatives to eventual consistency. I guess it all depends on the level of complexity that a developer or architect is willing to accept.
- Annatar 10y ago
  Yep, correct, it comes down to acceptance. And I've yet to see a large scale scenario where it was not possible to implement software which couldn't be both correct and performant (I specialize in large scale applications for a living). Therefore, I do not accept the premise that it is impossible to design systems which are both correct and fast on a large scale, and neither should anyone else have to accept such a thing. It's just not right.
  - autonomousLD 10y ago
    >And I've yet to see a large scale scenario where it was not possible to implement software which couldn't be both correct and performant CAP theorem says otherwise. I suggest you try implementing something as simple as distributed counters.
malux85 10y ago
At Uber scale (or Facebook scale, or Google scale, or any non-trivial scale) you simply cannot say "Atomicity or bust", because there isn't a distributed database that can give you atomicity at 1M writes a second : That technology simply doesn't exist at this price point, or it would be 10x increase in server costs. You DONT need atomicity all the time, sometimes you do (Financial Transactions), but a lot of the time you dont: Facebook likes. Do they need to be atomic? Does it matter if a web request gets a stale "like count" and the number of likes for an item is 12,544 instead of the 12,545 on someones page view because the database isn't atomic? No, it doesn't matter. Uber GPS data, Uber writes the GPS locations as your trip progresses - does it matter that the last 1 or 2 locations might not be visible to the app or external reports for a few seconds? No, it doesn't matter. Ad-click data, Does it matter that the last 1 or 2 hundred impressions are not visible in a report who's absolute values are in the millions or 10s of millions? No, it doesn't matter - even though this can represent money, the value of a couple of hundred impressions missing on the report is chump change, not even worth thinking about. With the exception of financial transactions, some safety critical machine data, and a few other cases, atomicity is not required, and that's where these distributed databases shine.
- Annatar 10y ago
  You DONT need atomicity all the time, sometimes you do (Financial Transactions) Bingo. No atomicity, bad money or no money. I'd have a problem. I also have this extreme dislike of the "garbage in, garbage out" principle, so either what I'm delivering is correct, or I should be fired, because using computers at that point is pointless. If we don't care about correctness or completeness at all, then we just discounted two of the three reasons why computers exist at all. That leaves us with speed, and at that point, what use is it?
  - zimpenfish 10y ago
    But if Uber went the "correct and complete" route, which would cost them either a lot more or drastically reduce the number of writes, that would actively hamper their business. At what point do you consider it a valid tradeoff to keep the business alive?
    - Annatar 10y ago
      I would consider correctness to be critical to the survival of any business; therefore, it is my unshaken conviction that this is a thing were no compromises can be made. I'm deeply convicted about that.
      - OskarS 10y ago
        Then you're simply wrong. Spending 10x the money on making sure all data is atomically written and freshly read is idiotic if there's no problem with reading slightly stale data.
        Annatar 10y ago
        You are assuming that ten times more money needs to be spent, but without any real basis, that's nothing but a meaningless assertion. It doesn't have to be expensive to be fast, reliable and correct.
  - 10y ago
bpicolo 10y ago
Do keep in mind that's entirely irrelevant for uniquely-keyed data points. (append-only database usage)
- Annatar 10y ago
  If one is only ever going to append to a database and never read it or use data in it, then one does not even need to store the data being appended, let alone need a database. However, since the use case here appears to be storing GPS data, it would be reasonable to conclude that data is being actively consumed.
  - deleted 10y ago
    [deleted]
  - bpicolo 10y ago
    Reading is fine in an append-only datastore. That's my point. There's no stale reads if your data for a key that's never changed given quorum-style reads as far as I understand.

by Annatar 10y ago

amelius 10y ago

Sometimes there are no good (efficient) alternatives to eventual consistency. I guess it all depends on the level of complexity that a developer or architect is willing to accept.

Annatar 10y ago
Yep, correct, it comes down to acceptance. And I've yet to see a large scale scenario where it was not possible to implement software which couldn't be both correct and performant (I specialize in large scale applications for a living). Therefore, I do not accept the premise that it is impossible to design systems which are both correct and fast on a large scale, and neither should anyone else have to accept such a thing. It's just not right.
- autonomousLD 10y ago
  >And I've yet to see a large scale scenario where it was not possible to implement software which couldn't be both correct and performant CAP theorem says otherwise. I suggest you try implementing something as simple as distributed counters.

malux85 10y ago

At Uber scale (or Facebook scale, or Google scale, or any non-trivial scale) you simply cannot say "Atomicity or bust", because there isn't a distributed database that can give you atomicity at 1M writes a second : That technology simply doesn't exist at this price point, or it would be 10x increase in server costs. You DONT need atomicity all the time, sometimes you do (Financial Transactions), but a lot of the time you dont: Facebook likes. Do they need to be atomic? Does it matter if a web request gets a stale "like count" and the number of likes for an item is 12,544 instead of the 12,545 on someones page view because the database isn't atomic? No, it doesn't matter. Uber GPS data, Uber writes the GPS locations as your trip progresses - does it matter that the last 1 or 2 locations might not be visible to the app or external reports for a few seconds? No, it doesn't matter. Ad-click data, Does it matter that the last 1 or 2 hundred impressions are not visible in a report who's absolute values are in the millions or 10s of millions? No, it doesn't matter - even though this can represent money, the value of a couple of hundred impressions missing on the report is chump change, not even worth thinking about. With the exception of financial transactions, some safety critical machine data, and a few other cases, atomicity is not required, and that's where these distributed databases shine.

Annatar 10y ago
You DONT need atomicity all the time, sometimes you do (Financial Transactions) Bingo. No atomicity, bad money or no money. I'd have a problem. I also have this extreme dislike of the "garbage in, garbage out" principle, so either what I'm delivering is correct, or I should be fired, because using computers at that point is pointless. If we don't care about correctness or completeness at all, then we just discounted two of the three reasons why computers exist at all. That leaves us with speed, and at that point, what use is it?
- zimpenfish 10y ago
  But if Uber went the "correct and complete" route, which would cost them either a lot more or drastically reduce the number of writes, that would actively hamper their business. At what point do you consider it a valid tradeoff to keep the business alive?
  - Annatar 10y ago
    I would consider correctness to be critical to the survival of any business; therefore, it is my unshaken conviction that this is a thing were no compromises can be made. I'm deeply convicted about that.
    - OskarS 10y ago
      Then you're simply wrong. Spending 10x the money on making sure all data is atomically written and freshly read is idiotic if there's no problem with reading slightly stale data.
      - Annatar 10y ago
        You are assuming that ten times more money needs to be spent, but without any real basis, that's nothing but a meaningless assertion. It doesn't have to be expensive to be fast, reliable and correct.
- 10y ago

bpicolo 10y ago

Do keep in mind that's entirely irrelevant for uniquely-keyed data points. (append-only database usage)

Annatar 10y ago
If one is only ever going to append to a database and never read it or use data in it, then one does not even need to store the data being appended, let alone need a database. However, since the use case here appears to be storing GPS data, it would be reasonable to conclude that data is being actively consumed.
- deleted 10y ago
  [deleted]
- bpicolo 10y ago
  Reading is fine in an append-only datastore. That's my point. There's no stale reads if your data for a key that's never changed given quorum-style reads as far as I understand.