3 ms·

Why would this be news? B+ trees do not scale, it's why log-structured merge trees were invented. Unfortunately, cross-table joins don't shard or scale well un

Why would this be news? B+ trees do not scale, it's why log-structured merge trees were invented. Unfortunately, cross-table joins don't shard or scale well unless the data is well suited to a large scale sharding (like "customer" or "region". You'll probably start doing "data lake" type frameworks for join calculations across shards. If you are sharding a database due to scale that modern big machines can't handle, you'll probably want to move your largest tables to cassandra or dynamo or other purely scalable approaches. "fastest-developing verticals" ... sigh.

_3u10 5y ago
Scaling isn’t some kind of universal property. For many problems Cassandra doesn’t scale. For many problems log structured merge trees don’t scale.
- ewuhic 5y ago
  Like for which exactly?
  - zbentley 5y ago
    Off the top of my head and in no particular order: - Super DELETE-heavy workloads. Tombstone cleanup can hurt. - Complex multidimensional indexes. Things that you'd use multiple semi-overlapping compound indexes on a single table for in a traditional RDBMS. - Operational pain of operating large Cassandra clusters. Recovering from multi-node failures while under high write load can be nail-biting as you watch the replication progress and the hint expiration "death clock" race for completion. - Even semi-frequent scans over huge (GBs in size) tables can cause unexpected hiccups in other ongoing CRUD operations. If the scans hit enough nodes/partitions, the impact on other workloads can be pervasive. - Emulating transactions (with an optimistic lock, full-cluster reads, and a cleanup program that expires interrupted transaction records) at scale can get costly. If you need this, cassandra was probably the wrong choice in the first place, but you asked for limitations and I have seen this one a couple of times.
    - AtlasBarfed 5y ago
      These are all quite true: Frequent DELETE: True, unless your data is time-windowed. The extension to this is frequent updates to the same "cell" of data, which also overloads the compaction and read paths. DELETEs have a big toll on relational databases at scale as well though, although SSDs mitigated a large part of those. DELETEs are very hard on B+tree indices without LSM features, and I think I've read that compaction issues exist in Postgres under heavy deletes: all those empty cells need to be cleaned up eventually... Complex Indexes: Well, you can just roll your own view table, although I don't know your exact use case. The original (heavily downvotes) comment about B+ trees is that SQL users are used to throwing a B+ tree index onto tables. You can't do that in cassandra, and I'm not aware of how you would do it in other distributed scenarios that aren't full-dataset-replication strategies. From what I've read, a B+ tree scaling and many other index scalings aren't Cassandra problems, they are fundamental CAP problems. Operational Pain: You are mostly right (Cassandra is a cranky beast), but once you get to heavy distributed data, it's all a PITA. At least Cassandra has some node loss tolerance. In a lot of times I hear people complain about Cassandra I ask, "well, did the database go down"? And they generally admit it stayed up. Scans suck: Yup. If you need scans, do a separate datacenter for the scans. On a sufficiently large distributed dataset, all scans / "big data queries" are approximations of current data state: there's no meaningful/practical way to tie to a transaction window. Is this a cassandra problem or a fundamental CAP problem? Transactions: Compare and Swap is about as good as it gets, and it's hard, and there's no way to span tables. Does anyone have a solution mathematically for sharded distributed data that isn't? As in is this a Cassandra problem or a fundamental CAP problem?
  - _3u10 5y ago
    The short comings of Cassandra are well documented on their homepage. If you need any of the things it doesn’t do well at it isn’t going to scale for you.
evanelias 5y ago
> Why would this be news? B+ trees do not scale, it's why log-structured merge trees were invented. I agree this article is low quality and often incorrect, but I don't see what the choice of data structure has to do with it. Sharding is orthogonal to data structure. For example, Facebook migrated their largest db fleet from sharded InnoDB (B+ trees) to sharded MyRocks (LSM). Still sharded either way.

_3u10 5y ago

Scaling isn’t some kind of universal property. For many problems Cassandra doesn’t scale. For many problems log structured merge trees don’t scale.

ewuhic 5y ago
Like for which exactly?
- zbentley 5y ago
  Off the top of my head and in no particular order: - Super DELETE-heavy workloads. Tombstone cleanup can hurt. - Complex multidimensional indexes. Things that you'd use multiple semi-overlapping compound indexes on a single table for in a traditional RDBMS. - Operational pain of operating large Cassandra clusters. Recovering from multi-node failures while under high write load can be nail-biting as you watch the replication progress and the hint expiration "death clock" race for completion. - Even semi-frequent scans over huge (GBs in size) tables can cause unexpected hiccups in other ongoing CRUD operations. If the scans hit enough nodes/partitions, the impact on other workloads can be pervasive. - Emulating transactions (with an optimistic lock, full-cluster reads, and a cleanup program that expires interrupted transaction records) at scale can get costly. If you need this, cassandra was probably the wrong choice in the first place, but you asked for limitations and I have seen this one a couple of times.
  - AtlasBarfed 5y ago
    These are all quite true: Frequent DELETE: True, unless your data is time-windowed. The extension to this is frequent updates to the same "cell" of data, which also overloads the compaction and read paths. DELETEs have a big toll on relational databases at scale as well though, although SSDs mitigated a large part of those. DELETEs are very hard on B+tree indices without LSM features, and I think I've read that compaction issues exist in Postgres under heavy deletes: all those empty cells need to be cleaned up eventually... Complex Indexes: Well, you can just roll your own view table, although I don't know your exact use case. The original (heavily downvotes) comment about B+ trees is that SQL users are used to throwing a B+ tree index onto tables. You can't do that in cassandra, and I'm not aware of how you would do it in other distributed scenarios that aren't full-dataset-replication strategies. From what I've read, a B+ tree scaling and many other index scalings aren't Cassandra problems, they are fundamental CAP problems. Operational Pain: You are mostly right (Cassandra is a cranky beast), but once you get to heavy distributed data, it's all a PITA. At least Cassandra has some node loss tolerance. In a lot of times I hear people complain about Cassandra I ask, "well, did the database go down"? And they generally admit it stayed up. Scans suck: Yup. If you need scans, do a separate datacenter for the scans. On a sufficiently large distributed dataset, all scans / "big data queries" are approximations of current data state: there's no meaningful/practical way to tie to a transaction window. Is this a cassandra problem or a fundamental CAP problem? Transactions: Compare and Swap is about as good as it gets, and it's hard, and there's no way to span tables. Does anyone have a solution mathematically for sharded distributed data that isn't? As in is this a Cassandra problem or a fundamental CAP problem?
- _3u10 5y ago
  The short comings of Cassandra are well documented on their homepage. If you need any of the things it doesn’t do well at it isn’t going to scale for you.

evanelias 5y ago

> Why would this be news? B+ trees do not scale, it's why log-structured merge trees were invented. I agree this article is low quality and often incorrect, but I don't see what the choice of data structure has to do with it. Sharding is orthogonal to data structure. For example, Facebook migrated their largest db fleet from sharded InnoDB (B+ trees) to sharded MyRocks (LSM). Still sharded either way.