3 ms·

How does this stack up against HBase and Cassandra, which seem to have gotten traction already in the same areas that FoundationDB seems best suited for?

Bahamut 9y ago
An interesting fact is that Apple is probably one of the biggest users, if not the biggest user, of Cassandra out there. Can't speak to HBase, but one thing Cassandra doesn't guarantee is ACID - I've seen some data consistency issues that has arisen from Cassandra in our usage, although it hasn't been a huge problem for us. That difference alone probably brings a lot of value to FoundationDB.
- Joeri 9y ago
  My understanding of hbase (which could be wrong) is that writes of a particular key always go to the region master first, so if you read from the master you always get the latest value of a key. The tricky part is that when the region master goes offline another one needs to take its place, and you can get inconsistency or unavailability depending on how it is set up. I’d like to see a deep dive of how foundationdb handles this. It has to trade-off consistency for availability at some point and it would be nice to know exactly where.
  - poooogles 9y ago
    From what I understand that shouldn't happen as writes are written to a WAL on HDFS (been a while since I've used it). From my memory writes within a row are atomic. It seems to pass Jepsen as well [1]. 1. https://www.google.co.uk/amp/s/yokota.blog/2015/09/30/call-me-maybe-hbase/amp/ https://www.google.co.uk/amp/s/yokota.blog/2015/09/30/call-m...
    - deleted 9y ago
      [deleted]
  - necubi 9y ago
    By default in HBase, each region is served by a single region server. Writes and reads always go to that regionserver, and are consistent across a single row (HBase is a CP system). In this mode, when a regionserver goes offline its regions are (eventually) reassigned to a different region server. During this process the region is unavailable, but there's no loss of consistency. Hbase 0.98 (I think?) introduced a feature called "timeline consistency" that allows reads from replica regionservers. This has to be enabled for a table and has to be specified on the query side. If it is, you have the option of falling back to the replica if the primary doesn't respond within a deadline. This may be a good tradeoff if you value availability over consistency.
voidmain 9y ago
HBase and Cassandra both provide much weaker guarantees. At best they can support compare-and-set operations that are local to a single region/"row", whereas FoundationDB lets you do optimistic transactions (and consistent read snapshots) on the entire database. And at least in cassandra even getting that level of consistency comes at a big performance cost. The benefit of ACID transactions isn't just safety at the application level, it's the ability to compose abstractions and build complex data models on top of simple ones. For example, indexes in FoundationDB are typically more scalable than in the majority of other systems, where index queries often have to be broadcast to all the systems in a cluster. Yet FoundationDB doesn't even have indexes as a feature - higher layers build and maintain index invariants using transactions. FoundationDB is also just really reliable and fault tolerant. Its testing story is drastically better than what the teams building these other products are doing.

Bahamut 9y ago

An interesting fact is that Apple is probably one of the biggest users, if not the biggest user, of Cassandra out there. Can't speak to HBase, but one thing Cassandra doesn't guarantee is ACID - I've seen some data consistency issues that has arisen from Cassandra in our usage, although it hasn't been a huge problem for us. That difference alone probably brings a lot of value to FoundationDB.

Joeri 9y ago
My understanding of hbase (which could be wrong) is that writes of a particular key always go to the region master first, so if you read from the master you always get the latest value of a key. The tricky part is that when the region master goes offline another one needs to take its place, and you can get inconsistency or unavailability depending on how it is set up. I’d like to see a deep dive of how foundationdb handles this. It has to trade-off consistency for availability at some point and it would be nice to know exactly where.
- poooogles 9y ago
  From what I understand that shouldn't happen as writes are written to a WAL on HDFS (been a while since I've used it). From my memory writes within a row are atomic. It seems to pass Jepsen as well [1]. 1. https://www.google.co.uk/amp/s/yokota.blog/2015/09/30/call-me-maybe-hbase/amp/ https://www.google.co.uk/amp/s/yokota.blog/2015/09/30/call-m...
  - deleted 9y ago
    [deleted]
- necubi 9y ago
  By default in HBase, each region is served by a single region server. Writes and reads always go to that regionserver, and are consistent across a single row (HBase is a CP system). In this mode, when a regionserver goes offline its regions are (eventually) reassigned to a different region server. During this process the region is unavailable, but there's no loss of consistency. Hbase 0.98 (I think?) introduced a feature called "timeline consistency" that allows reads from replica regionservers. This has to be enabled for a table and has to be specified on the query side. If it is, you have the option of falling back to the replica if the primary doesn't respond within a deadline. This may be a good tradeoff if you value availability over consistency.

voidmain 9y ago

HBase and Cassandra both provide much weaker guarantees. At best they can support compare-and-set operations that are local to a single region/"row", whereas FoundationDB lets you do optimistic transactions (and consistent read snapshots) on the entire database. And at least in cassandra even getting that level of consistency comes at a big performance cost. The benefit of ACID transactions isn't just safety at the application level, it's the ability to compose abstractions and build complex data models on top of simple ones. For example, indexes in FoundationDB are typically more scalable than in the majority of other systems, where index queries often have to be broadcast to all the systems in a cluster. Yet FoundationDB doesn't even have indexes as a feature - higher layers build and maintain index invariants using transactions. FoundationDB is also just really reliable and fault tolerant. Its testing story is drastically better than what the teams building these other products are doing.