4 ms·

FoundationDB is a key/value store, and well-known for being fast, scalable and incredibly reliable. It passed many of the distributed database tests unlike many

FoundationDB is a key/value store, and well-known for being fast, scalable and incredibly reliable. It passed many of the distributed database tests unlike many competitors. Here's a good HN thread when Apple open-sourced it a few years ago: https://news.ycombinator.com/item?id=16877395 https://news.ycombinator.com/item?id=16877395 CockroachDB implements postgres-compatible SQL layer on top of their own key/value store called Pebble. This is similar to TiDB which implements mysql on top of their TiKV key/value store. The latter databases offer easier programming with SQL interfaces, but since you can model just about anything on top of key/value store, there are already "layers" that can implement similar abstractions on top of FDB like the Record Layer[1], Document Layer [2], and even a SQL layer [3] at one point. 1) https://www.foundationdb.org/blog/announcing-record-layer/ https://www.foundationdb.org/blog/announcing-record-layer/ 2) https://foundationdb.github.io/fdb-document-layer/ https://foundationdb.github.io/fdb-document-layer/ 3) https://github.com/jaytaylor/sql-layer https://github.com/jaytaylor/sql-layer

EGreg 5y ago
What is better, FoundationDB, CockroachDB, or Hypercore?
- vvern 5y ago
  One of these is not like the others.
atombender 5y ago
An important difference between CockroachDB and TiDB is that the former has an embedded key-value store; everything runs in the same process. TiDB is split into multiple parts: TiDB proper (stateless SQL/query plan layer), TiKV (the key-value store), and PD (placement driver, the main control plane and metadata layer). TiKV also isn't a pure key-value store, as it handles some particular query operations that can be "pushed down" from TiDB to reduce network round trips. FoundationDB's role similar to TiKV. It's a pure key-value store with replication built in.
jandrewrogers 5y ago
There is one caveat I would add. While you can always build just about any kind of database or data model on top of a logical KV store, the scalability and performance is almost always going to be qualitatively worse for queries than using a storage engine purpose-built for the intended type of database or data model. It is simpler to implement, which is why people do it, but you give up a lot of selectivity and locality on the query side. The target design case for KV stores is when all your queries are essentially trivial. Layering SQL on top of it implies the expected use case is complex queries of the type where a KV store won't perform well. Good for a compatibility interface, but not performant.
- skrtskrt 5y ago
  though as with things like CAP theorem, there are always clever ways to to place a given implementation at various points along the spectrum, rather than just at one end or the other.
- sa46 5y ago
  Very curious, what’s an alternative implementation strategy or more appropriate mental model? My mental model (I am not a database engineer) is that every SQL database is fundamentally backed by a key value store. For Postgres, each value is a row of a relation, the key is the relation’s key, and the physical storage is the table file. Postgres builds b-trees on top to make it fast, but the b-tree itself is a relation, a key value store that Postgres works hard to keep sorted.
  - travisd 5y ago
    i’m also not a db engineer, but i think this is true-ish. however building and maintaining those index tables is hard and probably prone to issues if you can’t update multiple as part of the same transaction. the other major thing you’d miss is smarter joins. the distributed databases do a lot of work to be able to push down predicates as far down as possible to the physical nodes storing the data. there’s probably more as well.
    - 5y ago
legojoey17 5y ago
Adding clarifications to that, Pebble is a local K/V library akin to RocksDB/LevelDB and provides no services or distributed capabilities. Comparing the two does not make sense. On the other hand, TiKV is more comparable to the internal distributed transactional K/V layer to CockroachDB that its SQL layer is coupled to. That of course is not a usable external interface. You could utilize CockroachDB like a K/V store with simple table to forego the SQL functionality and planning (e.g. only primary col index, only key/value column, so on), but I am not sure what the practicality of that is as I have not kept up with CockroachDB development. (disclaimer: I interned at Cockroach several years back)
- manigandham 5y ago
  The comparison is about providing a relational data model on top of a key/value store. Yugabyte also does something similar on top of its underlying DocDB document store.
ddaalluu2 5y ago
It's actually not that fast. Since it has multiple layers and that implies overhead. It's about 4x as slow as MySQL. Personally I prefer cockroachdb, postgres compatible and easy scale.

EGreg 5y ago

What is better, FoundationDB, CockroachDB, or Hypercore?

vvern 5y ago
One of these is not like the others.

atombender 5y ago

An important difference between CockroachDB and TiDB is that the former has an embedded key-value store; everything runs in the same process. TiDB is split into multiple parts: TiDB proper (stateless SQL/query plan layer), TiKV (the key-value store), and PD (placement driver, the main control plane and metadata layer). TiKV also isn't a pure key-value store, as it handles some particular query operations that can be "pushed down" from TiDB to reduce network round trips. FoundationDB's role similar to TiKV. It's a pure key-value store with replication built in.

jandrewrogers 5y ago

There is one caveat I would add. While you can always build just about any kind of database or data model on top of a logical KV store, the scalability and performance is almost always going to be qualitatively worse for queries than using a storage engine purpose-built for the intended type of database or data model. It is simpler to implement, which is why people do it, but you give up a lot of selectivity and locality on the query side. The target design case for KV stores is when all your queries are essentially trivial. Layering SQL on top of it implies the expected use case is complex queries of the type where a KV store won't perform well. Good for a compatibility interface, but not performant.

skrtskrt 5y ago
though as with things like CAP theorem, there are always clever ways to to place a given implementation at various points along the spectrum, rather than just at one end or the other.
sa46 5y ago
Very curious, what’s an alternative implementation strategy or more appropriate mental model? My mental model (I am not a database engineer) is that every SQL database is fundamentally backed by a key value store. For Postgres, each value is a row of a relation, the key is the relation’s key, and the physical storage is the table file. Postgres builds b-trees on top to make it fast, but the b-tree itself is a relation, a key value store that Postgres works hard to keep sorted.
- travisd 5y ago
  i’m also not a db engineer, but i think this is true-ish. however building and maintaining those index tables is hard and probably prone to issues if you can’t update multiple as part of the same transaction. the other major thing you’d miss is smarter joins. the distributed databases do a lot of work to be able to push down predicates as far down as possible to the physical nodes storing the data. there’s probably more as well.
  - 5y ago

legojoey17 5y ago

Adding clarifications to that, Pebble is a local K/V library akin to RocksDB/LevelDB and provides no services or distributed capabilities. Comparing the two does not make sense. On the other hand, TiKV is more comparable to the internal distributed transactional K/V layer to CockroachDB that its SQL layer is coupled to. That of course is not a usable external interface. You could utilize CockroachDB like a K/V store with simple table to forego the SQL functionality and planning (e.g. only primary col index, only key/value column, so on), but I am not sure what the practicality of that is as I have not kept up with CockroachDB development. (disclaimer: I interned at Cockroach several years back)

manigandham 5y ago
The comparison is about providing a relational data model on top of a key/value store. Yugabyte also does something similar on top of its underlying DocDB document store.

ddaalluu2 5y ago

It's actually not that fast. Since it has multiple layers and that implies overhead. It's about 4x as slow as MySQL. Personally I prefer cockroachdb, postgres compatible and easy scale.