4 ms·

I mostly see etcd being used to store metadata & configuration data for distributed systems. Can etcd be also used as a general distributed database like Found

I mostly see etcd being used to store metadata & configuration data for distributed systems. Can etcd be also used as a general distributed database like FoundationDB or ScyllaDB? If so how does it compare to those other optiions?

philips 7y ago
etcd is used as the primary data store of applications like Kubernetes or CoreDNS. But, many use it for configuration like M3 or Vitess. You can see different users in the docs[1]. There are different databases like CockroachDB or Dgraph that use etcd's raft libraries but build more application focused (SQL, graphql, etc) APIs[2]. [1]: https://github.com/etcd-io/etcd/blob/master/Documentation/integrations.md https://github.com/etcd-io/etcd/blob/master/Documentation/in... [2]: https://github.com/etcd-io/etcd/tree/master/raft#notable-users https://github.com/etcd-io/etcd/tree/master/raft#notable-use...
justnoise 7y ago
I don't think etcd was designed with the idea of competing with larger distributed databases. For example, the maximum database size limit in etcd (as of v3.3) 10GB. This works for an application like Kubernetes where you're storing less than a million records but likely isn't something you want backing your wildly successful django application.
gregwebs 7y ago
Etcd is designed to be able to cache it's entire working set in memory. https://etcd.io/docs/v3.3.12/dev-guide/limit/ https://etcd.io/docs/v3.3.12/dev-guide/limit/ https://github.com/etcd-io/etcd/blob/master/Documentation/op-guide/hardware.md#memory https://github.com/etcd-io/etcd/blob/master/Documentation/op... TiKV, a distributed kv store that can store many terabytes of data, actually uses etcd internally for metadata storage.
- fulafel 7y ago
  Most databases fit in memory though.
  - aphyr 7y ago
    Many do, and even large DBs can be accomodated by throwing terabytes of RAM at the problem, but etcd is fundamentally targeted at a different problem domain than a general-purpose database. Everything in etcd is represented by a single-threaded state machine, which is great for simplicity and correctness, but that comes with performance limitations.

philips 7y ago

etcd is used as the primary data store of applications like Kubernetes or CoreDNS. But, many use it for configuration like M3 or Vitess. You can see different users in the docs[1]. There are different databases like CockroachDB or Dgraph that use etcd's raft libraries but build more application focused (SQL, graphql, etc) APIs[2]. [1]: https://github.com/etcd-io/etcd/blob/master/Documentation/integrations.md https://github.com/etcd-io/etcd/blob/master/Documentation/in... [2]: https://github.com/etcd-io/etcd/tree/master/raft#notable-users https://github.com/etcd-io/etcd/tree/master/raft#notable-use...

justnoise 7y ago

I don't think etcd was designed with the idea of competing with larger distributed databases. For example, the maximum database size limit in etcd (as of v3.3) 10GB. This works for an application like Kubernetes where you're storing less than a million records but likely isn't something you want backing your wildly successful django application.

gregwebs 7y ago

Etcd is designed to be able to cache it's entire working set in memory. https://etcd.io/docs/v3.3.12/dev-guide/limit/ https://etcd.io/docs/v3.3.12/dev-guide/limit/ https://github.com/etcd-io/etcd/blob/master/Documentation/op-guide/hardware.md#memory https://github.com/etcd-io/etcd/blob/master/Documentation/op... TiKV, a distributed kv store that can store many terabytes of data, actually uses etcd internally for metadata storage.

fulafel 7y ago
Most databases fit in memory though.
- aphyr 7y ago
  Many do, and even large DBs can be accomodated by throwing terabytes of RAM at the problem, but etcd is fundamentally targeted at a different problem domain than a general-purpose database. Everything in etcd is represented by a single-threaded state machine, which is great for simplicity and correctness, but that comes with performance limitations.