5 ms·

Rant: Why so much focus on Key Value stores? That's the easy part of the problem. I would like to know more about the interesting ones: secondary indexes, ran

Rant: Why so much focus on Key Value stores? That's the easy part of the problem. I would like to know more about the interesting ones: secondary indexes, range scans, performance on mixed workloads, robustness, operational complexity.

stochastic_monk 9y ago
R-trees (and their higher-dimensional counterparts) and B/B+-trees are great for supporting range queries. You’re right that kvs systems are limited, and I find the above quite effective at supporting more general operations.
snissn 9y ago
> Why so much focus on Key Value stores? That's the easy part of the problem. There aren't enough good/fast/reliable ones
- drb91 9y ago
  Is that a CAP joke or what?
- kamranjon 9y ago
  Does Redis not count?
  - deleted 9y ago
    [deleted]
  - deleted 9y ago
    [deleted]
  - doorbumper 9y ago
    Redis is an impressive piece of engineering, but it performs best as an in-memory kv-store on a single core. Its distributed capabilities target a different problem than other distributed kv-stores attempt to solve. Redis Cluster focuses on reasonable functionality for an in-memory store. However, Redis Cluster is neither highly available, nor consistent. There are multiple modes that can cause catastrophic data loss, so Redis Cluster works best in situations where losing data isn't a big deal. For it's intended use case, nothing else comes close to offering the same functionality, performance, reliability, and ease of use.
majidazimi 9y ago
Here is my 2 cent on why this is the case: On a general purpose hardware/network adding features as you mentioned, tends to be seriously difficult and tremendously workload dependent. At some point you need help from hardware designers which itself limits the use cases of your system. FPGA supported databases, In-network SQL processing, Server-less caches (an SSD directly connected to network without OS), Storage systems with mind blowing low energy consumption, ... are just bunch of absolutely amazing research that are being done today. however the limited public availability is badgering. With mass public usability in-mind, a general purpose K/V system is the highest summit you can really achieve.
zzzcpan 9y ago
Key-value store is not the focus of this research. And the problems it focuses on are not even in the same league as those you are interested in.
zilchers 9y ago
CockroachDB is implemented on top of a key / value store. I don’t know too much about the internals, but a kv store with lots of degrees of consistency seems like a good primitive to build more complex data stores on top. For instance, indexes could live as immediately consistent sharded keys, while data stays eventually consistent (or user configurable).

stochastic_monk 9y ago

R-trees (and their higher-dimensional counterparts) and B/B+-trees are great for supporting range queries. You’re right that kvs systems are limited, and I find the above quite effective at supporting more general operations.

snissn 9y ago

> Why so much focus on Key Value stores? That's the easy part of the problem. There aren't enough good/fast/reliable ones

drb91 9y ago
Is that a CAP joke or what?
kamranjon 9y ago
Does Redis not count?
- deleted 9y ago
  [deleted]
- deleted 9y ago
  [deleted]
- doorbumper 9y ago
  Redis is an impressive piece of engineering, but it performs best as an in-memory kv-store on a single core. Its distributed capabilities target a different problem than other distributed kv-stores attempt to solve. Redis Cluster focuses on reasonable functionality for an in-memory store. However, Redis Cluster is neither highly available, nor consistent. There are multiple modes that can cause catastrophic data loss, so Redis Cluster works best in situations where losing data isn't a big deal. For it's intended use case, nothing else comes close to offering the same functionality, performance, reliability, and ease of use.

majidazimi 9y ago

Here is my 2 cent on why this is the case: On a general purpose hardware/network adding features as you mentioned, tends to be seriously difficult and tremendously workload dependent. At some point you need help from hardware designers which itself limits the use cases of your system. FPGA supported databases, In-network SQL processing, Server-less caches (an SSD directly connected to network without OS), Storage systems with mind blowing low energy consumption, ... are just bunch of absolutely amazing research that are being done today. however the limited public availability is badgering. With mass public usability in-mind, a general purpose K/V system is the highest summit you can really achieve.

zzzcpan 9y ago

Key-value store is not the focus of this research. And the problems it focuses on are not even in the same league as those you are interested in.

zilchers 9y ago

CockroachDB is implemented on top of a key / value store. I don’t know too much about the internals, but a kv store with lots of degrees of consistency seems like a good primitive to build more complex data stores on top. For instance, indexes could live as immediately consistent sharded keys, while data stays eventually consistent (or user configurable).