4 ms·

The title contains "rocksdb and raft", but the article doesn't give any details on how the Keevo key-value store is designed using rocksdb + raft. You can not f

The title contains "rocksdb and raft", but the article doesn't give any details on how the Keevo key-value store is designed using rocksdb + raft. You can not figure out how they manage data with raft (how many raft groups in the system? do they split the group when it reaches certain stage?), there is no mention on the design choice of not using etcd raft which is the most popular one in Go. They didn't mention how they test the system, Jepsen or in-house monkey testing? They do have a Keevo link in the article but it says "Sorry, we couldn't find that page". On the Go side, since they mentioned that rocksdb is written in C++, I am interested to know the cgo issues they might had when using Go and rocksdb together, but of course they didn't mention that. Same goes for the GC stw pauses. I strongly believe that articles like this largely used as soft ads materials without much detailed technical meat should be banned from Hack News. If it is all about "we built system X using building blocks Y and Z in language T!", what is the real value for readers?

jnordwick 9y ago
Don't worry. This involves Go, a Google connection, so it is immune to criticism of bias.
espadrine 9y ago
> since they mentioned that rocksdb is written in C++, I am interested to know the cgo issues On that subject, is there a Badger[0] based distributed key-value store, equivalent to etcd, ready for production? [0]: https://github.com/dgraph-io/badger https://github.com/dgraph-io/badger
- xstartup 9y ago
  I wonder why people don't use badger over rocksdb.
tbarbugli 9y ago
We currently use the Raft Go implementation from Hashicorp (the article links to the Github project). We found Hashicorp implementation mature enough and easier to work with than the one from etcd. Using Rocksdb with Go has indeed its own challenges because CGO calls are far from cheap and have an important consequences on Go applications. For performance reasons, we ended up moving some logic from Go to C in a few hot-spots (eg. moving the retrieval of large amount of keys from Go to C). Disclaimer: I work for Stream and I am one the two co-founders
- StreamBright 9y ago
  Could you elaborate on Keevo? Could you compare it to Riak or Cassandra for example? Do you have any plans to open source it?
StreamBright 9y ago
I disagree. I think it is great to see how people build stuff and everybody can get the conclusion. I am super interested in Keevo and hoping to get to know more. Rocksdb is a rock solid foundation (no pun intended), some of the most amazing developers are working on it. The only missing part is how to create a service on the top, sort of like LevelDB + Riak. Actually it would make sense to Riak guys to look into using RocksDB instead of LevelDB.
- xstartup 9y ago
  Why not badger since it's clearly superior to RocksDB in terms of performance and written in Go: https://github.com/dgraph-io/badger https://github.com/dgraph-io/badger

jnordwick 9y ago

Don't worry. This involves Go, a Google connection, so it is immune to criticism of bias.

espadrine 9y ago

> since they mentioned that rocksdb is written in C++, I am interested to know the cgo issues On that subject, is there a Badger[0] based distributed key-value store, equivalent to etcd, ready for production? [0]: https://github.com/dgraph-io/badger https://github.com/dgraph-io/badger

xstartup 9y ago
I wonder why people don't use badger over rocksdb.

tbarbugli 9y ago

We currently use the Raft Go implementation from Hashicorp (the article links to the Github project). We found Hashicorp implementation mature enough and easier to work with than the one from etcd. Using Rocksdb with Go has indeed its own challenges because CGO calls are far from cheap and have an important consequences on Go applications. For performance reasons, we ended up moving some logic from Go to C in a few hot-spots (eg. moving the retrieval of large amount of keys from Go to C). Disclaimer: I work for Stream and I am one the two co-founders

StreamBright 9y ago
Could you elaborate on Keevo? Could you compare it to Riak or Cassandra for example? Do you have any plans to open source it?

StreamBright 9y ago

I disagree. I think it is great to see how people build stuff and everybody can get the conclusion. I am super interested in Keevo and hoping to get to know more. Rocksdb is a rock solid foundation (no pun intended), some of the most amazing developers are working on it. The only missing part is how to create a service on the top, sort of like LevelDB + Riak. Actually it would make sense to Riak guys to look into using RocksDB instead of LevelDB.

xstartup 9y ago
Why not badger since it's clearly superior to RocksDB in terms of performance and written in Go: https://github.com/dgraph-io/badger https://github.com/dgraph-io/badger