4 ms·

Still waiting for some benchmarks by the team. What I would be curious, is, how much faster the db would be if it was in c++/rust instead of java/go. I know bi

Still waiting for some benchmarks by the team. What I would be curious, is, how much faster the db would be if it was in c++/rust instead of java/go. I know bigtable claims 3x faster than hbase. Scylla claims ~10x (though also different architecture) faster than Cassandra. Trinity ~2x compared to Lucene. etc etc. Also: incoming comments about the name of the db.

jrs95 9y ago
Agreed. I love Go, but a database is one of the things in the stack that ought to be as optimized as possible in my opinion. C++ still seems preferable in these types of applications. At the same time, if the underlying architecture is enough of an improvement over older systems, it's still an upgrade.
- zzzcpan 9y ago
  You can't optimize anything with C++ if your database is stuck waiting for consensus with nodes 100 ms away. Close to the metal and low level languages are really poor choices here, even Go is too low level for CP databases.
  - ddorian43 9y ago
    Yes, but you'll have lower overhead on each query/request and more concurrency ? For each query the bottleneck is the network which you can't fix, but you can fix the 2,3,etc bottlenecks which are inside 1 machine (cpu/ram/disk).
    - jrs95 9y ago
      Yeah...I'm not really aware of a higher level language that mystically makes networking faster. As far as I'm aware, the most efficient network protocols tend to be implemented at a low level, too.
- wiremine 9y ago
  > C++ still seems preferable in these types of applications Agreed, but until there are some published performance metrics, I think it's an assumption more than a truth. Premature optimization...
  - jrs95 9y ago
    Well we already have metrics on what the languages themselves can do. And given that performance will definitely matter in this domain, it's not a premature optimization at all. A database isn't something that can be written easily enough to simply rewrite it in another language later as an optimization.
knz42 9y ago
Note that the underlying data store where a lot of the byte churning happens is RocksDB, written in C++. Also the Go/C++ interface in CockroachDB is coarse to reduce overheads.
- ddorian43 9y ago
  I know and that you take special care to bulk as much stuff to it for lower overhead. But then how much of % speed is left into GO ? Say GO is 2x slower, and the db is bottlenecked on cpu/ram, you're using 2x servers. While if it's bottlenecked on disk it's rocksdb which you can't make faster by using a different language. Like even TiDB (mysql protocol, golang) uses TiKB (kv in rust,rocksdb) to make it faster.
- ddorian43 9y ago
  In that I would be curious on RocksDB vs LMDB. There was an issue about this but it didn't progress: https://github.com/cockroachdb/cockroach/issues/14#issuecomment-41171979 https://github.com/cockroachdb/cockroach/issues/14#issuecomm... For some reason(s) nobody is using LMDB on distributed dbs (only know actordb). While for single-nodes there are many cases that have switched and been better.
eis 9y ago
I did a performance evaluation for a project recently and CockroachDB while awesome to setup and operate, was painfully slow. Couldn't get more than a couple thousand writes per second on a cluster consisting of more than half a dozen nodes all in the same datacenter. There seemed to be a huge amount of write amplification (nearly two orders of magnitude) and the load distribution across nodes wasn't really even either. Sadly had to give up on CockroachDB. They've got good engineers on their team and have some nice funding as well so I'm hopeful that they'll make big improvements in this department eventually. But it'll take quite a lot of work to bring it up to speed - excuse the pun :) Oh and I don't think Go is any limiting factor here. It's more of an architectural/engineering issue.
uluyol 9y ago
I would argue that data structure and system architecture play much more of a role than language choice. Scylla's claims only make sense for datasets that are completely in-memory. Cassandra explicitly targets much larger than memory datasets, and so your bottleneck should be disk throughput not CPU. If you plan to keep everything in memory and just snapshot to disk, you should probably change your choice of data structures. We're also talking about wide-area storage. Any latency from Go over C++ should be negligible: the Go GC is sub-ms latency now and will just get better [1] while for inter-DC network latency you can expect 50-2/300 ms latencies. The GC just doesn't matter here. Go also gives you control over memory layout, and has an ever-improving compiler [2]. C++ would only really be necessary for sub-ms tail latency systems which are really rare. I think the gap between BigTable and Hbase is likely due to Google spending more time optimizing BigTable. They do use it for basically everything not built on Spanner. [1] See recent activity on https://github.com/golang/go/issues/10958 https://github.com/golang/go/issues/10958 [2] Go has a ways to match GCC, but you can work around the language for now to get close to C. If you want to improve things here, maybe contribute to gccgo or gollvm so Go users can get all of LLVM & GCC's optimizations.
elvinyung 9y ago
Relevant: blog post from the TiDB team about why they didn't choose Go for the core distributed storage engine: https://pingcap.github.io/blog/2017/05/27/rust-in-tikv/#go https://pingcap.github.io/blog/2017/05/27/rust-in-tikv/#go

jrs95 9y ago

Agreed. I love Go, but a database is one of the things in the stack that ought to be as optimized as possible in my opinion. C++ still seems preferable in these types of applications. At the same time, if the underlying architecture is enough of an improvement over older systems, it's still an upgrade.

zzzcpan 9y ago
You can't optimize anything with C++ if your database is stuck waiting for consensus with nodes 100 ms away. Close to the metal and low level languages are really poor choices here, even Go is too low level for CP databases.
- ddorian43 9y ago
  Yes, but you'll have lower overhead on each query/request and more concurrency ? For each query the bottleneck is the network which you can't fix, but you can fix the 2,3,etc bottlenecks which are inside 1 machine (cpu/ram/disk).
  - jrs95 9y ago
    Yeah...I'm not really aware of a higher level language that mystically makes networking faster. As far as I'm aware, the most efficient network protocols tend to be implemented at a low level, too.
wiremine 9y ago
> C++ still seems preferable in these types of applications Agreed, but until there are some published performance metrics, I think it's an assumption more than a truth. Premature optimization...
- jrs95 9y ago
  Well we already have metrics on what the languages themselves can do. And given that performance will definitely matter in this domain, it's not a premature optimization at all. A database isn't something that can be written easily enough to simply rewrite it in another language later as an optimization.

knz42 9y ago

Note that the underlying data store where a lot of the byte churning happens is RocksDB, written in C++. Also the Go/C++ interface in CockroachDB is coarse to reduce overheads.

ddorian43 9y ago
I know and that you take special care to bulk as much stuff to it for lower overhead. But then how much of % speed is left into GO ? Say GO is 2x slower, and the db is bottlenecked on cpu/ram, you're using 2x servers. While if it's bottlenecked on disk it's rocksdb which you can't make faster by using a different language. Like even TiDB (mysql protocol, golang) uses TiKB (kv in rust,rocksdb) to make it faster.
ddorian43 9y ago
In that I would be curious on RocksDB vs LMDB. There was an issue about this but it didn't progress: https://github.com/cockroachdb/cockroach/issues/14#issuecomment-41171979 https://github.com/cockroachdb/cockroach/issues/14#issuecomm... For some reason(s) nobody is using LMDB on distributed dbs (only know actordb). While for single-nodes there are many cases that have switched and been better.

eis 9y ago

I did a performance evaluation for a project recently and CockroachDB while awesome to setup and operate, was painfully slow. Couldn't get more than a couple thousand writes per second on a cluster consisting of more than half a dozen nodes all in the same datacenter. There seemed to be a huge amount of write amplification (nearly two orders of magnitude) and the load distribution across nodes wasn't really even either. Sadly had to give up on CockroachDB. They've got good engineers on their team and have some nice funding as well so I'm hopeful that they'll make big improvements in this department eventually. But it'll take quite a lot of work to bring it up to speed - excuse the pun :) Oh and I don't think Go is any limiting factor here. It's more of an architectural/engineering issue.

uluyol 9y ago

I would argue that data structure and system architecture play much more of a role than language choice. Scylla's claims only make sense for datasets that are completely in-memory. Cassandra explicitly targets much larger than memory datasets, and so your bottleneck should be disk throughput not CPU. If you plan to keep everything in memory and just snapshot to disk, you should probably change your choice of data structures. We're also talking about wide-area storage. Any latency from Go over C++ should be negligible: the Go GC is sub-ms latency now and will just get better [1] while for inter-DC network latency you can expect 50-2/300 ms latencies. The GC just doesn't matter here. Go also gives you control over memory layout, and has an ever-improving compiler [2]. C++ would only really be necessary for sub-ms tail latency systems which are really rare. I think the gap between BigTable and Hbase is likely due to Google spending more time optimizing BigTable. They do use it for basically everything not built on Spanner. [1] See recent activity on https://github.com/golang/go/issues/10958 https://github.com/golang/go/issues/10958 [2] Go has a ways to match GCC, but you can work around the language for now to get close to C. If you want to improve things here, maybe contribute to gccgo or gollvm so Go users can get all of LLVM & GCC's optimizations.

elvinyung 9y ago

Relevant: blog post from the TiDB team about why they didn't choose Go for the core distributed storage engine: https://pingcap.github.io/blog/2017/05/27/rust-in-tikv/#go https://pingcap.github.io/blog/2017/05/27/rust-in-tikv/#go