4 ms·

Curious - would Rust be more appropriate than Go for such a task?

cultofmetatron 6y ago
rust would be more appropriate if your intention was to use this in production. Databases ad GC don't mix well. (its done but it makes tuning a nightmare) As much as I love rust, its learning curve is high and I'm sure Op doesn't want to spend half his article teaching all the intricacies of types and the borrow checker. Go is easy to learn over a weekend so its probably a better medium for illustrating the concepts as everything is laid out simply.
- arendtio 6y ago
  Sometimes I wonder how Go would look like with a borrow checker...
- eatonphil 6y ago
  Truth is I've written about how to parse in better languages before (JavaScript, Python, Standard ML, etc.) and I wanted to figure out a good approach for doing it in Go. As for databases and GC, you may be right but there are still major databases out there written in Java for example. https://www.quora.com/Which-databases-data-stores-are-written-in-Java https://www.quora.com/Which-databases-data-stores-are-writte...
- matttproud 6y ago
  A garbage collector is not inherently incompatible with a low-latency database. I was generally very happy with the performance of Go's garbage collector enough to have built Prometheus, the time series database, on it back in 2012, when the collector was considerably more naive. https://blog.golang.org/ismmkeynote https://blog.golang.org/ismmkeynote
speedgoose 6y ago
Golang works too. There is a few databases written in Golang, such as Prometheus, InfluxDB, CockroachDB, or tidb. https://github.com/topics/database?l=go&o=desc&s=stars https://github.com/topics/database?l=go&o=desc&s=stars
- orhanhh 6y ago
  Note that neither CockroachDB or TiDB use Golang for their actual storage engine, which is in both cases written in C (RocksDB). They do use Golang for SQL parsing though, which is what this post was mostly about.
  - cube2222 6y ago
    However, cockroachdb does all other work (including query execution) in Go too. There's also DGraph based on Badger as a storage engine which is an all-go stack. And badger does compare favorably to rocksdb under certain workloads.
  - irfansharif 6y ago
    > Note that neither CockroachDB [...] use Golang for their actual storage engine We do, now. We're looking to move away from RocksDB to https://github.com/cockroachdb/pebble/ https://github.com/cockroachdb/pebble/.
    - eatonphil 6y ago
      Woah, that's big news. Thanks for sharing.
  - valyala 6y ago
    VictoriaMetrics [1] is written entirely in Go. By default it uses canonical zstd library for compression (the library is written in C), but it supports pure Go mode when built with `make victoria-metrics-pure`. In this mode it uses zstd implementation written in Go [2]. [1] https://github.com/VictoriaMetrics/VictoriaMetrics https://github.com/VictoriaMetrics/VictoriaMetrics [2] https://github.com/klauspost/compress/tree/master/zstd#zstd https://github.com/klauspost/compress/tree/master/zstd#zstd

rust would be more appropriate if your intention was to use this in production. Databases ad GC don't mix well. (its done but it makes tuning a nightmare) As much as I love rust, its learning curve is high and I'm sure Op doesn't want to spend half his article teaching all the intricacies of types and the borrow checker. Go is easy to learn over a weekend so its probably a better medium for illustrating the concepts as everything is laid out simply.

arendtio 6y ago
Sometimes I wonder how Go would look like with a borrow checker...
eatonphil 6y ago
Truth is I've written about how to parse in better languages before (JavaScript, Python, Standard ML, etc.) and I wanted to figure out a good approach for doing it in Go. As for databases and GC, you may be right but there are still major databases out there written in Java for example. https://www.quora.com/Which-databases-data-stores-are-written-in-Java https://www.quora.com/Which-databases-data-stores-are-writte...
matttproud 6y ago
A garbage collector is not inherently incompatible with a low-latency database. I was generally very happy with the performance of Go's garbage collector enough to have built Prometheus, the time series database, on it back in 2012, when the collector was considerably more naive. https://blog.golang.org/ismmkeynote https://blog.golang.org/ismmkeynote

speedgoose 6y ago

Golang works too. There is a few databases written in Golang, such as Prometheus, InfluxDB, CockroachDB, or tidb. https://github.com/topics/database?l=go&o=desc&s=stars https://github.com/topics/database?l=go&o=desc&s=stars

orhanhh 6y ago
Note that neither CockroachDB or TiDB use Golang for their actual storage engine, which is in both cases written in C (RocksDB). They do use Golang for SQL parsing though, which is what this post was mostly about.
- cube2222 6y ago
  However, cockroachdb does all other work (including query execution) in Go too. There's also DGraph based on Badger as a storage engine which is an all-go stack. And badger does compare favorably to rocksdb under certain workloads.
- irfansharif 6y ago
  > Note that neither CockroachDB [...] use Golang for their actual storage engine We do, now. We're looking to move away from RocksDB to https://github.com/cockroachdb/pebble/ https://github.com/cockroachdb/pebble/.
  - eatonphil 6y ago
    Woah, that's big news. Thanks for sharing.
- valyala 6y ago
  VictoriaMetrics [1] is written entirely in Go. By default it uses canonical zstd library for compression (the library is written in C), but it supports pure Go mode when built with `make victoria-metrics-pure`. In this mode it uses zstd implementation written in Go [2]. [1] https://github.com/VictoriaMetrics/VictoriaMetrics https://github.com/VictoriaMetrics/VictoriaMetrics [2] https://github.com/klauspost/compress/tree/master/zstd#zstd https://github.com/klauspost/compress/tree/master/zstd#zstd