6 ms·
I'm shocked that they chose a garbage-collected language to implement a database with. Golang is great for building servers, but this is domain with potentially
by echelon 7y ago
I'm shocked that they chose a garbage-collected language to implement a database with. Golang is great for building servers, but this is domain with potentially much greater resource constraints.
How do they deal with GC pause? I'm sure that they have an answer, but they'd have so much more latitude if they had the facility to reason about this from the ground up. They've sort of painted themselves into a corner now.
C, C++, or Rust would have been a better match and given me more faith in their product.
- jordanlewis 7y agoWe wrote a (brief) blog post about this decision way back in 2015 that you might be interested in: https://www.cockroachlabs.com/blog/why-go-was-the-right-choice-for-cockroachdb/ https://www.cockroachlabs.com/blog/why-go-was-the-right-choi... We're generally pretty happy with our choice to use Go for the database, even when it causes us pain like the kind you see in this post. Rust wasn't really ready when we started working on CockroachDB, and we think that we wouldn't have been able to make progress as quickly as we did if we had used C(++).
- fouc 7y agoHow hard would it be to port the code to Rust? (Not trying to make it sound like a "Rewrite in Rust" thing)
- mjibson 7y agoSuper mega ultra hard. It would take so much time for us to learn rust, port everything including all tooling, and fix new bugs we introduce that we wouldn't add any new features (but lots of new bugs!) for like 2-4 years and the company would die.
- roberson87 7y agoThanks for your input. I have no exp at all on db internals, but could you expand your thoughts on the possibility of moving critical (storage layer?) parts to Rust and leave all the networking stuff/rest in Go? Similar to what TiDb have done?
- ryanworl 7y agoWriting a program in C++ or Rust will not just make it use memory more efficiently than Go with the GC for free. Many Rust programs do a lot of heap allocations, and lots of people in C++ liberally use reference counted pointers which both require heap allocation and atomics. It is possible to write Go programs which efficiently use memory using object pooling just as you would in C. Go additionally can stack allocate many variables that would require a heap allocation in Java, where the GC is a much bigger issue.
- timClicks 7y agoWhen you say that Rust uses lots of allocations, are you referring to the heavy use of Vec<_>? My understanding is that almost everything that can be allocated on the stack, is. Boxed types and trait objects are available, but they're opt-in features rather than the default.
- ken 7y agoA database is a server. Your malloc isn't deterministic, either. According to the Go Wikipedia article, as of 2017: > Garbage collection pauses should be significantly shorter than they were in Go 1.7, usually under 100 microseconds and often as low as 10 microseconds. According to [1], "A trivial SELECT can take in the order of 0.1ms to execute server-side", i.e., 100 microseconds. Any I/O will of course significantly increase that. This sounds like an entirely reasonable price to pay. Writing high-level functionality in C in 2019 would not give me much faith in their product. [1]: https://www.2ndquadrant.com/en/blog/postgresql-latency-pipelining-batching/ https://www.2ndquadrant.com/en/blog/postgresql-latency-pipel...
- aloknnikhil 7y agoFor me, the issue with GC never has been about how long it runs for. It's been about the unpredictability of when it can kick in. Granted a malloc can be unpredictable too (albeit very very rarely), but at least you know a malloc is the only time your process could stall waiting for memory. Databases make extensive use of scratch space for undo logs and redo logs and especially while processing JOINs. It's tremendous pressure on the GC that results in frequent performance dips. With C/C++/Rust, you'd pipeline the query stages to malloc/free that space outside the hot path to minimize query response times.
- irfansharif 7y agoI'll repeat what was mentioned elsewhere. The GC pauses in Go are in the order of microseconds. We've done a ton of work keeping our memory usage under tight bounds and keeping a close eye on exactly how the go GC processes interplay with query execution threads (what processors are consumed, GC pressure buildup, where/when the stalls are introduced, etc.) Consistently we've found that given you're invariably dealing with network latencies (it's a distributed db after all), the microsecond GC pauses are almost entirely dwarfed to matter (I'm not excluding p99s here).
- tom_mellior 7y ago> With C/C++/Rust, you'd pipeline the query stages to malloc/free that space outside the hot path to minimize query response times. I don't understand why you seem to assert that GC'd languages would force you to put allocations on the hot path. Surely in Go you can also allocate that space elsewhere?
- weberc2 7y agoGo’s GC pauses are on a microsecond scale. If you absolutely need to minimize latency at all costs, I’d be more worried that the Go compiler doesn’t optimize as aggressively as C/C++/Rust.
- snark42 7y agoInfluxDB is written in Golang as well. It's rather performant and scales out nicely, given my experience I don't think I'd consider Golang a big concern.
- valyala 7y agoThere is VictoriaMetrics, which is also written in Go and shows much higher performance numbers than InfluxDB, especially on high cardinality data. See [1], [2] and [3]. [1] https://medium.com/@valyala/measuring-vertical-scalability-for-time-series-databases-in-google-cloud-92550d78d8ae https://medium.com/@valyala/measuring-vertical-scalability-f... [2] https://medium.com/@valyala/insert-benchmarks-with-inch-influxdb-vs-victoriametrics-e31a41ae2893 https://medium.com/@valyala/insert-benchmarks-with-inch-infl... [3] https://medium.com/@valyala/high-cardinality-tsdb-benchmarks-victoriametrics-vs-timescaledb-vs-influxdb-13e6ee64dd6b https://medium.com/@valyala/high-cardinality-tsdb-benchmarks...