3 ms·

I am wondering the choice of go as language to build a database. I am a big fan of golang for some workload but the GC will kill performance in a database, wou

by siscia 10y ago

I am wondering the choice of go as language to build a database. I am a big fan of golang for some workload but the GC will kill performance in a database, wouldn't? I am not being a brat I am seriously wondering how much GC affect the overall performance.

rakoo 10y ago
Then you should be happy that Go has been putting a lot of effort into making its GC as non-blocking as possible. There was an interesting blog post from Pusher yesterday (https://blog.pusher.com/golangs-real-time-gc-in-theory-and-practice/ https://blog.pusher.com/golangs-real-time-gc-in-theory-and-p...) about how they had to learn how the GC works and how to make the most of it; the links give more information into golang's internal evolution.
maaaats 10y ago
> but the GC will kill performance in a database, wouldn't? Why do you assume it would?
daenney 10y ago
> I am a big fan of golang for some workload but the GC will kill performance in a database, wouldn't? Cassandra and HSQLDB are written in Java for example which usually suffer from much larger GC collections/pauses (though you can read a book or two on the subject and tune it). So is Neo4j, a graph database. Or Datomic in Clojure (which runs on the JVM). Then there's Mnesia written in Erlang. A time series database, or a graph database, is quite different from a more general purpose database like Postgres/MySQL and means you need to consider different things. GC isn't necessarily an issue for high throughput (read or write). A lot of it depends on how much pressure you're putting on the GC (this is true for any type of application). Being aware of how your chosen language's GC behaves paired with the (typical) usage/behaviour of your code is probably the more important thing to consider. Just because something is written in a language without a GC doesn't automatically make it performant either.
- daenney 10y ago
  Oh, and about Go's (1.8) garbage collector: https://twitter.com/brianhatfield/status/804355831080751104 https://twitter.com/brianhatfield/status/804355831080751104. I think it'll be alright.
philips 10y ago
If you are careful and measure you can get consistent performance. We do lots of measurement of GC pauses for etcd and have it in a really tight band by ensuring we don't needlessly generate garbage in the hot path. Here is our measurement dashboard for a functional test cluster: http://dash.etcd.io/dashboard/db/functional-tests?from=now-24h&to=now http://dash.etcd.io/dashboard/db/functional-tests?from=now-2...

by siscia 10y ago

rakoo 10y ago

Then you should be happy that Go has been putting a lot of effort into making its GC as non-blocking as possible. There was an interesting blog post from Pusher yesterday (https://blog.pusher.com/golangs-real-time-gc-in-theory-and-practice/ https://blog.pusher.com/golangs-real-time-gc-in-theory-and-p...) about how they had to learn how the GC works and how to make the most of it; the links give more information into golang's internal evolution.

maaaats 10y ago

> but the GC will kill performance in a database, wouldn't? Why do you assume it would?

daenney 10y ago

> I am a big fan of golang for some workload but the GC will kill performance in a database, wouldn't? Cassandra and HSQLDB are written in Java for example which usually suffer from much larger GC collections/pauses (though you can read a book or two on the subject and tune it). So is Neo4j, a graph database. Or Datomic in Clojure (which runs on the JVM). Then there's Mnesia written in Erlang. A time series database, or a graph database, is quite different from a more general purpose database like Postgres/MySQL and means you need to consider different things. GC isn't necessarily an issue for high throughput (read or write). A lot of it depends on how much pressure you're putting on the GC (this is true for any type of application). Being aware of how your chosen language's GC behaves paired with the (typical) usage/behaviour of your code is probably the more important thing to consider. Just because something is written in a language without a GC doesn't automatically make it performant either.

daenney 10y ago
Oh, and about Go's (1.8) garbage collector: https://twitter.com/brianhatfield/status/804355831080751104 https://twitter.com/brianhatfield/status/804355831080751104. I think it'll be alright.

philips 10y ago

If you are careful and measure you can get consistent performance. We do lots of measurement of GC pauses for etcd and have it in a really tight band by ensuring we don't needlessly generate garbage in the hot path. Here is our measurement dashboard for a functional test cluster: http://dash.etcd.io/dashboard/db/functional-tests?from=now-24h&to=now http://dash.etcd.io/dashboard/db/functional-tests?from=now-2...