4 ms·

Throughput optimization in database engines often relies on operation latency being visible and predictable to the execution scheduler. Disk I/O has this proper

Throughput optimization in database engines often relies on operation latency being visible and predictable to the execution scheduler. Disk I/O has this property, a GC does not, so they are not comparable in terms of their impact on throughput. An important class of schedule-based architectural optimizations are rendered ineffective if you are running a GC in the background. This only matters to databases that actually implement these optimizations. If a database does not then a GC has a much smaller impact. But it also means that, all other things being equal, the database will never be competitive with a non-GC design since those optimizations can provide an integer factor improvement in performance. Also, many database engines aren't doing malloc() at runtime. 100 microseconds of CPU time is an integer factor larger than many atomic database operations in a fast database engine. An operation taking an order of magnitude longer than the scheduler would expect based on runtime state has real adverse consequences.

mumblemumble 7y ago
So, before we theorize that Cockroach can't possibly be fast enough to meet anyone's needs, perhaps we should look at some performance figures? I haven't ever touched it myself, but, if the benchmarks that Cockroach Labs posts on their blog are to be believed, it's pretty respectable for what it is.
- jandrewrogers 7y ago
  You are misrepresenting what was written. Most databases don't need to be optimal to be useful, merely adequate, and few attempt to be optimal in any kind of absolute sense. I still use PostgreSQL regularly and it unambiguously falls into this "far from optimal" category. However, as data velocity and volume grow, optimization of absolute performance and hardware efficiency has an increasingly large impact on the cost of operating a database and the kinds of applications you can economically run. Databases that sacrifice major optimizations at an architectural level will have no chance to be competitive with databases that do not over the long term as average data volume and velocity grows. At the scale of database operations many companies are using today, these optimizations literally save them tens of millions of dollars per year on infrastructure. CockroachDB produces what looks like a fine product and has every right to make whatever technical decisions they wish. That the architecture has made substantial performance sacrifices is not theoretical though, and they don't pretend to operate in markets that require any kind of absolute operational efficiency.
  - mumblemumble 7y ago
    This seems to me, though, like a spot where the application of Knuth's law is critical. It's not enough to demonstrate that X is slower than it could be, you have to demonstrate that it is that, and that this is also having an appreciable impact on the overall behavior of the system. So, given that CockroachDB is a distributed system, I'd want to start by seeing some measurements to demonstrate that GC overhead is having a bigger impact than, say, network latency.
endymi0n 7y ago
This is slightly missing the point. Lots of successful large scale databases have been written in garbage collected languages, including for example Cassandra and ElasticSearch. They're completely fine. What's important to notice though is that what really matters about latency and predictability is _relative_ performance. Now there is a category of databases that are built for speed of response first, some of which have some pretty spaced out architecture under the hood. Aerospike comes to my mind, together with ScyllaDB or Redis maybe. They trade off this speed with massive compromises in complexity, consistency and scalability. GC would be disastrous for these, so they're all written in C(++). The main premise of CockroachDB is geographic replication, durability and scalability while maintaining full consistence and serialization. In order just to keep these promises, a large part of the design of Cockroach is that the DB needs to keep _waiting_ most of the time of any request until all geo replicated shards are consistent. We're talking dozens to hundreds of milliseconds here. And even if that wasn't the case, a full GC of 100 microseconds probably isn't even noticeable compared with the weight of complex SQL query planning and execution. You may stink a lot of valid points about Go, but this isn't one of them.
- stubish 7y ago
  Cassandra has huge problems with JVM garbage collection, and it still isn't rid of them in the decades it has been in development. The reason for the scary warnings if your partition sizes exceed 100MB is garbage collection. The reason you can't tune it to use all your RAM on a modern box is garbage collection. Picking JVM implementations and tuning mostly seems to revolve around garbage collection and its trade offs. A common symptom of poorly maintained clusters is second long garbage collection pauses. The trick with Go seems to be that programs don't seem to generate much garbage to collect, so I don't know if there is much use comparing with Java apps. But garbage collection overhead is certainly a major issue to be wary of for any application needing consistently low response times.

mumblemumble 7y ago

So, before we theorize that Cockroach can't possibly be fast enough to meet anyone's needs, perhaps we should look at some performance figures? I haven't ever touched it myself, but, if the benchmarks that Cockroach Labs posts on their blog are to be believed, it's pretty respectable for what it is.

jandrewrogers 7y ago
You are misrepresenting what was written. Most databases don't need to be optimal to be useful, merely adequate, and few attempt to be optimal in any kind of absolute sense. I still use PostgreSQL regularly and it unambiguously falls into this "far from optimal" category. However, as data velocity and volume grow, optimization of absolute performance and hardware efficiency has an increasingly large impact on the cost of operating a database and the kinds of applications you can economically run. Databases that sacrifice major optimizations at an architectural level will have no chance to be competitive with databases that do not over the long term as average data volume and velocity grows. At the scale of database operations many companies are using today, these optimizations literally save them tens of millions of dollars per year on infrastructure. CockroachDB produces what looks like a fine product and has every right to make whatever technical decisions they wish. That the architecture has made substantial performance sacrifices is not theoretical though, and they don't pretend to operate in markets that require any kind of absolute operational efficiency.
- mumblemumble 7y ago
  This seems to me, though, like a spot where the application of Knuth's law is critical. It's not enough to demonstrate that X is slower than it could be, you have to demonstrate that it is that, and that this is also having an appreciable impact on the overall behavior of the system. So, given that CockroachDB is a distributed system, I'd want to start by seeing some measurements to demonstrate that GC overhead is having a bigger impact than, say, network latency.

endymi0n 7y ago

This is slightly missing the point. Lots of successful large scale databases have been written in garbage collected languages, including for example Cassandra and ElasticSearch. They're completely fine. What's important to notice though is that what really matters about latency and predictability is _relative_ performance. Now there is a category of databases that are built for speed of response first, some of which have some pretty spaced out architecture under the hood. Aerospike comes to my mind, together with ScyllaDB or Redis maybe. They trade off this speed with massive compromises in complexity, consistency and scalability. GC would be disastrous for these, so they're all written in C(++). The main premise of CockroachDB is geographic replication, durability and scalability while maintaining full consistence and serialization. In order just to keep these promises, a large part of the design of Cockroach is that the DB needs to keep _waiting_ most of the time of any request until all geo replicated shards are consistent. We're talking dozens to hundreds of milliseconds here. And even if that wasn't the case, a full GC of 100 microseconds probably isn't even noticeable compared with the weight of complex SQL query planning and execution. You may stink a lot of valid points about Go, but this isn't one of them.

stubish 7y ago
Cassandra has huge problems with JVM garbage collection, and it still isn't rid of them in the decades it has been in development. The reason for the scary warnings if your partition sizes exceed 100MB is garbage collection. The reason you can't tune it to use all your RAM on a modern box is garbage collection. Picking JVM implementations and tuning mostly seems to revolve around garbage collection and its trade offs. A common symptom of poorly maintained clusters is second long garbage collection pauses. The trick with Go seems to be that programs don't seem to generate much garbage to collect, so I don't know if there is much use comparing with Java apps. But garbage collection overhead is certainly a major issue to be wary of for any application needing consistently low response times.