5 ms·

I don’t disagree that various automatic memory management strategies can be very effective and efficient along various metrics. However, there are cases where ‘

I don’t disagree that various automatic memory management strategies can be very effective and efficient along various metrics. However, there are cases where ‘throwing more hardware’ at a problem is not a viable solution. Typical examples, that I think are broadly understood and commercially available, are soft-real-time simulations with advanced graphics and audio (fancy words for video games) and audio creation/performance/recording. These two areas are typically bound to be using a large portion of consumer computing device resources and GC is typically just too much overhead in certain performance metrics.

suremarc 4y ago
My company offers a data feed over WebSocket that could reach 70MB/s if we allowed you to subscribe to the entire feed at once (we don’t). No one ever planned for it to handle that much data, but here we are. Because WebSocket servers do not scale horizontally we had to consume the entire feed on one machine. (Edit: ofc you can scale the # of clients handled, but not the data throughput!) Initially our Go app could only manage reading 10MB/s from Kafka, which is pretty sad. After switching from Shopify’s “Sarama” Go library to librdkafka (a Kafka client written in C), as well as some other arcane optimizations, we were able to 20x our throughput and handle up to 200MB/s, despite the overhead of interop between C and Go. On top of this, we used a library that wrapped librdkafka in Go bindings (confluent-kafka-go) and it turned out to have a much simpler interface than Sarama, in addition to it outperforming Sarama 20x. While we didn’t end up rewriting the thing in Rust, we did end up leveraging a library written in C for major performance wins, in a case where scaling out was not possible. And let us also remember that even when scaling horizontally is possible, it is not necessarily even desirable in all circumstances: distributed systems, aside from introducing some amount of overhead, are basically Pandora’s box wrt. failure modes.
- jillesvangurp 4y ago
  Go is nice but it certainly has its limitations. I've seen heavily optimized jvm based things run circles around the kind of numbers you are talking about. 70 MB/s is not a number that should scare anyone running things like Kafka at scale. I've worked with Elasticsearch clusters indexing close to a million documents per second from kafka. That takes a lot of CPU, memory, bare metal, etc. and it taxes the JVM garbage collection quite a bit. But it can be done. We basically maxed out CPUs across 30 or so nodes. Basically, when you get close to maxing out the disk IO, you know you are doing a decent job. People reach for C for a lot of reasons. And not always the right reasons. But if it works, it works. A big reason seems to simply be that interfacing with it from various other languages is pretty straightforward. That's why python developers don't worry about performance. They can always plug in something native if it becomes an issue.
  - suremarc 4y ago
    Again, scaling out to 30 nodes was not an option for us. It had to run on a single machine, unlike the rest of our data pipeline. While 200MB/s is nowhere near the theoretical limit of our hardware, it is near what Kafka’s stress testing tools achieved in the same setup, particularly with our record size (100 bytes).
- pjmlp 4y ago
  I am quite sure those arcane C optimizations are also possible in Go, it is a matter to learn how to properly use value types and the unsafe package.
  - suremarc 4y ago
    I meant that we did optimizations in Go, on top of using a Kafka client written in C. When language constructs like channels degrade performance, and when goroutine scheduling takes 20% of your cpu time, it begs the question of whether you’re using the right tool for the job. If you are using unsafe to thwart the garbage collector, this is probably even more true.
    - pjmlp 4y ago
      Same reasoning applies to C when using inline Assembler then.
jillesvangurp 4y ago
You are not wrong of course. But it's interesting to see many commercial 3d gaming engines use things like Lua, C# and other garbage collected languages as the main way for game developers to use their engines. I grew up in an era where all that stuff was done in assembly. That stopped being a thing decades ago. The vast majority of commercial software development can use off the shelf native components and pair them with garbage collected languages these days.

suremarc 4y ago

My company offers a data feed over WebSocket that could reach 70MB/s if we allowed you to subscribe to the entire feed at once (we don’t). No one ever planned for it to handle that much data, but here we are. Because WebSocket servers do not scale horizontally we had to consume the entire feed on one machine. (Edit: ofc you can scale the # of clients handled, but not the data throughput!) Initially our Go app could only manage reading 10MB/s from Kafka, which is pretty sad. After switching from Shopify’s “Sarama” Go library to librdkafka (a Kafka client written in C), as well as some other arcane optimizations, we were able to 20x our throughput and handle up to 200MB/s, despite the overhead of interop between C and Go. On top of this, we used a library that wrapped librdkafka in Go bindings (confluent-kafka-go) and it turned out to have a much simpler interface than Sarama, in addition to it outperforming Sarama 20x. While we didn’t end up rewriting the thing in Rust, we did end up leveraging a library written in C for major performance wins, in a case where scaling out was not possible. And let us also remember that even when scaling horizontally is possible, it is not necessarily even desirable in all circumstances: distributed systems, aside from introducing some amount of overhead, are basically Pandora’s box wrt. failure modes.

jillesvangurp 4y ago
Go is nice but it certainly has its limitations. I've seen heavily optimized jvm based things run circles around the kind of numbers you are talking about. 70 MB/s is not a number that should scare anyone running things like Kafka at scale. I've worked with Elasticsearch clusters indexing close to a million documents per second from kafka. That takes a lot of CPU, memory, bare metal, etc. and it taxes the JVM garbage collection quite a bit. But it can be done. We basically maxed out CPUs across 30 or so nodes. Basically, when you get close to maxing out the disk IO, you know you are doing a decent job. People reach for C for a lot of reasons. And not always the right reasons. But if it works, it works. A big reason seems to simply be that interfacing with it from various other languages is pretty straightforward. That's why python developers don't worry about performance. They can always plug in something native if it becomes an issue.
- suremarc 4y ago
  Again, scaling out to 30 nodes was not an option for us. It had to run on a single machine, unlike the rest of our data pipeline. While 200MB/s is nowhere near the theoretical limit of our hardware, it is near what Kafka’s stress testing tools achieved in the same setup, particularly with our record size (100 bytes).
pjmlp 4y ago
I am quite sure those arcane C optimizations are also possible in Go, it is a matter to learn how to properly use value types and the unsafe package.
- suremarc 4y ago
  I meant that we did optimizations in Go, on top of using a Kafka client written in C. When language constructs like channels degrade performance, and when goroutine scheduling takes 20% of your cpu time, it begs the question of whether you’re using the right tool for the job. If you are using unsafe to thwart the garbage collector, this is probably even more true.
  - pjmlp 4y ago
    Same reasoning applies to C when using inline Assembler then.

jillesvangurp 4y ago

You are not wrong of course. But it's interesting to see many commercial 3d gaming engines use things like Lua, C# and other garbage collected languages as the main way for game developers to use their engines. I grew up in an era where all that stuff was done in assembly. That stopped being a thing decades ago. The vast majority of commercial software development can use off the shelf native components and pair them with garbage collected languages these days.