4 ms·

That was my takeaway from folks who had moved from Go to Rust. I'm not a Go expert though, so feel free to correct me. https://blog.discord.com/why-discord-is-

by leerob 5y ago

That was my takeaway from folks who had moved from Go to Rust. I'm not a Go expert though, so feel free to correct me. https://blog.discord.com/why-discord-is-switching-from-go-to-rust-a190bbca2b1f https://blog.discord.com/why-discord-is-switching-from-go-to...

j03b 5y ago
The takeaways in this article are much more why Go's memory model becomes difficult at scale. When you're dealing with memory at Discord scale garbage collection is hard, for example on the mentioned Discord microservice: > There are millions of Users in each cache. There are tens of millions of Read States in each cache. There are hundreds of thousands of cache updates per second. This is something that is probably never going to be hit locally in the development toolchain. You can certainly prefer the Rust memory system to Go and have that be a valid reason to use Rust over Go for something like dev toolchains, but you're not going to hit scale problems like those mentioned in the article.
- foota 5y ago
  I don't know, building JavaScript things involves similar orders of magnitude.
- AaronFriel 5y ago
  Not just at scale, Go doesn't make it obvious when an allocation is on the heap or stack. The escape analysis can change (hopefully without regressions) from version to version of Go, and that can mean performance regressions can happen. It also means small refactors risk de-optimizing code without an obvious reason why that would be the case. I think more engineers are becoming skeptical of optimizing compilers with that sort of implicit, potential "spooky action at a distance" where nonlocal code can cause performance regressions. In Haskell, laziness and iterator fusion can do the same thing. Nonlocal code can cause local expressions to compile differently, resulting in - from an engineer's POV - nondeterministic performance, memory usage, GC pressure. "Space leaks" can also occur, again, due to nonlocal effects. In JavaScript, the JIT can similarly cause local regressions due to nonlocal code. A single expression deep in a call stack mutating an object can result in it no longer fitting an optimized "shape", resulting in function deoptimization. Just say no to spooky action at a distance in programming languages, in my opinion. It leads to tremendously difficult to debug regressions.
  - kgeist 5y ago
    >Not just at scale, Go doesn't make it obvious when an allocation is on the heap or stack. The escape analysis can change I thought heap allocations are only triggered by taking pointers, i.e. using reference semantics (including casting to interface and calling a method whose receiver is by-ref)? If I'm careful to only use variables by-value (for types that can afford it, i.e. excluding map/array), what else can trigger heap allocation?
    - jdreaver 5y ago
      Making a closure usually requires allocation. For example: package main import "fmt" func main() { closure := make_closure() fmt.Println("closure():", closure()) } func make_closure() func() int { x := 1 return func() int { return x } } This prints: $ go run main.go closure(): 1 If x were allocated on the stack, it would get nuked after we returned from make_closure(). In Rust you could move x to the closure, but I think in this Go example x would be heap allocated, assuming the Go compiler doesn't notice it can inline all of this and avoid allocating. Maybe assume a more complex example with a struct that had to be computed via a function argument or something :)
      - kgeist 5y ago
        Oh yes, closures too. Coming from C++ background, you kinda intuitively understand when things escape and when they don't, because you assume that Go would implement it in the most straightforward way (like it usually does), and in the case of closures, heap-allocating a captured variable is the simplest implementation (just let GC handle it!) considering the implicit reference semantics (that you can change "x" anywhere outside of the closure and it should be immediately visible in the closure, too). I.e. assume the worst (and don't expect Go to do some clever special-casing similar to move semantics in C++/Rust), and it's often the right answer :)
  - Thaxll 5y ago
    > the escape analysis can change (hopefully without regressions) from version to version of Go, and that can mean performance regressions can happen. Because it's not the case with Rust or any compiled language? LLVM has shown regressions wich is what Rust uses to compile code. 2sec search on google: https://github.com/rust-lang/rust/issues/24194 https://github.com/rust-lang/rust/issues/24194
kgeist 5y ago
Storing billions of states in garbage-collected memory sounds like using a wrong tool for the job (by the way, I wonder why they didn't use something like Redis? didn't scale well?). I think it's an edge case they ran into -- which doesn't mean Go's memory management is painful per se. It's not secret that large heaps with many old generation objects are problematic in presence of GC. And few projects have requirements of the scale of Discord. For stateless servers, Go is more than enough in my experience; in our services we do allocate garbage like there's no tomorrow and we're yet to run into GC problems, because all caching is delegated to a redis cluster and all garbage we produce during request processing is ephemeral so we don't have large object graphs that would slow down the GC. Rust is perfect for this kind of "core infrastructure" projects where you need maximum performance (Redis itself is written in C), but actual business logic on top of such infrastructure can easily be written in a simpler language like Go and GC is rarely a bottleneck there.
- dahfizz 5y ago
  According to Redis themselves, a high end and well tuned redis server will get you ~200k requests per second[1]. This is if all your application does is send requests in a loop. Am I correct that redis does not offer a native API and so forces you to craft commands as strings? Just having to run snprintf 500k per second wastes a considerable amount of CPU. Combined with the normal overhead of communicating over a socket, using redis is going to be considerably slower than just keeping that memory in your own process. That's assuming the overhead of the redis internals is 0. [1] https://redis.io/topics/benchmarks https://redis.io/topics/benchmarks
  - kgeist 5y ago
    In the article, they say: >The service we switched from Go to Rust is the “Read States” service. Its sole purpose is to keep track of which channels and messages you have read >On cache key eviction, we commit your Read States to the database. We also schedule a database commit for 30 seconds in the future whenever a Read State is updated. So it's a separate service whose only purpose is to store data in-memory with occasional commits to persisted DB, which sounded a lot like Redis. But of course, a special optimized service is always going to outperform a general purpose tool such as Redis. They also replaced HashMaps with BTreeMaps, reduced the number of memory copies they were doing, etc. -- which has nothing to do with Go, but also contributed to a performance increase
tialaramex 5y ago
Two things about this pain: * They incur the pain even when nothing needs doing. This is really terrible. If you spend ten minutes furiously reading data from a network device into a fixed RAM buffer, twiddling it, and writing it out again, in Rust (or C, C++ and various other explicit allocation languages) you neither allocated nor de-allocated anything, therefore you pay exactly zero for this work you didn't do. In Go you will incur the cost of the GC verifying that yup, everything is as it was, there's nothing to do, apparently five times. * The pain is swallowed in lumps, you can't spread it out. So you must either size hardware for lump swallowing, even though as we saw above we don't even want to swallow any lumps, or accept poor performance each time a lump is swallowed. With explicit allocation you can choose to swallow bigger lumps (e.g. arena allocation) to reduce overheads elsewhere, but you decide on the size of lumps you want for your allocation.

by leerob 5y ago

j03b 5y ago

The takeaways in this article are much more why Go's memory model becomes difficult at scale. When you're dealing with memory at Discord scale garbage collection is hard, for example on the mentioned Discord microservice: > There are millions of Users in each cache. There are tens of millions of Read States in each cache. There are hundreds of thousands of cache updates per second. This is something that is probably never going to be hit locally in the development toolchain. You can certainly prefer the Rust memory system to Go and have that be a valid reason to use Rust over Go for something like dev toolchains, but you're not going to hit scale problems like those mentioned in the article.

foota 5y ago
I don't know, building JavaScript things involves similar orders of magnitude.
AaronFriel 5y ago
Not just at scale, Go doesn't make it obvious when an allocation is on the heap or stack. The escape analysis can change (hopefully without regressions) from version to version of Go, and that can mean performance regressions can happen. It also means small refactors risk de-optimizing code without an obvious reason why that would be the case. I think more engineers are becoming skeptical of optimizing compilers with that sort of implicit, potential "spooky action at a distance" where nonlocal code can cause performance regressions. In Haskell, laziness and iterator fusion can do the same thing. Nonlocal code can cause local expressions to compile differently, resulting in - from an engineer's POV - nondeterministic performance, memory usage, GC pressure. "Space leaks" can also occur, again, due to nonlocal effects. In JavaScript, the JIT can similarly cause local regressions due to nonlocal code. A single expression deep in a call stack mutating an object can result in it no longer fitting an optimized "shape", resulting in function deoptimization. Just say no to spooky action at a distance in programming languages, in my opinion. It leads to tremendously difficult to debug regressions.
- kgeist 5y ago
  >Not just at scale, Go doesn't make it obvious when an allocation is on the heap or stack. The escape analysis can change I thought heap allocations are only triggered by taking pointers, i.e. using reference semantics (including casting to interface and calling a method whose receiver is by-ref)? If I'm careful to only use variables by-value (for types that can afford it, i.e. excluding map/array), what else can trigger heap allocation?
  - jdreaver 5y ago
    Making a closure usually requires allocation. For example: package main import "fmt" func main() { closure := make_closure() fmt.Println("closure():", closure()) } func make_closure() func() int { x := 1 return func() int { return x } } This prints: $ go run main.go closure(): 1 If x were allocated on the stack, it would get nuked after we returned from make_closure(). In Rust you could move x to the closure, but I think in this Go example x would be heap allocated, assuming the Go compiler doesn't notice it can inline all of this and avoid allocating. Maybe assume a more complex example with a struct that had to be computed via a function argument or something :)
    - kgeist 5y ago
      Oh yes, closures too. Coming from C++ background, you kinda intuitively understand when things escape and when they don't, because you assume that Go would implement it in the most straightforward way (like it usually does), and in the case of closures, heap-allocating a captured variable is the simplest implementation (just let GC handle it!) considering the implicit reference semantics (that you can change "x" anywhere outside of the closure and it should be immediately visible in the closure, too). I.e. assume the worst (and don't expect Go to do some clever special-casing similar to move semantics in C++/Rust), and it's often the right answer :)
- Thaxll 5y ago
  > the escape analysis can change (hopefully without regressions) from version to version of Go, and that can mean performance regressions can happen. Because it's not the case with Rust or any compiled language? LLVM has shown regressions wich is what Rust uses to compile code. 2sec search on google: https://github.com/rust-lang/rust/issues/24194 https://github.com/rust-lang/rust/issues/24194

kgeist 5y ago

Storing billions of states in garbage-collected memory sounds like using a wrong tool for the job (by the way, I wonder why they didn't use something like Redis? didn't scale well?). I think it's an edge case they ran into -- which doesn't mean Go's memory management is painful per se. It's not secret that large heaps with many old generation objects are problematic in presence of GC. And few projects have requirements of the scale of Discord. For stateless servers, Go is more than enough in my experience; in our services we do allocate garbage like there's no tomorrow and we're yet to run into GC problems, because all caching is delegated to a redis cluster and all garbage we produce during request processing is ephemeral so we don't have large object graphs that would slow down the GC. Rust is perfect for this kind of "core infrastructure" projects where you need maximum performance (Redis itself is written in C), but actual business logic on top of such infrastructure can easily be written in a simpler language like Go and GC is rarely a bottleneck there.

dahfizz 5y ago
According to Redis themselves, a high end and well tuned redis server will get you ~200k requests per second[1]. This is if all your application does is send requests in a loop. Am I correct that redis does not offer a native API and so forces you to craft commands as strings? Just having to run snprintf 500k per second wastes a considerable amount of CPU. Combined with the normal overhead of communicating over a socket, using redis is going to be considerably slower than just keeping that memory in your own process. That's assuming the overhead of the redis internals is 0. [1] https://redis.io/topics/benchmarks https://redis.io/topics/benchmarks
- kgeist 5y ago
  In the article, they say: >The service we switched from Go to Rust is the “Read States” service. Its sole purpose is to keep track of which channels and messages you have read >On cache key eviction, we commit your Read States to the database. We also schedule a database commit for 30 seconds in the future whenever a Read State is updated. So it's a separate service whose only purpose is to store data in-memory with occasional commits to persisted DB, which sounded a lot like Redis. But of course, a special optimized service is always going to outperform a general purpose tool such as Redis. They also replaced HashMaps with BTreeMaps, reduced the number of memory copies they were doing, etc. -- which has nothing to do with Go, but also contributed to a performance increase

tialaramex 5y ago

Two things about this pain: * They incur the pain even when nothing needs doing. This is really terrible. If you spend ten minutes furiously reading data from a network device into a fixed RAM buffer, twiddling it, and writing it out again, in Rust (or C, C++ and various other explicit allocation languages) you neither allocated nor de-allocated anything, therefore you pay exactly zero for this work you didn't do. In Go you will incur the cost of the GC verifying that yup, everything is as it was, there's nothing to do, apparently five times. * The pain is swallowed in lumps, you can't spread it out. So you must either size hardware for lump swallowing, even though as we saw above we don't even want to swallow any lumps, or accept poor performance each time a lump is swallowed. With explicit allocation you can choose to swallow bigger lumps (e.g. arena allocation) to reduce overheads elsewhere, but you decide on the size of lumps you want for your allocation.