5 ms·

It's always good to see a case-study/anecdote, but nothing in here is surprising. It also doesn't really invalidate Go in any way. Rust is faster than Go. Peop

by _bxg1 7y ago

It's always good to see a case-study/anecdote, but nothing in here is surprising. It also doesn't really invalidate Go in any way. Rust is faster than Go. People use Go, like any other technology, when the tradeoffs between developer iteration/throughput/latency/etc. make sense. When those cease to make sense, a hot path gets converted down to something more efficient. This is the natural way of things.

kerkeslager 7y ago
> It's always good to see a case-study/anecdote, but nothing in here is surprising. It also doesn't really invalidate Go in any way. Well, sure, because categorizing languages as "valid/invalid" doesn't make any sense. But it does show yet another example of how designing a language to solve Google's fairly-unique problems doesn't result in a general-purpose language suitable for solving most people's problems.
- kikimora 7y ago
  Long GC pauses caused by large collections/caches are decade long problem with no real wide spread solution so far. With Java and .NET you can resort to off-heap data. Not sure if this is possible with Go.
  - kerkeslager 7y ago
    > Long GC pauses caused by large collections/caches are decade long problem with no real wide spread solution so far. This is arguable, but the fact is that "large collections/caches" isn't Discord's situation.
  - dnautics 7y ago
    Erlang's basically solved it (and, arguably, solved it decades ago); relevant as discord uses erlang VM in places.
    - bsder 7y ago
      Erlang "solved" the problem by breaking having lots of little heaps so a GC can blast through the entire heap extremely quickly. Would that actually work in this instance? It seems like that LRU cache they're talking about is kind of large.
      - kerkeslager 7y ago
        > Would that actually work in this instance? It seems like that LRU cache they're talking about is kind of large. I can't say for sure without knowing what the contents of that heap is, but I suspect that yes, it would work. However, the reason the heaps are so small is that they're each a lightweight thread, and in Erlang, spinning up new threads is a way of life. It would be hard to overstate what a fundamentally different architecture this is.
  - throwaway34241 7y ago
    Yes, it's not particularly difficult with Go either. The default array/list type (slices) can point to unmanaged memory so it's easy to feed unmanaged data directly into most APIs. Also you can directly use off heap objects to support interfaces and pass references to them around just like objects allocated on the Go heap. Of course there are no generics yet so doing things like re-using a custom hash table implementation will be less convenient.
- vogre 7y ago
  Keeping LRU cache that large with these performance requirements is not a "most people's problem". Go is actually great to solve most people's problem with web servers, while Rust is better for edge cases.
  - kerkeslager 7y ago
    > Keeping LRU cache that large with these performance requirements is not a "most people's problem". Sure, but that's not what I said. Any program of sufficient complexity will run into at least one critical problem that isn't a "most people's problem". A well-written general-purpose language implementation will have been written in such a way that that problem isn't totally intractable. > Go is actually great to solve most people's problem with web servers, while Rust is better for edge cases. Most people's problem with web servers is writing a CRUD app, which is going to be easiest in something like Python/Django/PostGres/Apache. It's not the new shiny, but it includes all the usual wheels so you don't have to reinvent them in the name of "simple". Similar toolsets exist for Ruby/Java/.NET. Give it a few years and similar toolsets will be invented for Go, I'm sure.
    - majewsky 7y ago
      > Give it a few years and similar toolsets will be invented for Go, I'm sure. Go 1.0 is nearly 8 years old at this point, and these toolsets have existed for years.
- d1zzy 7y ago
  > But it does show yet another example of how designing a language to solve Google's fairly-unique problems doesn't result in a general-purpose language suitable for solving most people's problems. How much of Google's infrastructure actually runs on Go tho? :)
- spenczar5 7y ago
  No, I don't think so. You would need to demonstrate that this application is representative of "most people's problems," which doesn't seem clear to me. This shows only a single example where Go is not very suitable, but it doesn't prove a general case on its own.
calcifer 7y ago
This is a weirdly defensive comment, fighting against a strawman. The article doesn't claim it's surprising, that "invalidates" Go or that it isn't the "natural way of things".
- _bxg1 7y ago
  I'm not pushing back against the article, but against the comments that tend to appear below articles like this. The headline in particular, to someone who doesn't read the article, could be taken as "Discord has decided that Rust is better than Go and here's why", and run with.
- spenczar5 7y ago
  There's a long history of Go and Rust partisans arguing this point. It's unfortunate, but I think it's reasonable to fight that strawman preemptively.
Symmetry 7y ago
I think the way I'd put it is that languages with manual memory management, like Rust, have more scope for optimization than languages that don't. You can just use the gc crate in Rust and have almost the same ease of development but the same performance problems you do in Go.
- nicoburns 7y ago
  Notably, it didn't sound like the development in Rust was particularly difficult anyway. That's certainly been my experience in any case.
  - Symmetry 7y ago
    It also sounded like the developers had already thought carefully about their memory usage patterns and had been optimizing for performance as much as they could within the scope Go allowed them. Personally I've found Rust has a higher cognitive overhead than Go when I'm just banging something out only worrying about correctness but if you're thinking carefully about memory usage patterns in a way you need to to optimize performance there's no penalty.

by _bxg1 7y ago

kerkeslager 7y ago

> It's always good to see a case-study/anecdote, but nothing in here is surprising. It also doesn't really invalidate Go in any way. Well, sure, because categorizing languages as "valid/invalid" doesn't make any sense. But it does show yet another example of how designing a language to solve Google's fairly-unique problems doesn't result in a general-purpose language suitable for solving most people's problems.

kikimora 7y ago
Long GC pauses caused by large collections/caches are decade long problem with no real wide spread solution so far. With Java and .NET you can resort to off-heap data. Not sure if this is possible with Go.
- kerkeslager 7y ago
  > Long GC pauses caused by large collections/caches are decade long problem with no real wide spread solution so far. This is arguable, but the fact is that "large collections/caches" isn't Discord's situation.
- dnautics 7y ago
  Erlang's basically solved it (and, arguably, solved it decades ago); relevant as discord uses erlang VM in places.
  - bsder 7y ago
    Erlang "solved" the problem by breaking having lots of little heaps so a GC can blast through the entire heap extremely quickly. Would that actually work in this instance? It seems like that LRU cache they're talking about is kind of large.
    - kerkeslager 7y ago
      > Would that actually work in this instance? It seems like that LRU cache they're talking about is kind of large. I can't say for sure without knowing what the contents of that heap is, but I suspect that yes, it would work. However, the reason the heaps are so small is that they're each a lightweight thread, and in Erlang, spinning up new threads is a way of life. It would be hard to overstate what a fundamentally different architecture this is.
- throwaway34241 7y ago
  Yes, it's not particularly difficult with Go either. The default array/list type (slices) can point to unmanaged memory so it's easy to feed unmanaged data directly into most APIs. Also you can directly use off heap objects to support interfaces and pass references to them around just like objects allocated on the Go heap. Of course there are no generics yet so doing things like re-using a custom hash table implementation will be less convenient.
vogre 7y ago
Keeping LRU cache that large with these performance requirements is not a "most people's problem". Go is actually great to solve most people's problem with web servers, while Rust is better for edge cases.
- kerkeslager 7y ago
  > Keeping LRU cache that large with these performance requirements is not a "most people's problem". Sure, but that's not what I said. Any program of sufficient complexity will run into at least one critical problem that isn't a "most people's problem". A well-written general-purpose language implementation will have been written in such a way that that problem isn't totally intractable. > Go is actually great to solve most people's problem with web servers, while Rust is better for edge cases. Most people's problem with web servers is writing a CRUD app, which is going to be easiest in something like Python/Django/PostGres/Apache. It's not the new shiny, but it includes all the usual wheels so you don't have to reinvent them in the name of "simple". Similar toolsets exist for Ruby/Java/.NET. Give it a few years and similar toolsets will be invented for Go, I'm sure.
  - majewsky 7y ago
    > Give it a few years and similar toolsets will be invented for Go, I'm sure. Go 1.0 is nearly 8 years old at this point, and these toolsets have existed for years.
d1zzy 7y ago
> But it does show yet another example of how designing a language to solve Google's fairly-unique problems doesn't result in a general-purpose language suitable for solving most people's problems. How much of Google's infrastructure actually runs on Go tho? :)
spenczar5 7y ago
No, I don't think so. You would need to demonstrate that this application is representative of "most people's problems," which doesn't seem clear to me. This shows only a single example where Go is not very suitable, but it doesn't prove a general case on its own.

calcifer 7y ago

This is a weirdly defensive comment, fighting against a strawman. The article doesn't claim it's surprising, that "invalidates" Go or that it isn't the "natural way of things".

_bxg1 7y ago
I'm not pushing back against the article, but against the comments that tend to appear below articles like this. The headline in particular, to someone who doesn't read the article, could be taken as "Discord has decided that Rust is better than Go and here's why", and run with.
spenczar5 7y ago
There's a long history of Go and Rust partisans arguing this point. It's unfortunate, but I think it's reasonable to fight that strawman preemptively.

Symmetry 7y ago

I think the way I'd put it is that languages with manual memory management, like Rust, have more scope for optimization than languages that don't. You can just use the gc crate in Rust and have almost the same ease of development but the same performance problems you do in Go.

nicoburns 7y ago
Notably, it didn't sound like the development in Rust was particularly difficult anyway. That's certainly been my experience in any case.
- Symmetry 7y ago
  It also sounded like the developers had already thought carefully about their memory usage patterns and had been optimizing for performance as much as they could within the scope Go allowed them. Personally I've found Rust has a higher cognitive overhead than Go when I'm just banging something out only worrying about correctness but if you're thinking carefully about memory usage patterns in a way you need to to optimize performance there's no penalty.