6 ms·

Disagree. I can write a server in C++ that, with careful thought and a bit of planning, will be able to exploit the resources of the kind of 88-core dual-socke

Disagree. I can write a server in C++ that, with careful thought and a bit of planning, will be able to exploit the resources of the kind of 88-core dual-socket machines that are mainstream today. No amount of planning will allow me to write a Go program that does the same thing on the same machine. Small amounts of concurrency might seem easy in Go but lots of concurrency is hard.

cdelsolar 7y ago
Where's the evidence that Go can't handle large amounts of concurrency?
- shereadsthenews 7y ago
  All over my profiler. Go can deal with something embarrassingly parallel, same as any other language, but try getting it to scale up to high core counts with small RPCs is impossible, for the reasons the OP mentioned: you can't avoid its global allocator and you don't control the scheduler either. You will spend all of your CPU time in runtime.findrunnable and runtime.mallocgc.
  - stcredzero 7y ago
    You will spend all of your CPU time in runtime.findrunnable Profile your channel latency, then design around that. "Small RPCs?" With an 88 core server, should you be making that many remote procedure calls? Find the 20% most intensive code and take that out of the hands of the scheduler, as much as possible. and runtime.mallocgc Even with sync.Pool? Can you design those parts of the system to mostly use the stack?
    - kasey_junk 7y ago
      Not for nothing but channels become a bottleneck pretty early when you start doing highly concurrent golang. You can very carefully design around it using only channels but it quickly starts making more sense to just use a different concurrency abstraction.
      - chessturk 7y ago
        My experience has been channels are a really good way to make highly concurrent code readable for people who hate concurrency (ymmv). They aren't as efficient, but using them as a semaphore (i.e. only signaling) can facilitate fairly-efficient, easier-to-read shared memory. But I'm kind of proving the whole... > You can very carefully design around it only channels ...part of your post.
  - networkimprov 7y ago
    Is there a Github issue for this problem? If not, would you consider filing one? https://github.com/golang/go/issues https://github.com/golang/go/issues It should probably explain why one cannot "avoid its global allocator" by using pools and/or stack objects.
    - shereadsthenews 7y ago
      Stack is great, then you will spend more time in runtime.morestack because the Go authors have wisely decided that we are all too dumb to request specific stack sizes and we must rely on copying and doubling stacks larger than 2KiB even if we know in advance that 2KiB is not enough. This compares poorly with native threads where the stacks are dynamically allocated one page at a time without copying and we can specify the stack size _limit_ per thread if we need to. This is another place where the Go authors just arbitrarily decided that users cannot be trusted. sync.Pool is fun but it has known scalability limits that are yet to be addressed in a released runtime (see https://go-review.googlesource.com/c/go/+/166960/ https://go-review.googlesource.com/c/go/+/166960/ for example). You also can't use sync.Pool for anything that you need to be non-ephemeral, like CPU-local counters, because the GC can just blow them away at any time and the finalizers run at arbitrary future times or possibly never.
      - networkimprov 7y ago
        My point is that you should file a Github issue if there isn't one.
        icebraining 7y ago
        Why "should"? Nobody owes the writing of issues.
        networkimprov 7y ago
        If you depend on an open source project, and can articulate a problem with it that can be fixed, then I do feel you should file an issue to contribute your insight.
  - weberc2 7y ago
    I guarantee you the "little bit of planning" you're doing in C++ is far greater than the equivalent planning in Go to avoid the global allocator in your goroutines (i.e., allocate on the stack, use object pools, etc). I'm sure you still won't get to the same performance ceiling that C++ allows, but it doesn't mean "Go is bad at concurrency".
    - chessturk 7y ago
      The philosophy of Go, as far as I can tell, is opinionated simplicity. In this way, it is fantastic at concurrency. It gives you a simplified fork in goroutines, a way to safely pass by value or signal between these through channels, and structures like select to psuedo randomly deal with race conditions. That stuff is all awesome! But when you are talking about performance, opinionated simplicity is a bad philosophy. You are going to want the finer grained control the poster is asking for, and which Go deliberately doesn't allow for. C, C++, and Rust will always have a speed advantage because they lack the overhead of the GC and were designed with allowing the developer minute control of the system, which is the space to make such optimizations. That said, opininated simplicity means every Go codebase I drop into looks roughly the same. That is not true in the above langauges, precisely because they allow for more control. To me, Go's ideal use case is the domains of Java, Python, NodeJS, and even Elixir -- where it has a performance edge in many cases and where its consistant style really shine. All that to say, Go is bad at performance optimized concurrency, as designed.
      - weberc2 7y ago
        Your post doesn’t address mine. I gave specific advice for addressing the GP’s issue. It’s much easier to move your allocs out of the hot path in Go than it is to do virtually anything in C++. I’m not claiming Go will compete with C++ on performance, only that his argument isn’t a good reason to write off Go. (I’ve written a lot of both languages).
Thaxll 7y ago
You're talking a about a very specific scenario, out of the box Go is off course easier than C++ for concurrency. Who by the way runs applications on a 88 core server, why would you so they instead of splitting that into smaller chunks.
- shereadsthenews 7y ago
  You're making a different argument, that one does not need so much vertical scalability, and I might agree, but that doesn't negate the fact that Go has limits of vertical scalability that C++ doesn't face.
  - weberc2 7y ago
    The original claim was that concurrency is easier in Go, not that Go has a higher performance ceiling. You are the one making the different argument. :)
- derefr 7y ago
  > Who by the way runs applications on a 88 core server, why would you so they instead of splitting that into smaller chunks. Graph analysis and/or routing, for one. Distributed Dijkstra is pretty impractical (the optimal lower bound on the number of messages required equals the number of edges in the graph!) For example, I don't work for Google, but I can pretty much guarantee you that an individual Google Maps routing query runs on a single, extremely-high-memory (but not NUMA!) node, which has likely been optimized for speed using plain-old threading on a high-core-count CPU, with the upper bound on the number of useful cores being the system architecture's per-socket memory bandwidth.
  - foobiekr 7y ago
    I would be very, very curious if this is actually true. I agree with the intuition behind the statement but I am actually curious how it is done.
  - Thaxll 7y ago
    I highly doubt Google runs any workload on Borg with CPU count > 24. I don't work there, so I'm just assuming but it goes against all modern scaling / deployments patterns.
jerf 7y ago
"I can write a server in C++ that, with careful thought and a bit of planning, will be able to exploit the resources of the kind of 88-core dual-socket machines that are mainstream today." This is downvoted grey as I write this, and as someone who has been called a "Go shill" on occasion... it's exactly right. Go is a decent language for writing code in a fairly straightforward manner and getting pretty good performance out of it, but if you need to squeeze every bit of performance out of your hardware, it's a bad choice. (I can give you choices that are worse by an order of magnitude, or even more in some cases, but it's still a bad choice.) You have a fairly smooth optimization ride up to 1.5-2x slower than C for most use cases, a few pathological edge cases where it's grossly worse (many clustered around these "every drop of performance" problems!) and a few where it'll reach parity, and then you're going to hit a brick wall. An 88 core system is probably not impossible to sensibly use with Go, but you're going to be more constrained. I'd imagine it can probably serve web requests really well, but it's more likely to hit pathological cases if you hammer certain global resources. Arguably, precisely part of the point of Go was that C++ makes you pay for that level of performance in code complexity and cognitive overhead all the time, even when you don't remotely need it. (I can't prove this, but I'd guess the median "cloud service" is grotesquely overprovisioned on the smallest AWS instance. The "cloud services" that leap to mind are things like the AWS auth servers or Netflix content servers or the Google crawling or indexing servers, but while those are huge and important, they're also in many dimensions the exceptions. A good chunk of the popularity of "serverless" is probably a result of this.) When you do need every bit of performance, though, the list of viable options is short.
- chessturk 7y ago
  As a fellow gopher, I agree, Go isn't trying to compete with C/C++ on performance, it's trying to give you safety and ease of use of high level language. I think the point about concurrency typifies this, channels aren't as fast as mutexs and semaphores, but it makes sharing code with co workers easier to reason about. If you're in a domain where performance is still king, Go isn't trying to find a place there.
- fouc 7y ago
  How would Rust compare?
lenkite 7y ago
Could you describe how you would go about designing this and what libraries/frameworks would you use ? AFAIK co-routines is only coming with C++ 20, so I guess you would still use std::thread right ?
- alexhutcheson 7y ago
  std::thread has only been around since C++11, so most existing HPC codebases probably either use pthreads[1] directly, or an organization- or project-specific library wrapping pthreads. Some used boost::Thread, which was effectively the predecessor of std::thread. If you want to saturate every core on a very parallel problem like "handle as many packets as you can", a very rough first approach would be to spin up 88 threads (one per core). Within each thread, you could use either something like boost::Fiber[2] (similar to goroutines, except mapped N:1 to OS threads, rather than M:N) to avoid blocking the threads on IO. This paper[3] has a good overview of different concurrency approaches. If you're doing something that's not embarrassingly parallel[4], then often there is a well-researched approach for your specific domain. The same general ideas apply to keeping your cores busy, but you're often more bounded by communication between threads, memory bandwidth, etc. [1] https://en.wikipedia.org/wiki/POSIX_Threads https://en.wikipedia.org/wiki/POSIX_Threads [2] https://www.boost.org/doc/libs/1_70_0/libs/fiber/doc/html/fiber/overview.html https://www.boost.org/doc/libs/1_70_0/libs/fiber/doc/html/fi... [3] http://www.sosp.org/2001/papers/welsh.pdf http://www.sosp.org/2001/papers/welsh.pdf [4] https://en.wikipedia.org/wiki/Embarrassingly_parallel https://en.wikipedia.org/wiki/Embarrassingly_parallel

cdelsolar 7y ago

Where's the evidence that Go can't handle large amounts of concurrency?

shereadsthenews 7y ago
All over my profiler. Go can deal with something embarrassingly parallel, same as any other language, but try getting it to scale up to high core counts with small RPCs is impossible, for the reasons the OP mentioned: you can't avoid its global allocator and you don't control the scheduler either. You will spend all of your CPU time in runtime.findrunnable and runtime.mallocgc.
- stcredzero 7y ago
  You will spend all of your CPU time in runtime.findrunnable Profile your channel latency, then design around that. "Small RPCs?" With an 88 core server, should you be making that many remote procedure calls? Find the 20% most intensive code and take that out of the hands of the scheduler, as much as possible. and runtime.mallocgc Even with sync.Pool? Can you design those parts of the system to mostly use the stack?
  - kasey_junk 7y ago
    Not for nothing but channels become a bottleneck pretty early when you start doing highly concurrent golang. You can very carefully design around it using only channels but it quickly starts making more sense to just use a different concurrency abstraction.
    - chessturk 7y ago
      My experience has been channels are a really good way to make highly concurrent code readable for people who hate concurrency (ymmv). They aren't as efficient, but using them as a semaphore (i.e. only signaling) can facilitate fairly-efficient, easier-to-read shared memory. But I'm kind of proving the whole... > You can very carefully design around it only channels ...part of your post.
- networkimprov 7y ago
  Is there a Github issue for this problem? If not, would you consider filing one? https://github.com/golang/go/issues https://github.com/golang/go/issues It should probably explain why one cannot "avoid its global allocator" by using pools and/or stack objects.
  - shereadsthenews 7y ago
    Stack is great, then you will spend more time in runtime.morestack because the Go authors have wisely decided that we are all too dumb to request specific stack sizes and we must rely on copying and doubling stacks larger than 2KiB even if we know in advance that 2KiB is not enough. This compares poorly with native threads where the stacks are dynamically allocated one page at a time without copying and we can specify the stack size _limit_ per thread if we need to. This is another place where the Go authors just arbitrarily decided that users cannot be trusted. sync.Pool is fun but it has known scalability limits that are yet to be addressed in a released runtime (see https://go-review.googlesource.com/c/go/+/166960/ https://go-review.googlesource.com/c/go/+/166960/ for example). You also can't use sync.Pool for anything that you need to be non-ephemeral, like CPU-local counters, because the GC can just blow them away at any time and the finalizers run at arbitrary future times or possibly never.
    - networkimprov 7y ago
      My point is that you should file a Github issue if there isn't one.
      - icebraining 7y ago
        Why "should"? Nobody owes the writing of issues.
        networkimprov 7y ago
        If you depend on an open source project, and can articulate a problem with it that can be fixed, then I do feel you should file an issue to contribute your insight.
- weberc2 7y ago
  I guarantee you the "little bit of planning" you're doing in C++ is far greater than the equivalent planning in Go to avoid the global allocator in your goroutines (i.e., allocate on the stack, use object pools, etc). I'm sure you still won't get to the same performance ceiling that C++ allows, but it doesn't mean "Go is bad at concurrency".
  - chessturk 7y ago
    The philosophy of Go, as far as I can tell, is opinionated simplicity. In this way, it is fantastic at concurrency. It gives you a simplified fork in goroutines, a way to safely pass by value or signal between these through channels, and structures like select to psuedo randomly deal with race conditions. That stuff is all awesome! But when you are talking about performance, opinionated simplicity is a bad philosophy. You are going to want the finer grained control the poster is asking for, and which Go deliberately doesn't allow for. C, C++, and Rust will always have a speed advantage because they lack the overhead of the GC and were designed with allowing the developer minute control of the system, which is the space to make such optimizations. That said, opininated simplicity means every Go codebase I drop into looks roughly the same. That is not true in the above langauges, precisely because they allow for more control. To me, Go's ideal use case is the domains of Java, Python, NodeJS, and even Elixir -- where it has a performance edge in many cases and where its consistant style really shine. All that to say, Go is bad at performance optimized concurrency, as designed.
    - weberc2 7y ago
      Your post doesn’t address mine. I gave specific advice for addressing the GP’s issue. It’s much easier to move your allocs out of the hot path in Go than it is to do virtually anything in C++. I’m not claiming Go will compete with C++ on performance, only that his argument isn’t a good reason to write off Go. (I’ve written a lot of both languages).

Thaxll 7y ago

You're talking a about a very specific scenario, out of the box Go is off course easier than C++ for concurrency. Who by the way runs applications on a 88 core server, why would you so they instead of splitting that into smaller chunks.

shereadsthenews 7y ago
You're making a different argument, that one does not need so much vertical scalability, and I might agree, but that doesn't negate the fact that Go has limits of vertical scalability that C++ doesn't face.
- weberc2 7y ago
  The original claim was that concurrency is easier in Go, not that Go has a higher performance ceiling. You are the one making the different argument. :)
derefr 7y ago
> Who by the way runs applications on a 88 core server, why would you so they instead of splitting that into smaller chunks. Graph analysis and/or routing, for one. Distributed Dijkstra is pretty impractical (the optimal lower bound on the number of messages required equals the number of edges in the graph!) For example, I don't work for Google, but I can pretty much guarantee you that an individual Google Maps routing query runs on a single, extremely-high-memory (but not NUMA!) node, which has likely been optimized for speed using plain-old threading on a high-core-count CPU, with the upper bound on the number of useful cores being the system architecture's per-socket memory bandwidth.
- foobiekr 7y ago
  I would be very, very curious if this is actually true. I agree with the intuition behind the statement but I am actually curious how it is done.
- Thaxll 7y ago
  I highly doubt Google runs any workload on Borg with CPU count > 24. I don't work there, so I'm just assuming but it goes against all modern scaling / deployments patterns.

jerf 7y ago

"I can write a server in C++ that, with careful thought and a bit of planning, will be able to exploit the resources of the kind of 88-core dual-socket machines that are mainstream today." This is downvoted grey as I write this, and as someone who has been called a "Go shill" on occasion... it's exactly right. Go is a decent language for writing code in a fairly straightforward manner and getting pretty good performance out of it, but if you need to squeeze every bit of performance out of your hardware, it's a bad choice. (I can give you choices that are worse by an order of magnitude, or even more in some cases, but it's still a bad choice.) You have a fairly smooth optimization ride up to 1.5-2x slower than C for most use cases, a few pathological edge cases where it's grossly worse (many clustered around these "every drop of performance" problems!) and a few where it'll reach parity, and then you're going to hit a brick wall. An 88 core system is probably not impossible to sensibly use with Go, but you're going to be more constrained. I'd imagine it can probably serve web requests really well, but it's more likely to hit pathological cases if you hammer certain global resources. Arguably, precisely part of the point of Go was that C++ makes you pay for that level of performance in code complexity and cognitive overhead all the time, even when you don't remotely need it. (I can't prove this, but I'd guess the median "cloud service" is grotesquely overprovisioned on the smallest AWS instance. The "cloud services" that leap to mind are things like the AWS auth servers or Netflix content servers or the Google crawling or indexing servers, but while those are huge and important, they're also in many dimensions the exceptions. A good chunk of the popularity of "serverless" is probably a result of this.) When you do need every bit of performance, though, the list of viable options is short.

chessturk 7y ago
As a fellow gopher, I agree, Go isn't trying to compete with C/C++ on performance, it's trying to give you safety and ease of use of high level language. I think the point about concurrency typifies this, channels aren't as fast as mutexs and semaphores, but it makes sharing code with co workers easier to reason about. If you're in a domain where performance is still king, Go isn't trying to find a place there.
fouc 7y ago
How would Rust compare?

lenkite 7y ago

Could you describe how you would go about designing this and what libraries/frameworks would you use ? AFAIK co-routines is only coming with C++ 20, so I guess you would still use std::thread right ?

alexhutcheson 7y ago
std::thread has only been around since C++11, so most existing HPC codebases probably either use pthreads[1] directly, or an organization- or project-specific library wrapping pthreads. Some used boost::Thread, which was effectively the predecessor of std::thread. If you want to saturate every core on a very parallel problem like "handle as many packets as you can", a very rough first approach would be to spin up 88 threads (one per core). Within each thread, you could use either something like boost::Fiber[2] (similar to goroutines, except mapped N:1 to OS threads, rather than M:N) to avoid blocking the threads on IO. This paper[3] has a good overview of different concurrency approaches. If you're doing something that's not embarrassingly parallel[4], then often there is a well-researched approach for your specific domain. The same general ideas apply to keeping your cores busy, but you're often more bounded by communication between threads, memory bandwidth, etc. [1] https://en.wikipedia.org/wiki/POSIX_Threads https://en.wikipedia.org/wiki/POSIX_Threads [2] https://www.boost.org/doc/libs/1_70_0/libs/fiber/doc/html/fiber/overview.html https://www.boost.org/doc/libs/1_70_0/libs/fiber/doc/html/fi... [3] http://www.sosp.org/2001/papers/welsh.pdf http://www.sosp.org/2001/papers/welsh.pdf [4] https://en.wikipedia.org/wiki/Embarrassingly_parallel https://en.wikipedia.org/wiki/Embarrassingly_parallel