6 ms·

The Go authors have steadfastly refused to offer users any means of writing a CPU-local freelist, even though the utility of such things is obvious and evidence

The Go authors have steadfastly refused to offer users any means of writing a CPU-local freelist, even though the utility of such things is obvious and evidenced by the fact that they use such things all over the runtime package. They just think that _you_ are too stupid to be allowed to do such a thing. Honestly with the attitude that the Go authors treat their users I don't know why anyone tries to write high performing code in that language. C++ is there, after all.

adrianN 7y ago
Concurrency is a bit easier in Go than in C++.
- shereadsthenews 7y ago
  Disagree. I can write a server in C++ that, with careful thought and a bit of planning, will be able to exploit the resources of the kind of 88-core dual-socket machines that are mainstream today. No amount of planning will allow me to write a Go program that does the same thing on the same machine. Small amounts of concurrency might seem easy in Go but lots of concurrency is hard.
  - cdelsolar 7y ago
    Where's the evidence that Go can't handle large amounts of concurrency?
    - shereadsthenews 7y ago
      All over my profiler. Go can deal with something embarrassingly parallel, same as any other language, but try getting it to scale up to high core counts with small RPCs is impossible, for the reasons the OP mentioned: you can't avoid its global allocator and you don't control the scheduler either. You will spend all of your CPU time in runtime.findrunnable and runtime.mallocgc.
      - stcredzero 7y ago
        You will spend all of your CPU time in runtime.findrunnable Profile your channel latency, then design around that. "Small RPCs?" With an 88 core server, should you be making that many remote procedure calls? Find the 20% most intensive code and take that out of the hands of the scheduler, as much as possible. and runtime.mallocgc Even with sync.Pool? Can you design those parts of the system to mostly use the stack?
        kasey_junk 7y ago
        Not for nothing but channels become a bottleneck pretty early when you start doing highly concurrent golang. You can very carefully design around it using only channels but it quickly starts making more sense to just use a different concurrency abstraction.
        chessturk 7y ago
        My experience has been channels are a really good way to make highly concurrent code readable for people who hate concurrency (ymmv). They aren't as efficient, but using them as a semaphore (i.e. only signaling) can facilitate fairly-efficient, easier-to-read shared memory. But I'm kind of proving the whole... > You can very carefully design around it only channels ...part of your post.
      - networkimprov 7y ago
        Is there a Github issue for this problem? If not, would you consider filing one? https://github.com/golang/go/issues https://github.com/golang/go/issues It should probably explain why one cannot "avoid its global allocator" by using pools and/or stack objects.
        shereadsthenews 7y ago
        Stack is great, then you will spend more time in runtime.morestack because the Go authors have wisely decided that we are all too dumb to request specific stack sizes and we must rely on copying and doubling stacks larger than 2KiB even if we know in advance that 2KiB is not enough. This compares poorly with native threads where the stacks are dynamically allocated one page at a time without copying and we can specify the stack size _limit_ per thread if we need to. This is another place where the Go authors just arbitrarily decided that users cannot be trusted. sync.Pool is fun but it has known scalability limits that are yet to be addressed in a released runtime (see https://go-review.googlesource.com/c/go/+/166960/ https://go-review.googlesource.com/c/go/+/166960/ for example). You also can't use sync.Pool for anything that you need to be non-ephemeral, like CPU-local counters, because the GC can just blow them away at any time and the finalizers run at arbitrary future times or possibly never.
  - Thaxll 7y ago
    You're talking a about a very specific scenario, out of the box Go is off course easier than C++ for concurrency. Who by the way runs applications on a 88 core server, why would you so they instead of splitting that into smaller chunks.
    - shereadsthenews 7y ago
      You're making a different argument, that one does not need so much vertical scalability, and I might agree, but that doesn't negate the fact that Go has limits of vertical scalability that C++ doesn't face.
      - weberc2 7y ago
        The original claim was that concurrency is easier in Go, not that Go has a higher performance ceiling. You are the one making the different argument. :)
    - derefr 7y ago
      > Who by the way runs applications on a 88 core server, why would you so they instead of splitting that into smaller chunks. Graph analysis and/or routing, for one. Distributed Dijkstra is pretty impractical (the optimal lower bound on the number of messages required equals the number of edges in the graph!) For example, I don't work for Google, but I can pretty much guarantee you that an individual Google Maps routing query runs on a single, extremely-high-memory (but not NUMA!) node, which has likely been optimized for speed using plain-old threading on a high-core-count CPU, with the upper bound on the number of useful cores being the system architecture's per-socket memory bandwidth.
      - foobiekr 7y ago
        I would be very, very curious if this is actually true. I agree with the intuition behind the statement but I am actually curious how it is done.
      - Thaxll 7y ago
        I highly doubt Google runs any workload on Borg with CPU count > 24. I don't work there, so I'm just assuming but it goes against all modern scaling / deployments patterns.
  - jerf 7y ago
    "I can write a server in C++ that, with careful thought and a bit of planning, will be able to exploit the resources of the kind of 88-core dual-socket machines that are mainstream today." This is downvoted grey as I write this, and as someone who has been called a "Go shill" on occasion... it's exactly right. Go is a decent language for writing code in a fairly straightforward manner and getting pretty good performance out of it, but if you need to squeeze every bit of performance out of your hardware, it's a bad choice. (I can give you choices that are worse by an order of magnitude, or even more in some cases, but it's still a bad choice.) You have a fairly smooth optimization ride up to 1.5-2x slower than C for most use cases, a few pathological edge cases where it's grossly worse (many clustered around these "every drop of performance" problems!) and a few where it'll reach parity, and then you're going to hit a brick wall. An 88 core system is probably not impossible to sensibly use with Go, but you're going to be more constrained. I'd imagine it can probably serve web requests really well, but it's more likely to hit pathological cases if you hammer certain global resources. Arguably, precisely part of the point of Go was that C++ makes you pay for that level of performance in code complexity and cognitive overhead all the time, even when you don't remotely need it. (I can't prove this, but I'd guess the median "cloud service" is grotesquely overprovisioned on the smallest AWS instance. The "cloud services" that leap to mind are things like the AWS auth servers or Netflix content servers or the Google crawling or indexing servers, but while those are huge and important, they're also in many dimensions the exceptions. A good chunk of the popularity of "serverless" is probably a result of this.) When you do need every bit of performance, though, the list of viable options is short.
    - chessturk 7y ago
      As a fellow gopher, I agree, Go isn't trying to compete with C/C++ on performance, it's trying to give you safety and ease of use of high level language. I think the point about concurrency typifies this, channels aren't as fast as mutexs and semaphores, but it makes sharing code with co workers easier to reason about. If you're in a domain where performance is still king, Go isn't trying to find a place there.
    - fouc 7y ago
      How would Rust compare?
  - lenkite 7y ago
    Could you describe how you would go about designing this and what libraries/frameworks would you use ? AFAIK co-routines is only coming with C++ 20, so I guess you would still use std::thread right ?
    - alexhutcheson 7y ago
      std::thread has only been around since C++11, so most existing HPC codebases probably either use pthreads[1] directly, or an organization- or project-specific library wrapping pthreads. Some used boost::Thread, which was effectively the predecessor of std::thread. If you want to saturate every core on a very parallel problem like "handle as many packets as you can", a very rough first approach would be to spin up 88 threads (one per core). Within each thread, you could use either something like boost::Fiber[2] (similar to goroutines, except mapped N:1 to OS threads, rather than M:N) to avoid blocking the threads on IO. This paper[3] has a good overview of different concurrency approaches. If you're doing something that's not embarrassingly parallel[4], then often there is a well-researched approach for your specific domain. The same general ideas apply to keeping your cores busy, but you're often more bounded by communication between threads, memory bandwidth, etc. [1] https://en.wikipedia.org/wiki/POSIX_Threads https://en.wikipedia.org/wiki/POSIX_Threads [2] https://www.boost.org/doc/libs/1_70_0/libs/fiber/doc/html/fiber/overview.html https://www.boost.org/doc/libs/1_70_0/libs/fiber/doc/html/fi... [3] http://www.sosp.org/2001/papers/welsh.pdf http://www.sosp.org/2001/papers/welsh.pdf [4] https://en.wikipedia.org/wiki/Embarrassingly_parallel https://en.wikipedia.org/wiki/Embarrassingly_parallel
jnordwick 7y ago
"The key point here is our programmers are Googlers, they’re not researchers. They’re typically, fairly young, fresh out of school, probably learned Java, maybe learned C or C++, probably learned Python. They’re not capable of understanding a brilliant language but we want to use them to build good software. So, the language that we give them has to be easy for them to understand and easy to adopt." – Rob Pike
- wbl 7y ago
  For a company with supposedly high hiring standards. Granted domain knowledge also matters but that isn't what the people described have. So what does Google teach them?
  - erik_seaberg 7y ago
    Everyone I met there had my respect. I find "not capable of understanding a brilliant language" to be baseless contempt and I'm surprised he didn't get more heat over it. I wouldn't work with him, nor whatever people he's trying in vain to accommodate.
    - tensor 7y ago
      That's really quite ironic given: https://news.ycombinator.com/item?id=19825787 https://news.ycombinator.com/item?id=19825787
      - erik_seaberg 7y ago
        I should have worded that more gently. The greatest programmers in the world have demonstrated that even they can't use C reliably.
    - IOT_Apprentice 7y ago
      Yikes I think he is arguing against overwhelming programmers with complexity and the ability to hammer themselves with pointers, leaks and other banes of existence for average C programmers.
stcredzero 7y ago
Honestly with the attitude that the Go authors treat their users I don't know why anyone tries to write high performing code in that language. C++ is there, after all. I work in C++. I'm sometimes surprised at the way C++ treats its users. There seems to be a culture of pre-optimization. It's hard to write the most abstract application code without constantly thinking about performance under the surface. That's baked right in. Concepts which are used everyday by programmers can barely be cleanly summarized in a paragraph, even by well respected luminaries of the field. Trivia you "just have to remember," impinges on almost every single function or method you write. That said, there's also a lot of awesome things in C++. It just represents a particular set of cost/benefit dials. Golang represents another.
- MaxBarraclough 7y ago
  C++ is unique in that it aims to be feature-rich, while preserving at much as possible the property of if you don't use it, you don't pay for it. It breaks its own rule occasionally -- RTTI, exceptions, [0] standard library machinery with always-on thread safety -- but the C++ folks go to extreme lengths in the name of performance. There's no small irony in the way embedded folks write off C++ as too heavyweight, given that C++ tortures itself in the name of not forcing bloat upon the programmer. [0] https://llvm.org/docs/CodingStandards.html#do-not-use-rtti-or-exceptions https://llvm.org/docs/CodingStandards.html#do-not-use-rtti-o...
  - stcredzero 7y ago
    C++ is unique in that it aims to be feature-rich, while preserving at much as possible the property of if you don't use it, you don't pay for it. "You don't pay for it," should really be, you == the-CPU doesn't pay for it. On the other hand you == the-programmer has to think about it all the time. For some domains/contexts, yes this is actually very desirable. In that case C++ becomes a performance/optimization Swiss army knife. Other people might want it dialed in for, "You don't have to think too much about it, until it's time to optimize, and then you can't optimize all the way, but you get enough of the way there."
  - Const-me 7y ago

adrianN 7y ago

Concurrency is a bit easier in Go than in C++.

shereadsthenews 7y ago
Disagree. I can write a server in C++ that, with careful thought and a bit of planning, will be able to exploit the resources of the kind of 88-core dual-socket machines that are mainstream today. No amount of planning will allow me to write a Go program that does the same thing on the same machine. Small amounts of concurrency might seem easy in Go but lots of concurrency is hard.
- cdelsolar 7y ago
  Where's the evidence that Go can't handle large amounts of concurrency?
  - shereadsthenews 7y ago
    All over my profiler. Go can deal with something embarrassingly parallel, same as any other language, but try getting it to scale up to high core counts with small RPCs is impossible, for the reasons the OP mentioned: you can't avoid its global allocator and you don't control the scheduler either. You will spend all of your CPU time in runtime.findrunnable and runtime.mallocgc.
    - stcredzero 7y ago
      You will spend all of your CPU time in runtime.findrunnable Profile your channel latency, then design around that. "Small RPCs?" With an 88 core server, should you be making that many remote procedure calls? Find the 20% most intensive code and take that out of the hands of the scheduler, as much as possible. and runtime.mallocgc Even with sync.Pool? Can you design those parts of the system to mostly use the stack?
      - kasey_junk 7y ago
        Not for nothing but channels become a bottleneck pretty early when you start doing highly concurrent golang. You can very carefully design around it using only channels but it quickly starts making more sense to just use a different concurrency abstraction.
        chessturk 7y ago
        My experience has been channels are a really good way to make highly concurrent code readable for people who hate concurrency (ymmv). They aren't as efficient, but using them as a semaphore (i.e. only signaling) can facilitate fairly-efficient, easier-to-read shared memory. But I'm kind of proving the whole... > You can very carefully design around it only channels ...part of your post.
    - networkimprov 7y ago
      Is there a Github issue for this problem? If not, would you consider filing one? https://github.com/golang/go/issues https://github.com/golang/go/issues It should probably explain why one cannot "avoid its global allocator" by using pools and/or stack objects.
      - shereadsthenews 7y ago
        Stack is great, then you will spend more time in runtime.morestack because the Go authors have wisely decided that we are all too dumb to request specific stack sizes and we must rely on copying and doubling stacks larger than 2KiB even if we know in advance that 2KiB is not enough. This compares poorly with native threads where the stacks are dynamically allocated one page at a time without copying and we can specify the stack size _limit_ per thread if we need to. This is another place where the Go authors just arbitrarily decided that users cannot be trusted. sync.Pool is fun but it has known scalability limits that are yet to be addressed in a released runtime (see https://go-review.googlesource.com/c/go/+/166960/ https://go-review.googlesource.com/c/go/+/166960/ for example). You also can't use sync.Pool for anything that you need to be non-ephemeral, like CPU-local counters, because the GC can just blow them away at any time and the finalizers run at arbitrary future times or possibly never.
- Thaxll 7y ago
  You're talking a about a very specific scenario, out of the box Go is off course easier than C++ for concurrency. Who by the way runs applications on a 88 core server, why would you so they instead of splitting that into smaller chunks.
  - shereadsthenews 7y ago
    You're making a different argument, that one does not need so much vertical scalability, and I might agree, but that doesn't negate the fact that Go has limits of vertical scalability that C++ doesn't face.
    - weberc2 7y ago
      The original claim was that concurrency is easier in Go, not that Go has a higher performance ceiling. You are the one making the different argument. :)
  - derefr 7y ago
    > Who by the way runs applications on a 88 core server, why would you so they instead of splitting that into smaller chunks. Graph analysis and/or routing, for one. Distributed Dijkstra is pretty impractical (the optimal lower bound on the number of messages required equals the number of edges in the graph!) For example, I don't work for Google, but I can pretty much guarantee you that an individual Google Maps routing query runs on a single, extremely-high-memory (but not NUMA!) node, which has likely been optimized for speed using plain-old threading on a high-core-count CPU, with the upper bound on the number of useful cores being the system architecture's per-socket memory bandwidth.
    - foobiekr 7y ago
      I would be very, very curious if this is actually true. I agree with the intuition behind the statement but I am actually curious how it is done.
    - Thaxll 7y ago
      I highly doubt Google runs any workload on Borg with CPU count > 24. I don't work there, so I'm just assuming but it goes against all modern scaling / deployments patterns.
- jerf 7y ago
  "I can write a server in C++ that, with careful thought and a bit of planning, will be able to exploit the resources of the kind of 88-core dual-socket machines that are mainstream today." This is downvoted grey as I write this, and as someone who has been called a "Go shill" on occasion... it's exactly right. Go is a decent language for writing code in a fairly straightforward manner and getting pretty good performance out of it, but if you need to squeeze every bit of performance out of your hardware, it's a bad choice. (I can give you choices that are worse by an order of magnitude, or even more in some cases, but it's still a bad choice.) You have a fairly smooth optimization ride up to 1.5-2x slower than C for most use cases, a few pathological edge cases where it's grossly worse (many clustered around these "every drop of performance" problems!) and a few where it'll reach parity, and then you're going to hit a brick wall. An 88 core system is probably not impossible to sensibly use with Go, but you're going to be more constrained. I'd imagine it can probably serve web requests really well, but it's more likely to hit pathological cases if you hammer certain global resources. Arguably, precisely part of the point of Go was that C++ makes you pay for that level of performance in code complexity and cognitive overhead all the time, even when you don't remotely need it. (I can't prove this, but I'd guess the median "cloud service" is grotesquely overprovisioned on the smallest AWS instance. The "cloud services" that leap to mind are things like the AWS auth servers or Netflix content servers or the Google crawling or indexing servers, but while those are huge and important, they're also in many dimensions the exceptions. A good chunk of the popularity of "serverless" is probably a result of this.) When you do need every bit of performance, though, the list of viable options is short.
  - chessturk 7y ago
    As a fellow gopher, I agree, Go isn't trying to compete with C/C++ on performance, it's trying to give you safety and ease of use of high level language. I think the point about concurrency typifies this, channels aren't as fast as mutexs and semaphores, but it makes sharing code with co workers easier to reason about. If you're in a domain where performance is still king, Go isn't trying to find a place there.
  - fouc 7y ago
    How would Rust compare?
- lenkite 7y ago
  Could you describe how you would go about designing this and what libraries/frameworks would you use ? AFAIK co-routines is only coming with C++ 20, so I guess you would still use std::thread right ?
  - alexhutcheson 7y ago
    std::thread has only been around since C++11, so most existing HPC codebases probably either use pthreads[1] directly, or an organization- or project-specific library wrapping pthreads. Some used boost::Thread, which was effectively the predecessor of std::thread. If you want to saturate every core on a very parallel problem like "handle as many packets as you can", a very rough first approach would be to spin up 88 threads (one per core). Within each thread, you could use either something like boost::Fiber[2] (similar to goroutines, except mapped N:1 to OS threads, rather than M:N) to avoid blocking the threads on IO. This paper[3] has a good overview of different concurrency approaches. If you're doing something that's not embarrassingly parallel[4], then often there is a well-researched approach for your specific domain. The same general ideas apply to keeping your cores busy, but you're often more bounded by communication between threads, memory bandwidth, etc. [1] https://en.wikipedia.org/wiki/POSIX_Threads https://en.wikipedia.org/wiki/POSIX_Threads [2] https://www.boost.org/doc/libs/1_70_0/libs/fiber/doc/html/fiber/overview.html https://www.boost.org/doc/libs/1_70_0/libs/fiber/doc/html/fi... [3] http://www.sosp.org/2001/papers/welsh.pdf http://www.sosp.org/2001/papers/welsh.pdf [4] https://en.wikipedia.org/wiki/Embarrassingly_parallel https://en.wikipedia.org/wiki/Embarrassingly_parallel

jnordwick 7y ago

"The key point here is our programmers are Googlers, they’re not researchers. They’re typically, fairly young, fresh out of school, probably learned Java, maybe learned C or C++, probably learned Python. They’re not capable of understanding a brilliant language but we want to use them to build good software. So, the language that we give them has to be easy for them to understand and easy to adopt." – Rob Pike

wbl 7y ago
For a company with supposedly high hiring standards. Granted domain knowledge also matters but that isn't what the people described have. So what does Google teach them?
- erik_seaberg 7y ago
  Everyone I met there had my respect. I find "not capable of understanding a brilliant language" to be baseless contempt and I'm surprised he didn't get more heat over it. I wouldn't work with him, nor whatever people he's trying in vain to accommodate.
  - tensor 7y ago
    That's really quite ironic given: https://news.ycombinator.com/item?id=19825787 https://news.ycombinator.com/item?id=19825787
    - erik_seaberg 7y ago
      I should have worded that more gently. The greatest programmers in the world have demonstrated that even they can't use C reliably.
  - IOT_Apprentice 7y ago
    Yikes I think he is arguing against overwhelming programmers with complexity and the ability to hammer themselves with pointers, leaks and other banes of existence for average C programmers.

stcredzero 7y ago

Honestly with the attitude that the Go authors treat their users I don't know why anyone tries to write high performing code in that language. C++ is there, after all. I work in C++. I'm sometimes surprised at the way C++ treats its users. There seems to be a culture of pre-optimization. It's hard to write the most abstract application code without constantly thinking about performance under the surface. That's baked right in. Concepts which are used everyday by programmers can barely be cleanly summarized in a paragraph, even by well respected luminaries of the field. Trivia you "just have to remember," impinges on almost every single function or method you write. That said, there's also a lot of awesome things in C++. It just represents a particular set of cost/benefit dials. Golang represents another.

MaxBarraclough 7y ago
C++ is unique in that it aims to be feature-rich, while preserving at much as possible the property of if you don't use it, you don't pay for it. It breaks its own rule occasionally -- RTTI, exceptions, [0] standard library machinery with always-on thread safety -- but the C++ folks go to extreme lengths in the name of performance. There's no small irony in the way embedded folks write off C++ as too heavyweight, given that C++ tortures itself in the name of not forcing bloat upon the programmer. [0] https://llvm.org/docs/CodingStandards.html#do-not-use-rtti-or-exceptions https://llvm.org/docs/CodingStandards.html#do-not-use-rtti-o...
- stcredzero 7y ago
  C++ is unique in that it aims to be feature-rich, while preserving at much as possible the property of if you don't use it, you don't pay for it. "You don't pay for it," should really be, you == the-CPU doesn't pay for it. On the other hand you == the-programmer has to think about it all the time. For some domains/contexts, yes this is actually very desirable. In that case C++ becomes a performance/optimization Swiss army knife. Other people might want it dialed in for, "You don't have to think too much about it, until it's time to optimize, and then you can't optimize all the way, but you get enough of the way there."
- Const-me 7y ago