3 ms·

cgo is extremely slow, particularly in programs with many active goroutines. I regularly look sad at profiles of production cgo programs where findRunnable is

by raggi 2y ago

cgo is extremely slow, particularly in programs with many active goroutines. I regularly look sad at profiles of production cgo programs where findRunnable is in the top 10.

metadat 2y ago
I thought Go could handle a million-plus active goroutines? Are you saying it'll work but be poor performance compared to something equivalent in C++, JavaScript, or Rust? A quick search reveals cgo is mainly slow due to overhead when invoking non-pure go functions (i.e. C-functions). https://stackoverflow.com/questions/28272285/why-cgos-performance-is-so-slow-is-there-something-wrong-with-my-testing-code https://stackoverflow.com/questions/28272285/why-cgos-perfor... How does this relate and interact with goroutines in a performance impacting way? As in, why is the case with goroutines any different? Edit: @neonsunset: Thanks <3. The thread you linked covers erl/BEAM and C#/dotnet, and makes salient points of skepticism about the practical need for millions of routines. I'm still curious what makes the Go story worse or different?
- neonsunset 2y ago
  Related: https://news.ycombinator.com/item?id=40435220 https://news.ycombinator.com/item?id=40435220 (C# can handle millions of active tasks, and so can Elixir, though with greater overhead as it offers a different set of tradeoffs)
- jerf 2y ago
  Go can "handle" a million plus active goroutines, but that doesn't mean Go can handle a million goroutines doing literally anything a goroutine can do. They still have to fit into the resources of the computer. They can not all, for instance, have a gigabyte of their own active memory that they are doing something with, even though Go and a goroutine can individually "handle" a gigabyte of active working set. Cgo calls take a lot of resources, or, at least, take a lot of resources if you're trying to do a million of them simultaneously. (In absolute terms, Cgo slowdown is one of those things that programmers hear bandied about in conversations and can easily come away with the impression that a single Cgo call will consume 150 milliseconds or something and instantly bring your program to a crawl, but a single call is not noticeable. It's only really a problem when scaled somehow. See also the belief that GC guarantees that my program is going to be frozen for 250 milliseconds every couple of seconds.) In this context probably the most relevant one is they need a full OS thread to work in the C runtime, so a million simultaneous Cgo calls would require a million simultaneous OS threads, and that's still a pretty big expense today. For obvious reasons, Go can only "handle" millions of goroutines if all but some approximation of the number of CPUs available are doing nothing at any given time. The only common type of program that might have this circumstance is really network servers.
- raggi 2y ago
  Sure you can have a million goroutines, but you’ll now have more scheduler and gc overhead, and this is part of the issue. Calling out to ffi requires preparing both the stack and the runtime for that call. It’s theoretically possible to avoid the need for this preparation, but with some caveats - you have to block the gc view of parts of the world (the parts ffi could be using) and you need to ensure that your runtime isn’t interacting badly with the ffi environment (e.g. not sending it signals and so on). Go’s arrangement (as is common with most runtimes) needs to active work to prevent both problems, and the cost of cgo is largely the cost of doing those things. Most managed runtime/gc languages have these challenges and it’s one of the axes that people say makes them not systems languages (a poorly defined term for sure, but fair on an axis of “can’t make calls to other software modules at base call cost”)
  - neonsunset 2y ago
    The cause of this is not a technical limitation of "this is not possible" but rather "Go does this poorly due to tradeoffs* it insists on". C# trivially pins only select buffers or other data that need to be observed by whatever it is being called across FFI while GC is free to move and handle everything else (or, should the data need to be marshalled, it can be stack-allocated and pointer to such is passed instead, or it can be malloc and freed just like you can do in C). Years of work in this area made the impact of FFI as minimal as possible, with extra levers available to the user to completely erase the overhead (suppressing GC frame transition, using methods to allocate objects on the dedicated pinned heap, static linking for aot binaries, etc.). Go's GC implementation is comparatively simple and has other inefficiencies such as very expensive write barriers. *sometimes the tradeoff is just a product, which Go is, that was cheaper to produce, despite common beliefs
    - raggi 2y ago
      Yup, and far too much spinning going on in the runtime too.
neonsunset 2y ago
If only there was a high-level GC-based language with fast FFI and good performance, surely the industry would pick it based on the merits and not simply what is hyped...
- pjmlp 2y ago
  Sadly its owners have tainted a couple of generations to avoid anything from them, and the way they manage their OS isn't helping to change the point of view. That is how we end up using less capable alternatives.
ncruces 2y ago
Right, and both modernc and wazero are two highly sophisticated projects being used to avoid it. But cgo being slow (and above all displeasent to work with) is what I'd consider valid criticism of Go. Go has other advantages, FFI is not one of them.
- ctvo 2y ago
  Can you toss a reference to modernc? I can't seem to find the project.
  - ncruces 2y ago
    Most projects live in the gitlab below; modernc is the vanity import which alone doesn't resolve to anything. The most famous project is an SQLite driver. https://gitlab.com/cznic https://gitlab.com/cznic https://modern.org/sqlite https://modern.org/sqlite

by raggi 2y ago

cgo is extremely slow, particularly in programs with many active goroutines. I regularly look sad at profiles of production cgo programs where findRunnable is in the top 10.

metadat 2y ago

I thought Go could handle a million-plus active goroutines? Are you saying it'll work but be poor performance compared to something equivalent in C++, JavaScript, or Rust? A quick search reveals cgo is mainly slow due to overhead when invoking non-pure go functions (i.e. C-functions). https://stackoverflow.com/questions/28272285/why-cgos-performance-is-so-slow-is-there-something-wrong-with-my-testing-code https://stackoverflow.com/questions/28272285/why-cgos-perfor... How does this relate and interact with goroutines in a performance impacting way? As in, why is the case with goroutines any different? Edit: @neonsunset: Thanks <3. The thread you linked covers erl/BEAM and C#/dotnet, and makes salient points of skepticism about the practical need for millions of routines. I'm still curious what makes the Go story worse or different?

neonsunset 2y ago
Related: https://news.ycombinator.com/item?id=40435220 https://news.ycombinator.com/item?id=40435220 (C# can handle millions of active tasks, and so can Elixir, though with greater overhead as it offers a different set of tradeoffs)
jerf 2y ago
Go can "handle" a million plus active goroutines, but that doesn't mean Go can handle a million goroutines doing literally anything a goroutine can do. They still have to fit into the resources of the computer. They can not all, for instance, have a gigabyte of their own active memory that they are doing something with, even though Go and a goroutine can individually "handle" a gigabyte of active working set. Cgo calls take a lot of resources, or, at least, take a lot of resources if you're trying to do a million of them simultaneously. (In absolute terms, Cgo slowdown is one of those things that programmers hear bandied about in conversations and can easily come away with the impression that a single Cgo call will consume 150 milliseconds or something and instantly bring your program to a crawl, but a single call is not noticeable. It's only really a problem when scaled somehow. See also the belief that GC guarantees that my program is going to be frozen for 250 milliseconds every couple of seconds.) In this context probably the most relevant one is they need a full OS thread to work in the C runtime, so a million simultaneous Cgo calls would require a million simultaneous OS threads, and that's still a pretty big expense today. For obvious reasons, Go can only "handle" millions of goroutines if all but some approximation of the number of CPUs available are doing nothing at any given time. The only common type of program that might have this circumstance is really network servers.
raggi 2y ago
Sure you can have a million goroutines, but you’ll now have more scheduler and gc overhead, and this is part of the issue. Calling out to ffi requires preparing both the stack and the runtime for that call. It’s theoretically possible to avoid the need for this preparation, but with some caveats - you have to block the gc view of parts of the world (the parts ffi could be using) and you need to ensure that your runtime isn’t interacting badly with the ffi environment (e.g. not sending it signals and so on). Go’s arrangement (as is common with most runtimes) needs to active work to prevent both problems, and the cost of cgo is largely the cost of doing those things. Most managed runtime/gc languages have these challenges and it’s one of the axes that people say makes them not systems languages (a poorly defined term for sure, but fair on an axis of “can’t make calls to other software modules at base call cost”)
- neonsunset 2y ago
  The cause of this is not a technical limitation of "this is not possible" but rather "Go does this poorly due to tradeoffs* it insists on". C# trivially pins only select buffers or other data that need to be observed by whatever it is being called across FFI while GC is free to move and handle everything else (or, should the data need to be marshalled, it can be stack-allocated and pointer to such is passed instead, or it can be malloc and freed just like you can do in C). Years of work in this area made the impact of FFI as minimal as possible, with extra levers available to the user to completely erase the overhead (suppressing GC frame transition, using methods to allocate objects on the dedicated pinned heap, static linking for aot binaries, etc.). Go's GC implementation is comparatively simple and has other inefficiencies such as very expensive write barriers. *sometimes the tradeoff is just a product, which Go is, that was cheaper to produce, despite common beliefs
  - raggi 2y ago
    Yup, and far too much spinning going on in the runtime too.

neonsunset 2y ago

If only there was a high-level GC-based language with fast FFI and good performance, surely the industry would pick it based on the merits and not simply what is hyped...

pjmlp 2y ago
Sadly its owners have tainted a couple of generations to avoid anything from them, and the way they manage their OS isn't helping to change the point of view. That is how we end up using less capable alternatives.

ncruces 2y ago

Right, and both modernc and wazero are two highly sophisticated projects being used to avoid it. But cgo being slow (and above all displeasent to work with) is what I'd consider valid criticism of Go. Go has other advantages, FFI is not one of them.

ctvo 2y ago
Can you toss a reference to modernc? I can't seem to find the project.
- ncruces 2y ago
  Most projects live in the gitlab below; modernc is the vanity import which alone doesn't resolve to anything. The most famous project is an SQLite driver. https://gitlab.com/cznic https://gitlab.com/cznic https://modern.org/sqlite https://modern.org/sqlite