6 ms·

Not only does 3x seem quite significant, but thread context switches are also a large overhead and that goes untested here. If thread context switch overhead wa

by jchw 6y ago

Not only does 3x seem quite significant, but thread context switches are also a large overhead and that goes untested here. If thread context switch overhead was as low as usermode context switching, there would be no use for coroutines since you could just use threads instead; I doubt it’s non-trivial. (Of course, in Go, the scheduler also weaves in the GC IIRC, so an apples-apples comparison may be difficult. Micro benchmarks are just not that useful.) P.S.: this article seems to work under the assumption that 10,000 Goroutines is a reasonable upper limit, or at least it feels as though it implies that. However, you can definitely run apps with 100,000 or even 1,000,000.

refenestrator 6y ago
I thought the major cost was cold cache after switch in both cases? Kernel context switches are pretty light compared to the following slowdowns due to cache.
jeffbee 6y ago
The performance of the Go runtime with a million blocked goroutines is pretty OK, but its performance with even 1000 runnable goroutines is not great at all. You really need to think about which you are going to have.
- gnfargbl 6y ago
  True, but if you regularly have 1000 runnable goroutines then could you not reconfigure your app to have, say, 64 runnable goroutines, and get better throughput? Large numbers of goroutines do seem to be a good fit for problems which are mostly waiting on the network.
  - jeffbee 6y ago
    The architecture of Go forces you to have 1 goroutine servicing every socket, so the number of runnable goroutines will then be at the mercy of your packet inter-arrival process.
    - orisho 6y ago
      Does it? You could implement a classic select-based reader and writer in a single goroutine, to handle many sockets at once.
      - jeffbee 6y ago
        Please describe how you would actually do that in Go.
        lossolo 6y ago
        Using epoll directly (via syscalls) like here https://github.com/tidwall/evio https://github.com/tidwall/evio
        refenestrator 6y ago
        My information is outdated, but last I knew, Go always parked a goroutine when making a syscall. Seems like a lot of overhead to read bytes that you know are there from an epolled fd.
        coder543 6y ago
        Not exactly. It depends on the syscall. Syscalls that are known to be slow are treated the way you're describing, but syscalls that are often fast are treated optimistically in a way that allows the work queue to be stolen and spawned onto a new thread only if the syscall doesn't return quickly. I don't know which category epoll fits into, but it seems like it should be treated optimistically, since epoll is used for non-blocking I/O. https://utcc.utoronto.ca/~cks/space/blog/programming/GoSchedulerAndSyscalls https://utcc.utoronto.ca/~cks/space/blog/programming/GoSched...
        refenestrator 6y ago
        read(2) has to fit into the "might block" bucket, though, right? It would take a very smart runtime to figure out you just called epoll recently and established it won't block.
- EricBurnett 6y ago
  Is 1k runnable goroutines materially worse than 1k runnable threads? (Honest question, I don't know myself).
otabdeveloper4 6y ago
> If thread context switch overhead was as low as usermode context switching, there would be no use for coroutines since you could just use threads instead Two points: a) native threads switching overhead is extremely low, probably lower than usermode Go thread overhead; b) the utility of usermode threads isn't to lower overhead. It's a given that usermode threads introduce lots of overhead, if only because you're reimplementing the scheduling machinery that already exists in the kernel and that you're already paying the performance penalty for. But theoretically, you could implement specially tailored scheduling algorithms specific for your app and gain an advantage there - kernel schedulers are general-purpose and can introduce latencies, starvation and other undesired special effects. P.S. The Linux kernel also has interfaces for changing the thread scheduler algorithms. This gets you 99% of the way there - native threads + round-robin realtime thread scheduler gets you the performance benefits you want without having to rewrite schedulers and context switchers in usermode for every language runtime.
- Thaxll 6y ago
  "native threads switching overhead is extremely low, probably lower than usermode Go thread overhead" This is most likely very wrong.
  - otabdeveloper4 6y ago
    It isn't. Kernel context switching is the same thing as language runtime context switching, except you get access to kernel mode and you don't have to keep track of garbage collection stuff.
  - jaen 6y ago
    When you think about in depth, the kernel can context switch between threads essentially for free when returning from a syscall (the work to restore a userland context is the same regardless of which context it is switching to, disregarding cache effects). In most apps, context switches would happen only due to kernel->user transitions anyway (such as an I/O syscall finishing or a timer triggering), so in most apps kernel context switches are indeed almost free. To be significantly more efficient than the kernel there should be much more context switches than syscalls, which for non-synthetic programs is mostly only possible under heavy loads that can batch, with suitable I/O interfaces such as io_uring.
eliben 6y ago
Some measurements of context switching: https://eli.thegreenplace.net/2018/measuring-context-switching-and-memory-overheads-for-linux-threads/ https://eli.thegreenplace.net/2018/measuring-context-switchi...

by jchw 6y ago

refenestrator 6y ago

I thought the major cost was cold cache after switch in both cases? Kernel context switches are pretty light compared to the following slowdowns due to cache.

jeffbee 6y ago

The performance of the Go runtime with a million blocked goroutines is pretty OK, but its performance with even 1000 runnable goroutines is not great at all. You really need to think about which you are going to have.

gnfargbl 6y ago
True, but if you regularly have 1000 runnable goroutines then could you not reconfigure your app to have, say, 64 runnable goroutines, and get better throughput? Large numbers of goroutines do seem to be a good fit for problems which are mostly waiting on the network.
- jeffbee 6y ago
  The architecture of Go forces you to have 1 goroutine servicing every socket, so the number of runnable goroutines will then be at the mercy of your packet inter-arrival process.
  - orisho 6y ago
    Does it? You could implement a classic select-based reader and writer in a single goroutine, to handle many sockets at once.
    - jeffbee 6y ago
      Please describe how you would actually do that in Go.
      - lossolo 6y ago
        Using epoll directly (via syscalls) like here https://github.com/tidwall/evio https://github.com/tidwall/evio
        refenestrator 6y ago
        My information is outdated, but last I knew, Go always parked a goroutine when making a syscall. Seems like a lot of overhead to read bytes that you know are there from an epolled fd.
        coder543 6y ago
        Not exactly. It depends on the syscall. Syscalls that are known to be slow are treated the way you're describing, but syscalls that are often fast are treated optimistically in a way that allows the work queue to be stolen and spawned onto a new thread only if the syscall doesn't return quickly. I don't know which category epoll fits into, but it seems like it should be treated optimistically, since epoll is used for non-blocking I/O. https://utcc.utoronto.ca/~cks/space/blog/programming/GoSchedulerAndSyscalls https://utcc.utoronto.ca/~cks/space/blog/programming/GoSched...
        refenestrator 6y ago
        read(2) has to fit into the "might block" bucket, though, right? It would take a very smart runtime to figure out you just called epoll recently and established it won't block.
EricBurnett 6y ago
Is 1k runnable goroutines materially worse than 1k runnable threads? (Honest question, I don't know myself).

otabdeveloper4 6y ago

> If thread context switch overhead was as low as usermode context switching, there would be no use for coroutines since you could just use threads instead Two points: a) native threads switching overhead is extremely low, probably lower than usermode Go thread overhead; b) the utility of usermode threads isn't to lower overhead. It's a given that usermode threads introduce lots of overhead, if only because you're reimplementing the scheduling machinery that already exists in the kernel and that you're already paying the performance penalty for. But theoretically, you could implement specially tailored scheduling algorithms specific for your app and gain an advantage there - kernel schedulers are general-purpose and can introduce latencies, starvation and other undesired special effects. P.S. The Linux kernel also has interfaces for changing the thread scheduler algorithms. This gets you 99% of the way there - native threads + round-robin realtime thread scheduler gets you the performance benefits you want without having to rewrite schedulers and context switchers in usermode for every language runtime.

Thaxll 6y ago
"native threads switching overhead is extremely low, probably lower than usermode Go thread overhead" This is most likely very wrong.
- otabdeveloper4 6y ago
  It isn't. Kernel context switching is the same thing as language runtime context switching, except you get access to kernel mode and you don't have to keep track of garbage collection stuff.
- jaen 6y ago
  When you think about in depth, the kernel can context switch between threads essentially for free when returning from a syscall (the work to restore a userland context is the same regardless of which context it is switching to, disregarding cache effects). In most apps, context switches would happen only due to kernel->user transitions anyway (such as an I/O syscall finishing or a timer triggering), so in most apps kernel context switches are indeed almost free. To be significantly more efficient than the kernel there should be much more context switches than syscalls, which for non-synthetic programs is mostly only possible under heavy loads that can batch, with suitable I/O interfaces such as io_uring.

eliben 6y ago

Some measurements of context switching: https://eli.thegreenplace.net/2018/measuring-context-switching-and-memory-overheads-for-linux-threads/ https://eli.thegreenplace.net/2018/measuring-context-switchi...