5 ms·

I feel like this glossed over how when entering something like epoll, Go can optimize switching to another goroutine. Wouldn't the epoll block until the calling

by nsm 3y ago

I feel like this glossed over how when entering something like epoll, Go can optimize switching to another goroutine. Wouldn't the epoll block until the calling kernel thread had a file descriptor ready?

sharkbot 3y ago
I don’t have your answers, but the runtime source has some pointers that would likely get you the answer you seek: https://github.com/golang/go/blob/master/src/runtime/HACKING.md https://github.com/golang/go/blob/master/src/runtime/HACKING... It sounds like blocking calls switch to a system stack and return the Go stack to the executor pool, but I don’t have source links to back up that claim.
seabrookmx 3y ago
The key is that epoll _isn't_ blocking: epoll_create returns immediately with a file descriptor. Go can use it to issue some sort of i/o operation and rather than blocking the kernel thread, it swaps it to another goroutine that needs work done. When epoll notifies the go scheduler the task is complete via that file descriptor, it will then resume the original goroutine.
- nsm 3y ago
  The crucial operation here is epoll_wait, not epoll_create. Are you saying that the system call is performed on another kernel thread while goroutine continue running on the GOMAXPROCS threads?
  - gpderetta 3y ago
    So, I'm not familiar with the go event loop specifically, but I have written a few event loops myself. You can think of an event loop multiplexing across multiple sources of events: for example a local ready queue, a remote queue (for tasks coming from other threads), an io-queue, a timer queue. Epoll is used for the io-queue. If the local ready queue is empty (i.e. there are no runnable coroutines), the event loop will block in the epoll call, possibly until the next timer expiration if any; remote queue operations will wake-up the poll via somethink like event-fd. If the local queue is non-empty, epoll is called with a 0 timeout every N local queue pops (actual scheduling logic and priority is application specific of course). Polling epoll with a 0 timeout is a bit wasteful, for high performance applications you can use io_uring so checking for non-empty queue is very cheap and/or use some network bypass library that makes epoll cheap. There is also old-school readiness notification via signals but I doubt that still see any use. As you suggest, you could also have a dedicated thread per event loop just running poll (for example a sibling hyperthread), but it seems wasteful.
    - muxator 3y ago
      Thanks for this comment! It is a very helpful design document for improving one's future event loops architecture!
ridiculous_fish 3y ago
Go can multiplex goroutines into a single syscall invocation. Two goroutines waiting on distinct file descriptors will use a single epoll (or similar) syscall. However not all syscalls can be multiplexed this way. IMO this partially undermines the "Go doesn't have colored functions" claim. 1k goroutines calling read is fine, because read is multiplexed, but 1k goroutines calling stat will be painful, because you get 1k kernel threads. Go's "red functions" are those which require a dedicated kernel thread.
- insanitybit 3y ago
  I think it just undermines "colorless functions are good", not that go's functions are colored - they aren't. The problem is that "colors" (ie: "this is async") are helpful to the caller, they let them know which context they should be in when they perform that work.
- thumbuddy 3y ago
  I wish this was stated more. A lot of people new to go have this air of mystery surrounding what coroutines and goroutines are. They'll do something it'll be fast! Then do something else and feel like they are running hardware from the 90s. Not everyone has the know how to strace their applications and think through why something at that level is happening.
- gpderetta 3y ago
  Does Go spawn a thread for every non multiplexable system call? In principle, on linux, you should be able to multiplex stat anyway via io_uring.
wahern 3y ago
epoll_wait takes a timeout, which if 0 means it will return immediately even if no events are pending. You wouldn't necessarily need to make use of this feature to implement something like Go's scheduler. Jumping to the semantics of epoll skips most of the interesting bits about how Goroutines work. The source code will always beat a blog post. If you're curious, you could work backwards from here: https://go.dev/src/runtime/netpoll_epoll.go https://go.dev/src/runtime/netpoll_epoll.go Note that interface does indeed expose the timeout argument to the higher-level scheduling code; see line 92.
- nsm 3y ago
  I am aware of epoll_wait and friends (select/poll) accepting timeouts, and how that plugs into a general event loop like in libuv. However if memory serves, these calls are reasonably expensive (on the order of microseconds, and _worse_ if readiness requires putting the current thread to sleep and then waking it up again). However https://www.ardanlabs.com/blog/2018/08/scheduling-in-go-part2.html https://www.ardanlabs.com/blog/2018/08/scheduling-in-go-part... says goroutines are closer to 200ns. However the same source above does a decent job of explaining how epoll is being called "out of band". I am guessing when they talk about context switching costs, they are doing some sort of amortization of these system call costs. Since the actual OS threads try to remain CPU-affine (thread-per-core, LMAX disruptor etc. etc.), a lot of the OS scheduling costs can be skipped. Thank you for the pointer to the source code. It gives me a starting point.

by nsm 3y ago

sharkbot 3y ago

I don’t have your answers, but the runtime source has some pointers that would likely get you the answer you seek: https://github.com/golang/go/blob/master/src/runtime/HACKING.md https://github.com/golang/go/blob/master/src/runtime/HACKING... It sounds like blocking calls switch to a system stack and return the Go stack to the executor pool, but I don’t have source links to back up that claim.

seabrookmx 3y ago

The key is that epoll _isn't_ blocking: epoll_create returns immediately with a file descriptor. Go can use it to issue some sort of i/o operation and rather than blocking the kernel thread, it swaps it to another goroutine that needs work done. When epoll notifies the go scheduler the task is complete via that file descriptor, it will then resume the original goroutine.

nsm 3y ago
The crucial operation here is epoll_wait, not epoll_create. Are you saying that the system call is performed on another kernel thread while goroutine continue running on the GOMAXPROCS threads?
- gpderetta 3y ago
  So, I'm not familiar with the go event loop specifically, but I have written a few event loops myself. You can think of an event loop multiplexing across multiple sources of events: for example a local ready queue, a remote queue (for tasks coming from other threads), an io-queue, a timer queue. Epoll is used for the io-queue. If the local ready queue is empty (i.e. there are no runnable coroutines), the event loop will block in the epoll call, possibly until the next timer expiration if any; remote queue operations will wake-up the poll via somethink like event-fd. If the local queue is non-empty, epoll is called with a 0 timeout every N local queue pops (actual scheduling logic and priority is application specific of course). Polling epoll with a 0 timeout is a bit wasteful, for high performance applications you can use io_uring so checking for non-empty queue is very cheap and/or use some network bypass library that makes epoll cheap. There is also old-school readiness notification via signals but I doubt that still see any use. As you suggest, you could also have a dedicated thread per event loop just running poll (for example a sibling hyperthread), but it seems wasteful.
  - muxator 3y ago
    Thanks for this comment! It is a very helpful design document for improving one's future event loops architecture!

ridiculous_fish 3y ago

Go can multiplex goroutines into a single syscall invocation. Two goroutines waiting on distinct file descriptors will use a single epoll (or similar) syscall. However not all syscalls can be multiplexed this way. IMO this partially undermines the "Go doesn't have colored functions" claim. 1k goroutines calling read is fine, because read is multiplexed, but 1k goroutines calling stat will be painful, because you get 1k kernel threads. Go's "red functions" are those which require a dedicated kernel thread.

insanitybit 3y ago
I think it just undermines "colorless functions are good", not that go's functions are colored - they aren't. The problem is that "colors" (ie: "this is async") are helpful to the caller, they let them know which context they should be in when they perform that work.
thumbuddy 3y ago
I wish this was stated more. A lot of people new to go have this air of mystery surrounding what coroutines and goroutines are. They'll do something it'll be fast! Then do something else and feel like they are running hardware from the 90s. Not everyone has the know how to strace their applications and think through why something at that level is happening.
gpderetta 3y ago
Does Go spawn a thread for every non multiplexable system call? In principle, on linux, you should be able to multiplex stat anyway via io_uring.

wahern 3y ago

epoll_wait takes a timeout, which if 0 means it will return immediately even if no events are pending. You wouldn't necessarily need to make use of this feature to implement something like Go's scheduler. Jumping to the semantics of epoll skips most of the interesting bits about how Goroutines work. The source code will always beat a blog post. If you're curious, you could work backwards from here: https://go.dev/src/runtime/netpoll_epoll.go https://go.dev/src/runtime/netpoll_epoll.go Note that interface does indeed expose the timeout argument to the higher-level scheduling code; see line 92.

nsm 3y ago
I am aware of epoll_wait and friends (select/poll) accepting timeouts, and how that plugs into a general event loop like in libuv. However if memory serves, these calls are reasonably expensive (on the order of microseconds, and _worse_ if readiness requires putting the current thread to sleep and then waking it up again). However https://www.ardanlabs.com/blog/2018/08/scheduling-in-go-part2.html https://www.ardanlabs.com/blog/2018/08/scheduling-in-go-part... says goroutines are closer to 200ns. However the same source above does a decent job of explaining how epoll is being called "out of band". I am guessing when they talk about context switching costs, they are doing some sort of amortization of these system call costs. Since the actual OS threads try to remain CPU-affine (thread-per-core, LMAX disruptor etc. etc.), a lot of the OS scheduling costs can be skipped. Thank you for the pointer to the source code. It gives me a starting point.