5 ms·

Understanding the Go Runtime: The Scheduler

158 points by valyala 7mo ago

pss314 7mo ago
I enjoyed both these GopherCon talks: GopherCon 2018: The Scheduler Saga - Kavya Joshi https://www.youtube.com/watch?v=YHRO5WQGh0k https://www.youtube.com/watch?v=YHRO5WQGh0k GopherCon 2017: Understanding Channels - Kavya Joshi https://www.youtube.com/watch?v=KBZlN0izeiY https://www.youtube.com/watch?v=KBZlN0izeiY
- jvillegasd 7mo ago
  Good videos, thanks for sharing!
- c0balt 7mo ago
  https://m.youtube.com/watch?v=-K11rY57K7k https://m.youtube.com/watch?v=-K11rY57K7k - Dmitry Vyukov — Go scheduler: Implementing language with lightweight concurrency This one notably also explains the design considerations for golangs M:N:P in comparison to other schemes and which specific challenges it tries to address.
  - konart 7mo ago
    And by Dmitry himself.
withinboredom 7mo ago
My biggest issue with go is it’s incredibly unfair scheduler. No matter what load you have, P99 and especially P99.9 latency will be higher than any other language. The way that it steals work guarantees that requests “in the middle” will be served last. It’s a problem that only go can solve, but that means giving up some of your speed that are currently handled immediately that shouldn’t be. So overall latency will go up and P99 will drop precipitously. Thus, they’ll probably never fix it. If you have a system that requires predictable latency, go is not the right language for it.
- desdenova 7mo ago
  > If you have a system, go is not the right language for it. FTFY
- red_admiral 7mo ago
  > If you have a system that requires predictable latency, go is not the right language for it. I presume that's by design, to trade off against other things google designed it for?
  - withinboredom 7mo ago
    No clue. All I know is that people complain about it every time they benchmark.
- pjmlp 7mo ago
  It misses having a custom scheduler option, like Java and .NET runtimes offer, unfortunely that is too many knobs for the usual Go approach to language design. Having a interface for how it is supposed to behave, a runtime.SetScheduler() or something, but it won't happen.
  - MisterTea 7mo ago
    I find it hard to believe the people who built Go, coming from designing Plan 9 and Inferno, would build a language where it is difficult to swap out a component. I have this feeling that in their quest to make Go simple, they added complexity in other areas. Then again, this was built at Google, not Bell Labs so the culture of building absurdly complex things likely influenced this.
    - pjmlp 7mo ago
      The same people refused to support generics for several years, and the current design still has some issues to iron out. Go also lacks some of Limbo features, e.g. plugin package is kind of abandoned. Thus even though dynamic loading is supported, it is hardly usable.
GeertVL 7mo ago
This is an excellent idea as a blog. Kudos!
Horos 7mo ago
Isn't a dedicated worker pool with priority queues enough to get predictable P99 without leaving Go? If you fix N workers and control dispatch order yourself, the scheduler barely gets involved — no stealing, no surprises. The inter-goroutine handoff is ~50-100ns anyway. Isn't the real issue using `go f()` per request rather than something in the language itself?
- withinboredom 7mo ago
  No. Eventually the queues get full and go routines pause waiting to place the element onto the queue, landing you right back at unfair scheduling. https://github.com/php/frankenphp/pull/2016 https://github.com/php/frankenphp/pull/2016 if you want to see a “correctly behaving” implementation that becomes 100% cpu usage under contention.
  - Horos 7mo ago
    fair point on blocking sends — but that's an implementation detail, not a structural one. From my pov, the worker pool's job isn't to absorb saturation. it's to make capacity explicit so the layer above can route around it. a bounded queue that returns ErrQueueFull immediately is a signal, not a failure — it tells the load balancer to try another instance. saturation on a single instance isn't a scheduler problem, it's a provisioning signal. the fix is horizontal, not vertical. once you're running N instances behind something that understands queue depth, the "unfair scheduler under contention" scenario stops being reachable in production — by design, not by luck. the FrankenPHP case looks like a single-instance stress test pushed to the limit, which is a valid benchmark but not how you'd architect for HA.
- vlowther 7mo ago
  My usecase was building an append-only blob store with mandatory encryption, but using a semaphore + direct goroutine calls to limit background write concurrency instead of a channel + dedicated writer goroutines was a net win across a wide variety of write sizes and max concurrent inflight writes. It is interesting that frankenphp + caddy came up with almost the same conclusion despite vastly different work being done.
avabuildsdata 7mo ago
The unfair scheduling point resonates. I run a lot of concurrent HTTP workloads in Go (scraping, data pipelines) and the scheduler is honestly fine for throughput-oriented work where you don't care about tail latency. But the moment you need consistent response times under load it becomes a real problem. GOMAXPROCS tuning and runtime.LockOSThread help in narrow cases but they're band-aids. The lack of priority or fairness knobs is a deliberate design choice but it does push certain workloads toward other runtimes.
- valyala 7mo ago
  If the server cannot keep up with the given workload because of some bottleneck (CPU, network, disk IO), then it cannot guarantee any response times - incoming queries will be either rejected or queued in a long wait queue, which will lead to awfully big response times. This doesn't depend on the programming language or the framework the server written in. If you want response time guarantees, make sure the server has enough free resources for processing the given workload.
Someone 7mo ago
> a goroutine’s state is surprisingly small. The mcall() assembly function only saves 3 values — the stack pointer, the program counter, and the base pointer — into a tiny gobuf struct. That’s it. Why so few? Because goroutine switches happen at function call boundaries, and at those points the compiler has already spilled any important registers to the stack following normal calling conventions. Wouldn’t that mean go never uses registers to pass arguments to functions? If so, that seems in conflict with https://go.dev/src/cmd/compile/abi-internal#function-call-argument-and-result-passing https://go.dev/src/cmd/compile/abi-internal#function-call-ar..., which says “Because access to registers is generally faster than access to the stack, arguments and results are preferentially passed in registers” Or does the compiler always Go’s stable ABI, known as ABI0 in functions where it inserts code to potentially context switch, and only uses the (potentially) faster ABI that passes arguments in registers elsewhere?
- mknyszek 7mo ago
  The compiler generates code to spill arguments to the stack at synchronous preemption points (function entry). Signal-based preemption has a spill path that saves the full ABI register set.
capricio_one 7mo ago
Go missed a big opportunity to be Rust when we needed Rust more than anything. I have long since moved on from Go and C#/.NET is widely available nowadays and in many respects less held back by some strange political choices when it comes to DevEx (I am of course talking about generics).
- 9rx 7mo ago
  Rust is the older project of the two, kicking off in 2006. Go, which set sail in 2007, duplicating the work of Rust would have been pointless. We already had Rust. Go's objective was to become a faster Python. Which was something we also desperately needed at the time, and it has well succeeded on that front. Go has largely replaced all the non-data science things people were earlier doing with Python.
  - za3faran 7mo ago
    If you saw the early presentations, they complained about the slow compile times and high complexity of C++. It seems that they were targeting that, not Python.
    - 9rx 7mo ago
      I did see the early presentations. And since you did too, you will recall that one of the primary priorities was for it to "feel like a dynamically-typed language". You know, because it was trying to be a faster Python. What you might be confusing that with is that their assumption was that Google services were written in C++ because those services needed C++ performance, not because the developers wanted to write code in C++, and that those C++ developers would jump at the chance to use a Python-like language that still satisfies them performance-wise. It turns out they were wrong — the developers actually did want to write C++ — but you can understand the thinking when Google was already using Python heavily in less performance-critical areas. Guido van Rossum himself was even on the payroll at the time. For what it is worth, Google did create "Rust" after learning that a faster Python doesn't satisfy C++ developers. It's called Carbon. But it is telling that the earlier commenter has never heard of it, and it is unlikely it will ever leave the heap of esoteric languages because duplicating Rust was, and continues to be, pointless. We already had Rust.

158 points by valyala 7mo ago

I enjoyed both these GopherCon talks: GopherCon 2018: The Scheduler Saga - Kavya Joshi https://www.youtube.com/watch?v=YHRO5WQGh0k https://www.youtube.com/watch?v=YHRO5WQGh0k GopherCon 2017: Understanding Channels - Kavya Joshi https://www.youtube.com/watch?v=KBZlN0izeiY https://www.youtube.com/watch?v=KBZlN0izeiY

jvillegasd 7mo ago
Good videos, thanks for sharing!
c0balt 7mo ago
https://m.youtube.com/watch?v=-K11rY57K7k https://m.youtube.com/watch?v=-K11rY57K7k - Dmitry Vyukov — Go scheduler: Implementing language with lightweight concurrency This one notably also explains the design considerations for golangs M:N:P in comparison to other schemes and which specific challenges it tries to address.
- konart 7mo ago
  And by Dmitry himself.

withinboredom 7mo ago

My biggest issue with go is it’s incredibly unfair scheduler. No matter what load you have, P99 and especially P99.9 latency will be higher than any other language. The way that it steals work guarantees that requests “in the middle” will be served last. It’s a problem that only go can solve, but that means giving up some of your speed that are currently handled immediately that shouldn’t be. So overall latency will go up and P99 will drop precipitously. Thus, they’ll probably never fix it. If you have a system that requires predictable latency, go is not the right language for it.

desdenova 7mo ago
> If you have a system, go is not the right language for it. FTFY
red_admiral 7mo ago
> If you have a system that requires predictable latency, go is not the right language for it. I presume that's by design, to trade off against other things google designed it for?
- withinboredom 7mo ago
  No clue. All I know is that people complain about it every time they benchmark.
pjmlp 7mo ago
It misses having a custom scheduler option, like Java and .NET runtimes offer, unfortunely that is too many knobs for the usual Go approach to language design. Having a interface for how it is supposed to behave, a runtime.SetScheduler() or something, but it won't happen.
- MisterTea 7mo ago
  I find it hard to believe the people who built Go, coming from designing Plan 9 and Inferno, would build a language where it is difficult to swap out a component. I have this feeling that in their quest to make Go simple, they added complexity in other areas. Then again, this was built at Google, not Bell Labs so the culture of building absurdly complex things likely influenced this.
  - pjmlp 7mo ago
    The same people refused to support generics for several years, and the current design still has some issues to iron out. Go also lacks some of Limbo features, e.g. plugin package is kind of abandoned. Thus even though dynamic loading is supported, it is hardly usable.

GeertVL 7mo ago

This is an excellent idea as a blog. Kudos!

Horos 7mo ago

Isn't a dedicated worker pool with priority queues enough to get predictable P99 without leaving Go? If you fix N workers and control dispatch order yourself, the scheduler barely gets involved — no stealing, no surprises. The inter-goroutine handoff is ~50-100ns anyway. Isn't the real issue using `go f()` per request rather than something in the language itself?

withinboredom 7mo ago
No. Eventually the queues get full and go routines pause waiting to place the element onto the queue, landing you right back at unfair scheduling. https://github.com/php/frankenphp/pull/2016 https://github.com/php/frankenphp/pull/2016 if you want to see a “correctly behaving” implementation that becomes 100% cpu usage under contention.
- Horos 7mo ago
  fair point on blocking sends — but that's an implementation detail, not a structural one. From my pov, the worker pool's job isn't to absorb saturation. it's to make capacity explicit so the layer above can route around it. a bounded queue that returns ErrQueueFull immediately is a signal, not a failure — it tells the load balancer to try another instance. saturation on a single instance isn't a scheduler problem, it's a provisioning signal. the fix is horizontal, not vertical. once you're running N instances behind something that understands queue depth, the "unfair scheduler under contention" scenario stops being reachable in production — by design, not by luck. the FrankenPHP case looks like a single-instance stress test pushed to the limit, which is a valid benchmark but not how you'd architect for HA.
vlowther 7mo ago
My usecase was building an append-only blob store with mandatory encryption, but using a semaphore + direct goroutine calls to limit background write concurrency instead of a channel + dedicated writer goroutines was a net win across a wide variety of write sizes and max concurrent inflight writes. It is interesting that frankenphp + caddy came up with almost the same conclusion despite vastly different work being done.

avabuildsdata 7mo ago

The unfair scheduling point resonates. I run a lot of concurrent HTTP workloads in Go (scraping, data pipelines) and the scheduler is honestly fine for throughput-oriented work where you don't care about tail latency. But the moment you need consistent response times under load it becomes a real problem. GOMAXPROCS tuning and runtime.LockOSThread help in narrow cases but they're band-aids. The lack of priority or fairness knobs is a deliberate design choice but it does push certain workloads toward other runtimes.

valyala 7mo ago
If the server cannot keep up with the given workload because of some bottleneck (CPU, network, disk IO), then it cannot guarantee any response times - incoming queries will be either rejected or queued in a long wait queue, which will lead to awfully big response times. This doesn't depend on the programming language or the framework the server written in. If you want response time guarantees, make sure the server has enough free resources for processing the given workload.

Someone 7mo ago

> a goroutine’s state is surprisingly small. The mcall() assembly function only saves 3 values — the stack pointer, the program counter, and the base pointer — into a tiny gobuf struct. That’s it. Why so few? Because goroutine switches happen at function call boundaries, and at those points the compiler has already spilled any important registers to the stack following normal calling conventions. Wouldn’t that mean go never uses registers to pass arguments to functions? If so, that seems in conflict with https://go.dev/src/cmd/compile/abi-internal#function-call-argument-and-result-passing https://go.dev/src/cmd/compile/abi-internal#function-call-ar..., which says “Because access to registers is generally faster than access to the stack, arguments and results are preferentially passed in registers” Or does the compiler always Go’s stable ABI, known as ABI0 in functions where it inserts code to potentially context switch, and only uses the (potentially) faster ABI that passes arguments in registers elsewhere?

mknyszek 7mo ago
The compiler generates code to spill arguments to the stack at synchronous preemption points (function entry). Signal-based preemption has a spill path that saves the full ABI register set.

capricio_one 7mo ago

Go missed a big opportunity to be Rust when we needed Rust more than anything. I have long since moved on from Go and C#/.NET is widely available nowadays and in many respects less held back by some strange political choices when it comes to DevEx (I am of course talking about generics).

9rx 7mo ago
Rust is the older project of the two, kicking off in 2006. Go, which set sail in 2007, duplicating the work of Rust would have been pointless. We already had Rust. Go's objective was to become a faster Python. Which was something we also desperately needed at the time, and it has well succeeded on that front. Go has largely replaced all the non-data science things people were earlier doing with Python.
- za3faran 7mo ago
  If you saw the early presentations, they complained about the slow compile times and high complexity of C++. It seems that they were targeting that, not Python.
  - 9rx 7mo ago
    I did see the early presentations. And since you did too, you will recall that one of the primary priorities was for it to "feel like a dynamically-typed language". You know, because it was trying to be a faster Python. What you might be confusing that with is that their assumption was that Google services were written in C++ because those services needed C++ performance, not because the developers wanted to write code in C++, and that those C++ developers would jump at the chance to use a Python-like language that still satisfies them performance-wise. It turns out they were wrong — the developers actually did want to write C++ — but you can understand the thinking when Google was already using Python heavily in less performance-critical areas. Guido van Rossum himself was even on the payroll at the time. For what it is worth, Google did create "Rust" after learning that a faster Python doesn't satisfy C++ developers. It's called Carbon. But it is telling that the earlier commenter has never heard of it, and it is unlikely it will ever leave the heap of esoteric languages because duplicating Rust was, and continues to be, pointless. We already had Rust.