5 ms·

My biggest issue with go is it’s incredibly unfair scheduler. No matter what load you have, P99 and especially P99.9 latency will be higher than any other langu

My biggest issue with go is it’s incredibly unfair scheduler. No matter what load you have, P99 and especially P99.9 latency will be higher than any other language. The way that it steals work guarantees that requests “in the middle” will be served last. It’s a problem that only go can solve, but that means giving up some of your speed that are currently handled immediately that shouldn’t be. So overall latency will go up and P99 will drop precipitously. Thus, they’ll probably never fix it. If you have a system that requires predictable latency, go is not the right language for it.

desdenova 7mo ago
> If you have a system, go is not the right language for it. FTFY
red_admiral 7mo ago
> If you have a system that requires predictable latency, go is not the right language for it. I presume that's by design, to trade off against other things google designed it for?
- withinboredom 7mo ago
  No clue. All I know is that people complain about it every time they benchmark.
pjmlp 7mo ago
It misses having a custom scheduler option, like Java and .NET runtimes offer, unfortunely that is too many knobs for the usual Go approach to language design. Having a interface for how it is supposed to behave, a runtime.SetScheduler() or something, but it won't happen.
- MisterTea 7mo ago
  I find it hard to believe the people who built Go, coming from designing Plan 9 and Inferno, would build a language where it is difficult to swap out a component. I have this feeling that in their quest to make Go simple, they added complexity in other areas. Then again, this was built at Google, not Bell Labs so the culture of building absurdly complex things likely influenced this.
  - pjmlp 7mo ago
    The same people refused to support generics for several years, and the current design still has some issues to iron out. Go also lacks some of Limbo features, e.g. plugin package is kind of abandoned. Thus even though dynamic loading is supported, it is hardly usable.
melodyogonna 7mo ago
> If you have a system that requires predictable latency, go is not the right language for it. Having a garbage collector already make this the case, it is a known trade off.
- gf000 7mo ago
  This may have been practically true for a long time, but as Java's ZGC garbage collector proves, this is not a hard truth. You can have world pauses that are independent of heap size, and thus predictable latency (of course, trading off some throughput, but that is almost fundamental)
- pjmlp 7mo ago
  Not really, it is a matter of having the right implementation. - https://www.ptc.com/en/products/developer-tools/perc https://www.ptc.com/en/products/developer-tools/perc - https://www.aicas.com/products-services/jamaicavm https://www.aicas.com/products-services/jamaicavm - https://www.azul.com/products/prime https://www.azul.com/products/prime Not all GCs are born alike.
- kevin_thibedeau 7mo ago
  Nim's GC is deterministic when you need it.
  - melodyogonna 7mo ago
    How so?
    - kevin_thibedeau 7mo ago
      You can run it for fixed timeslices.
kjksf 7mo ago
> No matter what load you have, P99 and especially P99.9 latency will be higher than any other language I strongly call BS on that. Strong claim and evidence seems to be a hallucination in your own head. There are several writeups of large backends ported from node/python/ruby to Go which resulted in dramatic speedups, including drop in P99 and P99.9 latencies by 10x That's empirical evidence your claim is BS. What exactly is so unfair about Go scheduler and what do you compare it to? Node's lack of multi-threading? Python's and Ruby's GIL? Just leaving this to OS thread scheduler which, unlike Go, has no idea about i/o and therefore cannot optimize for it? Apparently the source of your claim is https://github.com/php/frankenphp/pull/2016 https://github.com/php/frankenphp/pull/2016 Which is optimizing for a very specific micro-benchmark of hammering std-lib http server with concurrent request. Which is not what 99% of go servers need to handle. And is exercising way more than a scheduler. And is not benchmarking against any other language, so the sweeping statement about "higher than any other language" is literally baseless. And you were able to make a change that trades throughput for P99 latency without changing the scheduler, which kind of shows it wasn't the scheduler but an interaction between a specific implementation of HTTP server and Go scheduler. And there are other HTTP servers in Go that focus on speed. It's just 99.9% of Go servers don't need any of that because the baseline is 10x faster than python/ruby/javascript and on-par with Java or C#.
- withinboredom 7mo ago
  Do I need to share the TLA+ spec that shows its unfair? Or do you have any actual proof to your claims?
  - 9rx 7mo ago
    It would be helpful for you to share a link to the Github issue you created. If the TLA+ spec you no doubt put a lot of time into creating is contained there, that would be additionally amazing, but more relevant will be the responses from the maintainers so that we're not stuck with one side of the story. Of course, expecting you to provide the link would be incredibly onerous. We can look it up ourselves just as easy as you can. Well, in theory we can. The only trouble is that I cannot find the issue you are talking about. I cannot find any issues in the Go issue tracker from your account. So, in the interest of good faith, perhaps you can help us out this one time and point us in the right direction?
pothamk 7mo ago
[dead]
mknyszek 7mo ago
> Thus, they’ll probably never fix it. I'm sorry you had a bad experience with Go. What makes you say this? Have you filed an issue upstream yet? If not, I encourage you to do so. I can't promise it'll be fixed or delved into immediately, but filing detailed feedback like this is really helpful for prioritizing work.
_rlh 7mo ago
“It’s a problem that only go can solve” I had this discussion a decade ago and concluded that a reasonable fair scheduler could be built on top of the go runtime scheduler by gating the work presented. The case was be made that the application is the proper, if not only, place to do this. Other than performance, if you encountered a runtime limitation then filing an issue is how the Go community moves forward.

desdenova 7mo ago

> If you have a system, go is not the right language for it. FTFY

red_admiral 7mo ago

> If you have a system that requires predictable latency, go is not the right language for it. I presume that's by design, to trade off against other things google designed it for?

withinboredom 7mo ago
No clue. All I know is that people complain about it every time they benchmark.

pjmlp 7mo ago

It misses having a custom scheduler option, like Java and .NET runtimes offer, unfortunely that is too many knobs for the usual Go approach to language design. Having a interface for how it is supposed to behave, a runtime.SetScheduler() or something, but it won't happen.

MisterTea 7mo ago
I find it hard to believe the people who built Go, coming from designing Plan 9 and Inferno, would build a language where it is difficult to swap out a component. I have this feeling that in their quest to make Go simple, they added complexity in other areas. Then again, this was built at Google, not Bell Labs so the culture of building absurdly complex things likely influenced this.
- pjmlp 7mo ago
  The same people refused to support generics for several years, and the current design still has some issues to iron out. Go also lacks some of Limbo features, e.g. plugin package is kind of abandoned. Thus even though dynamic loading is supported, it is hardly usable.

melodyogonna 7mo ago

> If you have a system that requires predictable latency, go is not the right language for it. Having a garbage collector already make this the case, it is a known trade off.

gf000 7mo ago
This may have been practically true for a long time, but as Java's ZGC garbage collector proves, this is not a hard truth. You can have world pauses that are independent of heap size, and thus predictable latency (of course, trading off some throughput, but that is almost fundamental)
pjmlp 7mo ago
Not really, it is a matter of having the right implementation. - https://www.ptc.com/en/products/developer-tools/perc https://www.ptc.com/en/products/developer-tools/perc - https://www.aicas.com/products-services/jamaicavm https://www.aicas.com/products-services/jamaicavm - https://www.azul.com/products/prime https://www.azul.com/products/prime Not all GCs are born alike.
kevin_thibedeau 7mo ago
Nim's GC is deterministic when you need it.
- melodyogonna 7mo ago
  How so?
  - kevin_thibedeau 7mo ago
    You can run it for fixed timeslices.

kjksf 7mo ago

> No matter what load you have, P99 and especially P99.9 latency will be higher than any other language I strongly call BS on that. Strong claim and evidence seems to be a hallucination in your own head. There are several writeups of large backends ported from node/python/ruby to Go which resulted in dramatic speedups, including drop in P99 and P99.9 latencies by 10x That's empirical evidence your claim is BS. What exactly is so unfair about Go scheduler and what do you compare it to? Node's lack of multi-threading? Python's and Ruby's GIL? Just leaving this to OS thread scheduler which, unlike Go, has no idea about i/o and therefore cannot optimize for it? Apparently the source of your claim is https://github.com/php/frankenphp/pull/2016 https://github.com/php/frankenphp/pull/2016 Which is optimizing for a very specific micro-benchmark of hammering std-lib http server with concurrent request. Which is not what 99% of go servers need to handle. And is exercising way more than a scheduler. And is not benchmarking against any other language, so the sweeping statement about "higher than any other language" is literally baseless. And you were able to make a change that trades throughput for P99 latency without changing the scheduler, which kind of shows it wasn't the scheduler but an interaction between a specific implementation of HTTP server and Go scheduler. And there are other HTTP servers in Go that focus on speed. It's just 99.9% of Go servers don't need any of that because the baseline is 10x faster than python/ruby/javascript and on-par with Java or C#.

withinboredom 7mo ago
Do I need to share the TLA+ spec that shows its unfair? Or do you have any actual proof to your claims?
- 9rx 7mo ago
  It would be helpful for you to share a link to the Github issue you created. If the TLA+ spec you no doubt put a lot of time into creating is contained there, that would be additionally amazing, but more relevant will be the responses from the maintainers so that we're not stuck with one side of the story. Of course, expecting you to provide the link would be incredibly onerous. We can look it up ourselves just as easy as you can. Well, in theory we can. The only trouble is that I cannot find the issue you are talking about. I cannot find any issues in the Go issue tracker from your account. So, in the interest of good faith, perhaps you can help us out this one time and point us in the right direction?

pothamk 7mo ago

[dead]

mknyszek 7mo ago

> Thus, they’ll probably never fix it. I'm sorry you had a bad experience with Go. What makes you say this? Have you filed an issue upstream yet? If not, I encourage you to do so. I can't promise it'll be fixed or delved into immediately, but filing detailed feedback like this is really helpful for prioritizing work.

_rlh 7mo ago

“It’s a problem that only go can solve” I had this discussion a decade ago and concluded that a reasonable fair scheduler could be built on top of the go runtime scheduler by gating the work presented. The case was be made that the application is the proper, if not only, place to do this. Other than performance, if you encountered a runtime limitation then filing an issue is how the Go community moves forward.