3 ms·
How do you apply back pressure? As far as I can tell Go will spawn a goroutine per request (well that's how the standard http lib and grpc seems to do it). I w
by bombela 6y ago
How do you apply back pressure? As far as I can tell Go will spawn a goroutine per request (well that's how the standard http lib and grpc seems to do it).
I would like to compute a load factor and refuse new requests. Or do some basic rate limiting. But the coroutine is spawned before I have a say...
- iampims 6y agothe package `netutil` has a LimitListener function which wraps the given listener and accepts at most N simultaneous connections.
- jerf 6y agoIn general, rate limiting of connections shouldn't take place in the server itself, it should be in some upstream load manager. If your system is too small to bother with that, it's almost certainly too small to worry about blowing out the goroutines. This is not a statement about Go, it's true in general. Even if you consult something per request, goroutines are fast enough to look at something and then decide not to do the work that, again, by the time that's a problem, the solution is to throttle upstream anyhow.
- bombela 6y agoI see so it seems that there is a curve of load handling behavior per language/lib/framework. As the load increase you enter the territory of queueing (add latenxy) up to a point. Then reject/drop requests. If the rejection is slower than the OS/hardware can provide the software crashes. For Go it appears that the curve is an almost straight vertical line to oblivion. But of course if you plan to scale and use a dedicated software to pace requests, there is no point into adding this complexity to the Go libraries. Maybe what I lament here. Is that this tradeof made by Go is not more advertised. Such that when the day comes that the dreaded out of memory killer shows up, you are ready to fight back.