3 ms·

The special sauce is you get work in from the network, split it into small pieces for a bunch of worker go routines and then make some of that work do down stre

The special sauce is you get work in from the network, split it into small pieces for a bunch of worker go routines and then make some of that work do down stream calls in easy to write CS 101 style network code; the go routines/channels exert back pressure exactly as you specify and the scheduler spreads the work across all the cores without you having to lock anything. And they have libraries for a bunch of modern things. Pretty good libraries too, written by people that care about error checking.

bombela 6y ago
How do you apply back pressure? As far as I can tell Go will spawn a goroutine per request (well that's how the standard http lib and grpc seems to do it). I would like to compute a load factor and refuse new requests. Or do some basic rate limiting. But the coroutine is spawned before I have a say...
- iampims 6y ago
  the package `netutil` has a LimitListener function which wraps the given listener and accepts at most N simultaneous connections.
- jerf 6y ago
  In general, rate limiting of connections shouldn't take place in the server itself, it should be in some upstream load manager. If your system is too small to bother with that, it's almost certainly too small to worry about blowing out the goroutines. This is not a statement about Go, it's true in general. Even if you consult something per request, goroutines are fast enough to look at something and then decide not to do the work that, again, by the time that's a problem, the solution is to throttle upstream anyhow.
  - bombela 6y ago
    I see so it seems that there is a curve of load handling behavior per language/lib/framework. As the load increase you enter the territory of queueing (add latenxy) up to a point. Then reject/drop requests. If the rejection is slower than the OS/hardware can provide the software crashes. For Go it appears that the curve is an almost straight vertical line to oblivion. But of course if you plan to scale and use a dedicated software to pace requests, there is no point into adding this complexity to the Go libraries. Maybe what I lament here. Is that this tradeof made by Go is not more advertised. Such that when the day comes that the dreaded out of memory killer shows up, you are ready to fight back.

bombela 6y ago

How do you apply back pressure? As far as I can tell Go will spawn a goroutine per request (well that's how the standard http lib and grpc seems to do it). I would like to compute a load factor and refuse new requests. Or do some basic rate limiting. But the coroutine is spawned before I have a say...

iampims 6y ago
the package `netutil` has a LimitListener function which wraps the given listener and accepts at most N simultaneous connections.
jerf 6y ago
In general, rate limiting of connections shouldn't take place in the server itself, it should be in some upstream load manager. If your system is too small to bother with that, it's almost certainly too small to worry about blowing out the goroutines. This is not a statement about Go, it's true in general. Even if you consult something per request, goroutines are fast enough to look at something and then decide not to do the work that, again, by the time that's a problem, the solution is to throttle upstream anyhow.
- bombela 6y ago
  I see so it seems that there is a curve of load handling behavior per language/lib/framework. As the load increase you enter the territory of queueing (add latenxy) up to a point. Then reject/drop requests. If the rejection is slower than the OS/hardware can provide the software crashes. For Go it appears that the curve is an almost straight vertical line to oblivion. But of course if you plan to scale and use a dedicated software to pace requests, there is no point into adding this complexity to the Go libraries. Maybe what I lament here. Is that this tradeof made by Go is not more advertised. Such that when the day comes that the dreaded out of memory killer shows up, you are ready to fight back.