5 ms·
Make resilient Go servers using timeouts, deadlines and context cancellation
- axaxs 7y agoYeah, I quit reading when I get the unblockable full page ad asking me about paying 50 dollars for a Go course. Good spam bot.
- Operyl 7y agoI did not get such an ad on my end? And this browser has no blockers of any kind. I scrolled through it in its entirety.
- atombender 7y agoAppears after a while. Not based on scrolling, as far as I can tell.
- yiyus 7y agoThat makes sense. Using a timeout is much more appropriate for this specific article.
- axaxs 7y agoSo you know I'm not lying... https://ibb.co/v43sMXv https://ibb.co/v43sMXv It's not actually unskippable, but on mobile you have to zoom out to click the X at top right.
- gigatexal 7y agoI scrolled through the whole article and didn’t get a blaring ad like that.
- regecks 7y agoI tried running Go HTTP servers bare to the internet (after Cloudflare promoted doing so in a blog post), but went back to using a reverse proxy the next time. The main benefit seems to be convenience. I can upgrade and graceful-restart nginx instead of having to rebuild and redeploy the Go server (involving a full app restart). Not having to worry about goroutine leaks because some jerk decided to send the request line @ 1 byte/sec is just an added bonus.
- tills13 7y agoCloudFlare itself offers request buffering, so you should be good on that front.
- jrockway 7y agoYou have to worry about the jerk sending requests at 1 byte per second no matter which webserver you use. It's always a problem to let an unlimited number of people ask for an unlimited amount of resources; it's just that things like goroutines are heavier than a file descriptor or a few bytes of RAM, so you'll notice wasted goroutines more quickly than wasted fds or memory. Typically, you need to consider the total amount of memory you want your web server to use, how much of that memory one request can use, and how long a request can use that memory. (File descriptors must also be considered.) Envoy has a section in their documentation about this here: https://www.envoyproxy.io/docs/envoy/latest/configuration/best_practices/edge#best-practices-edge https://www.envoyproxy.io/docs/envoy/latest/configuration/be... nginx similarly has a number of knobs to turn: https://www.nginx.com/blog/tuning-nginx/ https://www.nginx.com/blog/tuning-nginx/ I use Envoy as my web proxy and nginx to serve static content. My envoy configuration is complicated and my nginx configuration is simple, as a result. I imagine that if you are hosting a serious amount of traffic with Nginx as the edge proxy, more tuning is required. I've never tried, so I don't really know.
- zzzcpan 7y ago> You have to worry about the jerk sending requests at 1 byte per second no matter which webserver you use. Not necessarily. It's just free webservers don't bother dealing with it, but there are plenty of simple approaches. Like just dropping connections that are sending requests slower than some threshold or dropping the slowest connection when some total number of connections is reached. Or more complicated, which also works to protect from all kinds of attacks, dropping the highest malicious score or the lowest reputation score client when some resource usage threshold is reached. None of these are easy to implement with synchronous multithreaded networking code though, like in Go. Realistically it's only viable with asynchronous single threaded programming models or an actor model.
- diamondo25 7y agoOk, good, contexts are now making sure you can handle upcoming timeouts decided by an upper layer (caller function). But how about the time.After function? It'll still be running in the background? So you can still have a memory or 'processing power' leak?
- rakoo 7y agoThe function you write after time.After should use the same context, and check its Done channel before continuing execution
- diamondo25 7y agoSo you have to propagate the Context, hm. IIRC, go test will panic the test case when it times out. Not exactly sure tho. It would be nice if there was a kind of 'abort' feature to clean up subroutines spun off this thread
- thwarted 7y agoThe best there is with the context package is to make sure to call the cancel function given to you by contexts that have cancelation. Usually you do this via defer. The cancel function is a no-op if the context is finished otherwise. All this ends up doing though is making sure that things that clean themselves up know to clean themselves up eventually.
- morelisp 7y ago> Usually you do this via defer. I agree this is usually done by defer, but you probably should not do it that way unless your code is very simple. Consider a function body which I've seen variants of many times: ctx, cancel = context.WithTimeout(pctx, timeout) defer cancel() if resp, err := do(ctx, req); err == nil { process(resp) } return err Safe yes, but optimal? process doesn't use the context, and may take longer than the timeout. The context will continue running, with some associated resource cost (at the very least, the context's goroutine and timer). A minimal change is: ctx, cancel = context.WithTimeout(pctx, timeout) resp, err := do(ctx, req) cancel() if err == nil { process(resp) } return err Which disposes of those resources much earlier. (Depending on your Go compiler version there is also a potential cost associated simply with using defer; this is independent of that.)
- namanaggarwal 7y agoShouldn't read header timeout be less than read timeout?
- deleted 7y ago[deleted]
- mjpuser 7y agoNice intro to timeouts and context. Next step would be dealing with state changes that happen in a cancelled request.
- telendt 7y agoThere's an ugly bug in http.TimeoutHandler though - it obscures stack traces so that it's impossible to use them to locate panic in decorated handler: https://github.com/golang/go/issues/27375 https://github.com/golang/go/issues/27375
- awinter-py 7y agoabsolutely agree with the risk of slow clients saturating your connection limit when doing DB work with these, I'm a little shakier -- once I start a multistep DB write, I probably want it to finish. Yes I can use a transaction to roll back the whole thing, but I think there are cases where rollback is wrong and I'd rather keep the write. so while cancellation is cool, it's also a little fraught and hard to test.
- shhsshs 7y agoIn those rare cases you can choose not to propagate the same context through those operations. Only check for cancellation once the operations have all finished.
- JyB 7y agoAlmost all articles I've seen explaining the context pkg are done with net/http examples. That's fine but I feel like it might not be the best introduction for a novice as a lot of concepts are mixed together and they might miss the bigger picture. Context is not just for web servers. You don't have to know how net/http works. You could simply demonstrate the usefulness of the context package with showing how to properly clean up your program on a sigterm. Or even gracefully stop a long running operation so you are not afraid to stop/start your program at the "wrong" time.
- K0SM0S 7y ago> “Almost all articles I've seen explaining the context pkg are done with net/http examples. That's fine but I feel like it might not be the best introduction for a novice as a lot of concepts are mixed together and they might miss the bigger picture. Context is not just for web servers. You don't have to know how net/http works.” Indeed. But I think the seminal blog post¹ on the matter has somewhat skewed perception initially. The first sentence reads: > In Go servers, each incoming request is handled in its own goroutine. Request handlers often start additional goroutines to access backends such as databases and RPC services. That pretty much sets a mental context (pun unintended) within the server paradigm. People then just overlook the fact that any Go app can (and often probably should) make use of `context` — CLI anyone? Unless I'm mistaken, `context` is fundamentally about the underlying machine, not network (it just adds to confusion that the latter is represented as "context" within the former, i.e. files and folders and variables in a POSIX environment). > You could simply demonstrate the usefulness of the context package with showing how to properly clean up your program on a sigterm. Very nice one! I never saw a book titled "Linux Mechanics" but this would be akin to applying F⁻¹ to some object whose inherent motion comes from F. Like, the basics, "how do I stop this thing?" This is a cultural approach to computers that I think is fading along with the desktop culture, which itself got blurred by tons of layers of abstraction (think 1980-2020). In that vein I think it's very interesting to experiment with Linux namespaces² (notably cgroups and network namespaces). Create elementary "container components" (i.e. network namespace, UTS..) with Go and see from there how context applies (or not). You'll be effectively clean-rooming³ Docker, in some abstract sense. Distribute it and you've got a skeleton for Kubernetes. It's a really logical conclusion/next step when diving deep into Linux in the current context — DevOps etc. You just see that culture emerging from the tech. I think Go is incredibly well-positioned to learn these things as we speak: simple, efficient, easy to read, asks you to consider every step carefully (error handling), fluent with OS calls, etc. [1]: https://blog.golang.org/context https://blog.golang.org/context [2]: http://man7.org/linux/man-pages/man7/namespaces.7.html http://man7.org/linux/man-pages/man7/namespaces.7.html [3]: Ignoring common licensing or business concerns atrributed this approach, playing with "clean room" work is excellent technical training to master topics beyond intermediary level, well into expertise — that which no one can really directly impart on you, that which you must build for yourself, quite literally so at times. Can you build it, given Search and enough time? https://en.wikipedia.org/wiki/Clean_room_design https://en.wikipedia.org/wiki/Clean_room_design