5 ms·
I wouldn't call it my day-to-day, but sure, here are some examples: 1. We have a client component that makes HTTP requests to a server component of ours. We've
by idop 9y ago
I wouldn't call it my day-to-day, but sure, here are some examples:
1. We have a client component that makes HTTP requests to a server component of ours. We've had production downtime issues in the server caused by filehandle exhaustion. Turns out the client was creating a new HTTP connection for every request rather than reusing connections, and it did that because Go expects you to read the response body in its entirety even if you don't need it, otherwise connections aren't reused. And in many of the cases we thought that we were reading the body in its entirety, for example in locations where the JSON decoder was reading and decoding the response, but sometimes it simply did not really read it in full, despite definitely parsing the entire JSON structure returned from the server. A "bad" net/http client can easily bring down a net/http server.
2. We've had several instances of severe memory leaks, with the Go garbage collector not releasing or reusing memory ever. These were sometimes very difficult to track down with the available Go profiling tools (pprof for example), and even more difficult to understand. And in some of these cases we're still not even sure we understand despite fixing the issues, mostly because our code is not different than examples in the Go documentation itself. For example, we had functions that read a stream of bytes into a byte slice, then cast that byte slice into a string and return it. We would intuitively think that the memory allocated for that byte slice would be released, but apparently the casting performed constitutes an active reference of the data (rather than a copy of it), so the data is not marked for garbage collection. But the most obscure part is that it was never released, even after the cast data had no more references to it. Memory management in Go is very hard for us to reason, despite the language claiming that developers needn't worry about this subject as it is handled for them.
This is just the top my head, I can bring more.
- dpflan 9y agoThanks for sharing. Which language(s) did you previously use primarily? How did you learn to program in Go? What resources did you use to "ramp up"? I ask because I am wondering about how to be introduced to such issues early on and learn how to fix them before they become more serious like causing "production downtime" because the response body isn't being completely read and closed.
- idop 9y agoMy first resources were the Go website, specifically the following pages/articles: How to Write go Code [1], Effective Go [2] and Frequently Asked Questions [3]. I also read Alan Donovan and Brian Kerninghan's book "The Go Programming Language" [4]. So mostly these four, and of course simply writing code (along with the standard library documentation [5]). Since then I've also gained a lot of info and insight from various articles and blogs, such as "Debugging Performance Issues in Go Programs" [6] from an Intel blog, various posts from Peter Bourgon's blog [7] and Dave Cheney's blog [8], both of which I highly recommend. Oh, and "The complete guide to Go net/http timeouts" from the Cloudflare blog [9] is also worth reading. Previously I was mostly a Perl developer, but I also work(ed) a lot with JavaScript, and to a (much) less extent with Haskell, Racket and Nim. [1] https://golang.org/doc/code.html https://golang.org/doc/code.html [2] https://golang.org/doc/effective_go.html https://golang.org/doc/effective_go.html [3] https://golang.org/doc/faq https://golang.org/doc/faq [4] http://www.gopl.io/ http://www.gopl.io/ [5] https://golang.org/pkg/ https://golang.org/pkg/ [6] https://software.intel.com/en-us/blogs/2014/05/10/debugging-performance-issues-in-go-programs https://software.intel.com/en-us/blogs/2014/05/10/debugging-... [7] https://peter.bourgon.org/articles/ https://peter.bourgon.org/articles/ [8] https://dave.cheney.net/ https://dave.cheney.net/ [9] https://blog.cloudflare.com/the-complete-guide-to-golang-net-http-timeouts/ https://blog.cloudflare.com/the-complete-guide-to-golang-net...
- agnivade 9y agoYour #1 issue is mentioned clearly in the documentation. Mentioning here verbatim - > The http Client and Transport guarantee that Body is always non-nil, even on responses without a body or responses with a zero-length body. It is the caller's responsibility to close Body. The default HTTP client's Transport may not reuse HTTP/1.x "keep-alive" TCP connections if the Body is not read to completion and closed. Regarding #2, I think what you are taking about is this - https://blog.golang.org/go-slices-usage-and-internals https://blog.golang.org/go-slices-usage-and-internals (gotcha section). But maybe I am wrong.