3 ms·

It sounds like you have a memory leak in your golang app. That shouldn't be happening. Time to bust out pprof. I'd say though GC is what limits golang as a sys

by nvarsj 6y ago

It sounds like you have a memory leak in your golang app. That shouldn't be happening. Time to bust out pprof. I'd say though GC is what limits golang as a systems language. If you are doing lots of memory heavy things, I'd say golang may not be the correct language to use. [1] https://github.com/prometheus/prometheus/issues/6934 https://github.com/prometheus/prometheus/issues/6934

hardwaresofton 6y ago
Definitely a memory leak, but the code is like < 50 lines -- it's a memory leak (and/or my bone-headed misuse) in the underlying library, fasthttp. I got some help with fixing it from a reader though: https://gitlab.com/mrman/kcup-go/-/merge_requests/7 https://gitlab.com/mrman/kcup-go/-/merge_requests/7 Again, the goal was to serve a single file -- I should point out that v1 was using net/http and it did quite well (memory scaled up and down as necessary), but the combination of resource contraints (on my side, I only gave it 100MB of RAM), and use of fasthttp that wasn't ideal/correct caused the issues. It went something like this: v1 - net/http v2 - fasthttp (memory leak but I didn't notice at this point) v2.1 - fasthttp (use []bytes, still has the leak) v2.2 - fasthttp (leak fixed thanks to Pavel) All these solutions performed worse than their rust counterpart though, so I was surprised at how much work Go was making me do more than anything. GC is certainly what limits golang as a system language, and if I think about it I am certainly standing on top of a lot of abstraction on the Rust side (hyper, tokio, structopt), but this exploration made me more of a rust fan than anything. The ~30 mins I spent figuring out how to efficiently pass the references was painful at the time but the payoff has been hours of not futzing with the code.

3 ms·

It sounds like you have a memory leak in your golang app. That shouldn't be happening. Time to bust out pprof. I'd say though GC is what limits golang as a sys

by nvarsj 6y ago

hardwaresofton 6y ago
Definitely a memory leak, but the code is like < 50 lines -- it's a memory leak (and/or my bone-headed misuse) in the underlying library, fasthttp. I got some help with fixing it from a reader though: https://gitlab.com/mrman/kcup-go/-/merge_requests/7 https://gitlab.com/mrman/kcup-go/-/merge_requests/7 Again, the goal was to serve a single file -- I should point out that v1 was using net/http and it did quite well (memory scaled up and down as necessary), but the combination of resource contraints (on my side, I only gave it 100MB of RAM), and use of fasthttp that wasn't ideal/correct caused the issues. It went something like this: v1 - net/http v2 - fasthttp (memory leak but I didn't notice at this point) v2.1 - fasthttp (use []bytes, still has the leak) v2.2 - fasthttp (leak fixed thanks to Pavel) All these solutions performed worse than their rust counterpart though, so I was surprised at how much work Go was making me do more than anything. GC is certainly what limits golang as a system language, and if I think about it I am certainly standing on top of a lot of abstraction on the Rust side (hyper, tokio, structopt), but this exploration made me more of a rust fan than anything. The ~30 mins I spent figuring out how to efficiently pass the references was painful at the time but the payoff has been hours of not futzing with the code.