5 ms·
We run a cluster of P2P, GPU heavy machines that use Go to ingest byte streams of raw radar data, store that info in btrees, and render, cache & serve map tiles
by courtf 10y ago
We run a cluster of P2P, GPU heavy machines that use Go to ingest byte streams of raw radar data, store that info in btrees, and render, cache & serve map tiles that are drawn on the fly in response to http requests. We are not using much outside the stdlib (opengl and gdal bindings).
Garbage collection has become very fast in recent versions of the language. It's quite painless now, and we did struggle with it in the past. I'd take the current GC over dealing with other strategies unless you are a very specific, >60fps kind of situation. We literally don't think about the GC anymore.
Random stats from one of our busier nodes, over the past 20 min:
40GB currently in use, 391GB allocated (and freed) in the last 20min
average of 2.6ms GC pauses every 70.6s
95th percentile on GC time is 7.26ms
For dependency management, there's a lot of tools that make things more automated and I think probably offer people what they are expecting/used to from other environments. This has been a sticky point for sure. I've settled on a bare bones solution that seems to work well: just using git submodules in the vendor directory within the project repo. Does everything I need (pin to version of a fork, build immediately after a clone and a submodule update --init). As of Go1.5, the vendor directory is supported by the compiler, and it will look there first to resolve dependencies. While this does work with recursive dependencies, you will want to flatten the dependency graph as much as possible if types are being used across multiple deps because types are distinguished by their full import paths. This encourages breaking things into modules so I'm ok with it.
First-tier support is a problem I have no doubt, but not one we've suffered from personally.
I'd say that debugging comes up a lot too, and Go doesn't have an official, reliable option for debugging. The Delve project is getting close to that though it looks like. I personally debug with the stdlib pprof package (provides an http server that prints stack traces for all goroutines currently active, allows cpu/memory/blocking profiles to be requested), print statements, general testing/experimentation and newrelic.
- dalailambda 10y agoHow do you find the cgo overhead? Specifically with something like OpenGL, it seems like it would be a big pain point for rendering.
- courtf 10y agoI don't have great measurements for this, but we have done optimizations to reduce the data flowing over the Go<->C interface. One of our key measurements that does make a big impact on performance is how often we need to upload data (that's not already over there) to the GPUs. So that's something we have worked on reducing (buffer reuse, compression). We also have a series of caches on the other side, so we aren't drawing more than we need to. It's hard for me to tease apart how much of these optimizations (and others) are ultimately aimed at addressing the cgo overhead, and how many are just typical stuff. The data we work with is cumbersome and my intuition is that there's probably a lot of room for optimization in our drawing even still, regardless of cgo. I wouldn't be surprised if a direct port C/C++ implementation of the rendering pipeline was significantly faster than ours in getting data into and out of the GPUs, but a big part of the project is data storage/networking/serving/caching as well and Go has bridged the gap for us (a small team that needs to build reasonably fast things reasonably quickly :)).
- dalailambda 10y agoThat's interesting. The cgo overhead was the only thing holding me back from considering it for games, since I didn't want to write a lot of C wrappers around the C libraries I want to use just to have them be more efficient, which is a shame, since Go is pretty nice, barring the C interop in some cases.
- pjmlp 10y agoWhen I used to frequent gonuts, I raised the issue why they didn't went the FFI way as D, Rust, .NET, Delphi, FreePascal do, but sadly they rather use cgo as solution.
- dalailambda 10y agoI'm pretty sure the problem is Go's use of segmented stacks and runtime, which makes it hard to properly interface with C efficiently.
- mbgaxyz 10y agoWould the vendor management support deterministic builds?
- ereyes01 10y agoYes, that is the purpose of the vendor directory. You copy the source of each of your dependencies into vendor/ and commit it to your own source control so it can't change unless you do so yourself. I personally use git submodules to manage the contents of the vendor/ tree, but the go compiler/toolchain itself doesn't really care how you get the files into there. Edit: other comments below have mentioned some of the tools available to manage your vendor/ tree. Using git submodules manually can be crude, especially when adding dependencies that themselves have other dependencies.
- aj0strow 10y agoYes. I check my vendor into git, so it's as deterministic as source code.
- jre 10y agoWhat company are you working at ? Looks like you do some interesting geospatial/map rendering stuff. Are you hiring ? :-)