4 ms·
The main Go program I work on is CloudFlare's Railgun which is 18,000 lines of code. It builds in 4s. % make clean % time make make-listener make-sender
by jgrahamc 14y ago
The main Go program I work on is CloudFlare's Railgun which is 18,000 lines of code. It builds in 4s.
% make clean
% time make make-listener make-sender make-diag make-ssl...
3.42s user
1.05s system
105% cpu
4.240 total
% find src/ -name '*.go' | xargs wc -l | tail -1
17870 total
And that's building everything sequentially. If I switch to using -j4 I get a 2s build from clean.
% make clean
% time make -j4 make-listener make-sender make-diag make-ssl...
5.75s user
1.99s system
382% cpu
2.020 total
An incremental build (no op) of the entire tree is one second:
% time make -j4 make-listener make-sender make-diag make-ssl...
1.56s user
0.73s system
209% cpu
1.090 total
Yes, we wrap the go build tools with make. Not because they are lacking anything for Go but because we do other stuff (like build documentation and release packages).
(I should have not counted the tests in the above analysis. There are 7,290 lines of tests and 10,580 lines of code).
- stock_toaster 14y ago> Yes, we wrap the go build tools with make. Do you wrap `go build` or do you wrap the 6g/6l tools directly?
- jgrahamc 14y agogo build
- agentS 14y agoI assume you're calling the go command a few times in parallel, hence the improvement with -j4. I wonder if it would be faster to aggregate all the packages you need to build in a variable, and call the go command with all of them. i.e. instead of issuing "go build foo/mylib" and "go build bar/myotherlib" in parallel, just issuing "go build foo/mylib bar/myotherlib". I say this because I remember there being logic inside the go command to parallelize builds already, see the "-p" flag in "go help build". In particular, the go command will be able to avoid statting all the files, comparing mod times twice, recomputing the DAG, etc. Although, considering how fast it runs anyway, this speedup might be negligible...
- buro9 14y agoWe have similar times being reported on a codebase of 17,482 lines of code. : time go build -a real 0m3.980s user 0m5.012s sys 0m1.140s But that is unrealistic, in dev we skip the -a flag: : time go build real 0m1.854s user 0m1.760s sys 0m0.332s
- vidarh 14y agoGiven the way people are praising Go for compilation speed here I'm surprised at your numbers, which does not seem to support the claim it's all that fast, unless your build machine is antiquated... My home server is an AMD FX-4100 (quad core, 3GHz; this is one of the early Bulldozer CPU's and definitively not a speed daemon - I picked up the CPU and motherboard for <$150 a year or two ago). I just tested a 6k C program that compiled in 1s flat with gcc 4.7.2, or .44 seconds with -j4, and a 13k lines one that completed in 2 seconds, or .88 with -j4. Gcc isn't exactly renowned for being a fast c compiler. A "no op" build of the 13k project takes .004 seconds, though, perhaps that's the reason - if your make setup is horrendously complicated perhaps it's slowing your numbers down a lot. And it's certainly possible to make C compilation incredibly slow if you do lots of nasty pre-processor hacks. But then again whether 6k, 13k or 18k lines, both your project and mine are tiny.