5 ms·
One of the features of Go is suppose to be compilation speed. Can anyone compare to C, for example, or discuss compilation speed in large projects (e.g. >100k
by melling 14y ago
One of the features of Go is suppose to be compilation speed. Can anyone compare to C, for example, or discuss compilation speed in large projects (e.g. >100k loc)?
- enneff 14y agoThe Juju guys at Canonical said at GoSF that their >100k LoC project compiles in 6 seconds.
- azinman2 14y agoMuch better than SBT compiling my Scala project (20k? LoC) in 75 seconds (!) on my latest MBP. And that was not from scratch, but having changed about 10 lines of code.
- tptacek 14y agoThe Go compiler is so fast you can use it the way you would a scripting interpreter ("go run whatever.go"); it builds the whole standard library in seconds. Every program I've written in it (topping out at around 20kloc) compiles instantaneously --- I can't perceive the amount of time it takes to compile. C compilers are fast, but they aren't that fast. Go compiles so fast that it changes the way you write and organize code; you would never consider compilation overhead in any decision you made in a Go source tree, where you might do that in a C codebase. For instance, Go programmers routinely compile code, run it, and throw away the compiled binary! You wouldn't think it would be possible for compiler speed to be so much better than C that it would make a difference, but Go's compiler manages it.
- StavrosK 14y ago> Every program I've written in it (topping out at around 20kloc) compiles instantaneously --- I can't perceive the amount of time it takes to compile. Okay, Go is fast, but it's not that fast. My 1000 LoC program takes ~1 second to compile. While it's certainly fast enough for me, it's not imperceptible.
- justin66 14y agoIs this code up anywhere? It'd be interesting to see what's going on there.
- StavrosK 14y agoNo, it's closed-source unfortunately. It might be a bit inaccurate as I only measured the includes in my folder and not everything in the src folder, but it only imports from the stdlib and a redis lib.
- georgemcbay 14y agoThe one pathological case of kinda-slow Go compilation I have involves embedding a very big binary blob (FPGA firmware for a controller chip driving a Texas Instruments DMD array) as a const array of bytes that I can just dump down the usb connection when cold booting the device. I don't fault Go for this as I'm really abusing the language to simplify my single-file deployment without having to do too much work for the simple deploy. If I wanted to spend the time I could do something like append the blob to the end of the program executable instead of just clumsily jamming it into DATA, but this all exists in a library that doesn't need to change often so the fact that it takes ~10-15 seconds to compile is not a problem in practice.
- krasin 14y agoI would suggest to file an issue: https://code.google.com/p/go/issues/list https://code.google.com/p/go/issues/list At least, Go team will be aware of your pathological case and have a chance to speed up it.
- agentS 14y agoI vaguely remembered minux suggesting a way to do this more efficiently, and dug it up: golang.org/issue/3035 Maybe it'll work for your use case.
- georgemcbay 14y ago
- recuter 14y agoWhich is the entire raison d'être for the Go project and thus it can be dubbed a success. Any other advantages over C are just gravy. :) A nicer, more accessible (python/ruby/etc folks seem to like it a lot), faster compiling reasonable alternative to C is a worthy goal - I really don't get why some people get into flamewars over it.
- danenania 14y agoIf some people don't get into flamewars over a language, it's because no one's using it. :)
- georgemcbay 14y ago"You wouldn't think it would be possible for compiler speed to be so much better than C that it would make a difference" I would think that, even before I had used Go. Ignoring the smaller scanning benefits of Go that come about from the more rigid syntax for braces and such, the C preprocessor system introduces a complete nightmare when it comes to requiring multiple passes over the same code, especially for code that is naive about include guards, pre-compiled headers and other "best practices". C (and C++ is even worse) has a lot of very large low hanging fruit to cull when it comes to writing languages with faster compilers. Whatever speed C compilers have now is due to the geniuses who wrote the compilers and decades of shoulders to stand on, the language is actively working against them at nearly every turn when it comes to compilation speed. That aside, I agree. Go compiles very, very fast. So fast that it never occurred to me that "go get" did a compilation until it was pointed out to me that it did.
- haberman 14y ago> C (and C++ is even worse) has a lot of very large low hanging fruit to cull when it comes to writing languages with faster compilers. I look forward to seeing the speed boost that C and C++ get if/when modules are available: http://llvm.org/devmtg/2012-11/Gregor-Modules.pdf http://llvm.org/devmtg/2012-11/Gregor-Modules.pdf
- __david__ 14y ago> You wouldn't think it would be possible for compiler speed to be so much better than C that it would make a difference, but Go's compiler manages it. I did some ocaml work a few years ago and had a similar revelation. Its compiler is just as fast—my short little programs were more or less instantaneously compiled. Fast compiling is really, really nice, even though it seems like a silly thing to tout.
- vidarh 14y agoC compilers are largely slow because they are written with the expectations of a lot of optimization and large projects rather than with much concern for compile times. C compilers that are written to be fast rather than flexible can be extremely fast. E.g. see Fabrice Bellards "TCCBOOT", that he demonstrated compiling the Linux kernel at boot-time in 15 seconds on a 2.4GHz Pentium 4: http://bellard.org/tcc/tccboot_readme.html http://bellard.org/tcc/tccboot_readme.html TCC can also be treated much like an interpreter like your go example.
- pjmlp 14y agoAny language with modules is able to run circles around C compilation speeds, regardless of what optimizations you might be doing. It all boils down to having the parse each include file every time it appears, because the preprocessor might change the meaning of the file in each inclusion.
- vidarh 14y agoThat's something TCCBOOT demonstrated quite clearly isn't a major problem of C itself but rather an issue with common C compiler implementations, given the size of the Linux kernel and the time spent on a CPU which is by now ancient history. Yes, you need to pre-process and potentially parse them again, but in practice most C include files use include guards and are low cost to process if done even remotely properly.
- pjmlp 14y agoBut the compiler does not know the guards are there, so it needs to read the files anyway. This is why Plan9 C compilers have the strict rules that header files don't include other header files and you as a user need to include them all. I need to look into TCCBOOT in more detail.
- vidarh 14y agoTechnically it needs to read the file once, and keep the data around (of course it can keep re-reading the same files, but there's no technical reason other than memory availability that forces it to). If the compiler implementer cares about speed, it can easily keep the data together with tokenized pre-processed blocks that can be used to reduce the guard lookups to a hash table lookup unless someone has done something weird (in which case the worst case is to revert to pre-processing the file again). It is also perfectly possible to cache token streams or even pre-parsed (with some caveats) versions of the headers specialised by the value of used pre-processor definitions, which in most cases will prevent re-parsing of most include files (only causing re-parses when the pre-processor definitions actually change). The pathological worst case for C code can be horribly nasty, but real-world code is generally fairly well behaved and easy to speed up processing of on modern systems that aren't terribly memory starved. Even on systems that are memory starved, there have been plenty of C compilers that can "pre-compile" headers to speed up compilation.
- justin66 14y agoIf your memory goes back this far: it's Turbo Pascal fast. :)
- snaky 14y agoTurbo Pascal was so fast mostly because of not compiler but linker and .tpu files format. edit: s/tpl/tpu/ btw, the tp linker - http://turbopascal.org/linking-modules-and-creating-exe-file http://turbopascal.org/linking-modules-and-creating-exe-file
- justin66 14y agoWhich is an interesting point... I'm pretty sure Go has more in common with TP than ordinary C wrt linking, relying on modules, and using static binaries. http://prog21.dadgum.com/47.html http://prog21.dadgum.com/47.html edit: in fact a number of the things that Rob Pike has brought up in talks reminded me of that article
- _ak 14y agoIIRC, the compiler was single-pass, and thus, quite fast.
- vidarh 14y agoMost older Pascal compilers were, as most (all?) of Wirth's languages are explicitly designed to make single pass compilation easy.
- pjmlp 14y agoAnd it used modules.
- pacala 14y agoHaha, you've just sold me. I'm switching to Go for my next project :)
- icelancer 14y agoI've been reading this thread trying to find a reason to switch to Go. Then I read this comment and it brought back some very good memories (TP was my first language after command-line BASIC on the Apple IIe and TRS-80). Sigh. Looks like I have a new weekend project!
- jgrahamc 14y agoThe main Go program I work on is CloudFlare's Railgun which is 18,000 lines of code. It builds in 4s. % make clean % time make make-listener make-sender make-diag make-ssl... 3.42s user 1.05s system 105% cpu 4.240 total % find src/ -name '*.go' | xargs wc -l | tail -1 17870 total And that's building everything sequentially. If I switch to using -j4 I get a 2s build from clean. % make clean % time make -j4 make-listener make-sender make-diag make-ssl... 5.75s user 1.99s system 382% cpu 2.020 total An incremental build (no op) of the entire tree is one second: % time make -j4 make-listener make-sender make-diag make-ssl... 1.56s user 0.73s system 209% cpu 1.090 total Yes, we wrap the go build tools with make. Not because they are lacking anything for Go but because we do other stuff (like build documentation and release packages). (I should have not counted the tests in the above analysis. There are 7,290 lines of tests and 10,580 lines of code).
- stock_toaster 14y ago> Yes, we wrap the go build tools with make. Do you wrap `go build` or do you wrap the 6g/6l tools directly?
- jgrahamc 14y agogo build
- agentS 14y agoI assume you're calling the go command a few times in parallel, hence the improvement with -j4. I wonder if it would be faster to aggregate all the packages you need to build in a variable, and call the go command with all of them. i.e. instead of issuing "go build foo/mylib" and "go build bar/myotherlib" in parallel, just issuing "go build foo/mylib bar/myotherlib". I say this because I remember there being logic inside the go command to parallelize builds already, see the "-p" flag in "go help build". In particular, the go command will be able to avoid statting all the files, comparing mod times twice, recomputing the DAG, etc. Although, considering how fast it runs anyway, this speedup might be negligible...
- buro9 14y ago