3 ms·
A reason I've heard against parallelizing compilers is that you get parallelism for free in large builds as your build system will build separate code-gen units
by gpm 10y ago
A reason I've heard against parallelizing compilers is that you get parallelism for free in large builds as your build system will build separate code-gen units (files in C, crates in rust, I believe in go it's packages) that don't depend on eachother in parallel anyways. Adding it to the compiler as well just creates synchronization overhead.
- DSMan195276 10y agoI would agree. Basically all decent build systems will allow you to compile units that aren't dependent on each other in parallel. Threading the compiler may provide some gains, but it's likely that for a good portion of your build time your CPU is already fully utilized anyway (And without any locking overhead). I feel like the real gains would be with a threaded linker instead, but I can't say I have any data to back that up. But I mean, you can't link until everything is compiled, so presumably there are free CPU cores that could be used.
- uluyol 10y agoGo's compilation unit is a package, not a file. This limits the amount of parallelism you can get compared to say C outside of the compiler. You have bottleneck packages like the runtime which have to be compiled before other packages, and in the case where you do incremental compilation you probably only have a few packages to compile anyway. Some of this is briefly touched on in the CL (see comments on the amount of concurrently that should be enabled), and there are issues on the bug tracker that go into more detail.
- weberc2 10y agoStill, the benchmarks seem promising. 40% improvement on std lib packages is nothing to shake a stick at. The end of the CL explains that this is a best case scenario, one that will mostly benefit recompilation of a single package, which is the most common compilation activity.
- obstinate 10y agoSince most builds are incremental and only touch a single package, this should still be a win. And it may be a win whenever there is just one package on the critical path. (I don't know if go packages must be built after their dependencies, but it stands to reason if so.)