6 ms·

The only relevant info I could dig up was this issue from Kubernetes in 2016: https://github.com/kubernetes/kubernetes/issues/27444 https://github.com/kubernete

The only relevant info I could dig up was this issue from Kubernetes in 2016: https://github.com/kubernetes/kubernetes/issues/27444 https://github.com/kubernetes/kubernetes/issues/27444 It seems to be discussing the speed of building Kubernetes on their build-servers, targeting multiple architectures at once. Unfortunately all the build servers are internal to Google so I don't know exactly what's being included (or what's being run), but this comment (https://github.com/kubernetes/kubernetes/issues/27444#issuecomment-226438757 https://github.com/kubernetes/kubernetes/issues/27444#issuec...) details the breakdown of time spent: 30m of build 7m on cluster startup 12m on tests 7m on cluster teardown So, taken with an enormous hunk of rock salt, it seems like Kubernetes might take around 30m to build on a cluster, to build (possibly multiple) release artifacts. Seems in line with Rust.

melling 7y ago
What’s the state of parallel builds? We’re getting 16/32 core cpus. In a few years, we’re all going have Threadrippers. Of course everything can’t be done in parallel.
- granzymes 7y ago
  Almost all parts of the compiler (at least for LLVM) are still strictly single-threaded. You can't even use threads yourself if you write your own LLVM module pass.
  - fnord123 7y ago
    Some of the parts of a compiler are inherently serial. e.g. parsing a file. But if you have to parse multiple files, then hey presto, parallelism. The linker has to work as a spooler for writing the resulting executable or shared objects, but afaik, there's no reason for symbol resolution to not be done in parallel. And if you want to optimize your GOT then that could also be sorted by a page rank style optimization which is the algorithm which was the foundation of the phrase 'embarrassingly parallel'.
- pjmlp 7y ago
  Given that most sweet shop ITs provide i7 and i5 with 8GB and HDD, good luck getting that hardware adopted there.
ptomato 7y ago
That's compiling for 10 os/arch combos, so ~3m per.
- topspin 7y ago
  That makes a lot more sense. Go was specifically designed to compile fast.
- nneonneo 7y ago
  On the other hand, these are running on some big cluster with a whole bunch of CPUs, so I wonder if you’d still get 3m per build on something like a laptop?
  - ptomato 7y ago
    Any individual build they're referencing there runs on a single vm (the cluster referenced there is the k8s cluster being built up and torn down on that vm to run e2e tests on). It looks like at the time they were using GCP standard-8 instances, which are 4 core/8 thread w/30GB of ram. It's hard to say exactly what the current build time is, as it looks like they do some build caching now; on my desktop a fully clean build of current kubernetes master (3.2 million loc) for linux/amd64 takes just under 2 minutes.

melling 7y ago

What’s the state of parallel builds? We’re getting 16/32 core cpus. In a few years, we’re all going have Threadrippers. Of course everything can’t be done in parallel.

granzymes 7y ago
Almost all parts of the compiler (at least for LLVM) are still strictly single-threaded. You can't even use threads yourself if you write your own LLVM module pass.
- fnord123 7y ago
  Some of the parts of a compiler are inherently serial. e.g. parsing a file. But if you have to parse multiple files, then hey presto, parallelism. The linker has to work as a spooler for writing the resulting executable or shared objects, but afaik, there's no reason for symbol resolution to not be done in parallel. And if you want to optimize your GOT then that could also be sorted by a page rank style optimization which is the algorithm which was the foundation of the phrase 'embarrassingly parallel'.
pjmlp 7y ago
Given that most sweet shop ITs provide i7 and i5 with 8GB and HDD, good luck getting that hardware adopted there.

ptomato 7y ago

That's compiling for 10 os/arch combos, so ~3m per.

topspin 7y ago
That makes a lot more sense. Go was specifically designed to compile fast.
nneonneo 7y ago
On the other hand, these are running on some big cluster with a whole bunch of CPUs, so I wonder if you’d still get 3m per build on something like a laptop?
- ptomato 7y ago
  Any individual build they're referencing there runs on a single vm (the cluster referenced there is the k8s cluster being built up and torn down on that vm to run e2e tests on). It looks like at the time they were using GCP standard-8 instances, which are 4 core/8 thread w/30GB of ram. It's hard to say exactly what the current build time is, as it looks like they do some build caching now; on my desktop a fully clean build of current kubernetes master (3.2 million loc) for linux/amd64 takes just under 2 minutes.