3 ms·

Ran some benchmarking with cmake+ninja vs these bazel BUILD files on a soyoustart GAME-2. Build from scratch... - cmake: ~2900 files @ 28:21.07 total (https://

Ran some benchmarking with cmake+ninja vs these bazel BUILD files on a soyoustart GAME-2. Build from scratch... - cmake: ~2900 files @ 28:21.07 total (https://snapshot.raintank.io/dashboard/snapshot/ijkptI1dlteBvYvIsfernj6DYgiU8GcV?orgId=2 https://snapshot.raintank.io/dashboard/snapshot/ijkptI1dlteB...) - bazel: ~5k targets @ 23:55.13 total (https://snapshot.raintank.io/dashboard/snapshot/TbOP6L814R3qiYJEt6lYimHJW5ksRMFD https://snapshot.raintank.io/dashboard/snapshot/TbOP6L814R3q...) If anyone can recommend a file to modify I'd also be happy to test out incremental build times. Anyway, super glad to see google externalizing some of this stuff! I also have the metrics from this prometheus instance saved so if anyone wants to see any other metrics from node_exporter let me know! fwiw bazel "felt" slower because the number of targets is an estimate and that kept going up while cmake was constant progress to "finish". I'd love to see how the bazel time looks with remote build execution enabled to see how much faster that goes.

q3k 6y ago
Another fun thing you can do is to do another clone of the llvm-bazel repo, build that using Bazel again. Things should've cached correctly and the rebuild from a totally different checkout should be pretty much instant, as long as your ~/.cache/bazel is still there.
Joky 6y ago
I would double-check that it builds the exact same thing (there are lot of options when building LLVM, and CMake may auto-detect and enable support for things like Go bindings for example). Choice of compiler and compiler flags can make a difference as well.
siggi 6y ago
As another data point - I ran the same llvm bazel build using remote build execution on 128 cores. - clean, caching disabled @ 5.93m INFO: Elapsed time: 356.189s, Critical Path: 187.81s INFO: 4235 processes: 4235 remote. - clean, fully remote cached @ 27.33s INFO: Elapsed time: 27.330s, Critical Path: 15.93s INFO: 4235 processes: 4235 remote cache hit. Full disclosure - I co-founded BuildBuddy (YC W20) which provides bazel remote build execution and other open core bazel tooling
- cromwellian 6y ago
  Wow, this service looks amazing. I looked at setting up RBE on GCP and it looked like a pain to do it myself.
jlebar 6y ago
I would be extremely surprised if Bazel were 4m faster than cmake+ninja with the same build config. If this really is a correct result, I bet the ninja people would want to hear about it. (And, kudos to the Bazel folks!) But I think it's more likely measurement error. Maybe the build flags or host compiler aren't the same in the two configurations?
- simfoo 6y ago
  Most likely the CMake build is simply building more code than Bazel is. I am 100% sure that this is not comparing apples to apples
  - gravypod 6y ago
    Yea, you can also see from the linked dashboards that it's writing way more stuff at the end of the build. I was watching the log just now and it looks like it's linking some things. I hope someone with more Bazel and LLVM knowledge can give this a go. I just wanted to toss something up as a "what would it be like if I just compiled this project" perspective. This is absolutely not a definitive "X is better than Y" since X and Y are completely different things.
zmodem 6y ago
You may also want to try the GN build: https://github.com/llvm/llvm-project/blob/master/llvm/utils/gn/README.rst https://github.com/llvm/llvm-project/blob/master/llvm/utils/...

q3k 6y ago

Another fun thing you can do is to do another clone of the llvm-bazel repo, build that using Bazel again. Things should've cached correctly and the rebuild from a totally different checkout should be pretty much instant, as long as your ~/.cache/bazel is still there.

Joky 6y ago

I would double-check that it builds the exact same thing (there are lot of options when building LLVM, and CMake may auto-detect and enable support for things like Go bindings for example). Choice of compiler and compiler flags can make a difference as well.

siggi 6y ago

As another data point - I ran the same llvm bazel build using remote build execution on 128 cores. - clean, caching disabled @ 5.93m INFO: Elapsed time: 356.189s, Critical Path: 187.81s INFO: 4235 processes: 4235 remote. - clean, fully remote cached @ 27.33s INFO: Elapsed time: 27.330s, Critical Path: 15.93s INFO: 4235 processes: 4235 remote cache hit. Full disclosure - I co-founded BuildBuddy (YC W20) which provides bazel remote build execution and other open core bazel tooling

cromwellian 6y ago
Wow, this service looks amazing. I looked at setting up RBE on GCP and it looked like a pain to do it myself.

jlebar 6y ago

I would be extremely surprised if Bazel were 4m faster than cmake+ninja with the same build config. If this really is a correct result, I bet the ninja people would want to hear about it. (And, kudos to the Bazel folks!) But I think it's more likely measurement error. Maybe the build flags or host compiler aren't the same in the two configurations?

simfoo 6y ago
Most likely the CMake build is simply building more code than Bazel is. I am 100% sure that this is not comparing apples to apples
- gravypod 6y ago
  Yea, you can also see from the linked dashboards that it's writing way more stuff at the end of the build. I was watching the log just now and it looks like it's linking some things. I hope someone with more Bazel and LLVM knowledge can give this a go. I just wanted to toss something up as a "what would it be like if I just compiled this project" perspective. This is absolutely not a definitive "X is better than Y" since X and Y are completely different things.

zmodem 6y ago

You may also want to try the GN build: https://github.com/llvm/llvm-project/blob/master/llvm/utils/gn/README.rst https://github.com/llvm/llvm-project/blob/master/llvm/utils/...