4 ms·

Bazel (and Bazel-derivatives/alikes) is not unique in fixing all of these problems. For example, there is build2, which is arguably a lighter-weight and closer

by boris 7y ago

Bazel (and Bazel-derivatives/alikes) is not unique in fixing all of these problems. For example, there is build2, which is arguably a lighter-weight and closer to make (in spirit, not in syntax) solution (no Java dependency, etc). Here is an intro, if anyone is interested: https://build2.org/build2/doc/build2-build-system-manual.xhtml#intro https://build2.org/build2/doc/build2-build-system-manual.xht...

skybrian 7y ago
I did a quick skim and it doesn't seem to sandbox the build steps like Bazel? Maybe I missed something.
- boris 7y ago
  It doesn't do Bazel's style sandboxing where your entire compiler toolchain is part of your build system. It does what we call high-fidelity builds: it tracks changes not only to inputs but also to compile options, compiler itself (checksum), environment, etc., and if any of these auxiliary dependencies change, it triggers a rebuild. There are advantages and disadvantages to both approaches with build2's being lighter weight.
  - rienbdj 7y ago
    I don't see how build2 can be in the same spirit as Bazel when it misses the main advantage of Bazel
    - boris 7y ago
      I never said build2 is "in the same spirit" as Bazel (whatever that means). I said it fixes the same problems as OP listed.
      - weberc2 7y ago
        > I said it fixes the same problems as OP listed. The OP listed noted that hermeticity is required for fast builds, and that humans can't be relied upon to provide hermeticity as a property. How does build2 guarantee hermeticity or otherwise support fast builds without a clean room solution a la bazel?
        boris 7y ago
        The OP was vague on why exactly hermetic builds are required to achieve fast builds. The only concrete thing they mentioned is caching which doesn't require hermetic builds (in the strict sense, as in preventing any outside changes) provided you can detect changes accurately. To give a specific example, Bazel may prevent you from accidentally using a different version of the compiler while build2 will detect that you are attempting to use a different version.
        weberc2 7y ago
        Caching depends on a correct model of the dependency tree. If a dependency isn't included in the model, then a non-hermetic tool will silently succeed, incorrectly while a hermetic tool will do the correct thing and fail the build. Keeping the model accurate is too hard for humans, so in practice, non-hermetic build tools can't cache correctly.
klodolph 7y ago
It would be interesting to see how build2 achieves these things (hermeticity, cross-building, shared caches). I can’t see anything in the linked documentation that mentions these problems at all.
- boris 7y ago
  It currently doesn't do fully hermetic configurations (as in, where it is impossible for any inputs to come outside of the project). Instead build2 does high-fidelity builds where it makes sure that if any of the inputs change (including options, environment, compiler itself, etc), the target gets rebuilt. See my reply to a sibling comment for some details. We do plan to add support for hermetic builds though it won't be exactly like Bazel's -- the idea is to prevent and, where not possible, detect external changes and fail rather than rebuilding. Regarding caching (and distributed compilation), this is currently on the TODO list though a lot of the infrastructure is already there. For example, the same change detection that is used for high-fidelity builds will be used to decide if what's in the cache is usable in any particular build. While I agree we should mention these points in the documentation (things are still WIP on that front), I don't think cross-compilation deserves mentioning: for any modern build system it should just work. In build2 we simply implement things in the "cross-compile first" way, with native compilation being a special case (host == target).
  - evmar 7y ago
    Does build2 handle the case where code has #include "foo.h" and then, after a successful build, you create a new file named "foo.h" in a directory earlier in the search path than the foo.h that was used in the previous compile?
    - boris 7y ago
      It does but not in a way you probably expect it to be handled: i.e., with some filesystem mechanisms to make the update command only see what has been declared as target's dependencies -- I must admit I don't know Bazel does this in a cross-platform manner (Windows, Mac OS); copying seems way too heavy-handed. In any case, in build2 this is "handled" by not including headers as "foo.h" but as <libfoo/foo.h>, that is, with the project prefix. You can read more on this here: https://build2.org/build2-toolchain/doc/build2-toolchain-intro.xhtml#proj-struct https://build2.org/build2-toolchain/doc/build2-toolchain-int... And the proper fix will hopefully come with C++20 modules.

by boris 7y ago

skybrian 7y ago

I did a quick skim and it doesn't seem to sandbox the build steps like Bazel? Maybe I missed something.

boris 7y ago
It doesn't do Bazel's style sandboxing where your entire compiler toolchain is part of your build system. It does what we call high-fidelity builds: it tracks changes not only to inputs but also to compile options, compiler itself (checksum), environment, etc., and if any of these auxiliary dependencies change, it triggers a rebuild. There are advantages and disadvantages to both approaches with build2's being lighter weight.
- rienbdj 7y ago
  I don't see how build2 can be in the same spirit as Bazel when it misses the main advantage of Bazel
  - boris 7y ago
    I never said build2 is "in the same spirit" as Bazel (whatever that means). I said it fixes the same problems as OP listed.
    - weberc2 7y ago
      > I said it fixes the same problems as OP listed. The OP listed noted that hermeticity is required for fast builds, and that humans can't be relied upon to provide hermeticity as a property. How does build2 guarantee hermeticity or otherwise support fast builds without a clean room solution a la bazel?
      - boris 7y ago
        The OP was vague on why exactly hermetic builds are required to achieve fast builds. The only concrete thing they mentioned is caching which doesn't require hermetic builds (in the strict sense, as in preventing any outside changes) provided you can detect changes accurately. To give a specific example, Bazel may prevent you from accidentally using a different version of the compiler while build2 will detect that you are attempting to use a different version.
        weberc2 7y ago
        Caching depends on a correct model of the dependency tree. If a dependency isn't included in the model, then a non-hermetic tool will silently succeed, incorrectly while a hermetic tool will do the correct thing and fail the build. Keeping the model accurate is too hard for humans, so in practice, non-hermetic build tools can't cache correctly.

klodolph 7y ago

It would be interesting to see how build2 achieves these things (hermeticity, cross-building, shared caches). I can’t see anything in the linked documentation that mentions these problems at all.

boris 7y ago
It currently doesn't do fully hermetic configurations (as in, where it is impossible for any inputs to come outside of the project). Instead build2 does high-fidelity builds where it makes sure that if any of the inputs change (including options, environment, compiler itself, etc), the target gets rebuilt. See my reply to a sibling comment for some details. We do plan to add support for hermetic builds though it won't be exactly like Bazel's -- the idea is to prevent and, where not possible, detect external changes and fail rather than rebuilding. Regarding caching (and distributed compilation), this is currently on the TODO list though a lot of the infrastructure is already there. For example, the same change detection that is used for high-fidelity builds will be used to decide if what's in the cache is usable in any particular build. While I agree we should mention these points in the documentation (things are still WIP on that front), I don't think cross-compilation deserves mentioning: for any modern build system it should just work. In build2 we simply implement things in the "cross-compile first" way, with native compilation being a special case (host == target).
- evmar 7y ago
  Does build2 handle the case where code has #include "foo.h" and then, after a successful build, you create a new file named "foo.h" in a directory earlier in the search path than the foo.h that was used in the previous compile?
  - boris 7y ago
    It does but not in a way you probably expect it to be handled: i.e., with some filesystem mechanisms to make the update command only see what has been declared as target's dependencies -- I must admit I don't know Bazel does this in a cross-platform manner (Windows, Mac OS); copying seems way too heavy-handed. In any case, in build2 this is "handled" by not including headers as "foo.h" but as <libfoo/foo.h>, that is, with the project prefix. You can read more on this here: https://build2.org/build2-toolchain/doc/build2-toolchain-intro.xhtml#proj-struct https://build2.org/build2-toolchain/doc/build2-toolchain-int... And the proper fix will hopefully come with C++20 modules.