3 ms·

Uh, Either my understanding of Bazel is wrong, or everything you wrote is wrong. Bazel absolutely prevents network access and filesystem access (reads) from bu

by dijit 2y ago

Uh, Either my understanding of Bazel is wrong, or everything you wrote is wrong. Bazel absolutely prevents network access and filesystem access (reads) from builds. (only permitting explicit network includes from the WORKSPACE file, and access to files explicitly depended on in the BUILD files). Maybe you can write some “rules_” for languages that violate this, but it is designed purposely to be hermetic and bit-perfect reproducible. EDIT: From the FAQ[0]: > Will Bazel make my builds reproducible automatically? > For Java and C++ binaries, yes, assuming you do not change the toolchain. The issues with Docker's style of "reproducible" (meaning.. consistent environment; are also outlined in the same FAQ[1] > Doesn’t Docker solve the reproducibility problems? > Docker does not address reproducibility with regard to changes in the source code. Running Make with an imperfectly written Makefile inside a Docker container can still yield unpredictable results. [0]: https://bazel.build/about/faq#will_bazel_make_my_builds_reproducible_automatically https://bazel.build/about/faq#will_bazel_make_my_builds_repr... [1]: https://bazel.build/about/faq#doesn’t_docker_solve_the_reproducibility_problems https://bazel.build/about/faq#doesn’t_docker_solve_the_repro...

valcron1000 2y ago
I'm not familiar with Bazel at all so this might be obvious, but does Bazel check that the files listed in the BUILD file are the "right ones" (ex. through a checksum), and if so, is this always enforced (that is, this behavior cannot be disabled)?
- dijit 2y ago
  The contents of files are basically hashed, if the contents don't change of the file listed for a target then no change will happen, even if you modify metadata of the file (like last modified time by `touch` or so on.) Bazel is really sophisticated and I'd be lying if I said I understood it well, but I have spent time looking at it.
keithwinstein 2y ago
I think you're both right in a sense. Bazel doesn't (in general) prevent filesystem access, e.g. to library headers in /usr/include. If those headers change (maybe because a Debian package got upgraded or whatever), Bazel won't know it has to invalidate the build cache. I think the FAQ is still technically correct because upgrading the Debian package for a random library dependency counts as "chang[ing] the toolchain" in this context. But I don't think you'd call it hermetic by default. Check out the previous discussion at https://news.ycombinator.com/item?id=23184843 https://news.ycombinator.com/item?id=23184843 and below: > Under the hood there's a default auto-configured toolchain that finds whatever is installed locally in the system. Since it has no way of knowing what files an arbitrary "cc" might depend on, you lose hermeticity by using it.
amarshall 2y ago
AFAIK Bazel does not use the sandbox by default. Last time I experimented with it, the sandbox had some problematic holes, but I don’t remember exactly what, and it’s been a few years. The very doc you link hints at that, while also giving many caveats where the build will become non-reproducible. So it boils down to “yes, but only if you configure it correctly and do things right”.
- jchw 2y ago
  Yeah, I think you are right: by default, there is no OS-level sandboxing going on. According to documentation, the default spawn strategy is `local`[1], whereas it would need to be `sandboxed` for sandboxing to take effect. Meanwhile, if you want to forcibly block network access for a specific action, you can pass `block-network` as an execution requirement[2]. You can also explicitly block network access with flags, using --nosandbox_default_allow_network[3]. Interestingly though, an action can also `require-network` to bypass this, and I don't think there's any way to account for that. Maybe more importantly, Bazel lacks the concept of a fixed-output action, so when an impure action needs `require-network` the potentially-impure results could impact downstream dependents of actions. I was still ultimately incorrect to say that Bazel's sandbox can't sandbox the network. The actual reality is that it can. If you do enable the sandbox, while it's not exactly pervasive through the entire ecosystem, it does look like a fair number of projects at least set the `block-network` tag--about 700 as of writing this[4]. I think the broader point I was making (that Nix adheres to a stronger standard of "hermetic" than Bazel) is ultimately true, but I did miss on a bit of nuance initially. [1]: https://bazel.build/docs/user-manual#spawn-strategy https://bazel.build/docs/user-manual#spawn-strategy [2]: https://bazel.build/reference/be/common-definitions#common.tags https://bazel.build/reference/be/common-definitions#common.t... [3]: https://bazel.build/reference/command-line-reference#flag--sandbox_default_allow_network https://bazel.build/reference/command-line-reference#flag--s... [4]: https://github.com/search?q=language%3Abzl+%22block-network%22&type=code https://github.com/search?q=language%3Abzl+%22block-network%...
jchw 2y ago
I believe your understanding of Bazel is wrong. I don't see any documentation that suggests the Bazel sandbox prevents the toolchain from accessing the network. https://bazel.build/docs/sandboxing https://bazel.build/docs/sandboxing (Actually, it can: that documentation suggests it's optionally supported, at least on the Linux sandbox. That said, it's optional. There's definitely actions that use the network on purpose and can't participate in this.) This may seem pointless, because in many situations this would only matter in somewhat convoluted cases. In C++ the toolchain probably won't connect to the network. This isn't the case for e.g. Rust, where proc macros can access the network. (In practical terms, I believe the sqlx crate does this, connecting to a local Postgres instance to do type inference.) Likewise, you could do an absolute file inclusion, but that would be very much on purpose and not an accident. So it's reasonable to say that you get a level of reproducibility when you use Bazel for C++ builds... Kind of. It's not bit-for-bit because it uses the system toolchain, which is just an arbitrary choice. On Darwin it's even more annoying: with XCode installed via Mac App Store, the XCode version can change transparently under Bazel in the background, entirely breaking the hermeticity, and require you to purge the Bazel cache (because the dependency graph will be wrong and break the build. Usually.) Nix is different. The toolchain is built by Nix and undergoes the same sandboxed build process with sandboxing and cryptographically verified inputs. Bazel does not do that.
- paulddraper 2y ago
  It does. There are mechanisms for opting out/breaking that, just as with Nix or any other system. > macOS What does nix do on these systems?
  - jchw 2y ago
    Opt-out would be one thing, but it's actually opt-in for network isolation, and a project can disable all sandboxing with just a .bazelrc. Nix does have ways to opt-out of sandboxing, but you can't do it inside a Nix expression: if you ran Nix with sandbox = true, anything being able to escape or bypass the sandbox restrictions would be a security vulnerability and assigned a CVE. Disabling the sandbox can only be done by a trusted user, and it's entirely out-of-band from the builder. For Bazel, the sandbox is mostly just there to prevent accidental impurities, but it's not water tight by any means. Ultimately, I still think that Nix provides a greater degree of isolation and reproducibility than Bazel overall, and especially out of the box, but I was definitely incorrect when I said that Bazel's sandbox doesn't/can't block the network. I did dive a little deeper into the nuances in another comment.[1] > What does nix do on these systems? On macOS, Nix is not exactly as solid as it is on Linux. It uses sandbox-exec for sandboxing, which achieves most of what the Nix sandbox does on Linux, except it disallows all networking rather than just isolated networking. (Some derivations need local network access, so you can opt-in to having local network access per-derivation. This still doesn't give Internet access, though: internet access still requires a fixed-output derivation.) There's definitely some room for improvement there but it will be hard to do too much better since xnu doesn't have anything similar to network namespaces afaik. As for the toolchain, I'm not sure how the Nix bootstrap works on macOS. It seems like a lot of effort went in to making it work and it can function without XCode installed. (Can't find a source for this, but I was using it on a Mac Mini that I'm pretty sure didn't have XCode installed. So it clearly has its own hermetic toolchain setup just like Linux.) [1]: https://news.ycombinator.com/item?id=43032285 https://news.ycombinator.com/item?id=43032285
dietr1ch 2y ago
I remember that a system nagged about non-reproducible outputs, Blaze (not Bazel, but the internal thing) allowed looking into the outside-world through bad Starlark rules and compile time tricks could get you questioning why there's so much evil in the world. Maybe Bazel forbid these things right away and Googlers actually talking about Blaze will be inadvertently lying thinking they are similar enough.

by dijit 2y ago

valcron1000 2y ago

I'm not familiar with Bazel at all so this might be obvious, but does Bazel check that the files listed in the BUILD file are the "right ones" (ex. through a checksum), and if so, is this always enforced (that is, this behavior cannot be disabled)?

dijit 2y ago
The contents of files are basically hashed, if the contents don't change of the file listed for a target then no change will happen, even if you modify metadata of the file (like last modified time by `touch` or so on.) Bazel is really sophisticated and I'd be lying if I said I understood it well, but I have spent time looking at it.

keithwinstein 2y ago

I think you're both right in a sense. Bazel doesn't (in general) prevent filesystem access, e.g. to library headers in /usr/include. If those headers change (maybe because a Debian package got upgraded or whatever), Bazel won't know it has to invalidate the build cache. I think the FAQ is still technically correct because upgrading the Debian package for a random library dependency counts as "chang[ing] the toolchain" in this context. But I don't think you'd call it hermetic by default. Check out the previous discussion at https://news.ycombinator.com/item?id=23184843 https://news.ycombinator.com/item?id=23184843 and below: > Under the hood there's a default auto-configured toolchain that finds whatever is installed locally in the system. Since it has no way of knowing what files an arbitrary "cc" might depend on, you lose hermeticity by using it.

amarshall 2y ago

AFAIK Bazel does not use the sandbox by default. Last time I experimented with it, the sandbox had some problematic holes, but I don’t remember exactly what, and it’s been a few years. The very doc you link hints at that, while also giving many caveats where the build will become non-reproducible. So it boils down to “yes, but only if you configure it correctly and do things right”.

jchw 2y ago
Yeah, I think you are right: by default, there is no OS-level sandboxing going on. According to documentation, the default spawn strategy is `local`[1], whereas it would need to be `sandboxed` for sandboxing to take effect. Meanwhile, if you want to forcibly block network access for a specific action, you can pass `block-network` as an execution requirement[2]. You can also explicitly block network access with flags, using --nosandbox_default_allow_network[3]. Interestingly though, an action can also `require-network` to bypass this, and I don't think there's any way to account for that. Maybe more importantly, Bazel lacks the concept of a fixed-output action, so when an impure action needs `require-network` the potentially-impure results could impact downstream dependents of actions. I was still ultimately incorrect to say that Bazel's sandbox can't sandbox the network. The actual reality is that it can. If you do enable the sandbox, while it's not exactly pervasive through the entire ecosystem, it does look like a fair number of projects at least set the `block-network` tag--about 700 as of writing this[4]. I think the broader point I was making (that Nix adheres to a stronger standard of "hermetic" than Bazel) is ultimately true, but I did miss on a bit of nuance initially. [1]: https://bazel.build/docs/user-manual#spawn-strategy https://bazel.build/docs/user-manual#spawn-strategy [2]: https://bazel.build/reference/be/common-definitions#common.tags https://bazel.build/reference/be/common-definitions#common.t... [3]: https://bazel.build/reference/command-line-reference#flag--sandbox_default_allow_network https://bazel.build/reference/command-line-reference#flag--s... [4]: https://github.com/search?q=language%3Abzl+%22block-network%22&type=code https://github.com/search?q=language%3Abzl+%22block-network%...

jchw 2y ago

I believe your understanding of Bazel is wrong. I don't see any documentation that suggests the Bazel sandbox prevents the toolchain from accessing the network. https://bazel.build/docs/sandboxing https://bazel.build/docs/sandboxing (Actually, it can: that documentation suggests it's optionally supported, at least on the Linux sandbox. That said, it's optional. There's definitely actions that use the network on purpose and can't participate in this.) This may seem pointless, because in many situations this would only matter in somewhat convoluted cases. In C++ the toolchain probably won't connect to the network. This isn't the case for e.g. Rust, where proc macros can access the network. (In practical terms, I believe the sqlx crate does this, connecting to a local Postgres instance to do type inference.) Likewise, you could do an absolute file inclusion, but that would be very much on purpose and not an accident. So it's reasonable to say that you get a level of reproducibility when you use Bazel for C++ builds... Kind of. It's not bit-for-bit because it uses the system toolchain, which is just an arbitrary choice. On Darwin it's even more annoying: with XCode installed via Mac App Store, the XCode version can change transparently under Bazel in the background, entirely breaking the hermeticity, and require you to purge the Bazel cache (because the dependency graph will be wrong and break the build. Usually.) Nix is different. The toolchain is built by Nix and undergoes the same sandboxed build process with sandboxing and cryptographically verified inputs. Bazel does not do that.

paulddraper 2y ago
It does. There are mechanisms for opting out/breaking that, just as with Nix or any other system. > macOS What does nix do on these systems?
- jchw 2y ago
  Opt-out would be one thing, but it's actually opt-in for network isolation, and a project can disable all sandboxing with just a .bazelrc. Nix does have ways to opt-out of sandboxing, but you can't do it inside a Nix expression: if you ran Nix with sandbox = true, anything being able to escape or bypass the sandbox restrictions would be a security vulnerability and assigned a CVE. Disabling the sandbox can only be done by a trusted user, and it's entirely out-of-band from the builder. For Bazel, the sandbox is mostly just there to prevent accidental impurities, but it's not water tight by any means. Ultimately, I still think that Nix provides a greater degree of isolation and reproducibility than Bazel overall, and especially out of the box, but I was definitely incorrect when I said that Bazel's sandbox doesn't/can't block the network. I did dive a little deeper into the nuances in another comment.[1] > What does nix do on these systems? On macOS, Nix is not exactly as solid as it is on Linux. It uses sandbox-exec for sandboxing, which achieves most of what the Nix sandbox does on Linux, except it disallows all networking rather than just isolated networking. (Some derivations need local network access, so you can opt-in to having local network access per-derivation. This still doesn't give Internet access, though: internet access still requires a fixed-output derivation.) There's definitely some room for improvement there but it will be hard to do too much better since xnu doesn't have anything similar to network namespaces afaik. As for the toolchain, I'm not sure how the Nix bootstrap works on macOS. It seems like a lot of effort went in to making it work and it can function without XCode installed. (Can't find a source for this, but I was using it on a Mac Mini that I'm pretty sure didn't have XCode installed. So it clearly has its own hermetic toolchain setup just like Linux.) [1]: https://news.ycombinator.com/item?id=43032285 https://news.ycombinator.com/item?id=43032285

dietr1ch 2y ago

I remember that a system nagged about non-reproducible outputs, Blaze (not Bazel, but the internal thing) allowed looking into the outside-world through bad Starlark rules and compile time tricks could get you questioning why there's so much evil in the world. Maybe Bazel forbid these things right away and Googlers actually talking about Blaze will be inadvertently lying thinking they are similar enough.