6 ms·

I believe your understanding of Bazel is wrong. I don't see any documentation that suggests the Bazel sandbox prevents the toolchain from accessing the network.

by jchw 2y ago

I believe your understanding of Bazel is wrong. I don't see any documentation that suggests the Bazel sandbox prevents the toolchain from accessing the network. https://bazel.build/docs/sandboxing https://bazel.build/docs/sandboxing (Actually, it can: that documentation suggests it's optionally supported, at least on the Linux sandbox. That said, it's optional. There's definitely actions that use the network on purpose and can't participate in this.) This may seem pointless, because in many situations this would only matter in somewhat convoluted cases. In C++ the toolchain probably won't connect to the network. This isn't the case for e.g. Rust, where proc macros can access the network. (In practical terms, I believe the sqlx crate does this, connecting to a local Postgres instance to do type inference.) Likewise, you could do an absolute file inclusion, but that would be very much on purpose and not an accident. So it's reasonable to say that you get a level of reproducibility when you use Bazel for C++ builds... Kind of. It's not bit-for-bit because it uses the system toolchain, which is just an arbitrary choice. On Darwin it's even more annoying: with XCode installed via Mac App Store, the XCode version can change transparently under Bazel in the background, entirely breaking the hermeticity, and require you to purge the Bazel cache (because the dependency graph will be wrong and break the build. Usually.) Nix is different. The toolchain is built by Nix and undergoes the same sandboxed build process with sandboxing and cryptographically verified inputs. Bazel does not do that.

paulddraper 2y ago
It does. There are mechanisms for opting out/breaking that, just as with Nix or any other system. > macOS What does nix do on these systems?
- jchw 2y ago
  Opt-out would be one thing, but it's actually opt-in for network isolation, and a project can disable all sandboxing with just a .bazelrc. Nix does have ways to opt-out of sandboxing, but you can't do it inside a Nix expression: if you ran Nix with sandbox = true, anything being able to escape or bypass the sandbox restrictions would be a security vulnerability and assigned a CVE. Disabling the sandbox can only be done by a trusted user, and it's entirely out-of-band from the builder. For Bazel, the sandbox is mostly just there to prevent accidental impurities, but it's not water tight by any means. Ultimately, I still think that Nix provides a greater degree of isolation and reproducibility than Bazel overall, and especially out of the box, but I was definitely incorrect when I said that Bazel's sandbox doesn't/can't block the network. I did dive a little deeper into the nuances in another comment.[1] > What does nix do on these systems? On macOS, Nix is not exactly as solid as it is on Linux. It uses sandbox-exec for sandboxing, which achieves most of what the Nix sandbox does on Linux, except it disallows all networking rather than just isolated networking. (Some derivations need local network access, so you can opt-in to having local network access per-derivation. This still doesn't give Internet access, though: internet access still requires a fixed-output derivation.) There's definitely some room for improvement there but it will be hard to do too much better since xnu doesn't have anything similar to network namespaces afaik. As for the toolchain, I'm not sure how the Nix bootstrap works on macOS. It seems like a lot of effort went in to making it work and it can function without XCode installed. (Can't find a source for this, but I was using it on a Mac Mini that I'm pretty sure didn't have XCode installed. So it clearly has its own hermetic toolchain setup just like Linux.) [1]: https://news.ycombinator.com/item?id=43032285 https://news.ycombinator.com/item?id=43032285
  - paulddraper 2y ago
    > it's actually opt-in for network isolation Bazel enables sandboxing by default, including network isolation. [1] [2] The exception would be in environments that don't support it (Windows, unprivileged Docker container, etc.) [1] https://bazel.build/docs/user-manual#spawn-strategy https://bazel.build/docs/user-manual#spawn-strategy [2] https://stackoverflow.com/a/63635141 https://stackoverflow.com/a/63635141
    - jchw 2y ago
      My assertion that network isolation is opt-in is based on the fact that the --sandbox_default_allow_network defaults to true[1]. That suggests actions will have networking unless they are dispatched with `block-network`[2]. (It's hard to figure out exactly what's going on based on the documentation and some crawling around, but I wouldn't be surprised if specifically tests defaulted to blocking the network.) [1]: https://bazel.build/reference/command-line-reference#flag--sandbox_default_allow_network https://bazel.build/reference/command-line-reference#flag--s... [2]: https://bazel.build/reference/be/common-definitions#common.tags https://bazel.build/reference/be/common-definitions#common.t...

by jchw 2y ago

paulddraper 2y ago

It does. There are mechanisms for opting out/breaking that, just as with Nix or any other system. > macOS What does nix do on these systems?

jchw 2y ago
Opt-out would be one thing, but it's actually opt-in for network isolation, and a project can disable all sandboxing with just a .bazelrc. Nix does have ways to opt-out of sandboxing, but you can't do it inside a Nix expression: if you ran Nix with sandbox = true, anything being able to escape or bypass the sandbox restrictions would be a security vulnerability and assigned a CVE. Disabling the sandbox can only be done by a trusted user, and it's entirely out-of-band from the builder. For Bazel, the sandbox is mostly just there to prevent accidental impurities, but it's not water tight by any means. Ultimately, I still think that Nix provides a greater degree of isolation and reproducibility than Bazel overall, and especially out of the box, but I was definitely incorrect when I said that Bazel's sandbox doesn't/can't block the network. I did dive a little deeper into the nuances in another comment.[1] > What does nix do on these systems? On macOS, Nix is not exactly as solid as it is on Linux. It uses sandbox-exec for sandboxing, which achieves most of what the Nix sandbox does on Linux, except it disallows all networking rather than just isolated networking. (Some derivations need local network access, so you can opt-in to having local network access per-derivation. This still doesn't give Internet access, though: internet access still requires a fixed-output derivation.) There's definitely some room for improvement there but it will be hard to do too much better since xnu doesn't have anything similar to network namespaces afaik. As for the toolchain, I'm not sure how the Nix bootstrap works on macOS. It seems like a lot of effort went in to making it work and it can function without XCode installed. (Can't find a source for this, but I was using it on a Mac Mini that I'm pretty sure didn't have XCode installed. So it clearly has its own hermetic toolchain setup just like Linux.) [1]: https://news.ycombinator.com/item?id=43032285 https://news.ycombinator.com/item?id=43032285
- paulddraper 2y ago
  > it's actually opt-in for network isolation Bazel enables sandboxing by default, including network isolation. [1] [2] The exception would be in environments that don't support it (Windows, unprivileged Docker container, etc.) [1] https://bazel.build/docs/user-manual#spawn-strategy https://bazel.build/docs/user-manual#spawn-strategy [2] https://stackoverflow.com/a/63635141 https://stackoverflow.com/a/63635141
  - jchw 2y ago
    My assertion that network isolation is opt-in is based on the fact that the --sandbox_default_allow_network defaults to true[1]. That suggests actions will have networking unless they are dispatched with `block-network`[2]. (It's hard to figure out exactly what's going on based on the documentation and some crawling around, but I wouldn't be surprised if specifically tests defaulted to blocking the network.) [1]: https://bazel.build/reference/command-line-reference#flag--sandbox_default_allow_network https://bazel.build/reference/command-line-reference#flag--s... [2]: https://bazel.build/reference/be/common-definitions#common.tags https://bazel.build/reference/be/common-definitions#common.t...