5 ms·

People have complained about the build time of proc macros for ages in the community. This might be a misguided hack, but the response to this is bordering on a

by __jem 3y ago

People have complained about the build time of proc macros for ages in the community. This might be a misguided hack, but the response to this is bordering on a witch hunt, particularly when there is a glaring security hole (build.rs) that most people likely use without second thought every single day. I simply do not believe that most people commenting on this issue are auditing the builds of all their transitive dependencies.

mvolfik 3y ago
I'm quite ambivalent on this issue overall, but let me just point out: build.rs absolutely is a glaring security hole in the sense you say, but compared to that, this is much worse. You can verify the build.rs code that you download (at least in theory, and some people in banks or distro packages probably actually do), but binaries are orders of magnitude more difficult to inspect, and with the current Rust build system pretty much irreproducible.
- Ygg2 3y ago
  > build.rs absolutely is a glaring security hole in the sense you say, but compared to that, this is much worse. You can verify the build.rs code that you download In theory you can compile your own blob, but you'll need musl and whatnot to make a universal Linux build. Code for making the blob is there in the repo. build.rs is at best equal. It can access your locally available DB, and transmit your data.
  - WirelessGigabit 3y ago
    The problem here is that nothing in that build is pinned. It builds with nightly, but doesn't define which version / date. We also don't know how it's build. Ideally there is a Docker container out there that does just an import of source code and then builds. No apk install or apt install (you'd do that in a base published layer). Referenced with an SHA256. We then use this Docker container to pull in the source code AND its dependencies based on a Cargo.lock. Which... isn't there. So we don't know the exact dependencies that went in. (Even if there were a Cargo.lock, we need to make sure we actually respect it. I believe cargo install by default ignores the lock file and tries to get the latest version that matches).
mmastrac 3y ago
Yeah, there are already binaries in the crates.io ecosystem, and I'm certain that almost none of these people have audited a `build.rs` file or a proc macro implementation which effectively runs as you, completely unsandboxed. EDIT: I was wrong, this is not actually `watt` -- it may have been re-using code from the project. This one of those pile-ons where everyone gets excited about having a cause-du-jour to feel passionate about, while simultaneously ignoring issues that are far more pressing.
- monocasa 3y ago
  > almost none of these people have audited a `build.rs` file or a proc macro implementation which effectively runs as you, completely unsandboxed The biggest orgs tend to run all of their builds sandboxed, including what happens with build.rs. It's part of how they enforce dependency management day to day, but also helps protect against supply chain attacks. So not everyone does, but enough people do that you can rely on their complaints for the more well trodden parts of the ecosystem.
  - mmastrac 3y ago
    > The biggest orgs tend to run all of their builds sandboxed, including what happens with build.rs. It's part of how they enforce dependency management day to day, but also helps protect against supply chain attacks. Sandboxing doesn't completely prevent supply-chain attacks. You can avoid getting persistent malware on your CI machines, sure. Exfiltration of tokens and secrets during build? Maybe in a small handful of CI setups where admins have carefully split the source fetch, or limited CI network access to Nexus. Exfiltration of tokens and secrets and/or backdooring after the software has been deployed to production? No, build-time sandboxing doesn't help here at all. On all developer machines as well? No. Very few big orgs do this and only for mission-critical stuff. Some very important ones have docker-based sandboxed workflows or SSH-to-sandboxed-cluster workflows, or air-gapped laptops but that's very, very rare (I worked in air-gapped environment for a bit and it was a massive pain).
    - monocasa 3y ago
MarkMarine 3y ago
It’s probably true that most people commenting don’t audit the builds of transitive deps, but the original issue was a distro that couldn’t distribute precompiled binaries, I’m going to guess this has something to do with their license. I think having an exit path for those that want to compile from source is important, and I can’t understand the reluctance to provide that.
- __jem 3y ago
  Well, there is an exit path for those who want to compile from source. If you mean build from source for Cargo users, I believe there's issues with how feature flags interact with transitive dependencies that make this difficult. At least, there's comments on the issue that speak to this. Maybe someone more familiar with Cargo can chime in.
amluto 3y ago
At least Linux, OpenBSD, and (with more annoyance) Windows make it relatively straightforward to run things like build.rs in a sandbox. I wonder why Cargo doesn’t do this. There was also a project (that dtolnay was involved in, I believe!) a few years ago to compile proc macros to wasm.
- mmastrac 3y ago
  EDIT: I'm wrong, it's not watt -- just shared the same package name.
  - amluto 3y ago
    This looks like a better link: https://github.com/dtolnay/watt https://github.com/dtolnay/watt This would be a lot more compelling if it were integrated into rustc and cargo.
    - mmastrac 3y ago
      I suspect this move is partly to light a fire and force the Rust project to work on this sooner rather than later. We've needed this for years.
darthdeus 3y ago
build.rs is a source file that you can audit. A binary that has no reproducible build is not auditable even if anyone wanted to. A single person does not audit all of their dependency tree, but many people do read the source code of some if not many of their dependencies, and as a community we can figure out when something is fishy, like in this case. But when there are binaries involved, nobody can do anything. This isn't the same as installing a signed binary from a linux package manager that has a checksum and a verified build system. It's a random binary blob someone made in a way that nobody else can check, and it's just "trust me bro there's nothing bad in it".
- nullc 3y ago
  > and it's just "trust me bro there's nothing bad in it". The developer should be very concerned about what happens if his system(s) are compromised and the attacker slips a backdoor into these binaries-- it will be difficult to impossible to convince people that the developer himself didn't do it intentionally. Their opacity and immediacy make them much more interesting targets for attack than the source itself (and its associated build scripts). Saving a few seconds on the first compile on the some other developers computer hardly seems worth that risk. And at the meta level, we should probably worry about the security practices of someone who isn't worrying about that risk-- what else aren't they worrying about?

by __jem 3y ago

mvolfik 3y ago

I'm quite ambivalent on this issue overall, but let me just point out: build.rs absolutely is a glaring security hole in the sense you say, but compared to that, this is much worse. You can verify the build.rs code that you download (at least in theory, and some people in banks or distro packages probably actually do), but binaries are orders of magnitude more difficult to inspect, and with the current Rust build system pretty much irreproducible.

Ygg2 3y ago
> build.rs absolutely is a glaring security hole in the sense you say, but compared to that, this is much worse. You can verify the build.rs code that you download In theory you can compile your own blob, but you'll need musl and whatnot to make a universal Linux build. Code for making the blob is there in the repo. build.rs is at best equal. It can access your locally available DB, and transmit your data.
- WirelessGigabit 3y ago
  The problem here is that nothing in that build is pinned. It builds with nightly, but doesn't define which version / date. We also don't know how it's build. Ideally there is a Docker container out there that does just an import of source code and then builds. No apk install or apt install (you'd do that in a base published layer). Referenced with an SHA256. We then use this Docker container to pull in the source code AND its dependencies based on a Cargo.lock. Which... isn't there. So we don't know the exact dependencies that went in. (Even if there were a Cargo.lock, we need to make sure we actually respect it. I believe cargo install by default ignores the lock file and tries to get the latest version that matches).

mmastrac 3y ago

Yeah, there are already binaries in the crates.io ecosystem, and I'm certain that almost none of these people have audited a `build.rs` file or a proc macro implementation which effectively runs as you, completely unsandboxed. EDIT: I was wrong, this is not actually `watt` -- it may have been re-using code from the project. This one of those pile-ons where everyone gets excited about having a cause-du-jour to feel passionate about, while simultaneously ignoring issues that are far more pressing.

monocasa 3y ago
> almost none of these people have audited a `build.rs` file or a proc macro implementation which effectively runs as you, completely unsandboxed The biggest orgs tend to run all of their builds sandboxed, including what happens with build.rs. It's part of how they enforce dependency management day to day, but also helps protect against supply chain attacks. So not everyone does, but enough people do that you can rely on their complaints for the more well trodden parts of the ecosystem.
- mmastrac 3y ago
  > The biggest orgs tend to run all of their builds sandboxed, including what happens with build.rs. It's part of how they enforce dependency management day to day, but also helps protect against supply chain attacks. Sandboxing doesn't completely prevent supply-chain attacks. You can avoid getting persistent malware on your CI machines, sure. Exfiltration of tokens and secrets during build? Maybe in a small handful of CI setups where admins have carefully split the source fetch, or limited CI network access to Nexus. Exfiltration of tokens and secrets and/or backdooring after the software has been deployed to production? No, build-time sandboxing doesn't help here at all. On all developer machines as well? No. Very few big orgs do this and only for mission-critical stuff. Some very important ones have docker-based sandboxed workflows or SSH-to-sandboxed-cluster workflows, or air-gapped laptops but that's very, very rare (I worked in air-gapped environment for a bit and it was a massive pain).
  - monocasa 3y ago

MarkMarine 3y ago

It’s probably true that most people commenting don’t audit the builds of transitive deps, but the original issue was a distro that couldn’t distribute precompiled binaries, I’m going to guess this has something to do with their license. I think having an exit path for those that want to compile from source is important, and I can’t understand the reluctance to provide that.

__jem 3y ago
Well, there is an exit path for those who want to compile from source. If you mean build from source for Cargo users, I believe there's issues with how feature flags interact with transitive dependencies that make this difficult. At least, there's comments on the issue that speak to this. Maybe someone more familiar with Cargo can chime in.

amluto 3y ago

At least Linux, OpenBSD, and (with more annoyance) Windows make it relatively straightforward to run things like build.rs in a sandbox. I wonder why Cargo doesn’t do this. There was also a project (that dtolnay was involved in, I believe!) a few years ago to compile proc macros to wasm.

mmastrac 3y ago
EDIT: I'm wrong, it's not watt -- just shared the same package name.
- amluto 3y ago
  This looks like a better link: https://github.com/dtolnay/watt https://github.com/dtolnay/watt This would be a lot more compelling if it were integrated into rustc and cargo.
  - mmastrac 3y ago
    I suspect this move is partly to light a fire and force the Rust project to work on this sooner rather than later. We've needed this for years.

darthdeus 3y ago

build.rs is a source file that you can audit. A binary that has no reproducible build is not auditable even if anyone wanted to. A single person does not audit all of their dependency tree, but many people do read the source code of some if not many of their dependencies, and as a community we can figure out when something is fishy, like in this case. But when there are binaries involved, nobody can do anything. This isn't the same as installing a signed binary from a linux package manager that has a checksum and a verified build system. It's a random binary blob someone made in a way that nobody else can check, and it's just "trust me bro there's nothing bad in it".

nullc 3y ago
> and it's just "trust me bro there's nothing bad in it". The developer should be very concerned about what happens if his system(s) are compromised and the attacker slips a backdoor into these binaries-- it will be difficult to impossible to convince people that the developer himself didn't do it intentionally. Their opacity and immediacy make them much more interesting targets for attack than the source itself (and its associated build scripts). Saving a few seconds on the first compile on the some other developers computer hardly seems worth that risk. And at the meta level, we should probably worry about the security practices of someone who isn't worrying about that risk-- what else aren't they worrying about?