4 ms·

Java's Maven is an interesting case study, as it tried to be this:. A standard project layout, a standard dependency mechanism, pom.xml as standard metadata fil

Java's Maven is an interesting case study, as it tried to be this:. A standard project layout, a standard dependency mechanism, pom.xml as standard metadata file, and a standard workflow with standard target(clean/compile/test/deploy). Plugins for what's left. There might have been a time where it worked, but people started to use plugins for all kinds of reasons, quite good ones in most cases. Findbugs, code coverage, source code generation,... Today, a maven project without plugins is rare. Maven brought us 95%, but there is a long tail left to cover.

kibwen 3y ago
> Findbugs, code coverage, source code generation,... For the purpose of this conversation we mostly just care about the use case of someone grabbing the code and wanting to use it in their own project. For this use case, dev tools like findbugs and code coverage can be ignored, so it would suffice to have a version of the build system with plugins completely disabled. Code generation is the thornier one, and we can at least be more principled about it than "run some arbitrary code", and at least it should be trivial to say "this codegen process gets absolutely no I/O access whatsoever; you're a dumb text pipeline". But at the end of the day, we have to Just Say No to things like this. Even if it makes the codebase grodier to check in generated code, if I can't inspect and audit the source code, that's a problem, and arbitrary build-time codegen prevents that. Some trade-offs are worth making.
- hyperman1 3y ago
  The xz debacle happened partiallybecause the generated autoconf code was provided. Checking in generated code is not that much better. It's a bit more visible, but not much people will spend their limited time to validate it, as it's not worth it for generated code. xz also had checked in inscrutable test files, and nobody could know it was encrypted malware. I'm not a fan of generated code. It tends to cause misery, being in a no mans land between code and not-code. But it is usefull sometimes, e.g rust generating an API from the opengl XML specs. Sandboxing seems the least worst option, but it will still be uninspected half code that one day ends up in production.
  - kibwen 3y ago
    > The xz debacle happened partiallybecause the generated autoconf code was provided. The code was only provided in a roundabout way that was deliberately done to evade manual inspection, so that's not a failure of checking in generated code, that's a failure of actually building a binary from the artifacts that we expect it to be built from. Suffice to say, cutting out the Turing-complete crap from our build systems is only one of many things that we need to fix.
ptx 3y ago
Maybe now that we have things like GitHub Actions, Bitbucket Pipelines, etc., which can run steps in separate containers, maybe most of those things could be moved from the Maven build step to a different pipeline step? I'm not sure how well isolated the containers are (probably not very – I think GitHub gives access to the Docker socket) and you'd have to make sure they don't share secret tokens etc., but at least it might make things simpler to audit, and isolation could be improved in the future.
wongarsu 3y ago
Most of these could still be covered with IO limited to the project files though. There is a sizable movement in the Rust ecosystem to move all build-time scripts and procedural macros to (be combined to) WASM. This allows you to write turing-complete performant code to cover all use-cases people can reasonably think of, while also allowing trivially easy sandboxing. It's not perfect, for example some build scripts download content from the internet, which can be abused for information extraction. And code generation scripts could generate different code depending on where it thinks it's running. But it's a lot better than the unsandboxed code execution that's present in most current build systems, without introducing the constraints of a pure config file.
- RedShift1 3y ago
  How does the sandboxing help if the compiler and/or build scripts or whatever modifies its own output?
- comex 3y ago
  But the xz backdoor didn’t involve a build script that tried to compromise the machine the build was running on. It involved a build script that compromised the code being built. Sandboxing the build script wouldn’t have helped much if at all. Depending on the implementation, it might have prevented it from overwriting .o files that were already compiled, maybe. But there would still be all sorts of shenanigans it could have played to sneakily inject code. I’d argue that Rust’s biggest advantage here is that build scripts and procedural macros are written in Rust themselves, making them easier to read and thus harder to hide things in than autotools’ m4-to-shell gunk. But that means it’s important to build those things from source! Part of the movement you cite consists of watt, which ships proc macros as precompiled WebAssembly binaries mainly to save build time. But if watt were more widely adopted, it would actually make an attacker’s life much easier since they could backdoor the precompiled binary. Historically I’ve been sympathetic to dtolnay’s various attempts to use precompiled binaries this way (not just watt but also the serde thing that caused a hullabaloo); I hate slow builds. But after the xz backdoor I think this is untenable.
  - noctune 3y ago
shawnz 3y ago
When your programmatic build steps are isolated in plugins, then you can treat them like independent projects and apply your standard development practices like code review and unit tests to those plugins. Whereas when you stuff programmatic build steps into scripts that are bundled into existing projects, it's harder to make sure that your normal processes for assuring code quality get applied to those pieces of accessory code.
- dotancohen 3y ago
  My standard development practices like code review and unit tests do not scale to review and test every dependency of every dependency of my projects. Even at company-wide scale.
  - shawnz 3y ago
    I'm not saying that. I'm just saying that improved ability to apply such development practices is one benefit of using a plugin-style architecture for isolating the programmatic steps of your build pipeline. It's not perfect but in many ways it's still a significant improvement upon just allowing arbitrary code right in the pipeline definition.

kibwen 3y ago

> Findbugs, code coverage, source code generation,... For the purpose of this conversation we mostly just care about the use case of someone grabbing the code and wanting to use it in their own project. For this use case, dev tools like findbugs and code coverage can be ignored, so it would suffice to have a version of the build system with plugins completely disabled. Code generation is the thornier one, and we can at least be more principled about it than "run some arbitrary code", and at least it should be trivial to say "this codegen process gets absolutely no I/O access whatsoever; you're a dumb text pipeline". But at the end of the day, we have to Just Say No to things like this. Even if it makes the codebase grodier to check in generated code, if I can't inspect and audit the source code, that's a problem, and arbitrary build-time codegen prevents that. Some trade-offs are worth making.

hyperman1 3y ago
The xz debacle happened partiallybecause the generated autoconf code was provided. Checking in generated code is not that much better. It's a bit more visible, but not much people will spend their limited time to validate it, as it's not worth it for generated code. xz also had checked in inscrutable test files, and nobody could know it was encrypted malware. I'm not a fan of generated code. It tends to cause misery, being in a no mans land between code and not-code. But it is usefull sometimes, e.g rust generating an API from the opengl XML specs. Sandboxing seems the least worst option, but it will still be uninspected half code that one day ends up in production.
- kibwen 3y ago
  > The xz debacle happened partiallybecause the generated autoconf code was provided. The code was only provided in a roundabout way that was deliberately done to evade manual inspection, so that's not a failure of checking in generated code, that's a failure of actually building a binary from the artifacts that we expect it to be built from. Suffice to say, cutting out the Turing-complete crap from our build systems is only one of many things that we need to fix.

ptx 3y ago

Maybe now that we have things like GitHub Actions, Bitbucket Pipelines, etc., which can run steps in separate containers, maybe most of those things could be moved from the Maven build step to a different pipeline step? I'm not sure how well isolated the containers are (probably not very – I think GitHub gives access to the Docker socket) and you'd have to make sure they don't share secret tokens etc., but at least it might make things simpler to audit, and isolation could be improved in the future.

wongarsu 3y ago

Most of these could still be covered with IO limited to the project files though. There is a sizable movement in the Rust ecosystem to move all build-time scripts and procedural macros to (be combined to) WASM. This allows you to write turing-complete performant code to cover all use-cases people can reasonably think of, while also allowing trivially easy sandboxing. It's not perfect, for example some build scripts download content from the internet, which can be abused for information extraction. And code generation scripts could generate different code depending on where it thinks it's running. But it's a lot better than the unsandboxed code execution that's present in most current build systems, without introducing the constraints of a pure config file.

RedShift1 3y ago
How does the sandboxing help if the compiler and/or build scripts or whatever modifies its own output?
comex 3y ago
But the xz backdoor didn’t involve a build script that tried to compromise the machine the build was running on. It involved a build script that compromised the code being built. Sandboxing the build script wouldn’t have helped much if at all. Depending on the implementation, it might have prevented it from overwriting .o files that were already compiled, maybe. But there would still be all sorts of shenanigans it could have played to sneakily inject code. I’d argue that Rust’s biggest advantage here is that build scripts and procedural macros are written in Rust themselves, making them easier to read and thus harder to hide things in than autotools’ m4-to-shell gunk. But that means it’s important to build those things from source! Part of the movement you cite consists of watt, which ships proc macros as precompiled WebAssembly binaries mainly to save build time. But if watt were more widely adopted, it would actually make an attacker’s life much easier since they could backdoor the precompiled binary. Historically I’ve been sympathetic to dtolnay’s various attempts to use precompiled binaries this way (not just watt but also the serde thing that caused a hullabaloo); I hate slow builds. But after the xz backdoor I think this is untenable.
- noctune 3y ago

shawnz 3y ago

When your programmatic build steps are isolated in plugins, then you can treat them like independent projects and apply your standard development practices like code review and unit tests to those plugins. Whereas when you stuff programmatic build steps into scripts that are bundled into existing projects, it's harder to make sure that your normal processes for assuring code quality get applied to those pieces of accessory code.

dotancohen 3y ago
My standard development practices like code review and unit tests do not scale to review and test every dependency of every dependency of my projects. Even at company-wide scale.
- shawnz 3y ago
  I'm not saying that. I'm just saying that improved ability to apply such development practices is one benefit of using a plugin-style architecture for isolating the programmatic steps of your build pipeline. It's not perfect but in many ways it's still a significant improvement upon just allowing arbitrary code right in the pipeline definition.