4 ms·

Increasingly, my view is that there is only room for two quite separate things: * Task runners (rake, simple Make rules, shell scripts) etc. Basically convenie

Increasingly, my view is that there is only room for two quite separate things: * Task runners (rake, simple Make rules, shell scripts) etc. Basically convenience scripts for devs. * Hermetic build systems Hermetic build systems (Bazel, Nix package manager) can be 100% confident that there aren't any secret implicit dependencies and this unlocks some really great stuff such as: * Complete confidence in incremental builds at all time. It isn't possible for something secret to have changed * Excellent build caching, because you know all the inputs match the cache The classic example of this is C or C++. The build step is `gcc -c main.c -o main.o`, but that build command depends on a load of header files included by main.c. Both system ones and ones local to your project. If you change a header and rerun your build rule, it only knows about main.c, knows that it hasn't changed (by hash or timestamp) and doesn't do anything. You end up having to rebuild a load of stuff just in case. Your usual system (e.g. make) has no idea about this. Various solutions exist - Beast talks about the 'get gcc to tell you about the dependencies' approach but now you've got another file to generate and manage and have in your build system, and you still can't be sure it's right. There is a (good) argument to be made that these are problems that are only unmanageable in big software projects, and the rest of us should just take advantage of the simplicity of make (or newer better solutions like Beast). This is a similar argument to the one being made in another post about K8s and 'early optimisation'. But this is why every time someone posts a new build tool, I always hope it's going to be about 'power and reliability of Bazel' but as friendly to get going with a basic make rule.

JonChesterfield 4y ago
If determined, the system headers can be elided. Bundle parts of musl with the application or work in -ffreestanding. Then it's self contained except for the compiler, which I believe some embedded systems commit to the repo along with the source.
specialist 4y ago
> Hermetic build systems Wanting to know more, I found this: "Hermeticity: This page covers hermeticity, the benefits of using hermetic builds, and strategies for identifying non-hermetic behavior in your builds." https://docs.bazel.build/versions/main/hermeticity.html https://docs.bazel.build/versions/main/hermeticity.html Sounds great. Ages ago, my teams had a policy of "one button build". Install VS C++ on a new box, open the project (from source repo), hit "Build". Tada. We could rebuild any revision on demand. Terrific for reproducing regressions and delta-debugging. In the Java world, with (misuse of) maven, gradle, jenkins, etc. attaining reproducible one button builds is quixotic. For hermetic builds, everything would be digitally signed (SHA256), right? There's a spec for signing Linux kernels, which I can't quickly refind. But the idea is to apply that strategy to everything, right? That sounds perfect.
- rkangel 4y ago
  What you basically do is take all of the inputs (source files, compiler, exactly what command you're going to run) and then produce a hash of all of that together. Then the result of that becomes the input to the next thing etc. For caches you just say 'here's the SHA1 of the inputs and the cache server can just give you the output. Basel does this for each build command, e.g. for each object in a big C program. Nix does it for each package.
gravypod 4y ago
> I always hope it's going to be about 'power and reliability of Bazel' but as friendly to get going with a basic make rule. Have you seen https://please.build https://please.build? It is very close to what I think the answer to "What if Blaze was made today?" is. It is conceptually much simpler and doesn't suffer from some of the things that need to happen in Google-scale projects/teams. For example, they haven't done the same gross thing with proto_library that google did. You only define one and it only builds the bindings that are actually needed by the targets you are building.
- rkangel 4y ago
  This looks really interesting! Doesn't support Windows which is a bit of a shame, but maybe manageable. Will have to check it out properly at work tomorrow.

JonChesterfield 4y ago

If determined, the system headers can be elided. Bundle parts of musl with the application or work in -ffreestanding. Then it's self contained except for the compiler, which I believe some embedded systems commit to the repo along with the source.

specialist 4y ago

> Hermetic build systems Wanting to know more, I found this: "Hermeticity: This page covers hermeticity, the benefits of using hermetic builds, and strategies for identifying non-hermetic behavior in your builds." https://docs.bazel.build/versions/main/hermeticity.html https://docs.bazel.build/versions/main/hermeticity.html Sounds great. Ages ago, my teams had a policy of "one button build". Install VS C++ on a new box, open the project (from source repo), hit "Build". Tada. We could rebuild any revision on demand. Terrific for reproducing regressions and delta-debugging. In the Java world, with (misuse of) maven, gradle, jenkins, etc. attaining reproducible one button builds is quixotic. For hermetic builds, everything would be digitally signed (SHA256), right? There's a spec for signing Linux kernels, which I can't quickly refind. But the idea is to apply that strategy to everything, right? That sounds perfect.

rkangel 4y ago
What you basically do is take all of the inputs (source files, compiler, exactly what command you're going to run) and then produce a hash of all of that together. Then the result of that becomes the input to the next thing etc. For caches you just say 'here's the SHA1 of the inputs and the cache server can just give you the output. Basel does this for each build command, e.g. for each object in a big C program. Nix does it for each package.

gravypod 4y ago

> I always hope it's going to be about 'power and reliability of Bazel' but as friendly to get going with a basic make rule. Have you seen https://please.build https://please.build? It is very close to what I think the answer to "What if Blaze was made today?" is. It is conceptually much simpler and doesn't suffer from some of the things that need to happen in Google-scale projects/teams. For example, they haven't done the same gross thing with proto_library that google did. You only define one and it only builds the bindings that are actually needed by the targets you are building.

rkangel 4y ago
This looks really interesting! Doesn't support Windows which is a bit of a shame, but maybe manageable. Will have to check it out properly at work tomorrow.