3 ms·

99.9% of what is called "CI/CD" systems are NOT doing Continuous Integration. Great article here on "CBOI: Continuous Build, Occasional Integration": https://w

99.9% of what is called "CI/CD" systems are NOT doing Continuous Integration. Great article here on "CBOI: Continuous Build, Occasional Integration": https://www.aspect.dev/blog/cboi-continuous-build-occasional-integration https://www.aspect.dev/blog/cboi-continuous-build-occasional...

adonovan 5y ago
Another great article on this theme: https://gregoryszorc.com/blog/2021/04/07/modern-ci-is-too-complex-and-misdirected/ https://gregoryszorc.com/blog/2021/04/07/modern-ci-is-too-co... It is frustrating that we seem to be evolving towards the nesting of one build system inside another, such as "go build" inside "docker build". (One of the projects I work on uses janky, docker compose, docker build, Go build, Make, Rake, and Bundle.) This approach is opaque, coarse-grained, inefficient, unnecessarily sequential, and doesn't materialize "the" build graph as a single entity from which we can identify our software artifacts and their relationships and derive and automate all our workflows. Bazel addresses these problems with a single, canonical, first-class graph, and many projects that use Bazel achieve close-to-ideal build efficiency, scalability, and reproducibility, as well as a good foundation for other workflows. But switching to Bazel is a large undertaking for a project that has already become entangled in the kind of situation described above. Maintaining hand-written or generated BUILD files for dependencies can be a burden. (I led the design of Blaze in 2006, at which time Google was already using BUILD files to declare software artifacts and dependencies in its more primitive predecessor system; we simply could not have achieved Blaze even at Google's then scale otherwise.) I often dream of a future world in which it is expected that compiler writers will provide a formal definition of the toolchain's workflow in a form that allows rule-sets for Bazel and its successors to be derived. So there is a place for tools like Earthly that try to retrofit some of the goodness of pure functional builds onto the mess of Docker files, whose natural abstraction is not "consume inputs, produce outputs", but "mutate the file system", which doesn't compose. By strange coincidence I had just started playing last week---a mandatory vacation at my employer---with a proof-of-concept of a tool just like Earthly, so the timing of this announcement couldn't be better.

adonovan 5y ago

Another great article on this theme: https://gregoryszorc.com/blog/2021/04/07/modern-ci-is-too-complex-and-misdirected/ https://gregoryszorc.com/blog/2021/04/07/modern-ci-is-too-co... It is frustrating that we seem to be evolving towards the nesting of one build system inside another, such as "go build" inside "docker build". (One of the projects I work on uses janky, docker compose, docker build, Go build, Make, Rake, and Bundle.) This approach is opaque, coarse-grained, inefficient, unnecessarily sequential, and doesn't materialize "the" build graph as a single entity from which we can identify our software artifacts and their relationships and derive and automate all our workflows. Bazel addresses these problems with a single, canonical, first-class graph, and many projects that use Bazel achieve close-to-ideal build efficiency, scalability, and reproducibility, as well as a good foundation for other workflows. But switching to Bazel is a large undertaking for a project that has already become entangled in the kind of situation described above. Maintaining hand-written or generated BUILD files for dependencies can be a burden. (I led the design of Blaze in 2006, at which time Google was already using BUILD files to declare software artifacts and dependencies in its more primitive predecessor system; we simply could not have achieved Blaze even at Google's then scale otherwise.) I often dream of a future world in which it is expected that compiler writers will provide a formal definition of the toolchain's workflow in a form that allows rule-sets for Bazel and its successors to be derived. So there is a place for tools like Earthly that try to retrofit some of the goodness of pure functional builds onto the mess of Docker files, whose natural abstraction is not "consume inputs, produce outputs", but "mutate the file system", which doesn't compose. By strange coincidence I had just started playing last week---a mandatory vacation at my employer---with a proof-of-concept of a tool just like Earthly, so the timing of this announcement couldn't be better.