4 ms·

> Caching and parallel execution are age old concepts when it comes to CI and every modern build system can do it. It is much harder to set up caching/parallel

> Caching and parallel execution are age old concepts when it comes to CI and every modern build system can do it. It is much harder to set up caching/parallelization with GitHub Actions, Jenkins, Docker, and Make than it is with just Earthly

bastardoperator 3y ago
In your opinion, regardless I have to set those systems up anyways to use the Earthly system. Jenkins caches by default and has parallel(), actions uses actions/cache, actions is also async so it's parallel by default. Both can use docker and take advantage of dockers caching management and make is a great tool, but again, any build system can use make. I don't think it's harder at all, in fact in my mind Earthly is more money, more steps, more time and energy, and more vendor lock in for the same output you can get in just about any modern build system.
verdverm 3y ago
That caching is not earthly, it's buildkit Earthly is just a skin and a hosted service as I understand it
- shepherdjerred 3y ago
  Yes, you're right.
rewmie 3y ago
> It is much harder to set up caching/parallelization with GitHub Actions, Jenkins, Docker, and Make than it is with just Earthly I never used Earthly but I have to call nonsense on this take. Caching and parallelization is a solved problem in GitLab, GitHub, CircleCI, etc. It's so trivial that it's pipelines 101. Caching and parallelization might be easier in Earthly but any improvement over "trivial" is something not worth wasting time over.
- shepherdjerred 3y ago
  How do you usually do caching for complex, multi-language projects in this case? How do you guarantee _only_ changed targets are rebuilt when dependencies, like a language version or external library, change? With Earthly you just... write your Earthfile. You get everything for free. Caching and parallelization _also_ works locally, so I see that speedup in development.
  - howinteresting 3y ago
    I think there is certainly some value to that (the other posts don't appear to be fully grasping the complexity of the problem), but Earthly feels closer to an open source consultancy project than venture capital funded startup.
    - shepherdjerred 3y ago
      Yeah, I'm a little lost on how the value proposition of Earthly isn't being understood by others. I'm guessing that many of the posters don't understand that Earthly works locally.
  - Too 3y ago
    Dependency graph is a feature of the build system with better understanding of the language import rules than the CI system can have (or whatever terminology you want to use). Good ones can hermetically, concurrently and deterministic build only the files that were changed. Earthly is just a thin wrapper on top of docker multi stage builds[1]. Such docker layers are one-dimensional, meaning if one of the steps are calling “make”, that step will have to rebuild that entire layer, not just the one c++ file that changed. There are solutions to this with --mount,type=cache but they take away the reproducibility and and are equally possible with plain dockerfiles. [1] tbh I don’t know if this is the case, but that’s how others explain it. This is core problem with Earthly. They don’t say how it works, just that you will magically get 20x faster. Without understanding how, I don’t know how to take that seriously and I can only assume it’s snake oil. I mean, there is a need for a better Dockerfile syntax. But then market it as such! Not as magic 20x faster builds.
Klugistiono 3y ago
Most paralization is a junit configuration or parallel jobs. Which is super easy in Github Actions.

bastardoperator 3y ago

In your opinion, regardless I have to set those systems up anyways to use the Earthly system. Jenkins caches by default and has parallel(), actions uses actions/cache, actions is also async so it's parallel by default. Both can use docker and take advantage of dockers caching management and make is a great tool, but again, any build system can use make. I don't think it's harder at all, in fact in my mind Earthly is more money, more steps, more time and energy, and more vendor lock in for the same output you can get in just about any modern build system.

verdverm 3y ago

That caching is not earthly, it's buildkit Earthly is just a skin and a hosted service as I understand it

shepherdjerred 3y ago
Yes, you're right.

rewmie 3y ago

> It is much harder to set up caching/parallelization with GitHub Actions, Jenkins, Docker, and Make than it is with just Earthly I never used Earthly but I have to call nonsense on this take. Caching and parallelization is a solved problem in GitLab, GitHub, CircleCI, etc. It's so trivial that it's pipelines 101. Caching and parallelization might be easier in Earthly but any improvement over "trivial" is something not worth wasting time over.

shepherdjerred 3y ago
How do you usually do caching for complex, multi-language projects in this case? How do you guarantee _only_ changed targets are rebuilt when dependencies, like a language version or external library, change? With Earthly you just... write your Earthfile. You get everything for free. Caching and parallelization _also_ works locally, so I see that speedup in development.
- howinteresting 3y ago
  I think there is certainly some value to that (the other posts don't appear to be fully grasping the complexity of the problem), but Earthly feels closer to an open source consultancy project than venture capital funded startup.
  - shepherdjerred 3y ago
    Yeah, I'm a little lost on how the value proposition of Earthly isn't being understood by others. I'm guessing that many of the posters don't understand that Earthly works locally.
- Too 3y ago
  Dependency graph is a feature of the build system with better understanding of the language import rules than the CI system can have (or whatever terminology you want to use). Good ones can hermetically, concurrently and deterministic build only the files that were changed. Earthly is just a thin wrapper on top of docker multi stage builds[1]. Such docker layers are one-dimensional, meaning if one of the steps are calling “make”, that step will have to rebuild that entire layer, not just the one c++ file that changed. There are solutions to this with --mount,type=cache but they take away the reproducibility and and are equally possible with plain dockerfiles. [1] tbh I don’t know if this is the case, but that’s how others explain it. This is core problem with Earthly. They don’t say how it works, just that you will magically get 20x faster. Without understanding how, I don’t know how to take that seriously and I can only assume it’s snake oil. I mean, there is a need for a better Dockerfile syntax. But then market it as such! Not as magic 20x faster builds.

Klugistiono 3y ago

Most paralization is a junit configuration or parallel jobs. Which is super easy in Github Actions.