5 ms·

It failed because the marketing is outright bunk and overly dishonest. If I compile with Jenkins, Actions or Earthly, that compile time is going to be the same

It failed because the marketing is outright bunk and overly dishonest. If I compile with Jenkins, Actions or Earthly, that compile time is going to be the same under each build system assuming the same build node. Claiming you're 20x faster when CI is firing within seconds is kind of meaningless. Caching and parallel execution are age old concepts when it comes to CI and every modern build system can do it. CI is all about feedback, I didn't see much here in terms of collaboration and bubbling data up, I didn't look very hard, but this should be frontline and center. Lastly, I'm not interested in adopting a DSL for my builds ever again, sorry.

ravivyas 3y ago
Can you expand on the word "feedback"? What kind of feedback are you looking from CI?
- bastardoperator 3y ago
  It's one thing to know a build failed, it's another thing to know why and how a build failed, where it failed, and what the cause was. If I have to swim through or context shift to a separate build system's output to get that, what is this actually doing for me?
  - hedora 3y ago
    Also, which of my 1,000 test suites are flaky, and when did they start being flaky? Flamebox almost got this right, but groups by wall-clock instead of git sha: https://flamebox.sourceforge.net https://flamebox.sourceforge.net The screenshot is most of what there is to see. Clicking a test run opens the test log. Does anyone know of a currently-maintained alternative with a less-bad dashboard?
    - Phelinofist 3y ago
      That looks really cool, thanks for the link. Something like this well integrated into Jenkins would be super useful. Our DevOps team build something custom: they check all test logs of all builds of all branches and aggregate all that into a Grafana dashboard. We use it to monitor tests that are failing, to get a better grip on flakiness. Works okay but could be better.
  - MatthiasPortzel 3y ago
    From the post: > Earthly gives you write once, run anywhere build consistency, making it super easy to reproduce CI failures on your local computer.
    - marcinzm 3y ago
      That doesn't help with non-deterministic failures and I've also yet to see a true "write once, run anywhere" system ever. It may be 99% "write once, run anywhere" but there's always that 1% edge case.
      - bastardoperator 3y ago
        Exactly. This just reminds me of "works on my computer". Building locally is more advantageous for developing CI workflow, but I want to be as close to prod as possible and doing that on snowflake developer workstations is an exercise in futility.
        oftenwrong 3y ago
        Earthly makes use of BuildKit, which essentially executes the build steps in containers. It provides more isolation from the CI runner / dev workstation. Instead of having developers manage their own build tools, Earthly makes it easy to have the build definition manage them.
        vlovich123 3y ago
        All flakiness I’ve observed in code I’ve written has been dominated by things like “this job isn’t getting enough cpu cycles and the test assertion is too aggressive in such a scenario” or “lack of CPU cycles is triggering a race condition” or “statistical test X isn’t written robustly”. Not sure how containers solve these problems. My point being that this tool solves some problems for some teams and maybe for a lot of teams that struggle with this problem, but hard problems remain and this isn’t a silver bullet for that. You can’t outsource stability of the project-specific test infrastructure which is where most of the cost lies these days I think.
shepherdjerred 3y ago
> Caching and parallel execution are age old concepts when it comes to CI and every modern build system can do it. It is much harder to set up caching/parallelization with GitHub Actions, Jenkins, Docker, and Make than it is with just Earthly
- bastardoperator 3y ago
  In your opinion, regardless I have to set those systems up anyways to use the Earthly system. Jenkins caches by default and has parallel(), actions uses actions/cache, actions is also async so it's parallel by default. Both can use docker and take advantage of dockers caching management and make is a great tool, but again, any build system can use make. I don't think it's harder at all, in fact in my mind Earthly is more money, more steps, more time and energy, and more vendor lock in for the same output you can get in just about any modern build system.
- verdverm 3y ago
  That caching is not earthly, it's buildkit Earthly is just a skin and a hosted service as I understand it
  - shepherdjerred 3y ago
    Yes, you're right.
- rewmie 3y ago
  > It is much harder to set up caching/parallelization with GitHub Actions, Jenkins, Docker, and Make than it is with just Earthly I never used Earthly but I have to call nonsense on this take. Caching and parallelization is a solved problem in GitLab, GitHub, CircleCI, etc. It's so trivial that it's pipelines 101. Caching and parallelization might be easier in Earthly but any improvement over "trivial" is something not worth wasting time over.
  - shepherdjerred 3y ago
    How do you usually do caching for complex, multi-language projects in this case? How do you guarantee _only_ changed targets are rebuilt when dependencies, like a language version or external library, change? With Earthly you just... write your Earthfile. You get everything for free. Caching and parallelization _also_ works locally, so I see that speedup in development.

ravivyas 3y ago

Can you expand on the word "feedback"? What kind of feedback are you looking from CI?

bastardoperator 3y ago
It's one thing to know a build failed, it's another thing to know why and how a build failed, where it failed, and what the cause was. If I have to swim through or context shift to a separate build system's output to get that, what is this actually doing for me?
- hedora 3y ago
  Also, which of my 1,000 test suites are flaky, and when did they start being flaky? Flamebox almost got this right, but groups by wall-clock instead of git sha: https://flamebox.sourceforge.net https://flamebox.sourceforge.net The screenshot is most of what there is to see. Clicking a test run opens the test log. Does anyone know of a currently-maintained alternative with a less-bad dashboard?
  - Phelinofist 3y ago
    That looks really cool, thanks for the link. Something like this well integrated into Jenkins would be super useful. Our DevOps team build something custom: they check all test logs of all builds of all branches and aggregate all that into a Grafana dashboard. We use it to monitor tests that are failing, to get a better grip on flakiness. Works okay but could be better.
- MatthiasPortzel 3y ago
  From the post: > Earthly gives you write once, run anywhere build consistency, making it super easy to reproduce CI failures on your local computer.
  - marcinzm 3y ago
    That doesn't help with non-deterministic failures and I've also yet to see a true "write once, run anywhere" system ever. It may be 99% "write once, run anywhere" but there's always that 1% edge case.
    - bastardoperator 3y ago
      Exactly. This just reminds me of "works on my computer". Building locally is more advantageous for developing CI workflow, but I want to be as close to prod as possible and doing that on snowflake developer workstations is an exercise in futility.
      - oftenwrong 3y ago
        Earthly makes use of BuildKit, which essentially executes the build steps in containers. It provides more isolation from the CI runner / dev workstation. Instead of having developers manage their own build tools, Earthly makes it easy to have the build definition manage them.
        vlovich123 3y ago
        All flakiness I’ve observed in code I’ve written has been dominated by things like “this job isn’t getting enough cpu cycles and the test assertion is too aggressive in such a scenario” or “lack of CPU cycles is triggering a race condition” or “statistical test X isn’t written robustly”. Not sure how containers solve these problems. My point being that this tool solves some problems for some teams and maybe for a lot of teams that struggle with this problem, but hard problems remain and this isn’t a silver bullet for that. You can’t outsource stability of the project-specific test infrastructure which is where most of the cost lies these days I think.

shepherdjerred 3y ago

> Caching and parallel execution are age old concepts when it comes to CI and every modern build system can do it. It is much harder to set up caching/parallelization with GitHub Actions, Jenkins, Docker, and Make than it is with just Earthly

bastardoperator 3y ago
In your opinion, regardless I have to set those systems up anyways to use the Earthly system. Jenkins caches by default and has parallel(), actions uses actions/cache, actions is also async so it's parallel by default. Both can use docker and take advantage of dockers caching management and make is a great tool, but again, any build system can use make. I don't think it's harder at all, in fact in my mind Earthly is more money, more steps, more time and energy, and more vendor lock in for the same output you can get in just about any modern build system.
verdverm 3y ago
That caching is not earthly, it's buildkit Earthly is just a skin and a hosted service as I understand it
- shepherdjerred 3y ago
  Yes, you're right.
rewmie 3y ago
> It is much harder to set up caching/parallelization with GitHub Actions, Jenkins, Docker, and Make than it is with just Earthly I never used Earthly but I have to call nonsense on this take. Caching and parallelization is a solved problem in GitLab, GitHub, CircleCI, etc. It's so trivial that it's pipelines 101. Caching and parallelization might be easier in Earthly but any improvement over "trivial" is something not worth wasting time over.
- shepherdjerred 3y ago
  How do you usually do caching for complex, multi-language projects in this case? How do you guarantee _only_ changed targets are rebuilt when dependencies, like a language version or external library, change? With Earthly you just... write your Earthfile. You get everything for free. Caching and parallelization _also_ works locally, so I see that speedup in development.