4 ms·

I looked briefly at Earthly a while ago for work because I have to write CI integrations for a whole bunch of CI platforms (GitLab, Azure DevOps, Jenkins, and m

by pxc 3y ago

I looked briefly at Earthly a while ago for work because I have to write CI integrations for a whole bunch of CI platforms (GitLab, Azure DevOps, Jenkins, and maybe GitHub Actions). I ended up going with a very similar product (i.e., another nice BuildKit frontend with integrations into a bunch of CI systems), Dagger, because the DSL it used for defining pipelines at the time seemed better to me. I ended up deeply regretting this decision, as the Dagger devs have de facto abandoned that language entirely in favor of churning out SDKs for the most popular programming languages— all imperative, all Turing-complete, and all, imo, bad fits for the space. So now I'm back in hell, integrating with all of these systems by hand, hesitant to trust this kind of tooling to a startup again. To me this is still the most appealing use case of something like Earthly. Push-and-see-what-happens is still the norm on almost all CI systems, and it's an abominable workflow. If you are at a large organization and have to support teams that use a wide variety of CI/CD platforms, something like Earthly could potentially save quite a lot of pain. But the appeal is precisely in supporting existing CI platforms, rather than in adding another one.

verdverm 3y ago
CI/CD & provisioning are inherently order driven tasks, I actually prefer the SDKs over the CUE implementation. The main issue with the old CUE implementation is that they had to try and align BuildKits DAG solver with CUE's DAG solver, and they worked in opposite ways. I tried to help them through this about 3 years ago (being a CUE expert). I definitely think the SDKs is a much better solution for them. Whether we like it or not, a lot of the industry is moving this way. Pulumi is another example of this, though I'm sold on imperative cloud infra, builds do seem to have alignment here. Side note, I will be exploring a new CUE + Dagger setup, though it will work differently than the old Dagger engine
- pxc 3y ago
  I'm not totally opposed to powerful (Turing-complete) languages being used for configuration or defining builds— I think Nix works really well in that space, and someday Nickel may as well. But wherever I've seen build tools from a language ecosystem that just embed a DSL in the host language (e.g.: sbt, Gradle), I've also seen projects get way too clever with their builds and create total nightmares for downstream packaging efforts. I anticipate similar problems for other kinds of configuration tools based on general-purpose languages, like CI or cloud IaC. > Side note, I will be exploring a new CUE + Dagger setup, though it will work differently than the old Dagger engine Sounds cool! I'll keep an eye out for it. :)
shykes 3y ago
> Dagger devs have de facto abandoned that language entirely in favor of churning out SDKs for the most popular programming languages— all imperative, all Turing-complete, and all, imo, bad fits for the space. Disclaimer: I’m the CEO of Dagger. You’re right that we offer SDKs for popular languages, which happen to be imperative. But Dagger remains a declarative system, so what you liked about the earlier versions is still there. The trick is that we moved the declarative layer from static CUE configurations to dynamic GraphQL queries. Then we generated client libraries in various languages from the GraphQL schema. So not only can you still declaratively construct your DAG like before, you can do it in whatever language you prefer. You can even run a DAG directly in pure GraphQL: see https://play.dagger.cloud https://play.dagger.cloud which lets you do this straight from the browser. A useful analogy is SQL: it’s a declarative language that is commonly combined with another, often imperative, language. I hope you find this message useful and consider giving Dagger another chance.
- geggo98 3y ago
  While you are here: the Node.js syntax highlights on you QuickStart docs seems to be broken. Tested with Safari and Chrome on a recent macOS. That’s not a huge issue, but makes it a bit harder to grasp what’s going on and how the Dagger SDK actually works.
  - marcosnils 3y ago
    Thx for reporting. We'll fix ASAP

by pxc 3y ago

verdverm 3y ago

CI/CD & provisioning are inherently order driven tasks, I actually prefer the SDKs over the CUE implementation. The main issue with the old CUE implementation is that they had to try and align BuildKits DAG solver with CUE's DAG solver, and they worked in opposite ways. I tried to help them through this about 3 years ago (being a CUE expert). I definitely think the SDKs is a much better solution for them. Whether we like it or not, a lot of the industry is moving this way. Pulumi is another example of this, though I'm sold on imperative cloud infra, builds do seem to have alignment here. Side note, I will be exploring a new CUE + Dagger setup, though it will work differently than the old Dagger engine

pxc 3y ago
I'm not totally opposed to powerful (Turing-complete) languages being used for configuration or defining builds— I think Nix works really well in that space, and someday Nickel may as well. But wherever I've seen build tools from a language ecosystem that just embed a DSL in the host language (e.g.: sbt, Gradle), I've also seen projects get way too clever with their builds and create total nightmares for downstream packaging efforts. I anticipate similar problems for other kinds of configuration tools based on general-purpose languages, like CI or cloud IaC. > Side note, I will be exploring a new CUE + Dagger setup, though it will work differently than the old Dagger engine Sounds cool! I'll keep an eye out for it. :)

shykes 3y ago

> Dagger devs have de facto abandoned that language entirely in favor of churning out SDKs for the most popular programming languages— all imperative, all Turing-complete, and all, imo, bad fits for the space. Disclaimer: I’m the CEO of Dagger. You’re right that we offer SDKs for popular languages, which happen to be imperative. But Dagger remains a declarative system, so what you liked about the earlier versions is still there. The trick is that we moved the declarative layer from static CUE configurations to dynamic GraphQL queries. Then we generated client libraries in various languages from the GraphQL schema. So not only can you still declaratively construct your DAG like before, you can do it in whatever language you prefer. You can even run a DAG directly in pure GraphQL: see https://play.dagger.cloud https://play.dagger.cloud which lets you do this straight from the browser. A useful analogy is SQL: it’s a declarative language that is commonly combined with another, often imperative, language. I hope you find this message useful and consider giving Dagger another chance.

geggo98 3y ago
While you are here: the Node.js syntax highlights on you QuickStart docs seems to be broken. Tested with Safari and Chrome on a recent macOS. That’s not a huge issue, but makes it a bit harder to grasp what’s going on and how the Dagger SDK actually works.
- marcosnils 3y ago
  Thx for reporting. We'll fix ASAP