5 ms·

I tried using Earthly, great product. Unfortunately, we rejected it at our organization due to how verbose the commands are. I want my users to be able to say

I tried using Earthly, great product. Unfortunately, we rejected it at our organization due to how verbose the commands are. I want my users to be able to say `earthly plan-stage` for a terraform repo, and for it to work. But you cannot. You cannot have earthly automatically use environment variables from your env, you have to explicitly list the commands yourself using flags: $ earthly \ --build-arg AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID \ --build-arg AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY \ --build-arg AWS_DEFAULT_REGION=us-west-2 \ --build-arg AWS_SESSION_TOKEN=$AWS_SESSION_TOKEN \ --build-arg AWS_SECURITY_TOKEN=$AWS_SECURITY_TOKEN \ +plan-stage This means if I want to `terraform plan` something, I need to explicitly set all of my AWS environment variables. Compare this to `task plan-stage` or `make plan-stage`. It is much too verbose and will only get worse as we add features to our pipelines.

adamgordonbell 5y ago
You may want to use secrets like this: earthly \ --secret-file config=/home/user/.aws/config \ +build But yes, Earthly optimizes for reproducibility, so implicitly pulling in ENVs is not something it does.
- honkycat 5y ago
  I didn't use secrets because it is a prototype, and I need to explicitly give RUNs access to secret values, and I got sick of the boilerplate.
- andyonthewings 5y ago
  The required build arg and env var names are already listed in the Earthfile as `ARG` and `RUN --secret ...`. So I think specifying it in the commandline is redundant.
AlphaSite 5y ago
It’s a build tool, I’d be surprised if something like it worked well for non build related tasks. I suspect bazel would be equally verbose (if not even worse or impossible). I think you’re better of using something like taskfile w/ earthly for a really good workflow, it gives you the best of both worlds. (Actually I think I saw this on their slack)
- honkycat 5y ago
  Yeah, that was what I was doing in the prototype stage. But then I thought "why do we need two task runners? " and ended up just going with task files. They sell it as a make replacement. Bazel constantly says "you don't need hazel, it is a big tool" and earthly does not. Which is fine. Either way, I already said earthly is a great tool. This is just my feedback around why it doesnt fit my use case.
  - AlphaSite 5y ago
    Ah sorry, that’s fair. We went the other way from having just taskfile to having both and it seems quite nice so far.
    - honkycat 5y ago
      We may go back! Earthly is quite nice
lessthanseventy 5y ago
could combine it with something like: https://github.com/bbugyi200/funky https://github.com/bbugyi200/funky
no_circuit 5y ago
Interesting. Dev tools post always catch my attention. For me, your comment regarding `build-arg` is in the same bucket as the comment regarding the need for a `ls` command. Ultimately I predict that the requested feature set will eventually be equivalent to what is already in Bazel -- like `.bazelrc` for setting up build variations and environment variables, and `query` for figuring out what is going on with the project. If Bazel has been working on these features and issues for years, I'd expect the same here. Earthly seems very useful to work around issues like cross-compiling. A simple example that I'd expect a lot of people to run into is running into libc/stdlibc++ differences with Alpine containers. That is easy to come across when doing something as simple as pulling in a Python pip dependency that wants to compile native C/C++ code. I find this extremely difficult to learn to do in Bazel. However if you're in the league of complex cross-compiling, then you may already be on CMake and/or Bazel and have the dev resources to define the toolchains as needed.
andyonthewings 5y ago
That's my biggest complain about earthly as well. There is `.env` support that ease the problem a bit: https://docs.earthly.dev/docs/earthly-command#environment-variables-and-.env-file https://docs.earthly.dev/docs/earthly-command#environment-va... Worth watching the reported issue at https://github.com/earthly/earthly/issues/707 https://github.com/earthly/earthly/issues/707
- adamgordonbell 5y ago
  We do love getting all this input. Finding a way to make most use-cases ergonomic while still being reproducible is very important to us. Thumbs-ups on tickets and comments help us prioritize features. We already have a PR up for the `earthly ls` feature. https://github.com/earthly/earthly/pull/1472 https://github.com/earthly/earthly/pull/1472

adamgordonbell 5y ago

You may want to use secrets like this: earthly \ --secret-file config=/home/user/.aws/config \ +build But yes, Earthly optimizes for reproducibility, so implicitly pulling in ENVs is not something it does.

honkycat 5y ago
I didn't use secrets because it is a prototype, and I need to explicitly give RUNs access to secret values, and I got sick of the boilerplate.
andyonthewings 5y ago
The required build arg and env var names are already listed in the Earthfile as `ARG` and `RUN --secret ...`. So I think specifying it in the commandline is redundant.

AlphaSite 5y ago

It’s a build tool, I’d be surprised if something like it worked well for non build related tasks. I suspect bazel would be equally verbose (if not even worse or impossible). I think you’re better of using something like taskfile w/ earthly for a really good workflow, it gives you the best of both worlds. (Actually I think I saw this on their slack)

honkycat 5y ago
Yeah, that was what I was doing in the prototype stage. But then I thought "why do we need two task runners? " and ended up just going with task files. They sell it as a make replacement. Bazel constantly says "you don't need hazel, it is a big tool" and earthly does not. Which is fine. Either way, I already said earthly is a great tool. This is just my feedback around why it doesnt fit my use case.
- AlphaSite 5y ago
  Ah sorry, that’s fair. We went the other way from having just taskfile to having both and it seems quite nice so far.
  - honkycat 5y ago
    We may go back! Earthly is quite nice

lessthanseventy 5y ago

could combine it with something like: https://github.com/bbugyi200/funky https://github.com/bbugyi200/funky

no_circuit 5y ago

Interesting. Dev tools post always catch my attention. For me, your comment regarding `build-arg` is in the same bucket as the comment regarding the need for a `ls` command. Ultimately I predict that the requested feature set will eventually be equivalent to what is already in Bazel -- like `.bazelrc` for setting up build variations and environment variables, and `query` for figuring out what is going on with the project. If Bazel has been working on these features and issues for years, I'd expect the same here. Earthly seems very useful to work around issues like cross-compiling. A simple example that I'd expect a lot of people to run into is running into libc/stdlibc++ differences with Alpine containers. That is easy to come across when doing something as simple as pulling in a Python pip dependency that wants to compile native C/C++ code. I find this extremely difficult to learn to do in Bazel. However if you're in the league of complex cross-compiling, then you may already be on CMake and/or Bazel and have the dev resources to define the toolchains as needed.

andyonthewings 5y ago

That's my biggest complain about earthly as well. There is `.env` support that ease the problem a bit: https://docs.earthly.dev/docs/earthly-command#environment-variables-and-.env-file https://docs.earthly.dev/docs/earthly-command#environment-va... Worth watching the reported issue at https://github.com/earthly/earthly/issues/707 https://github.com/earthly/earthly/issues/707

adamgordonbell 5y ago
We do love getting all this input. Finding a way to make most use-cases ergonomic while still being reproducible is very important to us. Thumbs-ups on tickets and comments help us prioritize features. We already have a PR up for the `earthly ls` feature. https://github.com/earthly/earthly/pull/1472 https://github.com/earthly/earthly/pull/1472