3 ms·

except not really.. their tutorial [0] includes no-compile languages like python and JS, with only non-trivial compilation being and Java... and how do they do

by theamk 3y ago

except not really.. their tutorial [0] includes no-compile languages like python and JS, with only non-trivial compilation being and Java... and how do they do it? RUN gradle build At least in this case, Earthly has no insight into how repository is organized, and it _will_ recompile the entire repository with every single commit. So what's the real value of it? this basically seems like a better "docker build" alternative, giving nicer inter-layer caching. Maybe will save some time on dependencies if they take some time to install and devs were too lazy to set up own workers and used ephemeral runners instead. [0] https://docs.earthly.dev/basics/part-1-a-simple-earthfile https://docs.earthly.dev/basics/part-1-a-simple-earthfile

danudey 3y ago
That's step one, 'how to write a basic Earthfile that just does the thing', so they don't delve into the details of caching and how it works, but look at the whole block and not just that one line; it operates like a Dockerfile: build: COPY build.gradle ./ COPY src src RUN gradle build RUN gradle install SAVE ARTIFACT build/install/java-example/bin /bin SAVE ARTIFACT build/install/java-example/lib /lib The COPY phases copy files into the build context. If the files haven't changed, then the commands don't need to be run. In other words, if `build.gradle` and `src` are unchanged, it won't run `gradle build` or `gradle install`, and it'll just give you the artifacts from the previous build. They have a golang example in part three[0], which they redesign to use caching effectively: build: # Download deps before copying code. COPY go.mod go.sum . RUN go mod download # Copy and build code. COPY main.go . RUN go build -o output/example main.go SAVE ARTIFACT output/example AS LOCAL local-output/go-example They copy in go.mod and go.sum and then run `go mod download`. If the mod and sum files haven't changed, then `go mod download` doesn't need to be run. Then they copy `main.go` in, and run `go build`. If the `main.go` file hasn't changed, `go build` doesn't need to be run. [0] https://docs.earthly.dev/basics/part-3-adding-dependencies-with-caching https://docs.earthly.dev/basics/part-3-adding-dependencies-w...
- brabel 3y ago
  Gradle can do the caching of tasks itself in a level of granularity no CI can. If you keep your build cache "live" in the CI, Gradle knows to only recompile what has changed, and to only run tests which changed (or tests something which changed). I am not sure if many other build systems do that, but this is something I would absolutely expect from the build system, not CI. Do you not expect caching when building locally?? The whole idea with CI doing this is wrong IMO. All I want CI to do is call a couple of commands after setting up the environment (which would include the Gradle cache or whatever to speed up things - no need for your revolutionary CI build system).
- nvm0n2 3y ago
  A CI system like TeamCity will give you that sort of thing for free though, without needing an Earthfile. It just dedupes builds at the same commit, and keeps checkout directories between builds on the agents, so if the build tool has a cache then it's automatically reused. Works great. Given that it's hard to understand what exactly this is for? Build systems normally try to avoid redundant work for local purposes anyway. Is it some hack for companies that insist on resetting build environments to zero on every single build, and then decide they want caching back? If so, why not just ... let the builds take place in directories containing previous builds, and fix the problems? That's what we do at my company and it works great (using gradle), there are only rarely problems with unclean builds and it's easy to rerun with a clean checkout if that's suspected to be an issue (the bugs are mostly in our own code which does a layer of caching above the build system anyway).
  - oftenwrong 3y ago
    One of the chief benefits of Earthly is builds that are always clean and predictable. This is especially important on a shared build server.
    - nvm0n2 3y ago
      So they claim. And yet ultimately they do it by caching things, same as every other system.

by theamk 3y ago

danudey 3y ago

That's step one, 'how to write a basic Earthfile that just does the thing', so they don't delve into the details of caching and how it works, but look at the whole block and not just that one line; it operates like a Dockerfile: build: COPY build.gradle ./ COPY src src RUN gradle build RUN gradle install SAVE ARTIFACT build/install/java-example/bin /bin SAVE ARTIFACT build/install/java-example/lib /lib The COPY phases copy files into the build context. If the files haven't changed, then the commands don't need to be run. In other words, if `build.gradle` and `src` are unchanged, it won't run `gradle build` or `gradle install`, and it'll just give you the artifacts from the previous build. They have a golang example in part three[0], which they redesign to use caching effectively: build: # Download deps before copying code. COPY go.mod go.sum . RUN go mod download # Copy and build code. COPY main.go . RUN go build -o output/example main.go SAVE ARTIFACT output/example AS LOCAL local-output/go-example They copy in go.mod and go.sum and then run `go mod download`. If the mod and sum files haven't changed, then `go mod download` doesn't need to be run. Then they copy `main.go` in, and run `go build`. If the `main.go` file hasn't changed, `go build` doesn't need to be run. [0] https://docs.earthly.dev/basics/part-3-adding-dependencies-with-caching https://docs.earthly.dev/basics/part-3-adding-dependencies-w...

brabel 3y ago
Gradle can do the caching of tasks itself in a level of granularity no CI can. If you keep your build cache "live" in the CI, Gradle knows to only recompile what has changed, and to only run tests which changed (or tests something which changed). I am not sure if many other build systems do that, but this is something I would absolutely expect from the build system, not CI. Do you not expect caching when building locally?? The whole idea with CI doing this is wrong IMO. All I want CI to do is call a couple of commands after setting up the environment (which would include the Gradle cache or whatever to speed up things - no need for your revolutionary CI build system).
nvm0n2 3y ago
A CI system like TeamCity will give you that sort of thing for free though, without needing an Earthfile. It just dedupes builds at the same commit, and keeps checkout directories between builds on the agents, so if the build tool has a cache then it's automatically reused. Works great. Given that it's hard to understand what exactly this is for? Build systems normally try to avoid redundant work for local purposes anyway. Is it some hack for companies that insist on resetting build environments to zero on every single build, and then decide they want caching back? If so, why not just ... let the builds take place in directories containing previous builds, and fix the problems? That's what we do at my company and it works great (using gradle), there are only rarely problems with unclean builds and it's easy to rerun with a clean checkout if that's suspected to be an issue (the bugs are mostly in our own code which does a layer of caching above the build system anyway).
- oftenwrong 3y ago
  One of the chief benefits of Earthly is builds that are always clean and predictable. This is especially important on a shared build server.
  - nvm0n2 3y ago
    So they claim. And yet ultimately they do it by caching things, same as every other system.