21 ms·

We built the fastest CI and it failed

450 points by adamgordonbell 3y ago

adamgordonbell 3y ago
"People will buy a developer tool, but you can’t sell it."
- deepspace 3y ago
  That line caught my attention too, and it looked superficially true. I have certainly never purchased a tool based on a phone call from a vendor. However, that only pertains to cold-calling, which is the absolute worst way to sell anything. Advertising is a trillion dollar market for a reason – it works. The trick is to find the correct message and media for the audience. Effective advertising takes a lot of skill, a lot of money, or both. A large portion of HN posts (including this one) are contain "hidden" advertising, and they absolutely work. I have purchased more than one developer tool based on reddit and HN reviews, which originated as posts by the seller. On the "lot of money" side, a poster in the arrivals tunnel at SFO is an excellent way of raising brand awareness; even for developer tools.
- wrs 3y ago
  For me the killer quote was “if the prospect requires a demo, then they’re not worth going after” — because as a prospect, if your product requires a demo, I’m already halfway to no. I am so tired of “enterprise” products that have to be purchased as a pig in a poke because they can’t stand on their own during a trial period.
- nathanfig 3y ago
  I think HN will eat this up because few of us enjoy being sold to but like most absolute statements it's probably only applicable in specific circumstances. Enterprises sell tons of developer tools and move millions because of demos done well. AWS and Azure are definitely getting on planes and doing demos.
huijzer 3y ago
I have no personal experience with selling a SaaS, but read that AWS mostly sold to startups at first because there were no migration costs yet. Out of curiosity, has the same been attempted with Eathly and, if yes, did that work?
alexvitkov 3y ago
What does fast CI even mean? CI is an overgrown shell script running your build and telling you when it fails. In general build tooling has gotten so slow that the cost of whatever CI runner relative to it should be nil. If you want fast CI you need fast tsc, clang, rustc, etc... not a faster program that calls exec on them. A bit more on topic, if you're selling CI and your business fails it's because people you're providing no value - people can run their build scripts fine without you.
- jve 3y ago
  > If you want fast CI you need fast tsc, clang, rustc, etc... not a faster program that calls exec on them Yes and no. Caching stuff and knowing when to run tsc/clang/rustc also improves performance.
  - deleted 3y ago
    [deleted]
  - bluGill 3y ago
    Sure, but you have to be 100% perfect in guessing when the cache file must be rebuilt. If there is any situation at all where you use the cached file when you should have rebuilt your tool is not useful. Note that you can go too far. If you rebuild a file on linux even though the only changes were in a "#ifdef freebsd" (windows, mac...) section that is a waste of time, but not harmful. However we already have tools that do that.
    - esafak 3y ago
      This is a solved problem; modern build tools and workflow orchestrators face the same challenge. You declare your inputs for a given task and cache the output as long as the inputs are unchanged, as determined by their hash. So there is no wrong guessing, only an incomplete specification of inputs. It's an elegant solution, if I do say so, and has worked well at companies I've seen it used.
      - 3y ago
shuntress 3y ago
> Why pay both the CI vendor and us Exactly. Complexity, not speed, is the problem with builds. It makes perfect sense that "another layer of complexity that wraps up all the other layers" was tough to sell.
- pshirshov 3y ago
  In this particular domain, performance problems are tightly coupled with complexity.
  - shuntress 3y ago
    This is true to a certain extent. If a build lacks the automation required to run without significant human oversight then it will be slow because it is complex. But a normal build should be completely automatic (or at least close to it) and at that point whether it takes 30 seconds or 3 minutes doesn't really matter.
oooyay 3y ago
Is the concept that requiring a demo for developer tools is a negative signal that strange? CI is also notoriously expensive to migrate, which the author alluded to. The pumpkin spice latte of software engineering is disdain for things like Jira and Jenkins but what keeps them in their seat is all the custom integration work that has to be done.
- a1o 3y ago
  The thing with Jenkins is you can also make it custom enough so you have different teams handling different parts (say the building and the deployment, or the releases). In most CIs this is a bit harder to do - I think the Azure Classic pipeline also had interesting ways to delegate responsibilities. In the end the good thing of the different CIs is they apply to different problems and one can use them all at different parts of a company - teams that work in more innovative developments vs teams that maintain products that have to be trusted and adhere to many regulations.
consoomer 3y ago
We work in an industry where senior developers don't know the difference between git and Github. The same seniors don't know the difference between Github actions, Travis CI and have never heard of Jenkins. Hardly anyone cares. Are you going to sell this to a startup? I doubt it. Are you going to sell it to a smb or enterprise company? Maybe. Is there a market to make cash there? Sure. Is it what I would want to make and try selling? Hell no.
- prpl 3y ago
  Enterprise is moving to bazel or similar, often with build caches, and using partial builds on checkin based on what’s changed.
  - no_wizard 3y ago
    really depends on the enterprise. I know of quite a few that have buyers remorse over Bazel because they ended up having to have dedicated engineers to keep it working. There's alot of room for easier to manage Bazel alternatives and I believe enterprises are willing to pay for them.
  - hotnfresh 3y ago
    Enterprises with strong, opinionated, tech org leadership, maybe. The rest are shuffling around between solutions that haven’t been state-of-the-art in 15 years or more, and usually purchased or bundled from some giant vendor or another, as usual. They’re ahead of the curve if they have any standard widely-implemented CI solution that’s fully in use and not one of three (or more) in various stages of being adopted, maintained, and phased out, in fact.
- bluGill 3y ago
  Why should senior devs know that? Someone needs to know those differences, but on a large enterprise project you have a team that keeps the CI system working. The rest need to know how to checkin a file and how to check the CI output. They shouldn't care about those details. I expect any senior developer CAN learn the above things. However that doesn't mean they are worth knowing.
fukpaywalls2 3y ago
Another big "channel" is getting someone to proof "ready" the article.
pshirshov 3y ago
Judging by the description, that project seems to be another DAG-planning meta-program with input-based memoization. This is the superior approach to fight complexity but, unfortunately, most engineers just don't understand the problem and the solution.
- Guvante 3y ago
  It can be superior and not valuable to customers. People who don't have an existing build system don't have money. People who have an existing build system would need to migrate which can be super painful. And nothing in the article points to making it easier.
trumbitta2 3y ago
Stopped at "Imagine you live in a world where no part of the build has to repeat unless the changes actually impacted it." because I'm already in that world by using Nx and Nx Cloud.
- falcor84 3y ago
  I've never heard of Nx before, but what you describe sounds like nix, and I'm confused by the name similarity, as based on their site there appears to be no relation.
  - habitue 3y ago
    Nx came from the angular ecosystem, where "ng" is short for "angular". So there's no relation to nix, it's just coincidence (I use Nx at work and nix at home, and both are great in their respective domains.)
- marcosdumay 3y ago
  I am still convinced the author was trying to describe Make.
  - shepherdjerred 3y ago
    It's similar to Make, in that you're explicitly defining dependencies and that allows caching/parallelization, but much simpler to use, and each target can define its own dependencies (e.g. some target might need Go or NodeJS)
- no_wizard 3y ago
  I must admit I'm a bigger fan of the wireit[0] approach, the only pause I have is its a Google project, my temptation is to fork it. The code isn't terribly complex My biggest complaint with NX is: lack of a sane API for plugins, and it has more overhead than I'd care for. For the amount of complexity that NX has, I'd rather use Rush[1] which gives you everything NX does. My only complaint with Rush is that its development is really slow going, they really need to focus up on Rush plugins (they're good, but still experimental, and I'd love to see them clean up how `autoinstalls` work to be more intutive) I'm on the fence about turbo from Vercel [0]: https://github.com/google/wireit https://github.com/google/wireit [1]: https://rushjs.io/ https://rushjs.io/
chubot 3y ago
It was honestly a bit hard for me to understand what happened from this write-up. I read it a few times, and the terms are still bit confusing. Seems like there are at least two separate issues - migration of the CI configuration -- from existing CI YAML (github/gitlab) to Earthly (Makefile/Dockerfile hybrid) - migration of the job runners -- from existing CI to a service hosted by Earthly I would have thought the first thing is the hard thing. Changing the language could take months or even years. But I don't understand what the blog post was saying. I thought they validated that part? People were able to switch? But then it wasn't validated at the end? The customers were faced with 2 migrations, not 1? But now what's happening? You're still keeping the Earthly syntax but abandoning the CI? Isn't that the hard part to migrate to? I'm still confused.
- Terretta 3y ago
  It's not about migrating the syntax. It's that people's CI over time become some kind of amalgamated model encapsulating how a firm makes every individual piece of software it makes and lands it places, think (ab)using a CI as Airflow (arbitrary automation job runner), and that the migration is first reverse engineering what people used to know, before even starting untangling all of that to express it some different way. Nobody really wants to stop the world and sort that.
  - aftbit 3y ago
    I think the key is that nobody "wants" to spend any time at all on CI or their build system. It is one of those necessary evils that you have to do to have functioning software. Once you do it, even if it's a slow buggy pile of hacks, you will just ignore it until the pain gets very very bad. This was the same problem that caused our team to abandon CircleCI. They kept wanting us to rewrite our CI configs, touting various benefits that we just didn't care about. All we wanted was for our janky broken builds to keep working 90% of the time, like they did before Circle tried migrating this or that.
    - wpietri 3y ago
      Some people do want to spend time on that! It depends whether you see fast feedback as a competitive advantage. I once worked on a system where we had excellent test coverage and all our unit tests ran in under a minute. We worked hard to keep it that way, because it was a huge boon to productivity, especially when improving design issues. It also enabled frequent commits and merges, which made collaboration easier. It was a great experience. But yes, from the business perspective, the leading edge is a pretty small audience. (Circa 15%, per Moore.) And the bottom half of the audience by competence, which Moore called the Late Majority and the Laggards, can be well more than half your revenue in a SaaS context because they will create problems for themselves (like slow/buggy builds) that mean higher per-user revenue. And they may also have special feature needs where they'll pay more rather than clean up their messes. In that context, doing the right thing for your business can be very different than doing the right thing for users. Sort of like a physical therapist who makes most of their money from renting crutches and in-home meal delivery, and so doesn't push people to actually get better.
hardwaregeek 3y ago
At least for me, whenever I have to set up CI or touch CI, it's incredibly painful. I usually end up having to push a bunch of commits to debug stuff, and there's a lot of configuration hell. I don't know if Earthly solves this, perhaps it does. But if I were considering moving onto a new CI, I'd immediately think about having to redo this painful process. And that's at the team level. At the org level, switching CI would be even worse. That does make me wonder, how much of CI adoption is actually moving from one provider to another versus new projects adopting CI for the first time? I'd guess that barring a really extreme reason, most projects pick a CI provider and stick with it.
- mrweasel 3y ago
  CI systems are important to me, but my requirement is pretty much: Can it run a bash script. Every CI I've ever used has degraded into a bash script runner. It starts out well enough, you try to use all the build in features but at some point you fallback to running a shells script and then slowly more and more move to the script, because it's easier. Someone should just build a bash CI, something that can manage secrets and then run one or more shells scripts.
  - dpeckett 3y ago
    In a lot of ways that's pretty much what Earthly is, except those bash scripts run inside containers (that you can configure inside Earthly itself) and you pass files/artifacts between steps/containers.
  - deleted 3y ago
    [deleted]
  - kerblang 3y ago
    Once you have hooks into source control for commit-based callbacks into CI, everything else need only be shell. Like, if you build a triggers-shell-scripts-off-source-repo-hooks web service, does putting the results in a pretty web server really matter for anyone besides executives who want dashboards dashboards dashboards? I'm already sending script output to text files and our chat system via curl to boring web service APIs.
KETpXDDzR 3y ago
> 2-20X faster builds This appears several times over the whole article. What are you comparing to? Without any baseline, this phrase is worthless marketing gibberish.
- shepherdjerred 3y ago
  Uncached, unparalleled builds running on other platforms.
  - fhaldridge7 3y ago
    probably running on shared free runners of other platforms
dpeckett 3y ago
Part of the issue is that the Earthly is pretty much a CI system in itself. Given that the value proposition of a hosted CI product based on it is pretty minimal. It really is a radical mindset shift and I can see why converting legacy organizations is such an uphill battle.
wesleyyue 3y ago
For anyone looking for fast ci, I'd highly recommend webapp.io. it automatically caches layers if it detects it doesn't need to be run, and you can split a VM to shard test execution. Runs way faster than GitHub actions, you can ssh into any layer to debug, and it's just like a Linux machine/dockerfiles so no crazy syntax/mental model to learn. Support is great and we've been super happy with the experience. Best CI platform I've used by far.
- shepherdjerred 3y ago
  That's literally what Earthly does
  - wesleyyue 3y ago
    It's not. Have you tried both? I tried Earthly first and switched to webapp.io. It's a way better product.
SkyPuncher 3y ago
Unless CI is _really_ painful, it's just not that painful. I spend a day cursing about my build when I spin up a new project, then I forget about it for months. Specs run fast enough - especially locally where I only run failing or known-modified specs.
- ajmurmann 3y ago
  It very much depends on what you are building and testing. I work on a distributed database and while our build takes like a minute, the tests take hours and we run them on clusters we start on-demand. Getting our CI faster, has been a constant effort for years that quickly pays for itself.
- yencabulator 3y ago
  Last job where I worked on CI infrastructure, I set up a 200-machine physical server farm, that was then split on-demand into smaller clusters, and those ran tests for a distributed storage product -- including day-long stress tests. If you have trivial needs, trivial solutions are sufficient.
goodpoint 3y ago
5 minutes reading the website and I still don't know how it actually works. Marketing fluff and bits of Earthfile's ugly syntax.
linuxdude314 3y ago
Anyone else find it strange how the author uses CI as a noun? I've never heard anyone refer to a CI platform/system this way before and it sort of strikes me as a red flag. Also the idea of running CI on your laptop seems questionable at best, downright irresponsible and insecure at worst. Sure faster builds are nice, but by the time you're running your code through CI/CD its not usually after adding a few lines of for a feature, it's when the completed feature ready to merge or some sort of integration test is needed.
- xwowsersx 3y ago
  > Anyone else find it strange how the author uses CI as a noun? Do you mean using the term "CI" with no other modifiers/qualifiers? > Also the idea of running CI on your laptop seems questionable at best, downright irresponsible and insecure at worst. Didn't you also use CI as a noun here? And isn't it a noun? > Sure faster builds are nice, but by the time you're running your code through CI/CD its not usually after adding a few lines of for a feature, it's when the completed feature ready to merge or some sort of integration test is needed. I hear you, but not necessarily. You might have a test suite that runs in CI and after you open your PR, you continuing pushing to it and it's nice to have a fast pipeline.
- aftbit 3y ago
  I think the idea is to run the CI on your laptop for debugging and building the CI process itself. This dramatically shortens the build-test cycle (for the CI, not your product). Instead of "edit, git push, wait for runners, download logs, stare confusedly, repeat", you have "edit, run locally, watch logs, stare confusedly, repeat". Plus you can get real access to the actual runners, so maybe you can do things like strace, gdb, etc to see what is going wrong.
- ahtihn 3y ago
  I'm confused why you think fast CI/CD doesn't matter. In an ideal world, every commit you make goes through the full pipeline and ideally it's so fast that you get immediate feedback. The only reason that you're delaying code going through the pipeline is that it's slow.
jordemort 3y ago
I'm glad that they're just shutting down the service and not closing up shop entirely. I really like the tool (to the point that I regularly stop by their careers page); the syntax for an Earthfile is a very sensible and incremental evolution of Dockerfile syntax, and it makes a whole bunch of things easy that were either previously impossible or very awkward with a Dockerfile alone. I remember when Docker introduced BuildKit and buildx, they were trying to push Dockerfiles as a general-purpose build system (i.e. not just for containers but also for producing file artifacts and the like) - Earthly takes that idea and really delivers on it.
- verdverm 3y ago
  Try Dagger, it's better than Earthly I'm a happy user, not paying them yet
  - stavros 3y ago
    In what way? Last I tried both, I could only make Earthly work (though now I'm a fan of CUE because of Dagger).
    - verdverm 3y ago
      Dagger is language SDK based now, sad to see the CUE go myself, but I actually prefer this SDK way now
nbittich 3y ago
A quick look at the documentation and I knew I'll bother using it. We have Jenkins, Drone, woodpecker, GitHub action. Targeting devs feels like it's the worst market to get in, specially after years of VC baked web3/GPT/ terminal/ ide / programming language 300000x faster than python with no tradeoffs offers.
bastardoperator 3y ago
It failed because the marketing is outright bunk and overly dishonest. If I compile with Jenkins, Actions or Earthly, that compile time is going to be the same under each build system assuming the same build node. Claiming you're 20x faster when CI is firing within seconds is kind of meaningless. Caching and parallel execution are age old concepts when it comes to CI and every modern build system can do it. CI is all about feedback, I didn't see much here in terms of collaboration and bubbling data up, I didn't look very hard, but this should be frontline and center. Lastly, I'm not interested in adopting a DSL for my builds ever again, sorry.
- ravivyas 3y ago
  Can you expand on the word "feedback"? What kind of feedback are you looking from CI?
  - bastardoperator 3y ago
    It's one thing to know a build failed, it's another thing to know why and how a build failed, where it failed, and what the cause was. If I have to swim through or context shift to a separate build system's output to get that, what is this actually doing for me?
    - hedora 3y ago
      Also, which of my 1,000 test suites are flaky, and when did they start being flaky? Flamebox almost got this right, but groups by wall-clock instead of git sha: https://flamebox.sourceforge.net https://flamebox.sourceforge.net The screenshot is most of what there is to see. Clicking a test run opens the test log. Does anyone know of a currently-maintained alternative with a less-bad dashboard?
      - Phelinofist 3y ago
        That looks really cool, thanks for the link. Something like this well integrated into Jenkins would be super useful. Our DevOps team build something custom: they check all test logs of all builds of all branches and aggregate all that into a Grafana dashboard. We use it to monitor tests that are failing, to get a better grip on flakiness. Works okay but could be better.
reactordev 3y ago
This is a good write up of why you shouldn’t give away the house when you open source things. The issue was really this: Earthly being open-source, Earthy Satellite users were already seeing the benefit from 95% of Earthly CI. I’m a huge fan of open source, however, if your business model includes an open source model - you need a differentiator. Beyond blazingly fast(tm). You need a reason for people to offer up their credit cards, or better, their accounting PO process. You need a feature you withhold or a service you provide from the open source offering to make it appealing. GitLab restricts CI/CD to paying customers. Travis/CircleCI limit on build time/credits unless also open source. Azure DevOps is the devil. ArgoCD is complicated. GitHub actions are nice if you have hardware to run the runners. Swarm’s of Jenkins is what most are used to in the enterprise. As a former Director of DevOps, my immediate question will be what features make me want to buy vs host myself. I’m technically capable. Why should I buy with you when I can run my cloud and my DevOps pipelines for my workforce the way that makes the most sense for my business? Help me make the most sense. *EDIT* I'm not suggesting withholding features that are critical for the software to operate, but withholding business features that provide support or enterprise-level integration. Even making it tiered so the power users pay less because they support it more themselves?
- hhh 3y ago
  I feel almost certain someone could wrap ArgoCD into a product if they handled the 'patternization' of applicationsets. D2iQ already do it with Flux, but we bailed on D2iQ before we even got to try it.
  - gabereiser 3y ago
    ArgoCD is almost worthless if your DevOps teams are IaC wielders. Honestly, just a bash script to push a container to the pods is all that’s required for DevOps deployment. A build pipeline should be get artifacts -> put artifacts in container -> push container to registry -> trigger a pull on your pods. If you didn’t have a standard cloud architecture, I can understand the frustration and reasoning behind looking at ArgoCD. If you have a standardized cloud practice, it’s moot.
skinner927 3y ago
> Write once, run anywhere Except virtual machines of course. Does everyone only build web apps? What tools do systems developers use? You can’t reliably test system integration in a container.
- palata 3y ago
  > Does everyone only build web apps? Unfortunately I think most people do, which is why there is a big bias towards web tech. I don't, and I hate the fact that most people seem to think that web tech solves everything :-).
seanr88 3y ago
I'm having a very similar issue getting https://brisktest.com/ https://brisktest.com/ out there. People complain about things like the speed of their build system, but convincing people that your CI system is faster just seems like an uphill battle. Maybe the era of launching new indie dev tools to developers is over and you just need a massive marketing budget to bombard devs with advertising and lots of sponsored content.
jatins 3y ago
> We ended up with the most ridiculous negative qualification criteria I have ever heard of: if the prospect requires a demo, then they’re not worth going after Based on my experience at enterprise companies, this is _all_ enterprise sales. So sounds very counter intuitive to me.
theptip 3y ago
This is a good write-up, and contains some well-reasoned PMF and product strategy evaluations; grokking the thought process here will help you to survive in your own startup.
jameshart 3y ago
Tl;dr they made the mistake of assuming that since 95% of the time CI jobs spend is ‘running builds’ that a build system is 95% of a CI platform. This turns out not to be true.
- sharts 3y ago
  Who says ex-googlers aren't the sharpest tools in the shed?
pharmakom 3y ago
The issue is trust. I dislike trusting GitHub / GitLab with my code but I already do, so I choose them for CI. I can already get fast builds using Bazel (or maybe Buck 2) so why do I need another tool / provider?
- shepherdjerred 3y ago
  If Bazel works for you that's great, although it is still rather complex. For smaller projects Earthly is a better fit.
- maxloh 3y ago
  GitLab is open sourced. You can self-host GitLab (and their CI solution) yourself.
shepherdjerred 3y ago
I've used Earthly for a year and absolutely love it. It allows you to have two dependencies on the host system for any project -- just Docker and Earthly. I've used it to build cross-compilers, system software, web apps, my resume, school projects. The speed up is real. You can definitely get caching and parallelization and containizeration with other build systems, but Earthly's value is that it comes for free with a familiar syntax.
- 12_throw_away 3y ago
  Ditto, it's been a huge success where I work. Keeping my fingers crossed that they find a way to stay in business :)
  - dpeckett 3y ago
    I feel like it's just a patience game of staying alive long enough that organic traction takes off. Earthly is bloody fantastic, the flexibility and expressiveness IMO is hard to beat.
CodeL 3y ago
Innovation isn't just about speed; it's about understanding the nuances of user needs and market dynamics. Even the fastest CI can face hurdles if it doesn't align with the broader ecosystem's expectations and requirements.
- mcnichol 3y ago
  This right here. I have found some of the greatest products I have used came from: "I'm working on this project, the tools I currently have are just different forms of a hammer...I need a screwdriver. I built a screwdriver. This screwdriver is the only tool I'll consider when I face this situation." The tools sell because the people doing the work and feeling the pain react strongly while pointing at the incompetent layers above with purchasing power. Giving (or showing) someone the solution and then taking it away is probably the most common reason I have seen someone leave a situation.
  - CodeL 3y ago
    Absolutely. It's often those who are in the trenches, dealing with the day-to-day challenges, who recognize the need for better tools. They craft solutions tailored to their specific needs, which often resonate with others facing similar challenges. For instance: - Slack emerged because teams needed a more efficient way to communicate than traditional email chains. It was built as a communication tool for a gaming company and then pivoted to become the go-to messaging app for businesses. - Docker was developed out of the frustration of "it works on my machine" syndrome. It provided a consistent environment, making software deployment more predictable and efficient. - Trello was born out of the need for a visual project management tool that was intuitive and easy to use, unlike complex project management software. These tools became popular not just because they were innovative, but because they addressed real pain points felt by those on the ground. When decision-makers overlook these grassroots solutions, they risk alienating their most valuable assets: the people doing the work.
    - mcnichol 3y ago
      Couldn't have said it better. I have loved each one of those tools too. Docker maybe a little more love-hate but I think that has a lot more to do with the politics and salesy side of things and "Who owns what"
progx 3y ago
You get customer when you solve a real problem for them. CI-Systems exists, slow or fast doesn't be a problem for most of the devs, cause you run normally everything local and only on publish in a remote CI-system. So CI-Systems exists and solve their problem. I think only for a really small market the CI-performance is a real pain. Sure, everbody wants it fast, but that is not a reason to change a huge complex setup.
mcnichol 3y ago
Not to rain on the parade here but this is literally a copy-pasta product. Jenkins, Google Borg, Cloud Foundry, Concourse Pipelines...and surprise surprise when you look at where he came from...Ex-Google, Ex-VMW, RabbitMQ... If OP wasn't in and around the source of all these tools above then they were at the very least first cousins to the story. The sales cycle is long, integrations require multiple dimensions of executive buy-in, especially security and networking. I think they made something nice but it felt like a nothing-burger story about something that is constantliy oscillating between bespoke and commoditized due to upstream problems that are such a mix of issues. Lack of oversight to micromanagement, inexperience to too much experience that they cannot let go of "the way it's always done". It may be just my unpopular opinion but you are boiling an ocean of problems selling toolchains. Business and Tech are like water often finding the holes that lead to a path of least resistance, even when these erode the foundation of "core business". Toolchains that behave like guidelines and "parenting strategies" with removable guardrails have always offered the greatest rewards in my experience.
- rewmie 3y ago
  > Not to rain on the parade here but this is literally a copy-pasta product. More to the point, the product offered no compelling reason to use it, let alone pay for it. Being "fast" is not a selling point. Developers don't want slow pipelines, but that does not mean they want fast pipelines. The speed that the pipeline works is more depending on how the pipeline is setup than the overhead of the pipeline service, and other CICD services are already blazing fast. Take for instance CircleCI, which is already a tough sell over GitHub actions and GitLab CICD: how did the service stood out? Did it added tangible value over GitHub/Gitlab/CircleCI/etc? Shaving milliseconds off a minutes-long build ain't it.
  - pixl97 3y ago
    Reliable, configurable, and compatible. That's what, at least from the work I do, seems to be what the market is looking for. Most CI/CD for sizeable companies have integrations into other systems, such as static analysis and things lime SBOM. Those tools tend to take a bunch of time outside the CI. They also need to ensure the results from those tools can break builds along with other alerting. Eventually the CI itself becomes a small fraction of total execution time.
Anon_Forever 3y ago
Sometimes the best technology fails and the space is overtaken by lesser technologies. It can be something as silly and out of your control as timing. Maybe the competition came out a bit after you and grew faster as interest in the space rose astronomically in parallel.
kunalgupta 3y ago
Swapping single words in my pitches has taught me a lot about what people want
bluepuma77 3y ago
Why have examples with JavaScript, when no build process is needed? Why not use TypeScript or even something like SvelteKit, which has a raving fan-base which is obviously working with bleeding edge tech that is still rough around the edges?
davinci123 3y ago
I think the biggest moral of the story - don't build a product without considering the switching/adoption cost. Higher the switching/adoption cost, lower the product/market fit. I have heard this story again and again from so many startups who have built products which provide incremental benefits (1.3x cheaper or 1.5x faster) and not getting enough traction.
skybrian 3y ago
This article is confusing because the author is too close to the problem, and it needs to be explained from an outsider’s point of view before diving into the details. But it looks like the information is there. It sounds to me like they ended up with an encapsulation layer between language-specific build systems and the continuous build systems that kick them off. A comparison might be with something like Bazel, which does everything, but full adoption requires going all-in on using their build language to replace language-specific build systems, often including moving source files to make it work. This feels foreign compared to using a language’s own build system, but it might seem natural for C programmers, where the language doesn’t have its own build system. Similarly, Java went through several unfortunate build systems before unfortunately settling on Gradle. Language-specific build systems end up not doing everything because each language has its own conventions and ecosystem. Modern ones don’t try, they know what they’re good at and stick to it. So, the tool that actually knows about multiple languages and their artifacts is often a shell script, makefile, docker file, or the continuous build itself. Or even doing it by hand. That’s the layer they’re trying to improve. I don’t have a clear idea what their fundamental insight is that makes their way of doing it better, though.
unbalancedevh 3y ago
It really bugs me when authors use acronyms or abbreviations without expanding what they mean. It tells the reader, "I didn't write this article for you, so don't bother trying to understand it."
- fydorm 3y ago
  Which terms are you referring to? If you aren't familiar with "CI/CD", then this article really isn't for you, as you haven't run into the problems this startup is trying to solve.
  - unbalancedevh 3y ago
    Yes, CI/CD. It would be nice to know in the first paragraph that the article isn't for me, instead of having to read 1/3 of the article and then infer it, or even keep guessing what it's about. Sometimes that's fine, but I seem to be seeing it more often, and it discourages people new to the field or who are still exploring.
_whiteCaps_ 3y ago
> Imagine you live in a world where no part of the build has to repeat unless the changes actually impacted it. A world in which all builds happened with automatic parallelism. A world in which you could reproduce very reliably any part of the build on your laptop. That sounds similar to https://concourse-ci.org/ https://concourse-ci.org/ I quite like it, but it never seemed to gain traction outside of Cloud Foundry.
- AlphaWeaver 3y ago
  We spent some time evaluating this on my team, and we're still experimenting with it. I like it a lot, but the project appears to be mostly unmaintained since mid-2021, when the creator left it to work on a lispy CI/CD tool [0] that feels very complicated... not sure what's going on there. [0]: https://github.com/vito/bass https://github.com/vito/bass
  - gavmor 3y ago
    I enjoyed working with concourse as the foundations of Pivotal's release engineering systems. We tried several other platforms, but ultimately concourse was tailor-made for Pivotal-style dev/ops ie highly technical, prioritizing version-control and CLIs over GUIs, with the wall-mounted "build monitor" as a first-class user. While Concourse wasn't explicitly a Pivotal product, shouldn't Conway's Law encourage us not to understate the influences of the Pivotal milieu? I don't doubt that VMware failed to nurture Concourse's maintenance post-acquisition. I hesitate to make sweeping statements about the legendary virtualization behemoth, but I personally doubt their commitment XP dev/ops culture. Edit: don't take my word for it, read someone more closely involved to this day, Nat Bennett https://www.simplermachines.com/some-facts-about-concourse/ https://www.simplermachines.com/some-facts-about-concourse/
  - natbennett 3y ago
    There are folks still working on it but at this point it pretty much only gets updates that VMware needs to continue delivering Cloud Foundry. It looks dead-er than it is because the team hasn't explained this, so there's a giant list of work that won't ever get done just kinda... hanging out... on the README and in the issues list. I use it at work but it was already running when I got there. I also really like it but I'd be reluctant to stand it up somewhere new unless I was using it to run a Cloud Foundry. It has some operational issues and it is actually impossible to get an SLA if you're not also a VMware Cloud Foundry customer, and that costs a minimum of $50k per year.
ilyt 3y ago
> Why not just simplify the stack? Why pay both the CI vendor and us, when you can just pay us? Coz the CI they "pay" for comes with rest of the stack, and is not just CI product. Both Gitlab and Github have far more than just CI >These weren’t the raving fans we were used to talking with. New people would look at Earthly CI with a skeptical eye. They were mostly thinking that “all CIs are the same - they just have different syntax,” and then they would not really look any further as to why we might actually be different. we don't care about CI, we just need it and we need it to work. If we have working CI manifest for our app, the new app will have same manifest with few find-replace in it. The pain might be there to create it. And the example ones don't really look any easier than doing same thing in Gitlab. Frankly, putting some transpiler that takes Gitlab or Github CI config and just makes Earthfile on the fly would probably convince some people to at least try.
0xbadcafebee 3y ago
There is no real business advantage or value add to switching CI/CD systems. The only people that do it no longer have anyone working for them that knows how the existing CI/CD system works, so the new person just replaces it with something else. They're all basically the same crap. Run a job, evaluate its output, maybe run another job, send a notification. Whenever you run into a problem with a CI/CD system, it's because the system is "opinionated" and "tightly integrated", but of course lacks some obscure and subtle capability that you need to solve a problem. The fact that you are using a specific CI/CD system is what dooms you to eventually need another one to solve the problem that you current one cannot solve. It's ridiculous that here we are in 2023 and nobody has completed an open source project that replaces general use CI & CD (for modern best practices & use cases; Jenkins is a curse from the pit of hell). Both CI & CD are just loops of jobs and logic that then decides what next job to run. What those jobs do are all different, but so what? Make an open source library of jobs and logic. (for what it's worth, Drone.io is the best system I've ever used, and WoodpeckerCI is the open source equivalent, but it doesn't have a large community so work is very slow)
pxc 3y ago
I looked briefly at Earthly a while ago for work because I have to write CI integrations for a whole bunch of CI platforms (GitLab, Azure DevOps, Jenkins, and maybe GitHub Actions). I ended up going with a very similar product (i.e., another nice BuildKit frontend with integrations into a bunch of CI systems), Dagger, because the DSL it used for defining pipelines at the time seemed better to me. I ended up deeply regretting this decision, as the Dagger devs have de facto abandoned that language entirely in favor of churning out SDKs for the most popular programming languages— all imperative, all Turing-complete, and all, imo, bad fits for the space. So now I'm back in hell, integrating with all of these systems by hand, hesitant to trust this kind of tooling to a startup again. To me this is still the most appealing use case of something like Earthly. Push-and-see-what-happens is still the norm on almost all CI systems, and it's an abominable workflow. If you are at a large organization and have to support teams that use a wide variety of CI/CD platforms, something like Earthly could potentially save quite a lot of pain. But the appeal is precisely in supporting existing CI platforms, rather than in adding another one.
- verdverm 3y ago
  CI/CD & provisioning are inherently order driven tasks, I actually prefer the SDKs over the CUE implementation. The main issue with the old CUE implementation is that they had to try and align BuildKits DAG solver with CUE's DAG solver, and they worked in opposite ways. I tried to help them through this about 3 years ago (being a CUE expert). I definitely think the SDKs is a much better solution for them. Whether we like it or not, a lot of the industry is moving this way. Pulumi is another example of this, though I'm sold on imperative cloud infra, builds do seem to have alignment here. Side note, I will be exploring a new CUE + Dagger setup, though it will work differently than the old Dagger engine
  - pxc 3y ago
    I'm not totally opposed to powerful (Turing-complete) languages being used for configuration or defining builds— I think Nix works really well in that space, and someday Nickel may as well. But wherever I've seen build tools from a language ecosystem that just embed a DSL in the host language (e.g.: sbt, Gradle), I've also seen projects get way too clever with their builds and create total nightmares for downstream packaging efforts. I anticipate similar problems for other kinds of configuration tools based on general-purpose languages, like CI or cloud IaC. > Side note, I will be exploring a new CUE + Dagger setup, though it will work differently than the old Dagger engine Sounds cool! I'll keep an eye out for it. :)
verdverm 3y ago
There are a bunch of these next-gen CI / container building systems. They all pretty much build on BuildKit like Docker, but unlock way more of the potential, namely DAG layers (think multi-stage docker on steroids), and better / more granular caching. These next-gen CI tools are fighting to be the UX on top of BuildKit. Dagger is where it is at, and it's built by the creators of Docker, based on their lessons learned. If you're looking to try something out, try Dagger. You write in the same languages you do every day, no need for some weird syntax like Earthly or the flatness of a Dockerfile. Their CLI/TUI is pretty sweet too, but you don't even need it.
gwbas1c 3y ago
Now is a good time to read "Crossing the Chasm:" https://en.wikipedia.org/wiki/Crossing_the_Chasm https://en.wikipedia.org/wiki/Crossing_the_Chasm The book talks about the state of the business Earthly CI was in: They tried to go from enthusiasts to generalists... But with a completely new product! I don't think it was wrong to build a new CI; instead, I think the thing they did "wrong" was build a super-complicated CI; or that when they tried to cross the chasm, they tried to sell a (cough) completely different product than what their early users were using.
js4ever 3y ago
Writing a new DSL is doomed and OP started by that. You will have to bend the world to make devs using it. Give them something they want, and no one wants to learn a new DSL and rewrite everything.
plg 3y ago
Dumb question I guess, what is a CI???
augustk 3y ago
CI = Continuous Integration https://en.wikipedia.org/wiki/Continuous_integration https://en.wikipedia.org/wiki/Continuous_integration
px1999 3y ago
> it didn’t make sense for them to jump head-first into an expensive and time-consuming migration effort just because some startup they had never heard of promised to deliver the sun and the moon > we need to shut down Earthly CI Is it possible that it all boils down to these two points? The technical benefits don't matter if there's even a hint that your (fundamental to my business') service could fail and be removed. When I read this it sounds like a technically great idea which solved a problem in a way that missed some fundamental prerequisite for their target user group (and hence success).
Klugistiono 3y ago
I'm quite passioned about CI/CD and working in this field for the last 7 years. I'm keeping a close eye on jenkins, tekton, circleci, travis, github actions, gitlab etc. and never ever heard about earthly. Btw. one thing the person doesn't get: Everyone wants to have fast and reliable builds but no one cares. No one. CI/CD Systems in a developers eye are blackboxes. They don't care. And its already a solved problem somehow. I was waiting for Jenkins X to happen (something like jenkins working nicely in a cloud native way because Jenkins Plugins!) but that didn't happen. What happend instead is Github Actions. Holy shit are Github Actions good. Srsly. GitLab stuff was slow and effort. GitHub Actions just do what they should do 99% of the time. If you do not have a huge build, you will not care about your CI/CD System and everyone who has real build issues, has to solve it by another strategy than just switching the build system.
- throwawaaarrgh 3y ago
  GitHub Actions is the same as everything else. It doesn't work any better or worse. But the UI is clunkier, you can do less with it than other systems, it lacked important features forever (still does). The only reason you think it's significantly better is psychological. The "market" of actions is better than competitors because the incumbent gets an advantage, and it trades on the idea of a generic executable as a 3rd party plugin, unlike other systems that try to force you into a particular language or paradigm. But other systems do this too.
  - Klugistiono 3y ago
    I build enough pipelines in GitLab way to be able to tell you that GitHub Actions is much easier for me to implement. Its much closer than if you would have a second system like jenkins, i already have access to git and the packages. I do use much more container images than i did 6 or 10 years ago, that might have something to do with it but nonetheless, i do prefer GitHub Actions. I also stoped liking GitLab though. The Autodevops was horrendes, the security issues is concerning and plenty of ideas or features are half done. this is also something which changed. I liked GitLab much more than i do now. But looking at the year, i haven't looked at GitLab for at least 2 years now.
Tao3300 3y ago
Slow CI is a perk. https://xkcd.com/303 https://xkcd.com/303
stackzero 3y ago
Change the name to something short that sounds like it can be part of my stack and existed in linux forever e.g. ert Earthly sounds too mid 2010s and something extra/startupy
sharts 3y ago
[flagged]

"People will buy a developer tool, but you can’t sell it."

deepspace 3y ago
That line caught my attention too, and it looked superficially true. I have certainly never purchased a tool based on a phone call from a vendor. However, that only pertains to cold-calling, which is the absolute worst way to sell anything. Advertising is a trillion dollar market for a reason – it works. The trick is to find the correct message and media for the audience. Effective advertising takes a lot of skill, a lot of money, or both. A large portion of HN posts (including this one) are contain "hidden" advertising, and they absolutely work. I have purchased more than one developer tool based on reddit and HN reviews, which originated as posts by the seller. On the "lot of money" side, a poster in the arrivals tunnel at SFO is an excellent way of raising brand awareness; even for developer tools.
wrs 3y ago
For me the killer quote was “if the prospect requires a demo, then they’re not worth going after” — because as a prospect, if your product requires a demo, I’m already halfway to no. I am so tired of “enterprise” products that have to be purchased as a pig in a poke because they can’t stand on their own during a trial period.
nathanfig 3y ago
I think HN will eat this up because few of us enjoy being sold to but like most absolute statements it's probably only applicable in specific circumstances. Enterprises sell tons of developer tools and move millions because of demos done well. AWS and Azure are definitely getting on planes and doing demos.

I have no personal experience with selling a SaaS, but read that AWS mostly sold to startups at first because there were no migration costs yet. Out of curiosity, has the same been attempted with Eathly and, if yes, did that work?

alexvitkov 3y ago

What does fast CI even mean? CI is an overgrown shell script running your build and telling you when it fails. In general build tooling has gotten so slow that the cost of whatever CI runner relative to it should be nil. If you want fast CI you need fast tsc, clang, rustc, etc... not a faster program that calls exec on them. A bit more on topic, if you're selling CI and your business fails it's because people you're providing no value - people can run their build scripts fine without you.

jve 3y ago
> If you want fast CI you need fast tsc, clang, rustc, etc... not a faster program that calls exec on them Yes and no. Caching stuff and knowing when to run tsc/clang/rustc also improves performance.
- deleted 3y ago
  [deleted]
- bluGill 3y ago
  Sure, but you have to be 100% perfect in guessing when the cache file must be rebuilt. If there is any situation at all where you use the cached file when you should have rebuilt your tool is not useful. Note that you can go too far. If you rebuild a file on linux even though the only changes were in a "#ifdef freebsd" (windows, mac...) section that is a waste of time, but not harmful. However we already have tools that do that.
  - esafak 3y ago
    This is a solved problem; modern build tools and workflow orchestrators face the same challenge. You declare your inputs for a given task and cache the output as long as the inputs are unchanged, as determined by their hash. So there is no wrong guessing, only an incomplete specification of inputs. It's an elegant solution, if I do say so, and has worked well at companies I've seen it used.
    - 3y ago

shuntress 3y ago

> Why pay both the CI vendor and us Exactly. Complexity, not speed, is the problem with builds. It makes perfect sense that "another layer of complexity that wraps up all the other layers" was tough to sell.

pshirshov 3y ago
In this particular domain, performance problems are tightly coupled with complexity.
- shuntress 3y ago
  This is true to a certain extent. If a build lacks the automation required to run without significant human oversight then it will be slow because it is complex. But a normal build should be completely automatic (or at least close to it) and at that point whether it takes 30 seconds or 3 minutes doesn't really matter.

oooyay 3y ago

Is the concept that requiring a demo for developer tools is a negative signal that strange? CI is also notoriously expensive to migrate, which the author alluded to. The pumpkin spice latte of software engineering is disdain for things like Jira and Jenkins but what keeps them in their seat is all the custom integration work that has to be done.

a1o 3y ago
The thing with Jenkins is you can also make it custom enough so you have different teams handling different parts (say the building and the deployment, or the releases). In most CIs this is a bit harder to do - I think the Azure Classic pipeline also had interesting ways to delegate responsibilities. In the end the good thing of the different CIs is they apply to different problems and one can use them all at different parts of a company - teams that work in more innovative developments vs teams that maintain products that have to be trusted and adhere to many regulations.

consoomer 3y ago

We work in an industry where senior developers don't know the difference between git and Github. The same seniors don't know the difference between Github actions, Travis CI and have never heard of Jenkins. Hardly anyone cares. Are you going to sell this to a startup? I doubt it. Are you going to sell it to a smb or enterprise company? Maybe. Is there a market to make cash there? Sure. Is it what I would want to make and try selling? Hell no.

prpl 3y ago
Enterprise is moving to bazel or similar, often with build caches, and using partial builds on checkin based on what’s changed.
- no_wizard 3y ago
  really depends on the enterprise. I know of quite a few that have buyers remorse over Bazel because they ended up having to have dedicated engineers to keep it working. There's alot of room for easier to manage Bazel alternatives and I believe enterprises are willing to pay for them.
- hotnfresh 3y ago
  Enterprises with strong, opinionated, tech org leadership, maybe. The rest are shuffling around between solutions that haven’t been state-of-the-art in 15 years or more, and usually purchased or bundled from some giant vendor or another, as usual. They’re ahead of the curve if they have any standard widely-implemented CI solution that’s fully in use and not one of three (or more) in various stages of being adopted, maintained, and phased out, in fact.
bluGill 3y ago
Why should senior devs know that? Someone needs to know those differences, but on a large enterprise project you have a team that keeps the CI system working. The rest need to know how to checkin a file and how to check the CI output. They shouldn't care about those details. I expect any senior developer CAN learn the above things. However that doesn't mean they are worth knowing.

fukpaywalls2 3y ago

Another big "channel" is getting someone to proof "ready" the article.

pshirshov 3y ago

Judging by the description, that project seems to be another DAG-planning meta-program with input-based memoization. This is the superior approach to fight complexity but, unfortunately, most engineers just don't understand the problem and the solution.

Guvante 3y ago
It can be superior and not valuable to customers. People who don't have an existing build system don't have money. People who have an existing build system would need to migrate which can be super painful. And nothing in the article points to making it easier.

trumbitta2 3y ago

Stopped at "Imagine you live in a world where no part of the build has to repeat unless the changes actually impacted it." because I'm already in that world by using Nx and Nx Cloud.

falcor84 3y ago
I've never heard of Nx before, but what you describe sounds like nix, and I'm confused by the name similarity, as based on their site there appears to be no relation.
- habitue 3y ago
  Nx came from the angular ecosystem, where "ng" is short for "angular". So there's no relation to nix, it's just coincidence (I use Nx at work and nix at home, and both are great in their respective domains.)
marcosdumay 3y ago
I am still convinced the author was trying to describe Make.
- shepherdjerred 3y ago
  It's similar to Make, in that you're explicitly defining dependencies and that allows caching/parallelization, but much simpler to use, and each target can define its own dependencies (e.g. some target might need Go or NodeJS)
no_wizard 3y ago
I must admit I'm a bigger fan of the wireit[0] approach, the only pause I have is its a Google project, my temptation is to fork it. The code isn't terribly complex My biggest complaint with NX is: lack of a sane API for plugins, and it has more overhead than I'd care for. For the amount of complexity that NX has, I'd rather use Rush[1] which gives you everything NX does. My only complaint with Rush is that its development is really slow going, they really need to focus up on Rush plugins (they're good, but still experimental, and I'd love to see them clean up how `autoinstalls` work to be more intutive) I'm on the fence about turbo from Vercel [0]: https://github.com/google/wireit https://github.com/google/wireit [1]: https://rushjs.io/ https://rushjs.io/

chubot 3y ago

It was honestly a bit hard for me to understand what happened from this write-up. I read it a few times, and the terms are still bit confusing. Seems like there are at least two separate issues - migration of the CI configuration -- from existing CI YAML (github/gitlab) to Earthly (Makefile/Dockerfile hybrid) - migration of the job runners -- from existing CI to a service hosted by Earthly I would have thought the first thing is the hard thing. Changing the language could take months or even years. But I don't understand what the blog post was saying. I thought they validated that part? People were able to switch? But then it wasn't validated at the end? The customers were faced with 2 migrations, not 1? But now what's happening? You're still keeping the Earthly syntax but abandoning the CI? Isn't that the hard part to migrate to? I'm still confused.

Terretta 3y ago
It's not about migrating the syntax. It's that people's CI over time become some kind of amalgamated model encapsulating how a firm makes every individual piece of software it makes and lands it places, think (ab)using a CI as Airflow (arbitrary automation job runner), and that the migration is first reverse engineering what people used to know, before even starting untangling all of that to express it some different way. Nobody really wants to stop the world and sort that.
- aftbit 3y ago
  I think the key is that nobody "wants" to spend any time at all on CI or their build system. It is one of those necessary evils that you have to do to have functioning software. Once you do it, even if it's a slow buggy pile of hacks, you will just ignore it until the pain gets very very bad. This was the same problem that caused our team to abandon CircleCI. They kept wanting us to rewrite our CI configs, touting various benefits that we just didn't care about. All we wanted was for our janky broken builds to keep working 90% of the time, like they did before Circle tried migrating this or that.
  - wpietri 3y ago
    Some people do want to spend time on that! It depends whether you see fast feedback as a competitive advantage. I once worked on a system where we had excellent test coverage and all our unit tests ran in under a minute. We worked hard to keep it that way, because it was a huge boon to productivity, especially when improving design issues. It also enabled frequent commits and merges, which made collaboration easier. It was a great experience. But yes, from the business perspective, the leading edge is a pretty small audience. (Circa 15%, per Moore.) And the bottom half of the audience by competence, which Moore called the Late Majority and the Laggards, can be well more than half your revenue in a SaaS context because they will create problems for themselves (like slow/buggy builds) that mean higher per-user revenue. And they may also have special feature needs where they'll pay more rather than clean up their messes. In that context, doing the right thing for your business can be very different than doing the right thing for users. Sort of like a physical therapist who makes most of their money from renting crutches and in-home meal delivery, and so doesn't push people to actually get better.

hardwaregeek 3y ago

At least for me, whenever I have to set up CI or touch CI, it's incredibly painful. I usually end up having to push a bunch of commits to debug stuff, and there's a lot of configuration hell. I don't know if Earthly solves this, perhaps it does. But if I were considering moving onto a new CI, I'd immediately think about having to redo this painful process. And that's at the team level. At the org level, switching CI would be even worse. That does make me wonder, how much of CI adoption is actually moving from one provider to another versus new projects adopting CI for the first time? I'd guess that barring a really extreme reason, most projects pick a CI provider and stick with it.

mrweasel 3y ago
CI systems are important to me, but my requirement is pretty much: Can it run a bash script. Every CI I've ever used has degraded into a bash script runner. It starts out well enough, you try to use all the build in features but at some point you fallback to running a shells script and then slowly more and more move to the script, because it's easier. Someone should just build a bash CI, something that can manage secrets and then run one or more shells scripts.
- dpeckett 3y ago
  In a lot of ways that's pretty much what Earthly is, except those bash scripts run inside containers (that you can configure inside Earthly itself) and you pass files/artifacts between steps/containers.
- deleted 3y ago
  [deleted]
- kerblang 3y ago
  Once you have hooks into source control for commit-based callbacks into CI, everything else need only be shell. Like, if you build a triggers-shell-scripts-off-source-repo-hooks web service, does putting the results in a pretty web server really matter for anyone besides executives who want dashboards dashboards dashboards? I'm already sending script output to text files and our chat system via curl to boring web service APIs.

KETpXDDzR 3y ago

> 2-20X faster builds This appears several times over the whole article. What are you comparing to? Without any baseline, this phrase is worthless marketing gibberish.

shepherdjerred 3y ago
Uncached, unparalleled builds running on other platforms.
- fhaldridge7 3y ago
  probably running on shared free runners of other platforms

dpeckett 3y ago

Part of the issue is that the Earthly is pretty much a CI system in itself. Given that the value proposition of a hosted CI product based on it is pretty minimal. It really is a radical mindset shift and I can see why converting legacy organizations is such an uphill battle.

wesleyyue 3y ago

For anyone looking for fast ci, I'd highly recommend webapp.io. it automatically caches layers if it detects it doesn't need to be run, and you can split a VM to shard test execution. Runs way faster than GitHub actions, you can ssh into any layer to debug, and it's just like a Linux machine/dockerfiles so no crazy syntax/mental model to learn. Support is great and we've been super happy with the experience. Best CI platform I've used by far.

shepherdjerred 3y ago
That's literally what Earthly does
- wesleyyue 3y ago
  It's not. Have you tried both? I tried Earthly first and switched to webapp.io. It's a way better product.

SkyPuncher 3y ago

Unless CI is _really_ painful, it's just not that painful. I spend a day cursing about my build when I spin up a new project, then I forget about it for months. Specs run fast enough - especially locally where I only run failing or known-modified specs.

ajmurmann 3y ago
It very much depends on what you are building and testing. I work on a distributed database and while our build takes like a minute, the tests take hours and we run them on clusters we start on-demand. Getting our CI faster, has been a constant effort for years that quickly pays for itself.
yencabulator 3y ago
Last job where I worked on CI infrastructure, I set up a 200-machine physical server farm, that was then split on-demand into smaller clusters, and those ran tests for a distributed storage product -- including day-long stress tests. If you have trivial needs, trivial solutions are sufficient.

goodpoint 3y ago

5 minutes reading the website and I still don't know how it actually works. Marketing fluff and bits of Earthfile's ugly syntax.

linuxdude314 3y ago

Anyone else find it strange how the author uses CI as a noun? I've never heard anyone refer to a CI platform/system this way before and it sort of strikes me as a red flag. Also the idea of running CI on your laptop seems questionable at best, downright irresponsible and insecure at worst. Sure faster builds are nice, but by the time you're running your code through CI/CD its not usually after adding a few lines of for a feature, it's when the completed feature ready to merge or some sort of integration test is needed.

xwowsersx 3y ago
> Anyone else find it strange how the author uses CI as a noun? Do you mean using the term "CI" with no other modifiers/qualifiers? > Also the idea of running CI on your laptop seems questionable at best, downright irresponsible and insecure at worst. Didn't you also use CI as a noun here? And isn't it a noun? > Sure faster builds are nice, but by the time you're running your code through CI/CD its not usually after adding a few lines of for a feature, it's when the completed feature ready to merge or some sort of integration test is needed. I hear you, but not necessarily. You might have a test suite that runs in CI and after you open your PR, you continuing pushing to it and it's nice to have a fast pipeline.
aftbit 3y ago
I think the idea is to run the CI on your laptop for debugging and building the CI process itself. This dramatically shortens the build-test cycle (for the CI, not your product). Instead of "edit, git push, wait for runners, download logs, stare confusedly, repeat", you have "edit, run locally, watch logs, stare confusedly, repeat". Plus you can get real access to the actual runners, so maybe you can do things like strace, gdb, etc to see what is going wrong.
ahtihn 3y ago
I'm confused why you think fast CI/CD doesn't matter. In an ideal world, every commit you make goes through the full pipeline and ideally it's so fast that you get immediate feedback. The only reason that you're delaying code going through the pipeline is that it's slow.

jordemort 3y ago

I'm glad that they're just shutting down the service and not closing up shop entirely. I really like the tool (to the point that I regularly stop by their careers page); the syntax for an Earthfile is a very sensible and incremental evolution of Dockerfile syntax, and it makes a whole bunch of things easy that were either previously impossible or very awkward with a Dockerfile alone. I remember when Docker introduced BuildKit and buildx, they were trying to push Dockerfiles as a general-purpose build system (i.e. not just for containers but also for producing file artifacts and the like) - Earthly takes that idea and really delivers on it.

verdverm 3y ago
Try Dagger, it's better than Earthly I'm a happy user, not paying them yet
- stavros 3y ago
  In what way? Last I tried both, I could only make Earthly work (though now I'm a fan of CUE because of Dagger).
  - verdverm 3y ago
    Dagger is language SDK based now, sad to see the CUE go myself, but I actually prefer this SDK way now

nbittich 3y ago

A quick look at the documentation and I knew I'll bother using it. We have Jenkins, Drone, woodpecker, GitHub action. Targeting devs feels like it's the worst market to get in, specially after years of VC baked web3/GPT/ terminal/ ide / programming language 300000x faster than python with no tradeoffs offers.

bastardoperator 3y ago

It failed because the marketing is outright bunk and overly dishonest. If I compile with Jenkins, Actions or Earthly, that compile time is going to be the same under each build system assuming the same build node. Claiming you're 20x faster when CI is firing within seconds is kind of meaningless. Caching and parallel execution are age old concepts when it comes to CI and every modern build system can do it. CI is all about feedback, I didn't see much here in terms of collaboration and bubbling data up, I didn't look very hard, but this should be frontline and center. Lastly, I'm not interested in adopting a DSL for my builds ever again, sorry.

ravivyas 3y ago
Can you expand on the word "feedback"? What kind of feedback are you looking from CI?
- bastardoperator 3y ago
  It's one thing to know a build failed, it's another thing to know why and how a build failed, where it failed, and what the cause was. If I have to swim through or context shift to a separate build system's output to get that, what is this actually doing for me?
  - hedora 3y ago
    Also, which of my 1,000 test suites are flaky, and when did they start being flaky? Flamebox almost got this right, but groups by wall-clock instead of git sha: https://flamebox.sourceforge.net https://flamebox.sourceforge.net The screenshot is most of what there is to see. Clicking a test run opens the test log. Does anyone know of a currently-maintained alternative with a less-bad dashboard?
    - Phelinofist 3y ago
      That looks really cool, thanks for the link. Something like this well integrated into Jenkins would be super useful. Our DevOps team build something custom: they check all test logs of all builds of all branches and aggregate all that into a Grafana dashboard. We use it to monitor tests that are failing, to get a better grip on flakiness. Works okay but could be better.

reactordev 3y ago

This is a good write up of why you shouldn’t give away the house when you open source things. The issue was really this: Earthly being open-source, Earthy Satellite users were already seeing the benefit from 95% of Earthly CI. I’m a huge fan of open source, however, if your business model includes an open source model - you need a differentiator. Beyond blazingly fast(tm). You need a reason for people to offer up their credit cards, or better, their accounting PO process. You need a feature you withhold or a service you provide from the open source offering to make it appealing. GitLab restricts CI/CD to paying customers. Travis/CircleCI limit on build time/credits unless also open source. Azure DevOps is the devil. ArgoCD is complicated. GitHub actions are nice if you have hardware to run the runners. Swarm’s of Jenkins is what most are used to in the enterprise. As a former Director of DevOps, my immediate question will be what features make me want to buy vs host myself. I’m technically capable. Why should I buy with you when I can run my cloud and my DevOps pipelines for my workforce the way that makes the most sense for my business? Help me make the most sense. *EDIT* I'm not suggesting withholding features that are critical for the software to operate, but withholding business features that provide support or enterprise-level integration. Even making it tiered so the power users pay less because they support it more themselves?

hhh 3y ago
I feel almost certain someone could wrap ArgoCD into a product if they handled the 'patternization' of applicationsets. D2iQ already do it with Flux, but we bailed on D2iQ before we even got to try it.
- gabereiser 3y ago
  ArgoCD is almost worthless if your DevOps teams are IaC wielders. Honestly, just a bash script to push a container to the pods is all that’s required for DevOps deployment. A build pipeline should be get artifacts -> put artifacts in container -> push container to registry -> trigger a pull on your pods. If you didn’t have a standard cloud architecture, I can understand the frustration and reasoning behind looking at ArgoCD. If you have a standardized cloud practice, it’s moot.

skinner927 3y ago

> Write once, run anywhere Except virtual machines of course. Does everyone only build web apps? What tools do systems developers use? You can’t reliably test system integration in a container.

palata 3y ago
> Does everyone only build web apps? Unfortunately I think most people do, which is why there is a big bias towards web tech. I don't, and I hate the fact that most people seem to think that web tech solves everything :-).

seanr88 3y ago

I'm having a very similar issue getting https://brisktest.com/ https://brisktest.com/ out there. People complain about things like the speed of their build system, but convincing people that your CI system is faster just seems like an uphill battle. Maybe the era of launching new indie dev tools to developers is over and you just need a massive marketing budget to bombard devs with advertising and lots of sponsored content.

jatins 3y ago

> We ended up with the most ridiculous negative qualification criteria I have ever heard of: if the prospect requires a demo, then they’re not worth going after Based on my experience at enterprise companies, this is _all_ enterprise sales. So sounds very counter intuitive to me.

theptip 3y ago

This is a good write-up, and contains some well-reasoned PMF and product strategy evaluations; grokking the thought process here will help you to survive in your own startup.

jameshart 3y ago

Tl;dr they made the mistake of assuming that since 95% of the time CI jobs spend is ‘running builds’ that a build system is 95% of a CI platform. This turns out not to be true.

sharts 3y ago
Who says ex-googlers aren't the sharpest tools in the shed?

pharmakom 3y ago

The issue is trust. I dislike trusting GitHub / GitLab with my code but I already do, so I choose them for CI. I can already get fast builds using Bazel (or maybe Buck 2) so why do I need another tool / provider?

shepherdjerred 3y ago
If Bazel works for you that's great, although it is still rather complex. For smaller projects Earthly is a better fit.
maxloh 3y ago
GitLab is open sourced. You can self-host GitLab (and their CI solution) yourself.

shepherdjerred 3y ago

I've used Earthly for a year and absolutely love it. It allows you to have two dependencies on the host system for any project -- just Docker and Earthly. I've used it to build cross-compilers, system software, web apps, my resume, school projects. The speed up is real. You can definitely get caching and parallelization and containizeration with other build systems, but Earthly's value is that it comes for free with a familiar syntax.

12_throw_away 3y ago
Ditto, it's been a huge success where I work. Keeping my fingers crossed that they find a way to stay in business :)
- dpeckett 3y ago
  I feel like it's just a patience game of staying alive long enough that organic traction takes off. Earthly is bloody fantastic, the flexibility and expressiveness IMO is hard to beat.

CodeL 3y ago

Innovation isn't just about speed; it's about understanding the nuances of user needs and market dynamics. Even the fastest CI can face hurdles if it doesn't align with the broader ecosystem's expectations and requirements.

mcnichol 3y ago
This right here. I have found some of the greatest products I have used came from: "I'm working on this project, the tools I currently have are just different forms of a hammer...I need a screwdriver. I built a screwdriver. This screwdriver is the only tool I'll consider when I face this situation." The tools sell because the people doing the work and feeling the pain react strongly while pointing at the incompetent layers above with purchasing power. Giving (or showing) someone the solution and then taking it away is probably the most common reason I have seen someone leave a situation.
- CodeL 3y ago
  Absolutely. It's often those who are in the trenches, dealing with the day-to-day challenges, who recognize the need for better tools. They craft solutions tailored to their specific needs, which often resonate with others facing similar challenges. For instance: - Slack emerged because teams needed a more efficient way to communicate than traditional email chains. It was built as a communication tool for a gaming company and then pivoted to become the go-to messaging app for businesses. - Docker was developed out of the frustration of "it works on my machine" syndrome. It provided a consistent environment, making software deployment more predictable and efficient. - Trello was born out of the need for a visual project management tool that was intuitive and easy to use, unlike complex project management software. These tools became popular not just because they were innovative, but because they addressed real pain points felt by those on the ground. When decision-makers overlook these grassroots solutions, they risk alienating their most valuable assets: the people doing the work.
  - mcnichol 3y ago
    Couldn't have said it better. I have loved each one of those tools too. Docker maybe a little more love-hate but I think that has a lot more to do with the politics and salesy side of things and "Who owns what"

progx 3y ago

You get customer when you solve a real problem for them. CI-Systems exists, slow or fast doesn't be a problem for most of the devs, cause you run normally everything local and only on publish in a remote CI-system. So CI-Systems exists and solve their problem. I think only for a really small market the CI-performance is a real pain. Sure, everbody wants it fast, but that is not a reason to change a huge complex setup.

mcnichol 3y ago

Not to rain on the parade here but this is literally a copy-pasta product. Jenkins, Google Borg, Cloud Foundry, Concourse Pipelines...and surprise surprise when you look at where he came from...Ex-Google, Ex-VMW, RabbitMQ... If OP wasn't in and around the source of all these tools above then they were at the very least first cousins to the story. The sales cycle is long, integrations require multiple dimensions of executive buy-in, especially security and networking. I think they made something nice but it felt like a nothing-burger story about something that is constantliy oscillating between bespoke and commoditized due to upstream problems that are such a mix of issues. Lack of oversight to micromanagement, inexperience to too much experience that they cannot let go of "the way it's always done". It may be just my unpopular opinion but you are boiling an ocean of problems selling toolchains. Business and Tech are like water often finding the holes that lead to a path of least resistance, even when these erode the foundation of "core business". Toolchains that behave like guidelines and "parenting strategies" with removable guardrails have always offered the greatest rewards in my experience.

rewmie 3y ago
> Not to rain on the parade here but this is literally a copy-pasta product. More to the point, the product offered no compelling reason to use it, let alone pay for it. Being "fast" is not a selling point. Developers don't want slow pipelines, but that does not mean they want fast pipelines. The speed that the pipeline works is more depending on how the pipeline is setup than the overhead of the pipeline service, and other CICD services are already blazing fast. Take for instance CircleCI, which is already a tough sell over GitHub actions and GitLab CICD: how did the service stood out? Did it added tangible value over GitHub/Gitlab/CircleCI/etc? Shaving milliseconds off a minutes-long build ain't it.
- pixl97 3y ago
  Reliable, configurable, and compatible. That's what, at least from the work I do, seems to be what the market is looking for. Most CI/CD for sizeable companies have integrations into other systems, such as static analysis and things lime SBOM. Those tools tend to take a bunch of time outside the CI. They also need to ensure the results from those tools can break builds along with other alerting. Eventually the CI itself becomes a small fraction of total execution time.

Anon_Forever 3y ago

Sometimes the best technology fails and the space is overtaken by lesser technologies. It can be something as silly and out of your control as timing. Maybe the competition came out a bit after you and grew faster as interest in the space rose astronomically in parallel.

kunalgupta 3y ago

Swapping single words in my pitches has taught me a lot about what people want

bluepuma77 3y ago

Why have examples with JavaScript, when no build process is needed? Why not use TypeScript or even something like SvelteKit, which has a raving fan-base which is obviously working with bleeding edge tech that is still rough around the edges?

davinci123 3y ago

I think the biggest moral of the story - don't build a product without considering the switching/adoption cost. Higher the switching/adoption cost, lower the product/market fit. I have heard this story again and again from so many startups who have built products which provide incremental benefits (1.3x cheaper or 1.5x faster) and not getting enough traction.

skybrian 3y ago

This article is confusing because the author is too close to the problem, and it needs to be explained from an outsider’s point of view before diving into the details. But it looks like the information is there. It sounds to me like they ended up with an encapsulation layer between language-specific build systems and the continuous build systems that kick them off. A comparison might be with something like Bazel, which does everything, but full adoption requires going all-in on using their build language to replace language-specific build systems, often including moving source files to make it work. This feels foreign compared to using a language’s own build system, but it might seem natural for C programmers, where the language doesn’t have its own build system. Similarly, Java went through several unfortunate build systems before unfortunately settling on Gradle. Language-specific build systems end up not doing everything because each language has its own conventions and ecosystem. Modern ones don’t try, they know what they’re good at and stick to it. So, the tool that actually knows about multiple languages and their artifacts is often a shell script, makefile, docker file, or the continuous build itself. Or even doing it by hand. That’s the layer they’re trying to improve. I don’t have a clear idea what their fundamental insight is that makes their way of doing it better, though.

unbalancedevh 3y ago

It really bugs me when authors use acronyms or abbreviations without expanding what they mean. It tells the reader, "I didn't write this article for you, so don't bother trying to understand it."

fydorm 3y ago
Which terms are you referring to? If you aren't familiar with "CI/CD", then this article really isn't for you, as you haven't run into the problems this startup is trying to solve.
- unbalancedevh 3y ago
  Yes, CI/CD. It would be nice to know in the first paragraph that the article isn't for me, instead of having to read 1/3 of the article and then infer it, or even keep guessing what it's about. Sometimes that's fine, but I seem to be seeing it more often, and it discourages people new to the field or who are still exploring.

_whiteCaps_ 3y ago

> Imagine you live in a world where no part of the build has to repeat unless the changes actually impacted it. A world in which all builds happened with automatic parallelism. A world in which you could reproduce very reliably any part of the build on your laptop. That sounds similar to https://concourse-ci.org/ https://concourse-ci.org/ I quite like it, but it never seemed to gain traction outside of Cloud Foundry.

AlphaWeaver 3y ago
We spent some time evaluating this on my team, and we're still experimenting with it. I like it a lot, but the project appears to be mostly unmaintained since mid-2021, when the creator left it to work on a lispy CI/CD tool [0] that feels very complicated... not sure what's going on there. [0]: https://github.com/vito/bass https://github.com/vito/bass
- gavmor 3y ago
  I enjoyed working with concourse as the foundations of Pivotal's release engineering systems. We tried several other platforms, but ultimately concourse was tailor-made for Pivotal-style dev/ops ie highly technical, prioritizing version-control and CLIs over GUIs, with the wall-mounted "build monitor" as a first-class user. While Concourse wasn't explicitly a Pivotal product, shouldn't Conway's Law encourage us not to understate the influences of the Pivotal milieu? I don't doubt that VMware failed to nurture Concourse's maintenance post-acquisition. I hesitate to make sweeping statements about the legendary virtualization behemoth, but I personally doubt their commitment XP dev/ops culture. Edit: don't take my word for it, read someone more closely involved to this day, Nat Bennett https://www.simplermachines.com/some-facts-about-concourse/ https://www.simplermachines.com/some-facts-about-concourse/
- natbennett 3y ago
  There are folks still working on it but at this point it pretty much only gets updates that VMware needs to continue delivering Cloud Foundry. It looks dead-er than it is because the team hasn't explained this, so there's a giant list of work that won't ever get done just kinda... hanging out... on the README and in the issues list. I use it at work but it was already running when I got there. I also really like it but I'd be reluctant to stand it up somewhere new unless I was using it to run a Cloud Foundry. It has some operational issues and it is actually impossible to get an SLA if you're not also a VMware Cloud Foundry customer, and that costs a minimum of $50k per year.

ilyt 3y ago

> Why not just simplify the stack? Why pay both the CI vendor and us, when you can just pay us? Coz the CI they "pay" for comes with rest of the stack, and is not just CI product. Both Gitlab and Github have far more than just CI >These weren’t the raving fans we were used to talking with. New people would look at Earthly CI with a skeptical eye. They were mostly thinking that “all CIs are the same - they just have different syntax,” and then they would not really look any further as to why we might actually be different. we don't care about CI, we just need it and we need it to work. If we have working CI manifest for our app, the new app will have same manifest with few find-replace in it. The pain might be there to create it. And the example ones don't really look any easier than doing same thing in Gitlab. Frankly, putting some transpiler that takes Gitlab or Github CI config and just makes Earthfile on the fly would probably convince some people to at least try.

0xbadcafebee 3y ago

There is no real business advantage or value add to switching CI/CD systems. The only people that do it no longer have anyone working for them that knows how the existing CI/CD system works, so the new person just replaces it with something else. They're all basically the same crap. Run a job, evaluate its output, maybe run another job, send a notification. Whenever you run into a problem with a CI/CD system, it's because the system is "opinionated" and "tightly integrated", but of course lacks some obscure and subtle capability that you need to solve a problem. The fact that you are using a specific CI/CD system is what dooms you to eventually need another one to solve the problem that you current one cannot solve. It's ridiculous that here we are in 2023 and nobody has completed an open source project that replaces general use CI & CD (for modern best practices & use cases; Jenkins is a curse from the pit of hell). Both CI & CD are just loops of jobs and logic that then decides what next job to run. What those jobs do are all different, but so what? Make an open source library of jobs and logic. (for what it's worth, Drone.io is the best system I've ever used, and WoodpeckerCI is the open source equivalent, but it doesn't have a large community so work is very slow)

pxc 3y ago

I looked briefly at Earthly a while ago for work because I have to write CI integrations for a whole bunch of CI platforms (GitLab, Azure DevOps, Jenkins, and maybe GitHub Actions). I ended up going with a very similar product (i.e., another nice BuildKit frontend with integrations into a bunch of CI systems), Dagger, because the DSL it used for defining pipelines at the time seemed better to me. I ended up deeply regretting this decision, as the Dagger devs have de facto abandoned that language entirely in favor of churning out SDKs for the most popular programming languages— all imperative, all Turing-complete, and all, imo, bad fits for the space. So now I'm back in hell, integrating with all of these systems by hand, hesitant to trust this kind of tooling to a startup again. To me this is still the most appealing use case of something like Earthly. Push-and-see-what-happens is still the norm on almost all CI systems, and it's an abominable workflow. If you are at a large organization and have to support teams that use a wide variety of CI/CD platforms, something like Earthly could potentially save quite a lot of pain. But the appeal is precisely in supporting existing CI platforms, rather than in adding another one.

verdverm 3y ago
CI/CD & provisioning are inherently order driven tasks, I actually prefer the SDKs over the CUE implementation. The main issue with the old CUE implementation is that they had to try and align BuildKits DAG solver with CUE's DAG solver, and they worked in opposite ways. I tried to help them through this about 3 years ago (being a CUE expert). I definitely think the SDKs is a much better solution for them. Whether we like it or not, a lot of the industry is moving this way. Pulumi is another example of this, though I'm sold on imperative cloud infra, builds do seem to have alignment here. Side note, I will be exploring a new CUE + Dagger setup, though it will work differently than the old Dagger engine
- pxc 3y ago
  I'm not totally opposed to powerful (Turing-complete) languages being used for configuration or defining builds— I think Nix works really well in that space, and someday Nickel may as well. But wherever I've seen build tools from a language ecosystem that just embed a DSL in the host language (e.g.: sbt, Gradle), I've also seen projects get way too clever with their builds and create total nightmares for downstream packaging efforts. I anticipate similar problems for other kinds of configuration tools based on general-purpose languages, like CI or cloud IaC. > Side note, I will be exploring a new CUE + Dagger setup, though it will work differently than the old Dagger engine Sounds cool! I'll keep an eye out for it. :)

verdverm 3y ago

There are a bunch of these next-gen CI / container building systems. They all pretty much build on BuildKit like Docker, but unlock way more of the potential, namely DAG layers (think multi-stage docker on steroids), and better / more granular caching. These next-gen CI tools are fighting to be the UX on top of BuildKit. Dagger is where it is at, and it's built by the creators of Docker, based on their lessons learned. If you're looking to try something out, try Dagger. You write in the same languages you do every day, no need for some weird syntax like Earthly or the flatness of a Dockerfile. Their CLI/TUI is pretty sweet too, but you don't even need it.

gwbas1c 3y ago

Now is a good time to read "Crossing the Chasm:" https://en.wikipedia.org/wiki/Crossing_the_Chasm https://en.wikipedia.org/wiki/Crossing_the_Chasm The book talks about the state of the business Earthly CI was in: They tried to go from enthusiasts to generalists... But with a completely new product! I don't think it was wrong to build a new CI; instead, I think the thing they did "wrong" was build a super-complicated CI; or that when they tried to cross the chasm, they tried to sell a (cough) completely different product than what their early users were using.

js4ever 3y ago

Writing a new DSL is doomed and OP started by that. You will have to bend the world to make devs using it. Give them something they want, and no one wants to learn a new DSL and rewrite everything.

plg 3y ago

Dumb question I guess, what is a CI???

augustk 3y ago

CI = Continuous Integration https://en.wikipedia.org/wiki/Continuous_integration https://en.wikipedia.org/wiki/Continuous_integration

px1999 3y ago

> it didn’t make sense for them to jump head-first into an expensive and time-consuming migration effort just because some startup they had never heard of promised to deliver the sun and the moon > we need to shut down Earthly CI Is it possible that it all boils down to these two points? The technical benefits don't matter if there's even a hint that your (fundamental to my business') service could fail and be removed. When I read this it sounds like a technically great idea which solved a problem in a way that missed some fundamental prerequisite for their target user group (and hence success).

Klugistiono 3y ago

I'm quite passioned about CI/CD and working in this field for the last 7 years. I'm keeping a close eye on jenkins, tekton, circleci, travis, github actions, gitlab etc. and never ever heard about earthly. Btw. one thing the person doesn't get: Everyone wants to have fast and reliable builds but no one cares. No one. CI/CD Systems in a developers eye are blackboxes. They don't care. And its already a solved problem somehow. I was waiting for Jenkins X to happen (something like jenkins working nicely in a cloud native way because Jenkins Plugins!) but that didn't happen. What happend instead is Github Actions. Holy shit are Github Actions good. Srsly. GitLab stuff was slow and effort. GitHub Actions just do what they should do 99% of the time. If you do not have a huge build, you will not care about your CI/CD System and everyone who has real build issues, has to solve it by another strategy than just switching the build system.

throwawaaarrgh 3y ago
GitHub Actions is the same as everything else. It doesn't work any better or worse. But the UI is clunkier, you can do less with it than other systems, it lacked important features forever (still does). The only reason you think it's significantly better is psychological. The "market" of actions is better than competitors because the incumbent gets an advantage, and it trades on the idea of a generic executable as a 3rd party plugin, unlike other systems that try to force you into a particular language or paradigm. But other systems do this too.
- Klugistiono 3y ago
  I build enough pipelines in GitLab way to be able to tell you that GitHub Actions is much easier for me to implement. Its much closer than if you would have a second system like jenkins, i already have access to git and the packages. I do use much more container images than i did 6 or 10 years ago, that might have something to do with it but nonetheless, i do prefer GitHub Actions. I also stoped liking GitLab though. The Autodevops was horrendes, the security issues is concerning and plenty of ideas or features are half done. this is also something which changed. I liked GitLab much more than i do now. But looking at the year, i haven't looked at GitLab for at least 2 years now.

Tao3300 3y ago

Slow CI is a perk. https://xkcd.com/303 https://xkcd.com/303

stackzero 3y ago

Change the name to something short that sounds like it can be part of my stack and existed in linux forever e.g. ert Earthly sounds too mid 2010s and something extra/startupy

sharts 3y ago

[flagged]