4 ms·

I feel like it's getting harder to tell what dagger is _actually_ for these days. At first we'd hoped it could replace jenkins - it provided an alternative way

I feel like it's getting harder to tell what dagger is _actually_ for these days. At first we'd hoped it could replace jenkins - it provided an alternative way to run and debug CI pipelines - right on your machine! You could write in golang and just import what you needed. The dev direction feels more scattered now, trying to replace docker, be a new shell(?), and weirdly trying to be some kind of langchain? Doing something different doesn't imply better. A new set of complicated CLI args is no better than what we started with (shell scripts, or jenkinsfiles to integrate docker builds). I'm a little bummed that the project has seemingly drifted (in my view) from the original mission.

Imustaskforhelp 2y ago
theoretically you could always use the previous version / iterate on them. I think this approach of see what sticks and them trying out a lot of different things can be nice but not in its current form. Although I have installed dagger to give it a try and I am just not sure how to make it work , the quickstart / hello world doesn't' work. No I don't want to make an AI , I just want to see what you really are. And so I do agree with your statement!
- randomtechguy 2y ago
  Yup - Ever since the introduction of llm as a core API we're basically going to stop upgrading / using the project unless things start moving in a more sane direction.
  - levlaz 2y ago
    Why is that? It’s just an extra type that has no impact on any other types. It’s a tool in your toolbox that you can use when you need it and ignore otherwise.
    - randomtechguy 2y ago
      Asked the opposite way - why is this a core API type of a buildkit interface? Having API calls to various LLMs as core functionality of a build system is just straight up weird. Doesn't fit, confuses developers on my team when they try to read/contribute to our shared build libraries, worries me about the direction the project is going. Seems strange that especially with the push for modules this was integrated as a core type. It has nothing to do with buildkit or builds.
      - verdverm 2y ago
        Dagger isn't just a build system. I use it for all sorts of things, one example would be translating Go types to CUE, or the other way around. This happens in a container. With LLMs becoming core to the development workflow, it kinda makes sense to have a primitive, since LLM i/o is a bit different from other functions. I haven't tried it, but this thread had me go look at the details and now it's on the roadmap. Probably make something of an agentic workflow that uses a tool in a container, see how it works out. I'm still skeptical that Dagger is a good tool or ecosystem for LLM work without integrations to all the extra stuff you need around LLMs and agents.
      - levlaz 2y ago
        Dagger aims to be more than a build kit interface and more than just a build system. Many people have been using it outside of CI for years. But I can appreciate your perspective. Thank you for sharing your point of view.
clvx 2y ago
I feel Nix is eating their lunch. Even though Nix as a language could be tough, it's a simpler solution than Dagger. It provides a shell, almost complete reproducibility and isolation can be added through containers. Working with dependencies to build source code using nix is usually straightforward but having the right binary version that doesn't have the same love as major languages (i.e. terraform, flux, etc) is one thing Nix really needs to implement. Marcelo's package version search[1] is a way to discover the particular hash but then you need to explicitly use that nixpkgs version for that particular binary and do a lot of mental gymnastics in your nix files. Nix could implement a syntax where you define the package versions as attribute set and then internally does a discovery of the nixpkgs hash for that versions and installs it. Flox and DevBox follow this pattern but I don't see why you need and external tool where this can be embedded in Nix (the cli). [1] http://lazamar.github.io/download-specific-package-version-with-nix/ http://lazamar.github.io/download-specific-package-version-w...
- verdverm 2y ago
  I've found the Nix ecosystem to be lacking, missing packages, wrongly built packages (from the official upstream), and out of date versions. Homebrew still outclasses Nix in this regard (quality over quantity). After taking Nix for a spin, I cannot be bothered to learn another custom tool with a bespoke language when I already have containers for doing the same things. For Dagger, I can choose from a number of languages I already know and the Docker concepts map over nearly 1-1
  - clvx 2y ago
    > I've found the Nix ecosystem to be lacking, missing packages, wrongly built packages (from the official upstream), and out of date versions. Homebrew still outclasses Nix in this regard (quality over quantity). That's also the case with the Docker ecosystem. On top of that, you need to take into account the base image, versions, etc. At the end what I look for is for a project being able to build my source code with runtime dependencies and supporting tools that won't change overtime for the architecture that I need.
    - 2y ago
shykes 2y ago
If I may offer a different perspective: Dagger has always been a general-purpose composition engine, built on container tech. Its most successful use case is CI - specifically taking complex build and test environments, and making them more portable and reproducible. But we never claimed to replace Jenkins or any other CI platform, and we've always been very open with our community about our desire to expand the use of Dagger beyond CI. We also never claimed to replace Docker, or to "be a shell" (note that the title of this HN page doesn't reflect the title of our post in that regard). Every feature we ship is carefully designed for consistency with the overall design. For example, Dagger Shell is built on the same Dagger Engine that we've been steadily improving for years. It's just another client. Our goal is to build a platform that feels like Lego: each new piece makes all other pieces more useful, because they all can be composed together into a consistent system.
- randomtechguy 2y ago
  It's a fair point - My opinions and use case are my own, I didn't mean to imply or assume there were promises not kept. The dagger team has been nothing but supportive and I do think has built a great community. That said, in the early days it was definitely pitched for CI/CD - and this how we've implemented it. > What is it? > Programmable: develop your CI/CD pipelines as code, in the same programming language as your application. > Who is it for? > A developer wishing your CI pipelines were code instead of YAML https://github.com/dagger/dagger/blob/0620b658242fdf62c872c667623c9d47f79c1f6c/README.md https://github.com/dagger/dagger/blob/0620b658242fdf62c872c6... Edit: This functionality/interaction with the dagger engine still exists today, and is what we rely on. The original comment is more of an observation on the new directions the project has taken since then.
  - shykes 2y ago
    Yes it's a fair observation. In terms of use cases, we did focus exclusively on CI/CD, and only recently expanded our marketing to other use cases like AI agents. It's understandable that this expansion can be surprising, we're trying to explain it as clearly as possible, it's a work in progress. I just wanted to clarify that in terms of product design and engineering, there is unwavering focus and continuity. Everything we build is carefully designed to fit with the rest. We are emphatically not throwing unrelated products at the wall to see what sticks. For example, I saw your comment elsewhere about the LLM type not belonging in the core. That's a legitimate concern that we debated ourselves. In the end we think there is a good design reason to make it core; we may be wrong, but the point is that we take those kinds of design decisions seriously and we take all use cases into account when we make them.

Imustaskforhelp 2y ago

theoretically you could always use the previous version / iterate on them. I think this approach of see what sticks and them trying out a lot of different things can be nice but not in its current form. Although I have installed dagger to give it a try and I am just not sure how to make it work , the quickstart / hello world doesn't' work. No I don't want to make an AI , I just want to see what you really are. And so I do agree with your statement!

randomtechguy 2y ago
Yup - Ever since the introduction of llm as a core API we're basically going to stop upgrading / using the project unless things start moving in a more sane direction.
- levlaz 2y ago
  Why is that? It’s just an extra type that has no impact on any other types. It’s a tool in your toolbox that you can use when you need it and ignore otherwise.
  - randomtechguy 2y ago
    Asked the opposite way - why is this a core API type of a buildkit interface? Having API calls to various LLMs as core functionality of a build system is just straight up weird. Doesn't fit, confuses developers on my team when they try to read/contribute to our shared build libraries, worries me about the direction the project is going. Seems strange that especially with the push for modules this was integrated as a core type. It has nothing to do with buildkit or builds.
    - verdverm 2y ago
      Dagger isn't just a build system. I use it for all sorts of things, one example would be translating Go types to CUE, or the other way around. This happens in a container. With LLMs becoming core to the development workflow, it kinda makes sense to have a primitive, since LLM i/o is a bit different from other functions. I haven't tried it, but this thread had me go look at the details and now it's on the roadmap. Probably make something of an agentic workflow that uses a tool in a container, see how it works out. I'm still skeptical that Dagger is a good tool or ecosystem for LLM work without integrations to all the extra stuff you need around LLMs and agents.
    - levlaz 2y ago
      Dagger aims to be more than a build kit interface and more than just a build system. Many people have been using it outside of CI for years. But I can appreciate your perspective. Thank you for sharing your point of view.

clvx 2y ago

I feel Nix is eating their lunch. Even though Nix as a language could be tough, it's a simpler solution than Dagger. It provides a shell, almost complete reproducibility and isolation can be added through containers. Working with dependencies to build source code using nix is usually straightforward but having the right binary version that doesn't have the same love as major languages (i.e. terraform, flux, etc) is one thing Nix really needs to implement. Marcelo's package version search[1] is a way to discover the particular hash but then you need to explicitly use that nixpkgs version for that particular binary and do a lot of mental gymnastics in your nix files. Nix could implement a syntax where you define the package versions as attribute set and then internally does a discovery of the nixpkgs hash for that versions and installs it. Flox and DevBox follow this pattern but I don't see why you need and external tool where this can be embedded in Nix (the cli). [1] http://lazamar.github.io/download-specific-package-version-with-nix/ http://lazamar.github.io/download-specific-package-version-w...

verdverm 2y ago
I've found the Nix ecosystem to be lacking, missing packages, wrongly built packages (from the official upstream), and out of date versions. Homebrew still outclasses Nix in this regard (quality over quantity). After taking Nix for a spin, I cannot be bothered to learn another custom tool with a bespoke language when I already have containers for doing the same things. For Dagger, I can choose from a number of languages I already know and the Docker concepts map over nearly 1-1
- clvx 2y ago
  > I've found the Nix ecosystem to be lacking, missing packages, wrongly built packages (from the official upstream), and out of date versions. Homebrew still outclasses Nix in this regard (quality over quantity). That's also the case with the Docker ecosystem. On top of that, you need to take into account the base image, versions, etc. At the end what I look for is for a project being able to build my source code with runtime dependencies and supporting tools that won't change overtime for the architecture that I need.
  - 2y ago

shykes 2y ago

If I may offer a different perspective: Dagger has always been a general-purpose composition engine, built on container tech. Its most successful use case is CI - specifically taking complex build and test environments, and making them more portable and reproducible. But we never claimed to replace Jenkins or any other CI platform, and we've always been very open with our community about our desire to expand the use of Dagger beyond CI. We also never claimed to replace Docker, or to "be a shell" (note that the title of this HN page doesn't reflect the title of our post in that regard). Every feature we ship is carefully designed for consistency with the overall design. For example, Dagger Shell is built on the same Dagger Engine that we've been steadily improving for years. It's just another client. Our goal is to build a platform that feels like Lego: each new piece makes all other pieces more useful, because they all can be composed together into a consistent system.

randomtechguy 2y ago
It's a fair point - My opinions and use case are my own, I didn't mean to imply or assume there were promises not kept. The dagger team has been nothing but supportive and I do think has built a great community. That said, in the early days it was definitely pitched for CI/CD - and this how we've implemented it. > What is it? > Programmable: develop your CI/CD pipelines as code, in the same programming language as your application. > Who is it for? > A developer wishing your CI pipelines were code instead of YAML https://github.com/dagger/dagger/blob/0620b658242fdf62c872c667623c9d47f79c1f6c/README.md https://github.com/dagger/dagger/blob/0620b658242fdf62c872c6... Edit: This functionality/interaction with the dagger engine still exists today, and is what we rely on. The original comment is more of an observation on the new directions the project has taken since then.
- shykes 2y ago
  Yes it's a fair observation. In terms of use cases, we did focus exclusively on CI/CD, and only recently expanded our marketing to other use cases like AI agents. It's understandable that this expansion can be surprising, we're trying to explain it as clearly as possible, it's a work in progress. I just wanted to clarify that in terms of product design and engineering, there is unwavering focus and continuity. Everything we build is carefully designed to fit with the rest. We are emphatically not throwing unrelated products at the wall to see what sticks. For example, I saw your comment elsewhere about the LLM type not belonging in the core. That's a legitimate concern that we debated ourselves. In the end we think there is a good design reason to make it core; we may be wrong, but the point is that we take those kinds of design decisions seriously and we take all use cases into account when we make them.