3 ms·

Do you have any examples of your Devenv workflow you can share? I took a look at Dagger and really like the concept, but I'm trying to figure out the limitation

Do you have any examples of your Devenv workflow you can share? I took a look at Dagger and really like the concept, but I'm trying to figure out the limitations/why there's so much negativity in this thread. I currently manage my development environments via NixOS and Devenv, so if I could just keep that and achieve the same functionality, that sounds good to me.

pxc 10mo ago
At the time, Devenv was still pretty barebones compared to now (the rate of feature development has been great in the past few years), so it didn't have much in the way of support for workflows per se. Instead I used the scripts module to write shell scripts that bring along all of their dependencies. Thus all of our GitLab jobs are just one-line invocations of our wrapper scripts. We currently use a flake.nix-based Devenv and expose the scripts from the Devenv module as flake outputs, then just run them with `nix run`. Nowadays, though, Devenv actually directly supports some relevant functionality in the form of tasks¹, which can be written in any language, and support dependency relationships and run concurrently by default. We use those in a few places as well, though mostly in place of older, cruder `enterShell` integration. There, we use them: - to seamlessly ensure correct state of Git extensions: - all Git submodules are checked out - Git-LFS is configured and all large files are correctly checked out - to ensure valid OpenTofu caches since we use Nix instead of OpenTofu for managing providers - if .terraform dir doesn't exist, run `tofu init` - if Terraform providers lock file is older than devenv lock, run `tofu init -upgrade` - manage Poetry stubs and virtualenvs (most of this is configured behind the scenes upstream, but we do some extra stuff) - manage Bundler dependencies and binstubs for Ruby projects so executables under development can be involed naively from PATH (much of this is bespoke but upstreamable) So a lot of those Devenv tasks are things you might do in build or pre-build stages of a CI pipeline. You can define the requisite dependency relationships and then just run `devenv test` or `devenv tasks ...` as your CI job. What we don't have is any abstraction over the various CI platforms we use (currently GitLab, GitHub, and AzDO, with Jenkins support in the backlog). Unfortunately, for the sake of a better DX for our (internal) customers, we still integrate fairly directly with each CI platform, so we do have piles of vendor-specific YAML. Our tasks and scripts are currently just Bash, but they're relatively nice Bash because their dependencies are managed with resholve and they're linted by ShellCheck, thanks to some experimental changes I made to Devenv's scripts module. For our security/quality scans, we do, however, have some logic nestled in those Bash scripts which checks for a CI environment via env vars, then performs differential rather than full scans if we're in merge request contexts. Where the upstream tools don't directly support this, we basically perform "before" and "after" scans and take a diff. This is the direction I'd like to go in the future— all in on this, surfacing jobs in a "native" way be damned. Unfortunately I can't directly share these examples as they belong to my employer and live in proprietary projects. But what I can do is open a couple of WIP PRs for the upstreamable work (new or modified Devenv modules) so you can take a look at that at least. Also fwiw I think Dagger is still worth checking out. I just feel a bit burned by adopting it too early in its lifecycle, and generally advice caution about adopting tools developed by VC-backed startups. -- 1: https://devenv.sh/tasks/ https://devenv.sh/tasks/

pxc 10mo ago

At the time, Devenv was still pretty barebones compared to now (the rate of feature development has been great in the past few years), so it didn't have much in the way of support for workflows per se. Instead I used the scripts module to write shell scripts that bring along all of their dependencies. Thus all of our GitLab jobs are just one-line invocations of our wrapper scripts. We currently use a flake.nix-based Devenv and expose the scripts from the Devenv module as flake outputs, then just run them with `nix run`. Nowadays, though, Devenv actually directly supports some relevant functionality in the form of tasks¹, which can be written in any language, and support dependency relationships and run concurrently by default. We use those in a few places as well, though mostly in place of older, cruder `enterShell` integration. There, we use them: - to seamlessly ensure correct state of Git extensions: - all Git submodules are checked out - Git-LFS is configured and all large files are correctly checked out - to ensure valid OpenTofu caches since we use Nix instead of OpenTofu for managing providers - if .terraform dir doesn't exist, run `tofu init` - if Terraform providers lock file is older than devenv lock, run `tofu init -upgrade` - manage Poetry stubs and virtualenvs (most of this is configured behind the scenes upstream, but we do some extra stuff) - manage Bundler dependencies and binstubs for Ruby projects so executables under development can be involed naively from PATH (much of this is bespoke but upstreamable) So a lot of those Devenv tasks are things you might do in build or pre-build stages of a CI pipeline. You can define the requisite dependency relationships and then just run `devenv test` or `devenv tasks ...` as your CI job. What we don't have is any abstraction over the various CI platforms we use (currently GitLab, GitHub, and AzDO, with Jenkins support in the backlog). Unfortunately, for the sake of a better DX for our (internal) customers, we still integrate fairly directly with each CI platform, so we do have piles of vendor-specific YAML. Our tasks and scripts are currently just Bash, but they're relatively nice Bash because their dependencies are managed with resholve and they're linted by ShellCheck, thanks to some experimental changes I made to Devenv's scripts module. For our security/quality scans, we do, however, have some logic nestled in those Bash scripts which checks for a CI environment via env vars, then performs differential rather than full scans if we're in merge request contexts. Where the upstream tools don't directly support this, we basically perform "before" and "after" scans and take a diff. This is the direction I'd like to go in the future— all in on this, surfacing jobs in a "native" way be damned. Unfortunately I can't directly share these examples as they belong to my employer and live in proprietary projects. But what I can do is open a couple of WIP PRs for the upstreamable work (new or modified Devenv modules) so you can take a look at that at least. Also fwiw I think Dagger is still worth checking out. I just feel a bit burned by adopting it too early in its lifecycle, and generally advice caution about adopting tools developed by VC-backed startups. -- 1: https://devenv.sh/tasks/ https://devenv.sh/tasks/