5 ms·

Looks really good. A layer over Nix that makes it tolerable sounds like a godsend. I’ve had a TODO item for “somehow use Nix for our dev env” at Notion since 20

by jitl 4y ago

Looks really good. A layer over Nix that makes it tolerable sounds like a godsend. I’ve had a TODO item for “somehow use Nix for our dev env” at Notion since 2019, but I keep kicking the can down the road because its better to tolerate a 500 line setup script than learn all the Nix stuff. Of course we could use Docker, but then on Mac our dev process would be 3x slower. Congrats to Daniel and the team! Excited to see what’s next after this.

xvector 4y ago
Nix absolutely needed a wrapper like this to be used 'in the real world.' When you think about bringing it to production (eg getting dev teams to migrate to it), Nix goes from a genuinely interesting idea to an "oh, that's cute" experimental toy because no one is going to spend hours learning Nix's weird DSL. It's simply not approachable in its base form. I spent hours converting my devboxes to NixOS and managing my dev environments with home-manager and I still don't have a clue how any of it works. Errors are opaque and annoying to debug. Dev environments constantly break and change in ways that belay the "reproducible" nature of Nix. If someone actively interested in Nix can't easily grasp it, how does anyone expect it to catch on in the real world?
- mrich 4y ago
  When there are issues with debugging breakages of such environments, will a wrapper be sufficient? Does devbox have extra handling for such scenarios?
  - pxc 4y ago
    The opacity of Nix errors is largely (entirely?) constrained to the module systems (notably NixOS, Nix-Darwin, and Home Manager). (This is unfortunate, because these are actually some of the best parts of the Nix ecosystem.) The issue is essentially that configuration options all get merged into a global namespace, but there are no facilities to track where they came from. So when configuration mismatches of certain kinds occur, you get an error in some library code that's trying to merge or coerce two incompatible values, and nothing pointing you to the two places where the conflicting values are originally set. (This kind of error, the most common mostly-useless error message, is typically easily debugged by searching for the relevant options in your configuration and in the source code of that collection of modules. But that is still backwards and a chore, and deserves a real solution.) Anyway, the package definitions and builds in Nixpkgs don't use any such module system in any way. So this tool is not wrapping the functionality that is associated with opaque error messages. :)
- baby 4y ago
  My theory is that we need better learning resources (that’s why I’m working on https://mimoo.github.io/nixbyexample/ https://mimoo.github.io/nixbyexample/), better tooling (debugging or starting a project from scratch is hard), better integration with rust/js/etc. projects based on convention (and not configuration: I shouldn’t have to point to more than a Cargo.toml or package.json and nix should do the rest)
  - gamjQZnHT53AMa 4y ago
    This looks great. As someone who started learning Nix in the last few weeks this is definitely needed and will prove invaluable to future learners. I'm still completely lost on a lot of things but if I work something out that isn't covered on your site I'll try and put together a PR for it.
    - baby 4y ago
      That’d be great!
  - unicornmama 4y ago
    The problem is boundless complexity. Learning resources won’t help anyone to learn and manage boundless complexity.
    - ParetoOptimal 4y ago
      Are you really claiming Nix has boundless complexity?
- otabdeveloper4 4y ago
  I think you're confusing Nix, nixpkgs and NixOs. (Also Nix and Nix-flakes are two different things, like Javascript and React.)
  - nitsky 4y ago
    I think this attempt at clarification is just proving xvector’s point.
    - otabdeveloper4 4y ago
      It's not an attempt at clarification. It's best if you start with the problem you're trying to solve, then work backwards towards a solution. "Nix" isn't really a thing, no more than "Linux" is. It's a collection of tools and languages and frameworks people use to solve various very different problems.
      - jitl 4y ago
        See, this is why I would like to use a wrapper.
- ParetoOptimal 4y ago
  > Nix goes from a genuinely interesting idea to an "oh, that's cute" experimental toy because no one is going to spend hours learning Nix's weird DSL. It's simply not approachable in its base form. Here's an opposite opinion: > My hot take is that Nix actually has great syntax > In particular, the way Nix handles record syntax is exemplary and more languages should copy what Nix does in this regard And even more related to this discussion: > A lot of the times, when people say they hate "Nix's syntax", what they more likely mean is that they hate the domain-specific languages of the Nixpkgs overlay system and/or the NixOS module system https://mobile.twitter.com/GabriellaG439/status/1563001166568443904 https://mobile.twitter.com/GabriellaG439/status/156300116656...
nidnogg 4y ago
>Of course we could use Docker, but then on Mac our dev process would be 3x slower. Doesn't devbox depend on Docker, though? I figure any performance losses from Docker would happen with this too.
- Aeolun 4y ago
  Only for the build step (e.g. making a docker image). The dev step runs entirely on your machine. I got that from the README btw, no guarantees :)
- dloreto 4y ago
  The dependency on Docker only exists for when you want to turn your shell into a container – but it's not otherwise used when you're just running a shell locally. When writing javascript there's often a desire to have "isomorphic" or "universal" applications. Write the code once and run it in _either_ the client or the _server_. Devbox is taking a similar approach to the development environment: declare it once, run it locally as a shell, and when you're ready, turn it into a container without having to re-declare it. It's only the latter functionality that has a Docker dependency.
  - nidnogg 4y ago
    Hmmm, I'll be keeping an eye on this one then!
  - chupasaurus 4y ago
    It's actually not since BuildKit could be run as a standalone daemon, and you can run the result via containerd which is a dependency of Docker. Might write a PR over the weekend(s).
idontwantthis 4y ago
I just set up a docker dev environment on Mac and I would not call it slow anymore. If you were referring to volume mounts, they are 10x faster now. Definitely fast enough for our work.
- baby 4y ago
  Depends how much memory your container needs to build…
- greymalik 4y ago
  > they are 10x faster now What changed?
  - porker 4y ago
    Virtiofs
- ParetoOptimal 4y ago
  > they are 10x faster now. Definitely fast enough for our work. They are definitely faster and great improvements. But when I can use Linux through qemu and compile my companies Haskell application in 45s rather than 3m30s... The choice is obvious to use Linux.
mhio 4y ago
> Of course we could use Docker, but then on Mac our dev process would be 3x slower Out of interest, what in your dev process makes using containers so impactful?
- TheCapeGreek 4y ago
  Docker on Mac and Windows runs in a VM, not natively.
  - mhio 4y ago
    3x slower is not regular VM overhead range. Using the dodgy file share volume mounting, maybe for particular file access patterns.
- lozenge 4y ago
  They probably want their source code on their "real" filesystem which makes Docker painfully slow.
ianpurton 4y ago
> Of course we could use Docker, but then on Mac our dev process would be 3x slower. Not in my experience. If you set up volumes for node_modules or any folders where where dependencies are stored you get the same performance on Mac as anywhere else. For rust, which I use, I'm able to get better performance on Mac under docker than using rust tools natively. See https://purton.tech/blog/faster-rust-incremental-builds/ https://purton.tech/blog/faster-rust-incremental-builds/
matnosner 4y ago
If you like the concept you might also be interested in bob[1]. Same idea for build isolation crafted into a build system/task runner. (I'm the author). [1] https://github.com/benchkram/bob https://github.com/benchkram/bob

by jitl 4y ago

xvector 4y ago

Nix absolutely needed a wrapper like this to be used 'in the real world.' When you think about bringing it to production (eg getting dev teams to migrate to it), Nix goes from a genuinely interesting idea to an "oh, that's cute" experimental toy because no one is going to spend hours learning Nix's weird DSL. It's simply not approachable in its base form. I spent hours converting my devboxes to NixOS and managing my dev environments with home-manager and I still don't have a clue how any of it works. Errors are opaque and annoying to debug. Dev environments constantly break and change in ways that belay the "reproducible" nature of Nix. If someone actively interested in Nix can't easily grasp it, how does anyone expect it to catch on in the real world?

mrich 4y ago
When there are issues with debugging breakages of such environments, will a wrapper be sufficient? Does devbox have extra handling for such scenarios?
- pxc 4y ago
  The opacity of Nix errors is largely (entirely?) constrained to the module systems (notably NixOS, Nix-Darwin, and Home Manager). (This is unfortunate, because these are actually some of the best parts of the Nix ecosystem.) The issue is essentially that configuration options all get merged into a global namespace, but there are no facilities to track where they came from. So when configuration mismatches of certain kinds occur, you get an error in some library code that's trying to merge or coerce two incompatible values, and nothing pointing you to the two places where the conflicting values are originally set. (This kind of error, the most common mostly-useless error message, is typically easily debugged by searching for the relevant options in your configuration and in the source code of that collection of modules. But that is still backwards and a chore, and deserves a real solution.) Anyway, the package definitions and builds in Nixpkgs don't use any such module system in any way. So this tool is not wrapping the functionality that is associated with opaque error messages. :)
baby 4y ago
My theory is that we need better learning resources (that’s why I’m working on https://mimoo.github.io/nixbyexample/ https://mimoo.github.io/nixbyexample/), better tooling (debugging or starting a project from scratch is hard), better integration with rust/js/etc. projects based on convention (and not configuration: I shouldn’t have to point to more than a Cargo.toml or package.json and nix should do the rest)
- gamjQZnHT53AMa 4y ago
  This looks great. As someone who started learning Nix in the last few weeks this is definitely needed and will prove invaluable to future learners. I'm still completely lost on a lot of things but if I work something out that isn't covered on your site I'll try and put together a PR for it.
  - baby 4y ago
    That’d be great!
- unicornmama 4y ago
  The problem is boundless complexity. Learning resources won’t help anyone to learn and manage boundless complexity.
  - ParetoOptimal 4y ago
    Are you really claiming Nix has boundless complexity?
otabdeveloper4 4y ago
I think you're confusing Nix, nixpkgs and NixOs. (Also Nix and Nix-flakes are two different things, like Javascript and React.)
- nitsky 4y ago
  I think this attempt at clarification is just proving xvector’s point.
  - otabdeveloper4 4y ago
    It's not an attempt at clarification. It's best if you start with the problem you're trying to solve, then work backwards towards a solution. "Nix" isn't really a thing, no more than "Linux" is. It's a collection of tools and languages and frameworks people use to solve various very different problems.
    - jitl 4y ago
      See, this is why I would like to use a wrapper.
ParetoOptimal 4y ago
> Nix goes from a genuinely interesting idea to an "oh, that's cute" experimental toy because no one is going to spend hours learning Nix's weird DSL. It's simply not approachable in its base form. Here's an opposite opinion: > My hot take is that Nix actually has great syntax > In particular, the way Nix handles record syntax is exemplary and more languages should copy what Nix does in this regard And even more related to this discussion: > A lot of the times, when people say they hate "Nix's syntax", what they more likely mean is that they hate the domain-specific languages of the Nixpkgs overlay system and/or the NixOS module system https://mobile.twitter.com/GabriellaG439/status/1563001166568443904 https://mobile.twitter.com/GabriellaG439/status/156300116656...

nidnogg 4y ago

>Of course we could use Docker, but then on Mac our dev process would be 3x slower. Doesn't devbox depend on Docker, though? I figure any performance losses from Docker would happen with this too.

Aeolun 4y ago
Only for the build step (e.g. making a docker image). The dev step runs entirely on your machine. I got that from the README btw, no guarantees :)
dloreto 4y ago
The dependency on Docker only exists for when you want to turn your shell into a container – but it's not otherwise used when you're just running a shell locally. When writing javascript there's often a desire to have "isomorphic" or "universal" applications. Write the code once and run it in _either_ the client or the _server_. Devbox is taking a similar approach to the development environment: declare it once, run it locally as a shell, and when you're ready, turn it into a container without having to re-declare it. It's only the latter functionality that has a Docker dependency.
- nidnogg 4y ago
  Hmmm, I'll be keeping an eye on this one then!
- chupasaurus 4y ago
  It's actually not since BuildKit could be run as a standalone daemon, and you can run the result via containerd which is a dependency of Docker. Might write a PR over the weekend(s).

idontwantthis 4y ago

I just set up a docker dev environment on Mac and I would not call it slow anymore. If you were referring to volume mounts, they are 10x faster now. Definitely fast enough for our work.

baby 4y ago
Depends how much memory your container needs to build…
greymalik 4y ago
> they are 10x faster now What changed?
- porker 4y ago
  Virtiofs
ParetoOptimal 4y ago
> they are 10x faster now. Definitely fast enough for our work. They are definitely faster and great improvements. But when I can use Linux through qemu and compile my companies Haskell application in 45s rather than 3m30s... The choice is obvious to use Linux.

mhio 4y ago

> Of course we could use Docker, but then on Mac our dev process would be 3x slower Out of interest, what in your dev process makes using containers so impactful?

TheCapeGreek 4y ago
Docker on Mac and Windows runs in a VM, not natively.
- mhio 4y ago
  3x slower is not regular VM overhead range. Using the dodgy file share volume mounting, maybe for particular file access patterns.
lozenge 4y ago
They probably want their source code on their "real" filesystem which makes Docker painfully slow.

ianpurton 4y ago

> Of course we could use Docker, but then on Mac our dev process would be 3x slower. Not in my experience. If you set up volumes for node_modules or any folders where where dependencies are stored you get the same performance on Mac as anywhere else. For rust, which I use, I'm able to get better performance on Mac under docker than using rust tools natively. See https://purton.tech/blog/faster-rust-incremental-builds/ https://purton.tech/blog/faster-rust-incremental-builds/

matnosner 4y ago

If you like the concept you might also be interested in bob[1]. Same idea for build isolation crafted into a build system/task runner. (I'm the author). [1] https://github.com/benchkram/bob https://github.com/benchkram/bob