4 ms·

I was lead on a Nix adoption effort for a few years that was kind of like that: it solved real problems, unlocked far faster, smaller, and cheaper builds than w

I was lead on a Nix adoption effort for a few years that was kind of like that: it solved real problems, unlocked far faster, smaller, and cheaper builds than would have been possible any other way, and let us ship delta updates over crappy wifi connections to Linux computers on robots. Flakes were a perfect fit for our model, and we were just in time for stuff like up to date versions of cuda and tensorflow to be delivered via nixpkgs. In my mind, Nix was unstoppable, and I particularly loved how empowered I imagined developers would feel. No mystery-meat CI scripts pushing packages to distant infrastructure that no one understands or even has the permissions to interact with, just the entire build system in one repo, trivially cloneable and hackable... add patches or build steps to anything and it's the same build as always, build it locally or send it to Hydra, it doesn't matter. A devs "got" it and did leverage those things, used PRs to do safe evaluation of bumps to core dependencies, but I think on the whole it was regarded as cool but not tractable, and a few years after leaving, it sounds like plans are being laid to replace it all with something containers or whatever. Very frustrating, particularly in a world where it should be trivial to identify the 5-10 typical tasks that people want to do with the nix code, and set up Claude skills to handle those. Given how easy and self-contained the build-test loop is, it feels like an almost perfect fit for agent-led development.

deleted 2mo ago
[deleted]
SOLAR_FIELDS 2mo ago
Meanwhile I’m over here with my 100 line flake.nix that installs system dependencies with direnv across hundreds of dev users and dozens of repositories wondering why everyone is all up in arms about boring and stable technology that solves a specific problem
- mikepurvis 2mo ago
  In my personal life, I also like Nix for small projects, especially where you're crossing multiple ecosystems so that any one system's tooling/lockfiles aren't really enough to "enclose" the whole thing. For that work project though, the final closure was thousands of store paths, with hundreds of source repos being brought together across a multiple of languages and build systems. A lot of that was/is essential complexity, just the reality that robotics is hard, the tools aren't as mature as in other domains, and so you have to be able to develop features and bugfixes all over a huge software stack simultaneously. Doing that with conventional tooling basically gets you a rigid world where you have to tag/release intermediate packages all the time just to get changes into testing, or you have to build the world on every push (lol docker). With input addressing and hermetic building of the intermediate stages, Nix and Bazel are systems that don't make you make that choice.
forrestthewoods 2mo ago
My gut feeling is that Nix is just not quite the right abstraction level. I can’t quite articulate this. And hell maybe I’m wrong. I don’t want to configure a global environment. I just want a build system that works reliably in any environment and can cross-compile from any platform to any platform. Nix does too damn much. All I want is a build system that doesn’t suck. And I want to run it on windows + Mac + Linux as a first class citizen. And I want to arbitrarily target any platform. What this really comes down to is that Linux C/C++ toolchains are badly designed. Nix is a huge massive convoluted architecture to try and twist itself around that unfortunate reality. At least that’s my spicy unpopular opinion that is probably wrong but is at least has elements of truth.
- hugmynutus 2mo ago
  Shared Dynamic Libraries. Not to say they aren't useful. Keeping applications small, keeping security updates simple, sub linear ram scaling, yada-yada-yada, big wins cross the board, not debating that. The pervasive link detection structure of effectively everything on GNU/Linux assumes is (more-or-less) objectively incorrect behavior in any sane security minded context. Binaries should be able to declare the interface/contract they expect (args/types/abi/exceptions/cryptographic signatures/digests). Then the runtime linker "match" against the local system. The current system is basically 2 levels of string equality checking. Nix goes into the right direction by getting your runtime linker/elf-runtime & package manager "integrated". But this sort of just feels like putting 'lipstick on a pig' and dancing around the core issue that `foo.exe` cannot ever realize `foobar-v1.8` and `foobar-v1.7` are both installed on the same computer. Nix just manages the environment/symlinks such that `foo.exe` doesn't realize this fact.
  - forrestthewoods 2mo ago
    > lipstick on a pig Yes!! > dancing around the core issue that `foo.exe` cannot ever realize `foobar-v1.8` and `foobar-v1.7` are both installed on the same computer I’m curious if you have an opinion on how you think this should be handled? At this point I’m just on team static linking or Windows DLL black box style. The Linux approach of dynamic libraries which function same as static is just the worst of every world. I also blame C/C++ toolchains for being very very bad on Linux.
rossng 2mo ago
We're using it in the same way - running NixOS on robots. Overall I think it is succeeding, though it definitely doesn't 'just work' out of the box. We've invested a fair bit of effort into things like: - containerised services[1] - UI for branch deployment/service management - delta patching (at the byte level, to save bandwidth) The main problem was always that nixlang/nixpkgs are arcane and have a steep learning curve. The majority of your colleagues (in any workplace) don't care about build systems and just want things to work. I think that has been more or less solved by LLMs, and it's now more viable than ever to adopt Nix. [1] https://bou.ke/blog/nixos-containers/ https://bou.ke/blog/nixos-containers/
shakow 2mo ago
> let us ship delta updates over crappy wifi connections to Linux computers on robots How were you doing that? Running your own channels, or something more complex? Also, did you use a push or a pull model w.r.t. the robots?
- mikepurvis 2mo ago
  At the time I was there we hadn't made it all the way to NixOS on the targets, so it was an Ubuntu "base" + Nix managing the app workspace as a kind of pseudo container, though able to bring more of its own system configuration with it (https://github.com/numtide/system-manager https://github.com/numtide/system-manager) than the more typical nuisance setup of container + separate outer config managed by ansible or a deb or whatever. As a result of that, in "production" we still actually delivered full rootfs images just to be absolutely certain, and the delta upgrades were for smaller test fleets that could updated hourly just via simple push tooling. If I was building it from scratch though, I'd probably just do NixOS + colmena, and do a push model forever. It's not worth the saved SSH connection to not have those logs and status messages coming back to the central coordinator immediately.

deleted 2mo ago

[deleted]

SOLAR_FIELDS 2mo ago

Meanwhile I’m over here with my 100 line flake.nix that installs system dependencies with direnv across hundreds of dev users and dozens of repositories wondering why everyone is all up in arms about boring and stable technology that solves a specific problem

mikepurvis 2mo ago
In my personal life, I also like Nix for small projects, especially where you're crossing multiple ecosystems so that any one system's tooling/lockfiles aren't really enough to "enclose" the whole thing. For that work project though, the final closure was thousands of store paths, with hundreds of source repos being brought together across a multiple of languages and build systems. A lot of that was/is essential complexity, just the reality that robotics is hard, the tools aren't as mature as in other domains, and so you have to be able to develop features and bugfixes all over a huge software stack simultaneously. Doing that with conventional tooling basically gets you a rigid world where you have to tag/release intermediate packages all the time just to get changes into testing, or you have to build the world on every push (lol docker). With input addressing and hermetic building of the intermediate stages, Nix and Bazel are systems that don't make you make that choice.

forrestthewoods 2mo ago

My gut feeling is that Nix is just not quite the right abstraction level. I can’t quite articulate this. And hell maybe I’m wrong. I don’t want to configure a global environment. I just want a build system that works reliably in any environment and can cross-compile from any platform to any platform. Nix does too damn much. All I want is a build system that doesn’t suck. And I want to run it on windows + Mac + Linux as a first class citizen. And I want to arbitrarily target any platform. What this really comes down to is that Linux C/C++ toolchains are badly designed. Nix is a huge massive convoluted architecture to try and twist itself around that unfortunate reality. At least that’s my spicy unpopular opinion that is probably wrong but is at least has elements of truth.

hugmynutus 2mo ago
Shared Dynamic Libraries. Not to say they aren't useful. Keeping applications small, keeping security updates simple, sub linear ram scaling, yada-yada-yada, big wins cross the board, not debating that. The pervasive link detection structure of effectively everything on GNU/Linux assumes is (more-or-less) objectively incorrect behavior in any sane security minded context. Binaries should be able to declare the interface/contract they expect (args/types/abi/exceptions/cryptographic signatures/digests). Then the runtime linker "match" against the local system. The current system is basically 2 levels of string equality checking. Nix goes into the right direction by getting your runtime linker/elf-runtime & package manager "integrated". But this sort of just feels like putting 'lipstick on a pig' and dancing around the core issue that `foo.exe` cannot ever realize `foobar-v1.8` and `foobar-v1.7` are both installed on the same computer. Nix just manages the environment/symlinks such that `foo.exe` doesn't realize this fact.
- forrestthewoods 2mo ago
  > lipstick on a pig Yes!! > dancing around the core issue that `foo.exe` cannot ever realize `foobar-v1.8` and `foobar-v1.7` are both installed on the same computer I’m curious if you have an opinion on how you think this should be handled? At this point I’m just on team static linking or Windows DLL black box style. The Linux approach of dynamic libraries which function same as static is just the worst of every world. I also blame C/C++ toolchains for being very very bad on Linux.

rossng 2mo ago

We're using it in the same way - running NixOS on robots. Overall I think it is succeeding, though it definitely doesn't 'just work' out of the box. We've invested a fair bit of effort into things like: - containerised services[1] - UI for branch deployment/service management - delta patching (at the byte level, to save bandwidth) The main problem was always that nixlang/nixpkgs are arcane and have a steep learning curve. The majority of your colleagues (in any workplace) don't care about build systems and just want things to work. I think that has been more or less solved by LLMs, and it's now more viable than ever to adopt Nix. [1] https://bou.ke/blog/nixos-containers/ https://bou.ke/blog/nixos-containers/

shakow 2mo ago

> let us ship delta updates over crappy wifi connections to Linux computers on robots How were you doing that? Running your own channels, or something more complex? Also, did you use a push or a pull model w.r.t. the robots?

mikepurvis 2mo ago
At the time I was there we hadn't made it all the way to NixOS on the targets, so it was an Ubuntu "base" + Nix managing the app workspace as a kind of pseudo container, though able to bring more of its own system configuration with it (https://github.com/numtide/system-manager https://github.com/numtide/system-manager) than the more typical nuisance setup of container + separate outer config managed by ansible or a deb or whatever. As a result of that, in "production" we still actually delivered full rootfs images just to be absolutely certain, and the delta upgrades were for smaller test fleets that could updated hourly just via simple push tooling. If I was building it from scratch though, I'd probably just do NixOS + colmena, and do a push model forever. It's not worth the saved SSH connection to not have those logs and status messages coming back to the central coordinator immediately.