5 ms·

Okay I'm a Nix enthusiast but you'll have to trust me when I say that I'm not criticizing them for moving away from Nix; it isn't that strong of an emotional at

by jchw 1y ago

Okay I'm a Nix enthusiast but you'll have to trust me when I say that I'm not criticizing them for moving away from Nix; it isn't that strong of an emotional attachment. However, I'm not really sure I understand some of these complaints and they really could use more explanation. For example: > The biggest problem with Nix is its commit-based package versioning. Only the latest major version of each package is available, with versions tied to specific commits in the nixpkgs repo. While Nixpkgs is an amazing resource, Nix != Nixpkgs. Nixpkgs is highly unideal for cases where you want to be able to pull arbitrary versions of toolchains, but it is not the only way to go. For example, there is amazingly good Nix tooling for pulling an arbitrary version of Rust. Other Nix-based developer tools have shown how you can do this well. > no way of splitting up the Nix dependencies into separate layers That doesn't make any sense. You can literally just split them into separate layers in whatever arbitrary fashion you'd like. The built-in Nixpkgs docker tooling has some support for this even. > We also changed the codebase from Rust to Go because of the Buildkit libraries. This part is not related to Nix, but I find it interesting anyways. Obviously most people don't transition programming languages on a whim, it's generally something you do when you're already planning on building from scratch anyways. To me it almost sounds like different people worked on Railpacks vs Nixpacks. (I've definitely seen what happens when people not familiar with Nix wind up having to deal with unfinished Nix solutions within an organization. It is not pretty, as most people are unwilling to try to figure out Nix. I don't generally use Nix at work out of fear of causing this situation.)

cchance 1y ago
Honestly this feels more like rail... wants to make their own version, hence a new railX lol
- imtringued 1y ago
  Correct
dleslie 1y ago
I don't use Nix, however this seems dismissive: > While Nixpkgs is an amazing resource, Nix != Nixpkgs. If Nixpkgs is the default and alternatives require additional research and effort then for most users it _is_ Nix. > That doesn't make any sense. You can literally just split them into separate layers in whatever arbitrary fashion you'd like. The built-in Nixpkgs docker tooling has some support for this even. Is this obvious, simple, and default behaviour?
- whateveracct 1y ago
  Nixpkgs isn't Nix and in production you rarely just use Nixpkgs verbatim. It's trivial to overlay whatever versions you want (including forks), and I'd say it's expected for any company in production to manage their package set. We are talking about a company full of professionals. If they need something obvious, simple, and default to manage their build - the core business function that turns their text into deployable artifacts - maybe there is a skill culture issue. The industry is full of ineptitude though.
  - eddythompson80 1y ago
    > The industry is full of ineptitude though. While I disagree with the person you're replying to, I find your reply dismissive. I don't know the behind-the-scnene reasons for this, but I can very very easily apply a very similar situation to this from my experience. Nix is a full blown functional programming language along with a very rich (and poorly documented, niche, only second to C++ template in error comprehensibility[1]) ecosystem in itself. It's not like "docker" or "kubernetes" where you're mostly dealing with "data" files like yaml, json or Dockerfile. You're dealing with a complex programming project. With that in mind: - You have a core team with 1 or 2 people with Nix passion/expertise. - Those people do most of the heavy lifting in implementation. - They onboarding the team on to Nix - They evangelize Nix through the org/company - They mod and answer all the "#nix-discussions" channel questions Initially the system is fairly successful and everything is good. over the next 5-6 years it would accumulate a lot of feature asks. The original "Nix person" has long left. Most of the original people have moved either to other projects or not particularly that passionate about Nix. In fact, the "best" developer you have who has inherited the whole Nix thing has only really had to deal with all the shit parts of Nix and the system. They are they ones fixing issues, dealing with bugs, etc. All while maintaining 3 stacks, a Nix stack, a Go stack, and a Rust stack. Eventually that person/team that's annoyed by maintaining the Nix project wins. They want to own that code. They don't want to use Nix any more. They know what's needed, they want to implement it as part of their main Go stack that they are actively working on. They can optimize things for their special case without having to worry about "implementing it the Nix way" or "doing it upstream". They promise you (the management who is open to the idea, but trying to understand the ROI) feature parity + top 5 feature asks for the initial release. You trust the team enough to let them do what they think is best. [1]: LLMs are really good at suggesting a solution given an error message. Nix errors bring them to their knees. It's always "Hmmm.... it appears that there is an error in your configuration... have you tried a `git revert`?"
jrockway 1y ago
I think this is pretty well stated. I'll add that while nixpkgs isn't nix, nixpkgs is kind of the good part. I use NixOS and for the first time in my life, I'm using the latest version of the Linux kernel on release day. That's pretty excellent. While I've come to tolerate Debian Stable in my old age, it is always like stepping a few years into the past ;) The Nix language is something I could criticize for hours without getting bored, but it is what it is. It's old and they did the best they could and it's probably not worth changing. The Nix build system feels awfully primitive to me, often rebuilding stuff that doesn't need to be rebuilt for no good reason. (For example, my NixOS installer ISO has a ton of the build depend on the cmdline I pass to the kernel [just console=ttyS2,1500000n8], and so changing the speed of my serial port requires about 3 minutes of build time. It's goofy and makes me laugh, I'm not going to stop using Nix because of it... but it's also something that I wouldn't let happen in MY build.) Nix for Docker images is, in my opinion, what it's the worst at. A long time ago, I was writing some software in Go and needed to add the pg_dump binary from Postgres to my container image. The infrastructure team suggested using Nix, which I did, but our images blew up from 50MB of our compressed go binary to 1.5GB of God Knows What. pg_dump is 464K. I ended up doing things my way, with Bazel and rules_debian to install apt packages, and the result (on top of distroless) was much cleaner and more compact. My opinion with some actual Nix experience is that a Nix system always ends up being 1.4GB. My installer ISO is 1.4GB. My freshly installed machine is 1.4GB. That's just how it is, for whatever reason. Finally, the whole "I would like to build a large C++ project" situation is a well worn path. s/C++/Rust doesn't change anything material. There are build systems that exist to make the library situation more tolerable. They are all as complicated as Nix, but some work much better for this use case. Nix is trying to be a build system for building other people's software, supporting nixpkgs, and lands on the very generic side of things. Build systems that are designed for building your software tend to do better at that job. Personally, I'm happy with Bazel and probably wouldn't use anything else (except "go build" for go-only projects), but there are many, many, many other options. 99% of the time, you should use that instead of Nix (and write a flake so people can install the latest version of Your Thing with home-manager; or maybe I'm just the only person that uses their own software day to day and you don't actually need to do that...)
- internet_points 1y ago
baobun 1y ago
I think the part that's easy to miss is that their users are devs who will want to specify their own dependencies and versions for arbitrary packages. The way nix works with the way nixpkgs is structured, pinning a version of any package means pinning a commit of the entire nixpkgs tree. Since package builds of node/python/ruby packages depend on stuff outside of the package dir in the tree, you need that mapping between versions and commits. It is also a leaky abstraction, so they will need to expose that to their users, who now may run into situations where they need to align various states of the nixpkgs repo when they just wanted to "yarn add new-fancy-nodejs-package-with-linked–native-deps". Using nix without nixpkgs may be fine for more scoped use but seems hard to justify for a platform like Railway.
- Nullabillity 1y ago
  > Since package builds of node/python/ruby packages depend on stuff outside of the package dir in the tree, you need that mapping between versions and commits. > Using nix without nixpkgs may be fine for more scoped use but seems hard to justify for a platform like Railway. Nixpkgs isn't all-or-nothing. You're right that Nixpkgs itself rarely packages more than one version of something, but the standard approach for "language package managers" is that you use a tool like crate2nix[0] which automatically generates pinned derivations for all of your dependencies.[1] For system dependencies which aren't covered by your language package manager.. you're basically in the same position as for something like Debian: you can either pull it from Nixpkgs (and give up control beyond "which Nixpkgs am I on?") or you can write/fork your own package. Or you can pull specific packages from specific Nixpkgs checkouts and splice them into your "main" Nixpkgs version as an overlay (though this is definitely getting into the Weird Territory(tm)). [0]: https://github.com/nix-community/crate2nix https://github.com/nix-community/crate2nix [1]: For example: https://github.com/stackabletech/secret-operator/blob/30f0eb787857b47b832ff0ed06553aa21316d100/Cargo.nix https://github.com/stackabletech/secret-operator/blob/30f0eb...
  - deleted 1y ago
    [deleted]
- craftkiller 1y ago
gosub100 1y ago
> Nix != Nixpkgs I've been told this when trying FreeBSD in regards to freebsd ports. pkg generally works fine for me, but one day I tried to go off the beaten path and compile vim with some custom USE flags (I forget what they are called in freebsd) in the ports section. It pulled down 20+ dependencies and each `make menuconfig` kept asking me "would you like any of these options" I selected a few that seemed reasonable, and lo-and-behold, package 16 out of 23 fails because "this-requires-that and that-needs-Fubar3.32.1 and Fubar3 is deprecated for Fubar4" and I just gave up. I get that the Core OS devs can't support all 10k+ packages, but they should also be very clear that if you actually try to use them (i.e. enable custom features, not just compile stock code) there's a high chance they won't work. Another option would be to yank them from the ports list if they don't compile and require at least some standard, independently-produced build to succeed before they appear in portsnap fetch.
spease 1y ago
> While Nixpkgs is an amazing resource, Nix != Nixpkgs. Nixpkgs is highly unideal for cases where you want to be able to pull arbitrary versions of toolchains, but it is not the only way to go. For example, there is amazingly good Nix tooling for pulling an arbitrary version of Rust. Other Nix-based developer tools have shown how you can do this well. It’s so exhausting that every single time a basic use issue comes up with nix, the response is “but there’s a way to work around it (that you need tribal knowledge for and will require writing dozens to hundreds of lines of code to fix in a language that doesn’t work like any of the mainstreams with bad error messages and poorly documented standard libraries)”. People’s problems with nix are not that it isn’t turing-complete, it’s that it often creates more problems than it solves by refusing to provide a simple first-class API that interoperates with idiomatic projects in that ecosystem that just works. If every project you try to use nix for devolves into centering around trying to fix issues with nix that you have to write your own modules for, why even bother using nix instead of mainstream tools with good documentation? Exactly what happened in this case. In most cases people are probably just deciding to use docker instead. Nix’s refusal to address practical developer experience issues for a developer-facing product on a non-geologic timescale in favor of ideological pure flakes is rather frustrating. Yes, people are contributing their own time, but it’s so damn frustrating to see so much technical effort going into something that’s rendered practically unusable because of bad UX.
- __MatrixMan__ 1y ago
  You can use nixpkgs as a single versioned input, and you can also give certain things their own distinct input to be tracked separately in flake.lock, but I wouldn't describe either as a "workaround" for the other. It is a bit of a headache to have to pick which mode you want for each dependency, but I'm not sure that's a headache that can be dispensed with via UX improvements. > why even bother using nix instead of mainstream tools with good documentation? nix lets you have a single source of truth for all of it. It's not nix vs apt or nix vs pip, but rather nix vs (pip & (apt|brew)). So far as I know, the only other tool that scratches that itch is bazel, otherwise you're kind of stuck with multiple overlapping packaging strategies that don't communicate, which is a recipe for "works on my machine".
- jchw 1y ago
cpuguy83 1y ago
I don't understand the versioning argument. New to nix, but I definitely have packages from a specific commit.

by jchw 1y ago

cchance 1y ago

Honestly this feels more like rail... wants to make their own version, hence a new railX lol

imtringued 1y ago
Correct

dleslie 1y ago

I don't use Nix, however this seems dismissive: > While Nixpkgs is an amazing resource, Nix != Nixpkgs. If Nixpkgs is the default and alternatives require additional research and effort then for most users it _is_ Nix. > That doesn't make any sense. You can literally just split them into separate layers in whatever arbitrary fashion you'd like. The built-in Nixpkgs docker tooling has some support for this even. Is this obvious, simple, and default behaviour?

whateveracct 1y ago
Nixpkgs isn't Nix and in production you rarely just use Nixpkgs verbatim. It's trivial to overlay whatever versions you want (including forks), and I'd say it's expected for any company in production to manage their package set. We are talking about a company full of professionals. If they need something obvious, simple, and default to manage their build - the core business function that turns their text into deployable artifacts - maybe there is a skill culture issue. The industry is full of ineptitude though.
- eddythompson80 1y ago
  > The industry is full of ineptitude though. While I disagree with the person you're replying to, I find your reply dismissive. I don't know the behind-the-scnene reasons for this, but I can very very easily apply a very similar situation to this from my experience. Nix is a full blown functional programming language along with a very rich (and poorly documented, niche, only second to C++ template in error comprehensibility[1]) ecosystem in itself. It's not like "docker" or "kubernetes" where you're mostly dealing with "data" files like yaml, json or Dockerfile. You're dealing with a complex programming project. With that in mind: - You have a core team with 1 or 2 people with Nix passion/expertise. - Those people do most of the heavy lifting in implementation. - They onboarding the team on to Nix - They evangelize Nix through the org/company - They mod and answer all the "#nix-discussions" channel questions Initially the system is fairly successful and everything is good. over the next 5-6 years it would accumulate a lot of feature asks. The original "Nix person" has long left. Most of the original people have moved either to other projects or not particularly that passionate about Nix. In fact, the "best" developer you have who has inherited the whole Nix thing has only really had to deal with all the shit parts of Nix and the system. They are they ones fixing issues, dealing with bugs, etc. All while maintaining 3 stacks, a Nix stack, a Go stack, and a Rust stack. Eventually that person/team that's annoyed by maintaining the Nix project wins. They want to own that code. They don't want to use Nix any more. They know what's needed, they want to implement it as part of their main Go stack that they are actively working on. They can optimize things for their special case without having to worry about "implementing it the Nix way" or "doing it upstream". They promise you (the management who is open to the idea, but trying to understand the ROI) feature parity + top 5 feature asks for the initial release. You trust the team enough to let them do what they think is best. [1]: LLMs are really good at suggesting a solution given an error message. Nix errors bring them to their knees. It's always "Hmmm.... it appears that there is an error in your configuration... have you tried a `git revert`?"

jrockway 1y ago

I think this is pretty well stated. I'll add that while nixpkgs isn't nix, nixpkgs is kind of the good part. I use NixOS and for the first time in my life, I'm using the latest version of the Linux kernel on release day. That's pretty excellent. While I've come to tolerate Debian Stable in my old age, it is always like stepping a few years into the past ;) The Nix language is something I could criticize for hours without getting bored, but it is what it is. It's old and they did the best they could and it's probably not worth changing. The Nix build system feels awfully primitive to me, often rebuilding stuff that doesn't need to be rebuilt for no good reason. (For example, my NixOS installer ISO has a ton of the build depend on the cmdline I pass to the kernel [just console=ttyS2,1500000n8], and so changing the speed of my serial port requires about 3 minutes of build time. It's goofy and makes me laugh, I'm not going to stop using Nix because of it... but it's also something that I wouldn't let happen in MY build.) Nix for Docker images is, in my opinion, what it's the worst at. A long time ago, I was writing some software in Go and needed to add the pg_dump binary from Postgres to my container image. The infrastructure team suggested using Nix, which I did, but our images blew up from 50MB of our compressed go binary to 1.5GB of God Knows What. pg_dump is 464K. I ended up doing things my way, with Bazel and rules_debian to install apt packages, and the result (on top of distroless) was much cleaner and more compact. My opinion with some actual Nix experience is that a Nix system always ends up being 1.4GB. My installer ISO is 1.4GB. My freshly installed machine is 1.4GB. That's just how it is, for whatever reason. Finally, the whole "I would like to build a large C++ project" situation is a well worn path. s/C++/Rust doesn't change anything material. There are build systems that exist to make the library situation more tolerable. They are all as complicated as Nix, but some work much better for this use case. Nix is trying to be a build system for building other people's software, supporting nixpkgs, and lands on the very generic side of things. Build systems that are designed for building your software tend to do better at that job. Personally, I'm happy with Bazel and probably wouldn't use anything else (except "go build" for go-only projects), but there are many, many, many other options. 99% of the time, you should use that instead of Nix (and write a flake so people can install the latest version of Your Thing with home-manager; or maybe I'm just the only person that uses their own software day to day and you don't actually need to do that...)

internet_points 1y ago

baobun 1y ago

I think the part that's easy to miss is that their users are devs who will want to specify their own dependencies and versions for arbitrary packages. The way nix works with the way nixpkgs is structured, pinning a version of any package means pinning a commit of the entire nixpkgs tree. Since package builds of node/python/ruby packages depend on stuff outside of the package dir in the tree, you need that mapping between versions and commits. It is also a leaky abstraction, so they will need to expose that to their users, who now may run into situations where they need to align various states of the nixpkgs repo when they just wanted to "yarn add new-fancy-nodejs-package-with-linked–native-deps". Using nix without nixpkgs may be fine for more scoped use but seems hard to justify for a platform like Railway.

Nullabillity 1y ago
> Since package builds of node/python/ruby packages depend on stuff outside of the package dir in the tree, you need that mapping between versions and commits. > Using nix without nixpkgs may be fine for more scoped use but seems hard to justify for a platform like Railway. Nixpkgs isn't all-or-nothing. You're right that Nixpkgs itself rarely packages more than one version of something, but the standard approach for "language package managers" is that you use a tool like crate2nix[0] which automatically generates pinned derivations for all of your dependencies.[1] For system dependencies which aren't covered by your language package manager.. you're basically in the same position as for something like Debian: you can either pull it from Nixpkgs (and give up control beyond "which Nixpkgs am I on?") or you can write/fork your own package. Or you can pull specific packages from specific Nixpkgs checkouts and splice them into your "main" Nixpkgs version as an overlay (though this is definitely getting into the Weird Territory(tm)). [0]: https://github.com/nix-community/crate2nix https://github.com/nix-community/crate2nix [1]: For example: https://github.com/stackabletech/secret-operator/blob/30f0eb787857b47b832ff0ed06553aa21316d100/Cargo.nix https://github.com/stackabletech/secret-operator/blob/30f0eb...
- deleted 1y ago
  [deleted]
craftkiller 1y ago

gosub100 1y ago

> Nix != Nixpkgs I've been told this when trying FreeBSD in regards to freebsd ports. pkg generally works fine for me, but one day I tried to go off the beaten path and compile vim with some custom USE flags (I forget what they are called in freebsd) in the ports section. It pulled down 20+ dependencies and each `make menuconfig` kept asking me "would you like any of these options" I selected a few that seemed reasonable, and lo-and-behold, package 16 out of 23 fails because "this-requires-that and that-needs-Fubar3.32.1 and Fubar3 is deprecated for Fubar4" and I just gave up. I get that the Core OS devs can't support all 10k+ packages, but they should also be very clear that if you actually try to use them (i.e. enable custom features, not just compile stock code) there's a high chance they won't work. Another option would be to yank them from the ports list if they don't compile and require at least some standard, independently-produced build to succeed before they appear in portsnap fetch.

spease 1y ago

> While Nixpkgs is an amazing resource, Nix != Nixpkgs. Nixpkgs is highly unideal for cases where you want to be able to pull arbitrary versions of toolchains, but it is not the only way to go. For example, there is amazingly good Nix tooling for pulling an arbitrary version of Rust. Other Nix-based developer tools have shown how you can do this well. It’s so exhausting that every single time a basic use issue comes up with nix, the response is “but there’s a way to work around it (that you need tribal knowledge for and will require writing dozens to hundreds of lines of code to fix in a language that doesn’t work like any of the mainstreams with bad error messages and poorly documented standard libraries)”. People’s problems with nix are not that it isn’t turing-complete, it’s that it often creates more problems than it solves by refusing to provide a simple first-class API that interoperates with idiomatic projects in that ecosystem that just works. If every project you try to use nix for devolves into centering around trying to fix issues with nix that you have to write your own modules for, why even bother using nix instead of mainstream tools with good documentation? Exactly what happened in this case. In most cases people are probably just deciding to use docker instead. Nix’s refusal to address practical developer experience issues for a developer-facing product on a non-geologic timescale in favor of ideological pure flakes is rather frustrating. Yes, people are contributing their own time, but it’s so damn frustrating to see so much technical effort going into something that’s rendered practically unusable because of bad UX.

__MatrixMan__ 1y ago
You can use nixpkgs as a single versioned input, and you can also give certain things their own distinct input to be tracked separately in flake.lock, but I wouldn't describe either as a "workaround" for the other. It is a bit of a headache to have to pick which mode you want for each dependency, but I'm not sure that's a headache that can be dispensed with via UX improvements. > why even bother using nix instead of mainstream tools with good documentation? nix lets you have a single source of truth for all of it. It's not nix vs apt or nix vs pip, but rather nix vs (pip & (apt|brew)). So far as I know, the only other tool that scratches that itch is bazel, otherwise you're kind of stuck with multiple overlapping packaging strategies that don't communicate, which is a recipe for "works on my machine".
jchw 1y ago

cpuguy83 1y ago

I don't understand the versioning argument. New to nix, but I definitely have packages from a specific commit.