4 ms·

To date I've paid no attention to Flakes because whenever I look at them, they appear to require me to manually list all target architectures in the Flakes file

To date I've paid no attention to Flakes because whenever I look at them, they appear to require me to manually list all target architectures in the Flakes file, which is ridiculous. I'm not going to make the code I publish needlessly dependent on specific architectures. This problem seems to be solved commonly in two ways: - People thoughtlessly and senselessly listing only x86; - People using the popular flake utility repository. Having to rely on an unendorsed third party just to write architecture-independent code isn't very appealing to me. This seems like a serious deficiency in the design of Flakes, though it would be tolerable at least if this functionality was at least in Nixpkgs.

smoldesu 3y ago
I haven't personally found this to be an issue. I usually define the arches I intend to hit at the beginning of the flake, then pass that list to pkgs so it can track down the independent dependencies: allSystems = [ "x86_64-linux" "aarch64-linux" "x86_64-darwin" "aarch64-darwin" ]; forAllSystems = f: nixpkgs.lib.genAttrs allSystems (system: f { pkgs = import nixpkgs { inherit system; }; }); See? Easy as getting a tooth pulled!
pmarreck 3y ago
> they appear to require me to manually list all target architectures in the Flakes file, which is ridiculous Well, we're talking about evaluating a pure function (literally the entire philosophy of Nix is treating a build like a pure function). Those things are variable state as an input which affects the output, and therefore must be specified as inputs. I don't see the problem here (if you have already accepted that you want to treat builds as pure functions with total encapsulation of the state required to run them, and thus having deterministic outcomes).
- seabass-labrax 3y ago
  It would be nice to have a shorthand value that means "evaluate this pure function exactly the same way regardless of the 'system' parameter", which would be helpful for completely architecture-less packages (such as interpreted scripts).
  - pxc 3y ago
    Which deps they pull in is still determined by system because their interpreters are in the dependency closure and what arch Perl is built for affects that Perl package's hash, etc.
  - pmarreck 3y ago
    But as the other person said, you're still building in assumptions in that case, which is bad (from the perspective of the whole philosophy of the Nix design choices). Because the script assumes that the script runner (which is also compiled to a certain CPU and OS!) will work when given the script as input, for example- and it is not guaranteed to. If you include the OS and CPU arch, it is FAR MORE guaranteed to (although perhaps, not 100% provably). Perhaps you haven't dealt with enough Python packages that make assumptions about the availability and state/version of underlying OS-level libraries (and Python itself: See the transition from versions 2 to 3 as an example of this chaos) to understand how important it is to control for assumptions. What you gain in convenience today, you pay tenfold for as "interest" later (this is thus definitely one aspect of the "tech debt" problem).
    - seabass-labrax 3y ago
      I certainly have experienced my fair share of architecture-dependant Python horror stories, and I agree with your philosophy of convenience as a kind of loan against tech debt! However, if I package a Python script that appears not to use any system libraries or features (maybe I even wrote it myself), I would define the build for any architecture. One reason for doing this would be for when $DISTRO declares that they will now support a new architecture. What packages would then be expected to require patches? If existing packages were marked as 'all', those can reasonably be expected to succeed and should cause noisy errors if they don't. Conversely, packages which were explicitly marked for specific architectures (even if that is all the architectures that the distribution supported when they were created) can be expected not to work on the new architecture without modification. This does of course require the packager to actually know the difference between using the keywords and just marking every known architecture. And to be fair, what I was originally advocating for in my comment was more like Debian's 'any' wildcard, which is not quite as strong as the assertion that the package is completely architecture-independent. I would say that a specific tag for architecture-independent builds actually improves the situation: explicitly acknowledging the assumption that exotic architectures will never be an issue. Of course, as your sibling comment mentions, the interpreter is still part of the closure in Nix, so maybe this is incompatible with the existing choices made in the Nix ecosystem.
- hlandau 3y ago
  Except using NixOS without flakes doesn't require this manual enumeration so it's clearly not a requirement, but a design choice. Nothing is ultimately pure since you're running on top of a kernel which can affect computation. It's unclear to me what practical gain you get from trying to obsessively get "maximum purity", which is in any case illusory because you're still running on a kernel from outside the closure. Making it impossible to write portable code isn't a good design.
  - pmarreck 3y ago
    How does this make it "impossible to write portable code"? CPU and OS differences produce different binaries, which are fundamentally different output values. Are you not understanding exactly how this works, or something? If you have an old flake that only defines outputs to, say, ARM, and it's 100 years in the future and no active CPU derives from ARM anymore, that's not the flake's problem. You'd arguably find it far easier to work with the flake in that context (perhaps finding an old emulator to execute it on, and THEN trying to work on migrating that to whatever architectures are used then) than finding the old C code (or whatever) and expecting that to build on whatever tooling is available then. The idea that "C is portable" is laughable, already- it's only portable to the extent that the compiler for it is portable- and then you are right back at the original problem, the dependence on OS and CPU architectures as inputs! > obsessively get "maximum purity", which is in any case illusory I don't understand this argument. Should I stop using functional languages because a cosmic ray could potentially mutate a variable that I expect to be immutable? Could it be that expecting perfect purity is in fact the enemy of "good" in this case? The fact that I've almost universally had better experiences in languages that emphasize "purity" (despite running on "uncertain" underpinnings) seems to undermine this reasoning.
lillecarl 3y ago
It's like C++, you have boostlib which is separate from stdlib but you pretty much need it either way. Or in JavaScript, you need left-pad to align text to the right, which everyone and their uncle depends on. Nix is a programming language, where there are holes people will plug them with libraries, what's weird?

smoldesu 3y ago

I haven't personally found this to be an issue. I usually define the arches I intend to hit at the beginning of the flake, then pass that list to pkgs so it can track down the independent dependencies: allSystems = [ "x86_64-linux" "aarch64-linux" "x86_64-darwin" "aarch64-darwin" ]; forAllSystems = f: nixpkgs.lib.genAttrs allSystems (system: f { pkgs = import nixpkgs { inherit system; }; }); See? Easy as getting a tooth pulled!

pmarreck 3y ago

> they appear to require me to manually list all target architectures in the Flakes file, which is ridiculous Well, we're talking about evaluating a pure function (literally the entire philosophy of Nix is treating a build like a pure function). Those things are variable state as an input which affects the output, and therefore must be specified as inputs. I don't see the problem here (if you have already accepted that you want to treat builds as pure functions with total encapsulation of the state required to run them, and thus having deterministic outcomes).

seabass-labrax 3y ago
It would be nice to have a shorthand value that means "evaluate this pure function exactly the same way regardless of the 'system' parameter", which would be helpful for completely architecture-less packages (such as interpreted scripts).
- pxc 3y ago
  Which deps they pull in is still determined by system because their interpreters are in the dependency closure and what arch Perl is built for affects that Perl package's hash, etc.
- pmarreck 3y ago
  But as the other person said, you're still building in assumptions in that case, which is bad (from the perspective of the whole philosophy of the Nix design choices). Because the script assumes that the script runner (which is also compiled to a certain CPU and OS!) will work when given the script as input, for example- and it is not guaranteed to. If you include the OS and CPU arch, it is FAR MORE guaranteed to (although perhaps, not 100% provably). Perhaps you haven't dealt with enough Python packages that make assumptions about the availability and state/version of underlying OS-level libraries (and Python itself: See the transition from versions 2 to 3 as an example of this chaos) to understand how important it is to control for assumptions. What you gain in convenience today, you pay tenfold for as "interest" later (this is thus definitely one aspect of the "tech debt" problem).
  - seabass-labrax 3y ago
    I certainly have experienced my fair share of architecture-dependant Python horror stories, and I agree with your philosophy of convenience as a kind of loan against tech debt! However, if I package a Python script that appears not to use any system libraries or features (maybe I even wrote it myself), I would define the build for any architecture. One reason for doing this would be for when $DISTRO declares that they will now support a new architecture. What packages would then be expected to require patches? If existing packages were marked as 'all', those can reasonably be expected to succeed and should cause noisy errors if they don't. Conversely, packages which were explicitly marked for specific architectures (even if that is all the architectures that the distribution supported when they were created) can be expected not to work on the new architecture without modification. This does of course require the packager to actually know the difference between using the keywords and just marking every known architecture. And to be fair, what I was originally advocating for in my comment was more like Debian's 'any' wildcard, which is not quite as strong as the assertion that the package is completely architecture-independent. I would say that a specific tag for architecture-independent builds actually improves the situation: explicitly acknowledging the assumption that exotic architectures will never be an issue. Of course, as your sibling comment mentions, the interpreter is still part of the closure in Nix, so maybe this is incompatible with the existing choices made in the Nix ecosystem.
hlandau 3y ago
Except using NixOS without flakes doesn't require this manual enumeration so it's clearly not a requirement, but a design choice. Nothing is ultimately pure since you're running on top of a kernel which can affect computation. It's unclear to me what practical gain you get from trying to obsessively get "maximum purity", which is in any case illusory because you're still running on a kernel from outside the closure. Making it impossible to write portable code isn't a good design.
- pmarreck 3y ago
  How does this make it "impossible to write portable code"? CPU and OS differences produce different binaries, which are fundamentally different output values. Are you not understanding exactly how this works, or something? If you have an old flake that only defines outputs to, say, ARM, and it's 100 years in the future and no active CPU derives from ARM anymore, that's not the flake's problem. You'd arguably find it far easier to work with the flake in that context (perhaps finding an old emulator to execute it on, and THEN trying to work on migrating that to whatever architectures are used then) than finding the old C code (or whatever) and expecting that to build on whatever tooling is available then. The idea that "C is portable" is laughable, already- it's only portable to the extent that the compiler for it is portable- and then you are right back at the original problem, the dependence on OS and CPU architectures as inputs! > obsessively get "maximum purity", which is in any case illusory I don't understand this argument. Should I stop using functional languages because a cosmic ray could potentially mutate a variable that I expect to be immutable? Could it be that expecting perfect purity is in fact the enemy of "good" in this case? The fact that I've almost universally had better experiences in languages that emphasize "purity" (despite running on "uncertain" underpinnings) seems to undermine this reasoning.

lillecarl 3y ago

It's like C++, you have boostlib which is separate from stdlib but you pretty much need it either way. Or in JavaScript, you need left-pad to align text to the right, which everyone and their uncle depends on. Nix is a programming language, where there are holes people will plug them with libraries, what's weird?