3 ms·

Since I switched to nixos all these articles read like people fiddling with struct-packing and optimising their application memory layout. The compiler does it

Since I switched to nixos all these articles read like people fiddling with struct-packing and optimising their application memory layout. The compiler does it well enough now that we don't have to, so it is with nix and your application filesystem.

jeltz 2y ago
The C compiler does not optimize struct packing at all. Some languages like Rust allows optimizing struct layouts but even for Rust struct layout can matter if you care about cache locality and vector operations.
__MatrixMan__ 2y ago
I had a similar feeling during the recent crowdstrike incident. Hearing about how people couldn't operate their lathe or whatever because of an update, my initial reaction was: > Just boot yesterday's config and get on with your life But then, that's one of those NixOS things that we take for granted.
- sshine 2y ago
  Not just NixOS. Ubuntu with ZFS creates a snapshot of the system on every `apt install` command. But yeah, it’s pretty great to know that if your system fails, just `git restore --staged` and redeploy.
thot_experiment 2y ago
Using shared libraries is optimizing for a very different set of constraints than nixos, which iirc keeps like 90 versions of the same thing around just so everyone can have the one they want. There are still people who are space constrained. (I haven't touched nix in years so maybe i'm off base on this) > The compiler does it well enough now that we don't have to You know I see people say this and then I see some code with some nested loops running 2x as fast as code written with list comprehensions and I remember that it's actually. "The compiler does it well enough now that we don't have to as long as you understand the way the compiler works at a low enough level that you don't use patterns that will trip it up and even then you should still be benchmarking your perf because black magic doesn't always work the way you think it works" Struct packing too can still lead to speedups/space gains if you were previously badly aligned, which is absolutely something that can happen if you leave everything on auto.
- anotherhue 2y ago
  No argument, if you're perf sensitive and aren't benchmarking every change then it's a roll of a dice as to whether llvm will bless your build. The usual claim stands though, on a LoC basis a vanishingly small amount of code is perf sensitive (embedded likely more TBF)
- thomastjeffery 2y ago
  The problem with Nix is that it's a single monolithic package archive. Every conceivable package must go somewhere in the nixpkgs tree, and is expected to be as vanilla as possible. On top of that, there is the all-packages.nix global namespace, which implicitly urges everyone to use the same dependency versions; but in practice just results in a mess of redundant names like package_version.1.12_x-feature-enabled... The move toward flakes only replaces this problem with intentional fragmentation. Even so, flakes will probably end up being the best option if it ever gets coherent documentation.
- matrss 2y ago
  > Using shared libraries is optimizing for a very different set of constraints than nixos, which iirc keeps like 90 versions of the same thing around just so everyone can have the one they want. This isn't really true. One version of nixpkgs (i.e. a specific commit of https://github.com/NixOS/nixpkgs https://github.com/NixOS/nixpkgs) generally has one version of every package and other packages from the same nixpkgs version depending on it will use the same one as a dependency. Sometimes there are multiple versions (different major versions, different compile time options, etc.) but that is the same with other distros as well. In that sense, NixOS is very similar to a more traditional distribution, just that NixOS' functional package management better encapsulates the process of making changes to its package repository compared to the ad-hoc nature of a mutable set of binary packages like traditional distros and makes it possible to see and rebuild the dependency graph at every point in time while a more traditional distro doesn't give you e.g. the option to pretend that it's 10 days or months ago. You only really get multiple versions of the same packages if you start mixing different nixpkgs revisions, which is really only a good idea in edge cases. Old ones are also kept around for rollbacks, but those can be garbage collected.
cbmuser 2y ago
The compiler takes care of vulnerabilities? There was a recent talk that discussed the security nightmare with dozens of different versions of shared libraries in NixOS and how difficult it is for the distribution maintainers to track and update them.

jeltz 2y ago

The C compiler does not optimize struct packing at all. Some languages like Rust allows optimizing struct layouts but even for Rust struct layout can matter if you care about cache locality and vector operations.

__MatrixMan__ 2y ago

I had a similar feeling during the recent crowdstrike incident. Hearing about how people couldn't operate their lathe or whatever because of an update, my initial reaction was: > Just boot yesterday's config and get on with your life But then, that's one of those NixOS things that we take for granted.

sshine 2y ago
Not just NixOS. Ubuntu with ZFS creates a snapshot of the system on every `apt install` command. But yeah, it’s pretty great to know that if your system fails, just `git restore --staged` and redeploy.

thot_experiment 2y ago

Using shared libraries is optimizing for a very different set of constraints than nixos, which iirc keeps like 90 versions of the same thing around just so everyone can have the one they want. There are still people who are space constrained. (I haven't touched nix in years so maybe i'm off base on this) > The compiler does it well enough now that we don't have to You know I see people say this and then I see some code with some nested loops running 2x as fast as code written with list comprehensions and I remember that it's actually. "The compiler does it well enough now that we don't have to as long as you understand the way the compiler works at a low enough level that you don't use patterns that will trip it up and even then you should still be benchmarking your perf because black magic doesn't always work the way you think it works" Struct packing too can still lead to speedups/space gains if you were previously badly aligned, which is absolutely something that can happen if you leave everything on auto.

anotherhue 2y ago
No argument, if you're perf sensitive and aren't benchmarking every change then it's a roll of a dice as to whether llvm will bless your build. The usual claim stands though, on a LoC basis a vanishingly small amount of code is perf sensitive (embedded likely more TBF)
thomastjeffery 2y ago
The problem with Nix is that it's a single monolithic package archive. Every conceivable package must go somewhere in the nixpkgs tree, and is expected to be as vanilla as possible. On top of that, there is the all-packages.nix global namespace, which implicitly urges everyone to use the same dependency versions; but in practice just results in a mess of redundant names like package_version.1.12_x-feature-enabled... The move toward flakes only replaces this problem with intentional fragmentation. Even so, flakes will probably end up being the best option if it ever gets coherent documentation.
matrss 2y ago
> Using shared libraries is optimizing for a very different set of constraints than nixos, which iirc keeps like 90 versions of the same thing around just so everyone can have the one they want. This isn't really true. One version of nixpkgs (i.e. a specific commit of https://github.com/NixOS/nixpkgs https://github.com/NixOS/nixpkgs) generally has one version of every package and other packages from the same nixpkgs version depending on it will use the same one as a dependency. Sometimes there are multiple versions (different major versions, different compile time options, etc.) but that is the same with other distros as well. In that sense, NixOS is very similar to a more traditional distribution, just that NixOS' functional package management better encapsulates the process of making changes to its package repository compared to the ad-hoc nature of a mutable set of binary packages like traditional distros and makes it possible to see and rebuild the dependency graph at every point in time while a more traditional distro doesn't give you e.g. the option to pretend that it's 10 days or months ago. You only really get multiple versions of the same packages if you start mixing different nixpkgs revisions, which is really only a good idea in edge cases. Old ones are also kept around for rollbacks, but those can be garbage collected.

cbmuser 2y ago

The compiler takes care of vulnerabilities? There was a recent talk that discussed the security nightmare with dozens of different versions of shared libraries in NixOS and how difficult it is for the distribution maintainers to track and update them.