4 ms·

I'm just waiting for them to solve the problem of the package manager taking multiple gigabytes of memory to run. My last experience with it involved a lot of "

I'm just waiting for them to solve the problem of the package manager taking multiple gigabytes of memory to run. My last experience with it involved a lot of "unable to fork: Cannot allocate memory". It's insane that a package manager would even exceed even 100mb of memory usage, let alone run into multiple gigabytes. No way there's that much information to manage.

Filligree 7y ago
That's because it isn't a package manager, it's an interpreted programming language. The package management requires running a huge program in that language. Of course knowing why it happens didn't make it less frustrating, but it's hard to fix. What you can do is build your system remotely, on a more capable machine, perhaps with nixos-rebuild --target-host. (That's a shell script, and fairly readable.)
- kstenerud 7y ago
  To me this sounds absolutely insane. Package managers that work in a couple of megabytes of ram is a solved problem. How can this be unsolvable?
  - Filligree 7y ago
    It's not really the package-building process that takes tons of RAM. It's the OS-building process, where it pulls everything necessary to describe the current state of your machine into RAM, simultaneously. This typically means several hundred packages at least, plus a lot of code that's not as easily classifiable as "packages". The config file generators are written in the same language. Other operating systems punt on the challenge; there's no equivalent in e.g. Debian.
    - kstenerud 7y ago
      I get that, but it ran out of ram on a machine with 2gb of ram. Even if there were 10 million things to keep track of, that's 200 bytes per thing! If we're talking about a file system with 100,000 files, that's 20 kb of metadata per file. Actually, more than that because it exhausted the ram. Those numbers just don't add up.
taktoa 7y ago
That's actually just a command line infelicity; the advice to use `nix-env -i foobar` is really misguided, and you can avoid excessive memory usage by doing `nix-env -iA nixos.foobar` (or `nixpkgs.foobar` if you are using it on a non-NixOS machine). The difference is that in the former case, `foobar` is a regular expression that is being matched against the `name` field of every package in nixpkgs, whereas in the latter case, `foobar` is an attribute path that is used to traverse the nested Nix objects that make up nixpkgs. Since Nix is lazy, the former results in evaluating a LOT of stuff, whereas the latter does not.
- kstenerud 7y ago
  Even if it were evaluating the entire library of Congress, there's no valid reason for it to consume gigabytes of memory. I really can't understand why this is so...

Filligree 7y ago

That's because it isn't a package manager, it's an interpreted programming language. The package management requires running a huge program in that language. Of course knowing why it happens didn't make it less frustrating, but it's hard to fix. What you can do is build your system remotely, on a more capable machine, perhaps with nixos-rebuild --target-host. (That's a shell script, and fairly readable.)

kstenerud 7y ago
To me this sounds absolutely insane. Package managers that work in a couple of megabytes of ram is a solved problem. How can this be unsolvable?
- Filligree 7y ago
  It's not really the package-building process that takes tons of RAM. It's the OS-building process, where it pulls everything necessary to describe the current state of your machine into RAM, simultaneously. This typically means several hundred packages at least, plus a lot of code that's not as easily classifiable as "packages". The config file generators are written in the same language. Other operating systems punt on the challenge; there's no equivalent in e.g. Debian.
  - kstenerud 7y ago
    I get that, but it ran out of ram on a machine with 2gb of ram. Even if there were 10 million things to keep track of, that's 200 bytes per thing! If we're talking about a file system with 100,000 files, that's 20 kb of metadata per file. Actually, more than that because it exhausted the ram. Those numbers just don't add up.

taktoa 7y ago

That's actually just a command line infelicity; the advice to use `nix-env -i foobar` is really misguided, and you can avoid excessive memory usage by doing `nix-env -iA nixos.foobar` (or `nixpkgs.foobar` if you are using it on a non-NixOS machine). The difference is that in the former case, `foobar` is a regular expression that is being matched against the `name` field of every package in nixpkgs, whereas in the latter case, `foobar` is an attribute path that is used to traverse the nested Nix objects that make up nixpkgs. Since Nix is lazy, the former results in evaluating a LOT of stuff, whereas the latter does not.

kstenerud 7y ago
Even if it were evaluating the entire library of Congress, there's no valid reason for it to consume gigabytes of memory. I really can't understand why this is so...