3 ms·

You missed what he was saying. Nix solves this problem in multiple different ways. One of which is packaging, but also the entire /nix/store is mounted read-on

You missed what he was saying. Nix solves this problem in multiple different ways. One of which is packaging, but also the entire /nix/store is mounted read-only. A sudo pip install would have no ability to break the system or remove something that was installed by nix.

BiteCode_dev 5y ago
I get that, but since our entire legacy stack is stateful, there is no way around that unless you nixify the entire stuff. Immutable works only if the entire chain is.
- chriswarbo 5y ago
  > our entire legacy stack is stateful, there is no way around that unless you nixify the entire stuff AFAIK Docker images aren't stateful: they're immutable snapshots, content-addressed by their cryptographic "digest". That's an elegant idea, which works perfectly well for legacy systems without "nixifying" anything (although personally I prefer to generate container images using Nix rather than Docker, but that's besides the point ;) ) Images are typically used as a "base", with further "layers" on top; those layers are also immutable: they use the same content-addressable "digest" mechanism, then get union-mounted together. This lets us use a known-good base, and add our application-specific parts on top, like a /var/www/html directory or whatever. The problem with creating a layer via 'apt-get dist-upgrade' is that we no longer have any idea what will be in the image (it depends on when we build it, as repos get updated); we have to store/transmit both the old and new versions (the base and our layer); any security audits for the base image are invalidated; etc. This isn't a "legacy" problem either: it's simply a rejection of the 'base image + layers' concept. It's fine to reject that concept, but there are cleaner ways to do it like debootstrap (which is the legacy solution, around far longer than Dockerfiles running 'apt-get dist-upgrade'). Also, the Dockerfiles controlling this stuff are usually kept in git repos and hence are also immutable. However, there's another anti-pattern of pushing their results to some ":latest" tag, which defeats much of the point of versioning. These sorts of bodges are everywhere. Consider the problem actually being discussed in this article: that files managed by 'pip install' may conflict with those managed by 'apt-get'. APT has been able to detect such conflicts for decades (even longer than Nix!). The article even mentions 'dpkg-divert' (part of APT's conflict-resolution system), but only as a patch for Dockerfiles to avoid the new pip-install conflict detection! A slightly different patch (e.g. running 'dpkg-deb -b' on the pip-installed libraries) would not only avoid problems from the new pip-install behaviour, but would also allow APT to tell us if we're breaking things. I don't see these issues as a 'legacy problem' either; it's more like a WorseIsBetter approach which (a) undermines improvements when they're introduced, (b) propagates throughout the industry as "best practice", then (c) has to be supported forever, even by those who don't care to use it.
  - BiteCode_dev 5y ago
    Each things you talk about are plateform dependent, but the solution for pip is platform independent and those way easier to implement. A perfect solution you can't implement is of no use unfortunately. It's not just about finding a technical solution. A lot of packaging stuff is dealing with organizational barriers.

BiteCode_dev 5y ago

I get that, but since our entire legacy stack is stateful, there is no way around that unless you nixify the entire stuff. Immutable works only if the entire chain is.

chriswarbo 5y ago
> our entire legacy stack is stateful, there is no way around that unless you nixify the entire stuff AFAIK Docker images aren't stateful: they're immutable snapshots, content-addressed by their cryptographic "digest". That's an elegant idea, which works perfectly well for legacy systems without "nixifying" anything (although personally I prefer to generate container images using Nix rather than Docker, but that's besides the point ;) ) Images are typically used as a "base", with further "layers" on top; those layers are also immutable: they use the same content-addressable "digest" mechanism, then get union-mounted together. This lets us use a known-good base, and add our application-specific parts on top, like a /var/www/html directory or whatever. The problem with creating a layer via 'apt-get dist-upgrade' is that we no longer have any idea what will be in the image (it depends on when we build it, as repos get updated); we have to store/transmit both the old and new versions (the base and our layer); any security audits for the base image are invalidated; etc. This isn't a "legacy" problem either: it's simply a rejection of the 'base image + layers' concept. It's fine to reject that concept, but there are cleaner ways to do it like debootstrap (which is the legacy solution, around far longer than Dockerfiles running 'apt-get dist-upgrade'). Also, the Dockerfiles controlling this stuff are usually kept in git repos and hence are also immutable. However, there's another anti-pattern of pushing their results to some ":latest" tag, which defeats much of the point of versioning. These sorts of bodges are everywhere. Consider the problem actually being discussed in this article: that files managed by 'pip install' may conflict with those managed by 'apt-get'. APT has been able to detect such conflicts for decades (even longer than Nix!). The article even mentions 'dpkg-divert' (part of APT's conflict-resolution system), but only as a patch for Dockerfiles to avoid the new pip-install conflict detection! A slightly different patch (e.g. running 'dpkg-deb -b' on the pip-installed libraries) would not only avoid problems from the new pip-install behaviour, but would also allow APT to tell us if we're breaking things. I don't see these issues as a 'legacy problem' either; it's more like a WorseIsBetter approach which (a) undermines improvements when they're introduced, (b) propagates throughout the industry as "best practice", then (c) has to be supported forever, even by those who don't care to use it.
- BiteCode_dev 5y ago
  Each things you talk about are plateform dependent, but the solution for pip is platform independent and those way easier to implement. A perfect solution you can't implement is of no use unfortunately. It's not just about finding a technical solution. A lot of packaging stuff is dealing with organizational barriers.