3 ms·

> Package management, library management, etc are solved problems when it comes to Linux distros. The only practical improvement you can make is containers (or

> Package management, library management, etc are solved problems when it comes to Linux distros. The only practical improvement you can make is containers (or similar). To me this basically reads as "we added a bunch of complexity on top of the problem without actually solving it". Package managers are supposed to prevent conflicts and manage giant dependency graphs, but they're so shit at doing that while actually allowing people to use whatever software they want that we've invented containers to keep everything separate again.

0xbadcafebee 6y ago
> "we added a bunch of complexity on top of the problem without actually solving it" It solved it as much as is possible within the current state of software design. > Package managers are supposed to prevent conflicts and manage giant dependency graphs Actually, they can't do that. They can look for the conflicts they're programmed to look for by packages, and they can walk dependency graphs that are generated by the packages in the repos maintained by a person. The root of the problem is that modern software architecture (even for trendy languages) cannot express the dependencies in the first place. I wrote you an explanation, but it ran to 1,355 words, so I Gist'd it here https://gist.github.com/peterwwillis/e96854532f471c739983c0bba7608834 https://gist.github.com/peterwwillis/e96854532f471c739983c0b...
smilliken 6y ago
Nix[1] is a package manager for linux and macos that solves this problem beautifully. You can have any variation (version, build options, dependencies, etc) of a package installed natively without conflicting with eachother because the filesystem path includes a hash of the build instructions. It works really, really well for reproducibility and eliminating the "works on my machine" problem. I've been using it exckusively for dev machines and servers for years. [1] https://github.com/NixOS/nixpkgs https://github.com/NixOS/nixpkgs
- 0xbadcafebee 6y ago
  I worked for a company that built the same thing in the mid-2000s, and it did not solve all dependency issues, because there's multiple kinds of issues. They're actually kind of unsolveable. I list some of the reasons here https://gist.github.com/peterwwillis/e96854532f471c739983c0bba7608834 https://gist.github.com/peterwwillis/e96854532f471c739983c0b...
  - smilliken 6y ago
    Thanks for sharing. Agreed that runtime dependency conflicts are unavoidable for most programming languages. Your example of sqlite database format conflicts is also a good one. I definitely won't claim that nix has solved runtime dependency conflicts, but it does decisively solve build-time conflicts. Similar to containers like you describe, but reproducible and don't require storing and shipping opaque binary images around. Reproducibility is really the killer feature, IMO. I inherited a bunch of containers that don't build anymore because the build instructions weren't reproducible. Having an old container image that still runs is just tech debt if yiu can't edit the build instructions and genrrate a new one.
    - SQLite 6y ago
      All versions of SQLite from 3.0.0 (2004-06-18) through 3.34.0 (the latest) use the same database file format. So if two applications use different versions of SQLite, it shouldn't matter. The database file will be the same. Now, if one application uses a more recent version of SQLite and also makes use of some new feature (say, for example, generated columns which were added in release 3.31.0) then that might result in a database file that is unreadable by older versions since the older version won't be able to interpret the generated column syntax. But if you assume that all different versions of SQLite that you use have support for all of the features used in the database file (a reasonable assumption in most cases) then the database files are completely portable across versions. Different versions of SQLite can read/write the same database file concurrently. And newer versions of SQLite are always able to read/write databases created by older versions of SQLite, without exception or precondition. So perhaps the "sqlite format conflict" example was not the best, in as much as it will always work as long as the package manager installs the most recent version of SQLite. All of the above is also true for the API.

0xbadcafebee 6y ago

> "we added a bunch of complexity on top of the problem without actually solving it" It solved it as much as is possible within the current state of software design. > Package managers are supposed to prevent conflicts and manage giant dependency graphs Actually, they can't do that. They can look for the conflicts they're programmed to look for by packages, and they can walk dependency graphs that are generated by the packages in the repos maintained by a person. The root of the problem is that modern software architecture (even for trendy languages) cannot express the dependencies in the first place. I wrote you an explanation, but it ran to 1,355 words, so I Gist'd it here https://gist.github.com/peterwwillis/e96854532f471c739983c0bba7608834 https://gist.github.com/peterwwillis/e96854532f471c739983c0b...

smilliken 6y ago

Nix[1] is a package manager for linux and macos that solves this problem beautifully. You can have any variation (version, build options, dependencies, etc) of a package installed natively without conflicting with eachother because the filesystem path includes a hash of the build instructions. It works really, really well for reproducibility and eliminating the "works on my machine" problem. I've been using it exckusively for dev machines and servers for years. [1] https://github.com/NixOS/nixpkgs https://github.com/NixOS/nixpkgs

0xbadcafebee 6y ago
I worked for a company that built the same thing in the mid-2000s, and it did not solve all dependency issues, because there's multiple kinds of issues. They're actually kind of unsolveable. I list some of the reasons here https://gist.github.com/peterwwillis/e96854532f471c739983c0bba7608834 https://gist.github.com/peterwwillis/e96854532f471c739983c0b...
- smilliken 6y ago
  Thanks for sharing. Agreed that runtime dependency conflicts are unavoidable for most programming languages. Your example of sqlite database format conflicts is also a good one. I definitely won't claim that nix has solved runtime dependency conflicts, but it does decisively solve build-time conflicts. Similar to containers like you describe, but reproducible and don't require storing and shipping opaque binary images around. Reproducibility is really the killer feature, IMO. I inherited a bunch of containers that don't build anymore because the build instructions weren't reproducible. Having an old container image that still runs is just tech debt if yiu can't edit the build instructions and genrrate a new one.
  - SQLite 6y ago
    All versions of SQLite from 3.0.0 (2004-06-18) through 3.34.0 (the latest) use the same database file format. So if two applications use different versions of SQLite, it shouldn't matter. The database file will be the same. Now, if one application uses a more recent version of SQLite and also makes use of some new feature (say, for example, generated columns which were added in release 3.31.0) then that might result in a database file that is unreadable by older versions since the older version won't be able to interpret the generated column syntax. But if you assume that all different versions of SQLite that you use have support for all of the features used in the database file (a reasonable assumption in most cases) then the database files are completely portable across versions. Different versions of SQLite can read/write the same database file concurrently. And newer versions of SQLite are always able to read/write databases created by older versions of SQLite, without exception or precondition. So perhaps the "sqlite format conflict" example was not the best, in as much as it will always work as long as the package manager installs the most recent version of SQLite. All of the above is also true for the API.