3 ms·

This one is frustrating because this is an issue that has been solved many times before and I hate seeing it repeated in every new package manager. A vendor nam

by moojd 5y ago

This one is frustrating because this is an issue that has been solved many times before and I hate seeing it repeated in every new package manager. A vendor name should always be required and the top level should be reserved for official/standard packages. I want all of the following from a package manager: 1. Required vendor/namespace for third party packages 2. No multiple package versions. If there is a version conflict between transitive dependencies of a package because of semver, you should not be able to install that package. 3. Lock file and a separate 'install' command for installing the locked versions and an 'upgrade' command for updating versions via semver 4. Upgrade command should support a --dry-run option that lists the packages and versions that are to be updated and a --diff that lets you preview the code changes.

pcwalton 5y ago
Why are multiple package versions a problem? In Rust they Just Work.
- stubish 5y ago
  It can mess up when you are dealing with global resources. ie. two different versions of the same package both attempting to initialize global state (memory, filesystem, databases, whatever) will usually conflict with itself. Often in ways that will crash or even cause data loss.
  - pcwalton 5y ago
    I've never had that happen in Rust because multiple versions of a crate effectively live in completely separate worlds: they have their own copies of globals, for example. Can you name an example of multiple versions conflicting at runtime in Rust?
    - stubish 5y ago
      By globals, I mean global resources outside of the codes namespace. It may not even be a resource in the process, such as a log file or temporary directory or a database. If you have two versions of a crate in their completely separate worlds, and call both of their init_logging() functions to log to a file specified by an environment variable, things are likely to go pear shaped when they stomp over each others log file. I'm a Rust novice, but the example I tripped over in Go was https://github.com/golang/glog https://github.com/golang/glog. It has a module level init() initialization routine that makes calls to the stdlib flags package, manipulating the default command line flags (a global resource). If you ended up with multiple versions of glog via transient dependencies, your program would panic on startup as the second version's init() would make calls only allowed to be called once. Rust thankfully avoids this particular one by requiring initialization to be called by main() (apart from the hack described in the article).
      - cute_boi 5y ago
        First of all, in Rust global are highly discouraged and if there are shared resources then they should be defined or initialized by binary writer not the library writer Otherwise the library is not of good quality. Regarding go I can't say as I have never programmed go.
clon 5y ago
For all the animosity that PHP gets these days, every single item on your list (granted, of very basic demands) aligns with PHP's composer. I am surprised that Rust is that much worse off than PHP in this regard.
- moojd 5y ago
  I don't think composer has a diff option to dump the actual code differences before you update yet but yes most of this list comes from my past experience with composer. My current company doesn't use PHP but I look back fondly at how easy it was to audit my dependencies manually and be explicit about upgrades and transitive dependencies.
  - clon 5y ago
    It does offer a diff option when you have local edits in the /vendor (for whatever insane reason). Always assumed it could be triggered manually as well. TIL. I also love how easy it is to declare conflicts [1]. Some sub sub sub dependency down the tree had a bad 0.0.1 release? Just declare a conflict and have the tool do the work. [1] https://getcomposer.org/doc/04-schema.md#conflict https://getcomposer.org/doc/04-schema.md#conflict
epage 5y ago
> 2. No multiple package versions. If there is a version conflict between transitive dependencies of a package because of semver, you should not be able to install that package. I am grateful Rust allows this unlike C++ or Python. While I ideally minimize repeat dependencies, it is a big help to not be constrained to only on version. We've already had cases in Rust where some people were overly restrictive on dependency declarations (since Rust does block some versions as too similar) and it has caused a bit of pain.
dahfizz 5y ago
> 2. No multiple package versions. If there is a version conflict between transitive dependencies of a package because of semver, you should not be able to install that package. I think this depends highly on your environment. In npm land, where a typical project has hundreds or thousands of dependencies of dubious quality, this would be a nightmare. It guarantees that every single deployment will be different and risky. This model works much better in linux, where packages are maintained by maintainers and there is not an explosion in the dependency network. Especially on a Debian or Centos box, you can be confident that upgrading packages won't break stuff.
- moojd 5y ago
  If you are in node land I highly recommend using 'yarn install --flat' and I desperately wish this was the default in npm from the beginning. It would have radically altered the package development culture in a good way. The way npm currently handles version conflicts is one of the primary reasons why using npm is currently a nightmare. The average node project will have dozens of abandoned or ancient package versions precisely because allowing multiple versions to exist means that these packages never get forked or updated. Each one of those packages is a ticking time bomb waiting to be taken over by a malicious actor. Forking is a better solution than pulling in unmaintained packages with out of date dependencies.

by moojd 5y ago

pcwalton 5y ago

Why are multiple package versions a problem? In Rust they Just Work.

stubish 5y ago
It can mess up when you are dealing with global resources. ie. two different versions of the same package both attempting to initialize global state (memory, filesystem, databases, whatever) will usually conflict with itself. Often in ways that will crash or even cause data loss.
- pcwalton 5y ago
  I've never had that happen in Rust because multiple versions of a crate effectively live in completely separate worlds: they have their own copies of globals, for example. Can you name an example of multiple versions conflicting at runtime in Rust?
  - stubish 5y ago
    By globals, I mean global resources outside of the codes namespace. It may not even be a resource in the process, such as a log file or temporary directory or a database. If you have two versions of a crate in their completely separate worlds, and call both of their init_logging() functions to log to a file specified by an environment variable, things are likely to go pear shaped when they stomp over each others log file. I'm a Rust novice, but the example I tripped over in Go was https://github.com/golang/glog https://github.com/golang/glog. It has a module level init() initialization routine that makes calls to the stdlib flags package, manipulating the default command line flags (a global resource). If you ended up with multiple versions of glog via transient dependencies, your program would panic on startup as the second version's init() would make calls only allowed to be called once. Rust thankfully avoids this particular one by requiring initialization to be called by main() (apart from the hack described in the article).
    - cute_boi 5y ago
      First of all, in Rust global are highly discouraged and if there are shared resources then they should be defined or initialized by binary writer not the library writer Otherwise the library is not of good quality. Regarding go I can't say as I have never programmed go.

clon 5y ago

For all the animosity that PHP gets these days, every single item on your list (granted, of very basic demands) aligns with PHP's composer. I am surprised that Rust is that much worse off than PHP in this regard.

moojd 5y ago
I don't think composer has a diff option to dump the actual code differences before you update yet but yes most of this list comes from my past experience with composer. My current company doesn't use PHP but I look back fondly at how easy it was to audit my dependencies manually and be explicit about upgrades and transitive dependencies.
- clon 5y ago
  It does offer a diff option when you have local edits in the /vendor (for whatever insane reason). Always assumed it could be triggered manually as well. TIL. I also love how easy it is to declare conflicts [1]. Some sub sub sub dependency down the tree had a bad 0.0.1 release? Just declare a conflict and have the tool do the work. [1] https://getcomposer.org/doc/04-schema.md#conflict https://getcomposer.org/doc/04-schema.md#conflict

epage 5y ago

> 2. No multiple package versions. If there is a version conflict between transitive dependencies of a package because of semver, you should not be able to install that package. I am grateful Rust allows this unlike C++ or Python. While I ideally minimize repeat dependencies, it is a big help to not be constrained to only on version. We've already had cases in Rust where some people were overly restrictive on dependency declarations (since Rust does block some versions as too similar) and it has caused a bit of pain.

dahfizz 5y ago

> 2. No multiple package versions. If there is a version conflict between transitive dependencies of a package because of semver, you should not be able to install that package. I think this depends highly on your environment. In npm land, where a typical project has hundreds or thousands of dependencies of dubious quality, this would be a nightmare. It guarantees that every single deployment will be different and risky. This model works much better in linux, where packages are maintained by maintainers and there is not an explosion in the dependency network. Especially on a Debian or Centos box, you can be confident that upgrading packages won't break stuff.

moojd 5y ago
If you are in node land I highly recommend using 'yarn install --flat' and I desperately wish this was the default in npm from the beginning. It would have radically altered the package development culture in a good way. The way npm currently handles version conflicts is one of the primary reasons why using npm is currently a nightmare. The average node project will have dozens of abandoned or ancient package versions precisely because allowing multiple versions to exist means that these packages never get forked or updated. Each one of those packages is a ticking time bomb waiting to be taken over by a malicious actor. Forking is a better solution than pulling in unmaintained packages with out of date dependencies.