3 ms·

I don't know anything about Rust, but probably there is a huge difference, because Poetry need to download and install all packages just to get their version an

I don't know anything about Rust, but probably there is a huge difference, because Poetry need to download and install all packages just to get their version and only after that can it resolve dependencies.

jstrong 4y ago
I don't understand - if it didn't know what version until after download/install - which version would it download/install?
- kissgyorgy 4y ago
  Oh sorry, I was talking about dependency resolution, OP was talking about install :/
- ywain 4y ago
  If I'm not mistaken, it needs to download a package to know its dependencies and version constraints. So when you have many packages with many dependencies, which themselves have dependencies etc. it can take a while for poetry to assemble the full dependency graph and determine whether there are any unsolvable constraints (e.g. package foo depends on package bar with version >= 2, but package baz depends on package bar with version < 2). Not sure how other package managers avoid that. Maybe the central package repositories can expose the dependencies metadata without needing to download the actual package?
  - hobofan 4y ago
    > If I'm not mistaken, it needs to download a package to know its dependencies and version constraints. It's even worse than that. It needs to execute a python script (setup.py?) per package to get a list of it's dependencies and constraints. As that script may contain arbitrary platform-dependent logic (and in the case of ML-related packages often does), it can be impossible to resolve dependencies for other platforms. > Not sure how other package managers avoid that. Maybe the central package repositories can expose the dependencies metadata without needing to download the actual package? Yes exactly. For dependency resolution, cargo uses only a git based index[0] which is optimized to contain only the information required for dependency resolution (omitting other package metadata such as e.g. authors). So it syncs the git repository and after that it is just lookups in local files of the index. Only after dependency resolution does it need to consult an external server for retrieval of the actual package contents. [0]: https://github.com/rust-lang/crates.io-index https://github.com/rust-lang/crates.io-index
    - BerislavLopac 4y ago
      > It needs to execute a python script (setup.py?) per package to get a list of it's dependencies and constraints. Only for packages that use setup.py (which is still heavily used; not sure whether it's still a majority). It is slowly being replaced by setup.cfg [0] and, more recently, pyproject.toml [1], which both contain dependencies in a declarative format. [0] https://setuptools.pypa.io/en/latest/userguide/declarative_config.html https://setuptools.pypa.io/en/latest/userguide/declarative_c... [1] https://peps.python.org/pep-0631/ https://peps.python.org/pep-0631/
  - deleted 4y ago
    [deleted]
gazpacho 4y ago
Yes and no: it needs to download it to discover its dependencies. The version is encoded in the wheel file name.

jstrong 4y ago

I don't understand - if it didn't know what version until after download/install - which version would it download/install?

kissgyorgy 4y ago
Oh sorry, I was talking about dependency resolution, OP was talking about install :/
ywain 4y ago
If I'm not mistaken, it needs to download a package to know its dependencies and version constraints. So when you have many packages with many dependencies, which themselves have dependencies etc. it can take a while for poetry to assemble the full dependency graph and determine whether there are any unsolvable constraints (e.g. package foo depends on package bar with version >= 2, but package baz depends on package bar with version < 2). Not sure how other package managers avoid that. Maybe the central package repositories can expose the dependencies metadata without needing to download the actual package?
- hobofan 4y ago
  > If I'm not mistaken, it needs to download a package to know its dependencies and version constraints. It's even worse than that. It needs to execute a python script (setup.py?) per package to get a list of it's dependencies and constraints. As that script may contain arbitrary platform-dependent logic (and in the case of ML-related packages often does), it can be impossible to resolve dependencies for other platforms. > Not sure how other package managers avoid that. Maybe the central package repositories can expose the dependencies metadata without needing to download the actual package? Yes exactly. For dependency resolution, cargo uses only a git based index[0] which is optimized to contain only the information required for dependency resolution (omitting other package metadata such as e.g. authors). So it syncs the git repository and after that it is just lookups in local files of the index. Only after dependency resolution does it need to consult an external server for retrieval of the actual package contents. [0]: https://github.com/rust-lang/crates.io-index https://github.com/rust-lang/crates.io-index
  - BerislavLopac 4y ago
    > It needs to execute a python script (setup.py?) per package to get a list of it's dependencies and constraints. Only for packages that use setup.py (which is still heavily used; not sure whether it's still a majority). It is slowly being replaced by setup.cfg [0] and, more recently, pyproject.toml [1], which both contain dependencies in a declarative format. [0] https://setuptools.pypa.io/en/latest/userguide/declarative_config.html https://setuptools.pypa.io/en/latest/userguide/declarative_c... [1] https://peps.python.org/pep-0631/ https://peps.python.org/pep-0631/
- deleted 4y ago
  [deleted]

gazpacho 4y ago

Yes and no: it needs to download it to discover its dependencies. The version is encoded in the wheel file name.