8 ms·

Pip and cargo are not the same

81 points by pabs3 4y ago

pxc 4y ago
Cargo has the benefit of being much later to the game, and has thoughtfully incorporated lessons from the past. It's easier to work with 'downstream', and seems like a good model for new languages/runtimes to start with for their own packaging ecosystems.
- no_wizard 4y ago
  Python had multiple different points in its history to fix its env and package problems. For reasons I don’t understand the PSF has not taken any hard stance on a solid way forward, it’s always pushed back as a community issue. The problem is lack of leadership on this more than anything
  - maegul 4y ago
    It certainly seems that way. Guido wasn’t interested in solving it and it was probably one of the things, like a gradual typing system which he was interested in, that he should have made a core issue because of its centrality. Curiously, I wonder whether current leadership are unwilling to acknowledge that the current problems are essentially a leadership problem. See for example my short conversation with Pradyun on Mastodon (https://mas.to/@maegul/109726564552419983 https://mas.to/@maegul/109726564552419983) where I’m not sure they were being open and rational (though they clearly know more than me). This thread was in response to their blog post in Python packaging: https://pradyunsg.me/blog/2023/01/21/thoughts-on-python-packaging/ https://pradyunsg.me/blog/2023/01/21/thoughts-on-python-pack...
  - reidrac 4y ago
    They included virtual envs in Python 3, perhaps they should have done the same with some of the packaging solutions.
    - ElectricalUnion 4y ago
      > perhaps they should have done the same with some of the packaging solutions. The packaging solution is pip? (although for some reason it doesn't come with the now standard wheel, so it's less useful that it should be)
  - BiteCode_dev 4y ago
    It has. So many things were fixed in python packaging. - "-m" was introduced. - Wheel replaced eggs. - Manylinux target now exists. - pypi was scaled uo, a new api introduced and 2fa added. - setup.py got replaced by setup.cfg and now pyproject.toml is gaining speed. - the py launcher is a thing. - Import priority changed. - Import lib changed. - Zipapps were added. - Venv ships with python. - Pip replaced easy_install, then ensurepip was created. - The new dependancy solver was funded. - Distutils has been sunsetted. Setuptools is on its way. This doesn't include the dozens of third party projects that target improving package that came out in the last 2 decades. Pip-tools by itself has changed the game. A lot has been done, and the situation is 10 times better than it used to be. It is still bad, but not because nothing was done, simply because there is still a lot to do. People dont realize the sheer scale of the task, and the inertia that comes with a language as successful and old as python. You have a language that is 4 years older than java, highly dynamic, yet uses fortran/c/assembly code in very popular lib (scipy/numpy) accross all popular OS, and webassembly and arm, in 32 and 64 bits. People routinely install multiple versions of python on their machine, linux repos freeze the upgrades and split the packages, mac and windows stores ship broken versions, and half the python userbase is composed of people that are not professional coders that can't use a terminal. Of course companies will complain if you change anything that breaks their 5 years old server, and devs will complain if they don't get the last new great feature. It's a really, really hard problem, dealt by a FOSS community that 10 years ago was still running on mostly volunteers and the budget of a small start up. Of course, just to get heat by comments on social media and no thanks for all the effort, as if you own total strangers your free work, with no flaws on top of that. Not to mention you can imagine packaging is not the only things the core devs have to work on.
    - noisy_boy 4y ago
      > This doesn't include the dozens of third party projects that target improving package that came out in the last 2 decades. One would think that two decades of package improvement initiatives should have been a strong enough signal to PSF to prioritize building an official standard solution.
  - sdiacom 4y ago
    Not disputing that there's a lack of leadership on the matter, but there's also conflicting use-cases: for example, the requirements for the "Python is just a tool that comes with my OS, and if I need any additional modules I'll install them with my OS package manager" mindset, and the "I'm developing a web application with Python and I want an isolated development environment with controlled dependencies" mindset, are quite different.
    - pxc 4y ago
      > "Python is just a tool that comes with my OS, and if I need any additional modules I'll install them with my OS package manager Imo this isn't really viable because you will eventually run into version conflicts in the transitive dependencies of your the Python applications you're using/developing on your system, on most operating systems. The version(s) that ships with an OS should only be used for shipping applications that are themselves part of the OS/distro.
- pjmlp 4y ago
  It failed to learn from binary repos used in C and C++ builds, which are the main reason why C++ with all its issues can still manage to compile faster in many cases.
  - kibwen 4y ago
    No, Cargo does support this, via sccache. Binaries just aren't provided by crates.io itself, for reasons of economics.
    - pjmlp 4y ago
      No it doesn't, as that is a 3rd party dependency.
      - kibwen 4y ago
        Cargo deliberately supports configurable caches in order to let users provide caching layers.
    - mid-kid 4y ago
      ccache is not a suitable replacement for a stable ABI that allows binaries to be reused across systems.
      - kibwen 4y ago
        I'm referring to sccache, not ccache. The former is a distributed ccache that allows artifacts to be reused across systems.
  - pxc 4y ago
    Ecosystems where binary redistribution at the language level is the norm tend to have really gnarly downstream packaging issues, so that dependency management by a general purpose/distro package manager is basically an unsupported use case (.NET) or that for large projects, there may not be an known, working fully from-source build (Java). I would much rather have binary redistribution left up to downstreams.
- lloeki 4y ago
  Looking at this from the Ruby world, cargo appears to be mostly a clone of the bundler model (with a sprinkle of rake folded in) which has existed for a looooong while ;) It's quite funny to see languages fumbling on dependency management and somehow "refusing" to look at how bundler operates, then stumbling on a very close solution and then community rejoices at their package management being the best thing since slice bread.
  - Tade0 4y ago
    How is the approach in Ruby different/better? Asking out of ignorance.
  - geewee 4y ago
    I'm pretty sure that Rust hired the people who worked on bundler to build cargo - so it would make sense that they're pretty similar.
  - kibwen 4y ago
    Mozilla literally contracted the Bundler authors to create Cargo, so the similarity is intentional. :P
    - lloeki 4y ago
      I'm quite sure it is! I did not know the bundler authors were actively contacted though, thanks for that piece of info!
- oblio 4y ago
  It's not perfect but pip (and npm...) could have learned a thing or two from Maven. You know, prior art that appeared about 5 years later and had huge adoption. Insularity is a software ecosystem disease.
  - POPOSYS 4y ago
    Please describe what is better in Maven. Or please send a source with a comparison or some explanation. Thanks!
    - oblio 4y ago
      Maven uses groupIds and artifactIds to group dependencies. GroupIds are reverse DNS notation based on a subdomain you control. They both group dependencies (duh!) and avoid the top-level squatting problem. It's much harder to mix up http-utils like in npm and pypi, since there's no http-utils. There's com.google:http-utils and there's org.apache:http-utils (all the names are made up for example purposes). Maven has a local centralized artifact repository/cache where artifacts are downloaded and then referred to by every project. In Java there isn't even a need for symlinks, since Java has a dynamic classpath aka the places where the libraries are searched. Though Python could do the same thing with the PYTHONPATH. Maven is plugin based and it's reasonably easy to build your own, so it's been extended like crazy to do all sorts of wonderful and loony things like build C++. It's actually expected to be extended, the tool itself implements Java builds through plugins, the core doesn't do that. Oh, from the start it had tools to cache/proxy your own packages. So you'd be able to have a mini-centralized repo for your company with just the stuff you need, in case the main repo is down. Artifactory, Nexus, there are others. And not just cache/mirror, but proxy: you'd call your local repo, and if it wouldn't find the thing, it would download it from internet repos you configure. Another thing, the package format and repo structure are simple and straightforward. So Gradle, sbt, other fancier Java build tools just use Maven repos. There are a million other things like that, it's a very robust tool with a very robust ecosystem. The main pain point is the early-2000s style XML configuration format :-| That makes a lot of people avoid it due to verbosity. But you know what? The format is stable. It's quite readable. There are a gazillion tools to manage it, IDEs have advanced autocompletion for it, etc. And I'm not saying that other folks should copy the config file format, just the rest of the ideas. Heck, even Maven has Polyglot Maven, to use different config file formats (not super adopted, but it's there).
smeagull 4y ago
I do wish more attention was paid to the ballooning disk space requirements for both. I think Cargo is doing better, but I've yet to get a shared directory working for every build on a system. I shouldn't have to spend 2G per Rust project.
- beebmam 4y ago
  It's a lot more than that if you're doing cross-platform (many) builds. It's a real pain.
  - jcelerier 4y ago
    Same with vcpkg for c++ development... Many of my students still have laptops with 128G of storage total, good luck making a whole $world fit in this when c:\Windows already uses half of it and they have their own documents & stuff
- kelnos 4y ago
  Yeah it seems super weird to me that cargo doesn't cache built artifacts based on crate name + version + enabled features + rustc version (and anything else that might make it unique). It seems silly that I have many built copies of the same dependency scattered across different projects on my machine. Then again, maybe it's actually fairly rare that the things that make a crate build unique are all the same all that often, and the gains vs. complexities of caching just aren't worth it.
  - jackosdev 4y ago
    sccache works really well and there’s only two steps to install it and enable it globally, speeds up compilation time a lot as well: https://github.com/mozilla/sccache https://github.com/mozilla/sccache
    - the_duke 4y ago
      sccache includes the absolute path for each compilation, so it doesn't help with caching the same dependency across different projects.
renewiltord 4y ago
These are some good differences. Personally, the one thing I do miss from the modern package managers is the Java-style namespacing that piggy-backs on DNS. I wonder if there is some project out there that has decided to piggy-back on ENS namespacing. And then you can have eth.renewiltord and have your packages under that, and then when your projects are all quite established, your ENS domain will have a public value that matches the exploit surface if someone were to get it. And then you can sell it and reap the rewards of EEEEVIL!
- kelnos 4y ago
  As much as it can be annoying to type out those imports in Java, I agree. All the "first one who asks for the name gets it" package managers suffer from problems (some of them security-related) that anything that talks to Maven Central does not. If you want an account on Sonatype's OSS repository (which allows you to push to Maven Central), you need to prove that you have control of the domain you want to use (in reversed form) as the "group ID" you publish libraries under (IIRC, by setting a TXT record in DNS). Meanwhile, to publish to crates.io, you just link your GitHub account (which is required, gross), and then you can publish to any package name that hasn't yet been taken, and then it's yours, forever. That seems terrible, somehow. (Granted, that just covers the group ID for depending on the library; you can put Java classes in that library wherever you want, even in "other people's" namespaces. In practice, though, I don't think this ends up being a big issue.) (Also granted, if you let your domain registration lapse, someone else could snap it up and possibly hijack your namespace. But I wouldn't be surprised if the Sonatype folks would try to get in touch with you if someone else tried to claim your namespace, even if they had control over the domain.)
  - devman0 4y ago
    I worked for many years in the enterprise java world, and when I went to do other things I was surprised how bad or incomplete package management is in other language ecosystems. The maven packaging system (as distinct from the Maven tool itself) makes managing dependencies for building and shipping java applications much more sane.
thejosh 4y ago
Poetry installs from a lockfike, which is different for every environment. So it takes. While to resolve... Cargo says "here is the package I need, at this version, go grab it", so it's quick.
- jstrong 4y ago
  Cargo does semantic versioning-based dependency resolution, similar to Poetry, it's just a lot faster because it's written in a Rust. I don't think there's a big difference in the work that happens.
  - kissgyorgy 4y ago
    I don't know anything about Rust, but probably there is a huge difference, because Poetry need to download and install all packages just to get their version and only after that can it resolve dependencies.
    - jstrong 4y ago
      I don't understand - if it didn't know what version until after download/install - which version would it download/install?
      - kissgyorgy 4y ago
        Oh sorry, I was talking about dependency resolution, OP was talking about install :/
      - ywain 4y ago
        If I'm not mistaken, it needs to download a package to know its dependencies and version constraints. So when you have many packages with many dependencies, which themselves have dependencies etc. it can take a while for poetry to assemble the full dependency graph and determine whether there are any unsolvable constraints (e.g. package foo depends on package bar with version >= 2, but package baz depends on package bar with version < 2). Not sure how other package managers avoid that. Maybe the central package repositories can expose the dependencies metadata without needing to download the actual package?
LegionMammal978 4y ago
> There is a 1 to 1 relationship between cargo crate names and what gets used in the rust file. With pip your pypi package moo can include Python package foo, or whatever else it likes. This is false: by default, the crate name (i.e., the name of the lib target) will be the package name with hyphens replaced by underscores, but it can be overridden to any other name [0]. [0] https://doc.rust-lang.org/cargo/reference/cargo-targets.html#the-name-field https://doc.rust-lang.org/cargo/reference/cargo-targets.html...
- pie_flavor 4y ago
  That's still 1 to 1.
  - LegionMammal978 4y ago
    No, multiple packages can (and do) declare the same crate name. For instance, both the "md5" package [0] and "md-5" package [1] declare the "md5" crate. You can even add both of them as a dependency at the same time: you'll only get a compile error when you try to refer to "md5" from the code. So a Cargo package may declare at most one crate name, but this name is not unique. [0] https://crates.io/crates/md5 https://crates.io/crates/md5 [1] https://crates.io/crates/md-5 https://crates.io/crates/md-5
    - pie_flavor 4y ago
      I suppose, but one can also creatively define 'what gets used in the rust file'. If you somehow need to interface with both of those packages, you can say `md_5 = { package = "md-5" }`, and now in your code it will be renamed `md_5`.
pjmlp 4y ago
> Similarly you can’t use something in your rust code that you haven’t asked for in your Cargo.toml - transitive dependencies of your dependencies (mostly) don’t affect you If they land on the final executable they certainly affect me.
- db48x 4y ago
  Only indirectly. They don’t add additional types, methods, functions, etc that you must deal with in your code. The interface provided by the crate you directly depend on will, but nothing from the crates _it_ depends on will bleed through accidentally.
  - nibbleshifter 4y ago
    They do add extra filesize though.
  - pjmlp 4y ago
    The naive statement from someone that never did security assements, porting code across multiple platforms, license checks,...
    - kibwen 4y ago
      This appears to be deliberately missing the author's point.
      - pjmlp 4y ago
        This misses why dependency analysis matters.
cabirum 4y ago
Python ecosystem is too old to be fixed without breaking everything else. That said, use pipenv to get lockfiles and almost sane dep management. And, use global.require-virtualenv=true to prevent global package pollution.
conaclos 4y ago
Making dependency management easy, you end up with large tree of dependencies. It is one of the main concern I have with modern package manager. In my opinion, it lacks a kind of trust management. Packages from a same team/author could belong to a same trust group. When a package get updated, we have to audit dependencies from new trust groups. This could create a culture of reduced and audited third-party dependencies.
- Yoric 4y ago
  I have been thinking for years of a different trust mechanism for packages, one that is closer to capabilities, i.e. what Android is doing these days for applications. Both approaches could certainly coexist!
  - conaclos 4y ago
    Capabilities are great also! In particular at runtime.
- mid-kid 4y ago
  This is unfortunately the reality, and the primary reason why I prefer ecosystems with a higher barrier of entry to publish your library or program on, like linux distributions. Libraries simply don't get on there unless there's a program using it, in demand by the users of the distribution. Similarly, it makes it less attractive for application developers to rely on a library that's not already in the repository, as this increases packaging and install friction.
- Yoric 4y ago
  I have just informally pitched both your idea and mine to the Rust security team :) We'll see what they think about it! If it somehow works, feel free to use this message as the proof that I got the idea from you :)
  - conaclos 4y ago
    Nice! Is there any place where the discussion is going?

81 points by pabs3 4y ago

pxc 4y ago

Cargo has the benefit of being much later to the game, and has thoughtfully incorporated lessons from the past. It's easier to work with 'downstream', and seems like a good model for new languages/runtimes to start with for their own packaging ecosystems.

no_wizard 4y ago
Python had multiple different points in its history to fix its env and package problems. For reasons I don’t understand the PSF has not taken any hard stance on a solid way forward, it’s always pushed back as a community issue. The problem is lack of leadership on this more than anything
- maegul 4y ago
  It certainly seems that way. Guido wasn’t interested in solving it and it was probably one of the things, like a gradual typing system which he was interested in, that he should have made a core issue because of its centrality. Curiously, I wonder whether current leadership are unwilling to acknowledge that the current problems are essentially a leadership problem. See for example my short conversation with Pradyun on Mastodon (https://mas.to/@maegul/109726564552419983 https://mas.to/@maegul/109726564552419983) where I’m not sure they were being open and rational (though they clearly know more than me). This thread was in response to their blog post in Python packaging: https://pradyunsg.me/blog/2023/01/21/thoughts-on-python-packaging/ https://pradyunsg.me/blog/2023/01/21/thoughts-on-python-pack...
- reidrac 4y ago
  They included virtual envs in Python 3, perhaps they should have done the same with some of the packaging solutions.
  - ElectricalUnion 4y ago
    > perhaps they should have done the same with some of the packaging solutions. The packaging solution is pip? (although for some reason it doesn't come with the now standard wheel, so it's less useful that it should be)
- BiteCode_dev 4y ago
  It has. So many things were fixed in python packaging. - "-m" was introduced. - Wheel replaced eggs. - Manylinux target now exists. - pypi was scaled uo, a new api introduced and 2fa added. - setup.py got replaced by setup.cfg and now pyproject.toml is gaining speed. - the py launcher is a thing. - Import priority changed. - Import lib changed. - Zipapps were added. - Venv ships with python. - Pip replaced easy_install, then ensurepip was created. - The new dependancy solver was funded. - Distutils has been sunsetted. Setuptools is on its way. This doesn't include the dozens of third party projects that target improving package that came out in the last 2 decades. Pip-tools by itself has changed the game. A lot has been done, and the situation is 10 times better than it used to be. It is still bad, but not because nothing was done, simply because there is still a lot to do. People dont realize the sheer scale of the task, and the inertia that comes with a language as successful and old as python. You have a language that is 4 years older than java, highly dynamic, yet uses fortran/c/assembly code in very popular lib (scipy/numpy) accross all popular OS, and webassembly and arm, in 32 and 64 bits. People routinely install multiple versions of python on their machine, linux repos freeze the upgrades and split the packages, mac and windows stores ship broken versions, and half the python userbase is composed of people that are not professional coders that can't use a terminal. Of course companies will complain if you change anything that breaks their 5 years old server, and devs will complain if they don't get the last new great feature. It's a really, really hard problem, dealt by a FOSS community that 10 years ago was still running on mostly volunteers and the budget of a small start up. Of course, just to get heat by comments on social media and no thanks for all the effort, as if you own total strangers your free work, with no flaws on top of that. Not to mention you can imagine packaging is not the only things the core devs have to work on.
  - noisy_boy 4y ago
    > This doesn't include the dozens of third party projects that target improving package that came out in the last 2 decades. One would think that two decades of package improvement initiatives should have been a strong enough signal to PSF to prioritize building an official standard solution.
- sdiacom 4y ago
  Not disputing that there's a lack of leadership on the matter, but there's also conflicting use-cases: for example, the requirements for the "Python is just a tool that comes with my OS, and if I need any additional modules I'll install them with my OS package manager" mindset, and the "I'm developing a web application with Python and I want an isolated development environment with controlled dependencies" mindset, are quite different.
  - pxc 4y ago
    > "Python is just a tool that comes with my OS, and if I need any additional modules I'll install them with my OS package manager Imo this isn't really viable because you will eventually run into version conflicts in the transitive dependencies of your the Python applications you're using/developing on your system, on most operating systems. The version(s) that ships with an OS should only be used for shipping applications that are themselves part of the OS/distro.
pjmlp 4y ago
It failed to learn from binary repos used in C and C++ builds, which are the main reason why C++ with all its issues can still manage to compile faster in many cases.
- kibwen 4y ago
  No, Cargo does support this, via sccache. Binaries just aren't provided by crates.io itself, for reasons of economics.
  - pjmlp 4y ago
    No it doesn't, as that is a 3rd party dependency.
    - kibwen 4y ago
      Cargo deliberately supports configurable caches in order to let users provide caching layers.
  - mid-kid 4y ago
    ccache is not a suitable replacement for a stable ABI that allows binaries to be reused across systems.
    - kibwen 4y ago
      I'm referring to sccache, not ccache. The former is a distributed ccache that allows artifacts to be reused across systems.
- pxc 4y ago
  Ecosystems where binary redistribution at the language level is the norm tend to have really gnarly downstream packaging issues, so that dependency management by a general purpose/distro package manager is basically an unsupported use case (.NET) or that for large projects, there may not be an known, working fully from-source build (Java). I would much rather have binary redistribution left up to downstreams.
lloeki 4y ago
Looking at this from the Ruby world, cargo appears to be mostly a clone of the bundler model (with a sprinkle of rake folded in) which has existed for a looooong while ;) It's quite funny to see languages fumbling on dependency management and somehow "refusing" to look at how bundler operates, then stumbling on a very close solution and then community rejoices at their package management being the best thing since slice bread.
- Tade0 4y ago
  How is the approach in Ruby different/better? Asking out of ignorance.
- geewee 4y ago
  I'm pretty sure that Rust hired the people who worked on bundler to build cargo - so it would make sense that they're pretty similar.
- kibwen 4y ago
  Mozilla literally contracted the Bundler authors to create Cargo, so the similarity is intentional. :P
  - lloeki 4y ago
    I'm quite sure it is! I did not know the bundler authors were actively contacted though, thanks for that piece of info!
oblio 4y ago
It's not perfect but pip (and npm...) could have learned a thing or two from Maven. You know, prior art that appeared about 5 years later and had huge adoption. Insularity is a software ecosystem disease.
- POPOSYS 4y ago
  Please describe what is better in Maven. Or please send a source with a comparison or some explanation. Thanks!
  - oblio 4y ago
    Maven uses groupIds and artifactIds to group dependencies. GroupIds are reverse DNS notation based on a subdomain you control. They both group dependencies (duh!) and avoid the top-level squatting problem. It's much harder to mix up http-utils like in npm and pypi, since there's no http-utils. There's com.google:http-utils and there's org.apache:http-utils (all the names are made up for example purposes). Maven has a local centralized artifact repository/cache where artifacts are downloaded and then referred to by every project. In Java there isn't even a need for symlinks, since Java has a dynamic classpath aka the places where the libraries are searched. Though Python could do the same thing with the PYTHONPATH. Maven is plugin based and it's reasonably easy to build your own, so it's been extended like crazy to do all sorts of wonderful and loony things like build C++. It's actually expected to be extended, the tool itself implements Java builds through plugins, the core doesn't do that. Oh, from the start it had tools to cache/proxy your own packages. So you'd be able to have a mini-centralized repo for your company with just the stuff you need, in case the main repo is down. Artifactory, Nexus, there are others. And not just cache/mirror, but proxy: you'd call your local repo, and if it wouldn't find the thing, it would download it from internet repos you configure. Another thing, the package format and repo structure are simple and straightforward. So Gradle, sbt, other fancier Java build tools just use Maven repos. There are a million other things like that, it's a very robust tool with a very robust ecosystem. The main pain point is the early-2000s style XML configuration format :-| That makes a lot of people avoid it due to verbosity. But you know what? The format is stable. It's quite readable. There are a gazillion tools to manage it, IDEs have advanced autocompletion for it, etc. And I'm not saying that other folks should copy the config file format, just the rest of the ideas. Heck, even Maven has Polyglot Maven, to use different config file formats (not super adopted, but it's there).

smeagull 4y ago

I do wish more attention was paid to the ballooning disk space requirements for both. I think Cargo is doing better, but I've yet to get a shared directory working for every build on a system. I shouldn't have to spend 2G per Rust project.

beebmam 4y ago
It's a lot more than that if you're doing cross-platform (many) builds. It's a real pain.
- jcelerier 4y ago
  Same with vcpkg for c++ development... Many of my students still have laptops with 128G of storage total, good luck making a whole $world fit in this when c:\Windows already uses half of it and they have their own documents & stuff
kelnos 4y ago
Yeah it seems super weird to me that cargo doesn't cache built artifacts based on crate name + version + enabled features + rustc version (and anything else that might make it unique). It seems silly that I have many built copies of the same dependency scattered across different projects on my machine. Then again, maybe it's actually fairly rare that the things that make a crate build unique are all the same all that often, and the gains vs. complexities of caching just aren't worth it.
- jackosdev 4y ago
  sccache works really well and there’s only two steps to install it and enable it globally, speeds up compilation time a lot as well: https://github.com/mozilla/sccache https://github.com/mozilla/sccache
  - the_duke 4y ago
    sccache includes the absolute path for each compilation, so it doesn't help with caching the same dependency across different projects.

renewiltord 4y ago

These are some good differences. Personally, the one thing I do miss from the modern package managers is the Java-style namespacing that piggy-backs on DNS. I wonder if there is some project out there that has decided to piggy-back on ENS namespacing. And then you can have eth.renewiltord and have your packages under that, and then when your projects are all quite established, your ENS domain will have a public value that matches the exploit surface if someone were to get it. And then you can sell it and reap the rewards of EEEEVIL!

kelnos 4y ago
As much as it can be annoying to type out those imports in Java, I agree. All the "first one who asks for the name gets it" package managers suffer from problems (some of them security-related) that anything that talks to Maven Central does not. If you want an account on Sonatype's OSS repository (which allows you to push to Maven Central), you need to prove that you have control of the domain you want to use (in reversed form) as the "group ID" you publish libraries under (IIRC, by setting a TXT record in DNS). Meanwhile, to publish to crates.io, you just link your GitHub account (which is required, gross), and then you can publish to any package name that hasn't yet been taken, and then it's yours, forever. That seems terrible, somehow. (Granted, that just covers the group ID for depending on the library; you can put Java classes in that library wherever you want, even in "other people's" namespaces. In practice, though, I don't think this ends up being a big issue.) (Also granted, if you let your domain registration lapse, someone else could snap it up and possibly hijack your namespace. But I wouldn't be surprised if the Sonatype folks would try to get in touch with you if someone else tried to claim your namespace, even if they had control over the domain.)
- devman0 4y ago
  I worked for many years in the enterprise java world, and when I went to do other things I was surprised how bad or incomplete package management is in other language ecosystems. The maven packaging system (as distinct from the Maven tool itself) makes managing dependencies for building and shipping java applications much more sane.

thejosh 4y ago

Poetry installs from a lockfike, which is different for every environment. So it takes. While to resolve... Cargo says "here is the package I need, at this version, go grab it", so it's quick.

jstrong 4y ago
Cargo does semantic versioning-based dependency resolution, similar to Poetry, it's just a lot faster because it's written in a Rust. I don't think there's a big difference in the work that happens.
- kissgyorgy 4y ago
  I don't know anything about Rust, but probably there is a huge difference, because Poetry need to download and install all packages just to get their version and only after that can it resolve dependencies.
  - jstrong 4y ago
    I don't understand - if it didn't know what version until after download/install - which version would it download/install?
    - kissgyorgy 4y ago
      Oh sorry, I was talking about dependency resolution, OP was talking about install :/
    - ywain 4y ago
      If I'm not mistaken, it needs to download a package to know its dependencies and version constraints. So when you have many packages with many dependencies, which themselves have dependencies etc. it can take a while for poetry to assemble the full dependency graph and determine whether there are any unsolvable constraints (e.g. package foo depends on package bar with version >= 2, but package baz depends on package bar with version < 2). Not sure how other package managers avoid that. Maybe the central package repositories can expose the dependencies metadata without needing to download the actual package?

LegionMammal978 4y ago

> There is a 1 to 1 relationship between cargo crate names and what gets used in the rust file. With pip your pypi package moo can include Python package foo, or whatever else it likes. This is false: by default, the crate name (i.e., the name of the lib target) will be the package name with hyphens replaced by underscores, but it can be overridden to any other name [0]. [0] https://doc.rust-lang.org/cargo/reference/cargo-targets.html#the-name-field https://doc.rust-lang.org/cargo/reference/cargo-targets.html...

pie_flavor 4y ago
That's still 1 to 1.
- LegionMammal978 4y ago
  No, multiple packages can (and do) declare the same crate name. For instance, both the "md5" package [0] and "md-5" package [1] declare the "md5" crate. You can even add both of them as a dependency at the same time: you'll only get a compile error when you try to refer to "md5" from the code. So a Cargo package may declare at most one crate name, but this name is not unique. [0] https://crates.io/crates/md5 https://crates.io/crates/md5 [1] https://crates.io/crates/md-5 https://crates.io/crates/md-5
  - pie_flavor 4y ago
    I suppose, but one can also creatively define 'what gets used in the rust file'. If you somehow need to interface with both of those packages, you can say `md_5 = { package = "md-5" }`, and now in your code it will be renamed `md_5`.

pjmlp 4y ago

> Similarly you can’t use something in your rust code that you haven’t asked for in your Cargo.toml - transitive dependencies of your dependencies (mostly) don’t affect you If they land on the final executable they certainly affect me.

db48x 4y ago
Only indirectly. They don’t add additional types, methods, functions, etc that you must deal with in your code. The interface provided by the crate you directly depend on will, but nothing from the crates _it_ depends on will bleed through accidentally.
- nibbleshifter 4y ago
  They do add extra filesize though.
- pjmlp 4y ago
  The naive statement from someone that never did security assements, porting code across multiple platforms, license checks,...
  - kibwen 4y ago
    This appears to be deliberately missing the author's point.
    - pjmlp 4y ago
      This misses why dependency analysis matters.

cabirum 4y ago

Python ecosystem is too old to be fixed without breaking everything else. That said, use pipenv to get lockfiles and almost sane dep management. And, use global.require-virtualenv=true to prevent global package pollution.

conaclos 4y ago

Making dependency management easy, you end up with large tree of dependencies. It is one of the main concern I have with modern package manager. In my opinion, it lacks a kind of trust management. Packages from a same team/author could belong to a same trust group. When a package get updated, we have to audit dependencies from new trust groups. This could create a culture of reduced and audited third-party dependencies.

Yoric 4y ago
I have been thinking for years of a different trust mechanism for packages, one that is closer to capabilities, i.e. what Android is doing these days for applications. Both approaches could certainly coexist!
- conaclos 4y ago
  Capabilities are great also! In particular at runtime.
mid-kid 4y ago
This is unfortunately the reality, and the primary reason why I prefer ecosystems with a higher barrier of entry to publish your library or program on, like linux distributions. Libraries simply don't get on there unless there's a program using it, in demand by the users of the distribution. Similarly, it makes it less attractive for application developers to rely on a library that's not already in the repository, as this increases packaging and install friction.
Yoric 4y ago
I have just informally pitched both your idea and mine to the Rust security team :) We'll see what they think about it! If it somehow works, feel free to use this message as the proof that I got the idea from you :)
- conaclos 4y ago
  Nice! Is there any place where the discussion is going?