3 ms·

Julia and Rust seem to have package systems that are fine and manageable. I think these are really just major problems in Javascript, Python, and C/C++ (exactly

Julia and Rust seem to have package systems that are fine and manageable. I think these are really just major problems in Javascript, Python, and C/C++ (exactly the languages you mention) because the kind of widespread OSS code sharing just didn't exist to the extent it does today back when those languages were designed. People shared code through email, but didn't expect one button to pull in 200 Github repositories, and thus weren't built with the expectations required to make that be stable. Back when those languages were designed, you'd manually download the few modules you need, if you downloaded any packages at all. In C you'd normally build your own world, since it came before the www times, and C++ kind of inherited that. But languages which came out later decided that we now live in a world where most of the code that is executed is packages, most likely packages which live on Github. So Julia and Rust build this into the language. Julia in particular with the Project.toml and Manifest.jl for fully reproducing environments, its package manager simply uses git and lets you grab the full repository with `]dev packagename`, its package registry system lets you extend with private package worlds. I think the issue is that dependencies are central, so you can never remove old package systems because if that's where the old (and rarely updated) dependencies live, then you need to keep it around. But for dependencies to work well, you need all dependencies to be resolved using the same package system. So package systems don't tend to move very fast in any language, whatever you had early has too much momentum.

Hendrikto 4y ago
Tying package management to GitHub seems convenient in the short term, but will be the baggage of the next generation. I cringe hard when I see projects depending on git repos, without pinning a version or commit.
- staunton 4y ago
  It's not actually tied to GitHub. If GitHub died tomorrow, they would easily be able to move on and host the packages somewhere else. There's no way to do this without hosting. Also, there is a way to pin a version or commit. Julia for example always stores the exact commit information for all packages in the "Manifest" file. There are also straightforward ways to demand certain versions and package maintainers have the opportunity to specify compatibility and requirements precisely.
  - simiones 4y ago
    There will be a mess of dependencies of old projects that depend of projects that have since moved off GitHub, there will be name clashes between GitHub projects and BitBucket projects etc. These ecosystems are very much dependent on GitHub, and if/when anything permanent changes, it will be a mess to move off of it.
    - ChrisRackauckas 4y ago
      > there will be name clashes between GitHub projects and BitBucket projects etc. The package manager requires a canonical URL for the git repo. The location of dependencies doesn't matter, for example Github repos with Bitbucket dependencies work fine. The FastLapack packages lived on BitBucket at one point, and then migrated to Github, and that happened without a hitch to upstream packages like LinearSolve.jl or DifferentialEquations.jl. Also, it takes a one line change to General change a package's from Github to BitBucket. It didn't take me more than 2 minutes when I tried. Did you not have a similar experience? Can you show the PR where you gave it a try? Or is your response just speculation? The package server mirrors the repos, so even if these services went down it would not break the dependency chain (developers wouldn't be able to easily update the packages, but the installations and all would still work). So if tomorrow Github decided to make all of the code private for some odd reason, Julia's package server would continue to serve the current versions.
      - simiones 4y ago
        I was mostly speculating, and thinking more of how Go does Github dependencies. I am happy to hear that Cargo doesn't have the same sort of issues. I really should have looked into it more before commenting, especially since I knew Cargo is actually a well-designed and considered package system.
        staunton 4y ago
        Just to make sure there are no misunderstandings. ChrisRackauckas was talking about Julia's package manager, (which is called Pkg, if you need to name it, although I would just call it "the package manager"). Cargo is the Rust package manager.
- ChrisRackauckas 4y ago
  Julia and Rust all work with using versions tied to git tags. Package management isn't tied to Github but git: packages can live on Gitlab or BitBucket, though they generally don't and that's the choice of package developers. Because of that there are tie-ins to make Github really nice to use, but for example with Julia the only piece that is truly Github based is the fact that the General registry lives in a Github repo (https://github.com/JuliaRegistries/General https://github.com/JuliaRegistries/General), but could easily migrate to another git platform if it needed to.

Hendrikto 4y ago

Tying package management to GitHub seems convenient in the short term, but will be the baggage of the next generation. I cringe hard when I see projects depending on git repos, without pinning a version or commit.

staunton 4y ago
It's not actually tied to GitHub. If GitHub died tomorrow, they would easily be able to move on and host the packages somewhere else. There's no way to do this without hosting. Also, there is a way to pin a version or commit. Julia for example always stores the exact commit information for all packages in the "Manifest" file. There are also straightforward ways to demand certain versions and package maintainers have the opportunity to specify compatibility and requirements precisely.
- simiones 4y ago
  There will be a mess of dependencies of old projects that depend of projects that have since moved off GitHub, there will be name clashes between GitHub projects and BitBucket projects etc. These ecosystems are very much dependent on GitHub, and if/when anything permanent changes, it will be a mess to move off of it.
  - ChrisRackauckas 4y ago
    > there will be name clashes between GitHub projects and BitBucket projects etc. The package manager requires a canonical URL for the git repo. The location of dependencies doesn't matter, for example Github repos with Bitbucket dependencies work fine. The FastLapack packages lived on BitBucket at one point, and then migrated to Github, and that happened without a hitch to upstream packages like LinearSolve.jl or DifferentialEquations.jl. Also, it takes a one line change to General change a package's from Github to BitBucket. It didn't take me more than 2 minutes when I tried. Did you not have a similar experience? Can you show the PR where you gave it a try? Or is your response just speculation? The package server mirrors the repos, so even if these services went down it would not break the dependency chain (developers wouldn't be able to easily update the packages, but the installations and all would still work). So if tomorrow Github decided to make all of the code private for some odd reason, Julia's package server would continue to serve the current versions.
    - simiones 4y ago
      I was mostly speculating, and thinking more of how Go does Github dependencies. I am happy to hear that Cargo doesn't have the same sort of issues. I really should have looked into it more before commenting, especially since I knew Cargo is actually a well-designed and considered package system.
      - staunton 4y ago
        Just to make sure there are no misunderstandings. ChrisRackauckas was talking about Julia's package manager, (which is called Pkg, if you need to name it, although I would just call it "the package manager"). Cargo is the Rust package manager.
ChrisRackauckas 4y ago
Julia and Rust all work with using versions tied to git tags. Package management isn't tied to Github but git: packages can live on Gitlab or BitBucket, though they generally don't and that's the choice of package developers. Because of that there are tie-ins to make Github really nice to use, but for example with Julia the only piece that is truly Github based is the fact that the General registry lives in a Github repo (https://github.com/JuliaRegistries/General https://github.com/JuliaRegistries/General), but could easily migrate to another git platform if it needed to.