4 ms·

> Dep does not support using multiple major versions of a program in a single build. This alone is a complete showstopper. Go is meant for large-scale work, and

> Dep does not support using multiple major versions of a program in a single build. This alone is a complete showstopper. Go is meant for large-scale work, and in a large-scale program different parts will inevitably need different versions of some isolated dependency. This seems to be a major sticking point but I'm having trouble understanding why. I'm not familiar with any other package management system which supports simultaneous major versions in a single project. If two things require different versions of a package, those seem like separate projects to me. Is this to support dependencies which rely on the same package at different versions? If so, why not just call it a version conflict and fail until it's resolved?

deleted 8y ago
[deleted]
MrBuddyCasino 8y ago
> why not just call it a version conflict and fail until it's resolved Because its unnecessary and it doesn't scale. In Rust, versions become part of the symbol name, and the whole thing just becomes a non-issue. Go did the right thing here.
majewsky 8y ago
> If two things require different versions of a package, those seem like separate projects to me. You may be thinking of packages on the scale of frameworks. For example, if you have a Ruby on Rails app that mixes packages from Rails 3 and Rails 5, well sure, then you're in for some serious trouble. But a lot of packages are much more narrow in scope. Rust has a lot of small crates for basic data structures that have their APIs change betweeen versions. So you could end up with your logging package internally depending on some_data_structure=1.2, while your RabbitMQ client library internally depends on some_data_structure=2.3. As long as they do not expose some_data_structure in their API, you will never even notice (except maybe through increased binary size).
derefr 8y ago
> Is this to support dependencies which rely on the same package at different versions? Yes. > If two things require different versions of a package, those seem like separate projects to me Basically we’re talking about libs that have version conflicts, but also very slow release schedules, such that the version conflict isn’t going to be reconciled any time soon; or where one of the deps may even be effectively abandonware, but “works just fine” stand-alone so nobody’s going to update it. It just depends on extremely old versions of common shared deps. These are, effectively, irreconcilable version conflicts. The only thing to do, in most languages, is to fork one or the other dep to fix the conflict; or to split the project into microservices such that the conflicting components can live on opposite sides of a process memory boundary. Even these are sometimes impossible for office-political reasons. > I'm not familiar with any other package management system which supports simultaneous major versions in a single project. Node’s npm. Versions aren’t globally resolved; instead, each package effectively gets its own version resolution. Your root-level package gets a node_modules/ directory with your direct dependencies checked out into it; but your deps' deps are checked out into further node_modules/ directories that exist within their parent dep's checkout directory. It's somewhat as if each parent dep had vendored its deps, and had a vendor/ subdirectory; except that, in this case, the vendoring process is recursive. When a given package require()s a library by name, then, it’s actually importing its own npm-vendorified copy of the library from its own node_modules/ directory, rather than looking in your package's root node_modules/ directory. (In fact, there is no root level; you have "package isotropy", where a dep NPM checked out has the exact same NPM-populated subdirs as a project you git clone'd yourself.) Since these libs aren’t entered into a global namespace by require(), rather just have the module returned as a runtime object for the parent scope to bind to a local variable, there are no runtime conflicts between the separately-vendored versions, either. In a language without such runtime support, though, you would need your package-ecosystem tooling to reach into your downloaded lib and do some name-mangling of the exported symbols, and then either create a router symbol that resolves calls based on the lexical scope, or further mangle your deps' parents to call the dep by its mangled name rather than its original name. I think that’s what’s being proposed here, since Go has no such runtime support. (Effectively, in Go terms, you can simply think of it as all the references to git-http URL depspecs in import statements, being amended to qualify the repo name with the full git SHA, such that two different commits of the same project act as if they were two differently-named projects from Go's perspective.)
- rpeden 8y ago
  It's a relatively minor nitpick, but since version 3, npm flattens the package tree as much as it can and tries to bring everything possible to the top level node_modules directory. It'll still use nested node_modules where necessary to handle dependency version conflicts. A bit more info here: http://npm.github.io/how-npm-works-docs/npm3/how-npm3-works.html http://npm.github.io/how-npm-works-docs/npm3/how-npm3-works....
DougBTX 8y ago
> Is this to support dependencies which rely on the same package at different versions? Yes, that's right. See for example NPM's docs on how this works: http://npm.github.io/how-npm-works-docs/npm3/how-npm3-works.html http://npm.github.io/how-npm-works-docs/npm3/how-npm3-works....
aasasd 8y ago
A Java bro told me they have lots of pain with version conflicts, since Java world is so reliant both on tons of libraries and on strict interfaces. Apparently there's something called 'dependency shading' to overcome this issue, though seems like it has to be done by the dependency vendor, not the user. NPM notably doesn't have any problems with different versions (afaik), since imported modules are simply variables in the importing ones, and not in a global namespace. I think it's also possible to implement that in Python due to similar semantics, though it's not currently available.
Merovius 8y ago
> If so, why not just call it a version conflict and fail until it's resolved? Because resolving it, in general, doesn't scale. If the dependency graph is A -> B -> D A -> C -> D B and C have to coordinate their upgrade to Dv2, so as to not break A. But B and C might not know of each other and A might not even know about their dependencies on D. Worse, C might not be able or willing to upgrade to Dv2 in the foreseeable future, for lack of bandwidth, so B is kept from upgrading to Dv2 or has to break A. If you allow both Dv1 and Dv2 to coexist, B can upgrade to Dv2, A stays unbroken and at some point, C can upgrade to Dv2 too. No need to coordinate the timing or undue rush. Now multiply that graph with a hundred dependencies and a thousand projects A, and it becomes clear that avoiding the need of coordinated upgrades is a good thing.

deleted 8y ago

[deleted]

MrBuddyCasino 8y ago

> why not just call it a version conflict and fail until it's resolved Because its unnecessary and it doesn't scale. In Rust, versions become part of the symbol name, and the whole thing just becomes a non-issue. Go did the right thing here.

majewsky 8y ago

> If two things require different versions of a package, those seem like separate projects to me. You may be thinking of packages on the scale of frameworks. For example, if you have a Ruby on Rails app that mixes packages from Rails 3 and Rails 5, well sure, then you're in for some serious trouble. But a lot of packages are much more narrow in scope. Rust has a lot of small crates for basic data structures that have their APIs change betweeen versions. So you could end up with your logging package internally depending on some_data_structure=1.2, while your RabbitMQ client library internally depends on some_data_structure=2.3. As long as they do not expose some_data_structure in their API, you will never even notice (except maybe through increased binary size).

derefr 8y ago

> Is this to support dependencies which rely on the same package at different versions? Yes. > If two things require different versions of a package, those seem like separate projects to me Basically we’re talking about libs that have version conflicts, but also very slow release schedules, such that the version conflict isn’t going to be reconciled any time soon; or where one of the deps may even be effectively abandonware, but “works just fine” stand-alone so nobody’s going to update it. It just depends on extremely old versions of common shared deps. These are, effectively, irreconcilable version conflicts. The only thing to do, in most languages, is to fork one or the other dep to fix the conflict; or to split the project into microservices such that the conflicting components can live on opposite sides of a process memory boundary. Even these are sometimes impossible for office-political reasons. > I'm not familiar with any other package management system which supports simultaneous major versions in a single project. Node’s npm. Versions aren’t globally resolved; instead, each package effectively gets its own version resolution. Your root-level package gets a node_modules/ directory with your direct dependencies checked out into it; but your deps' deps are checked out into further node_modules/ directories that exist within their parent dep's checkout directory. It's somewhat as if each parent dep had vendored its deps, and had a vendor/ subdirectory; except that, in this case, the vendoring process is recursive. When a given package require()s a library by name, then, it’s actually importing its own npm-vendorified copy of the library from its own node_modules/ directory, rather than looking in your package's root node_modules/ directory. (In fact, there is no root level; you have "package isotropy", where a dep NPM checked out has the exact same NPM-populated subdirs as a project you git clone'd yourself.) Since these libs aren’t entered into a global namespace by require(), rather just have the module returned as a runtime object for the parent scope to bind to a local variable, there are no runtime conflicts between the separately-vendored versions, either. In a language without such runtime support, though, you would need your package-ecosystem tooling to reach into your downloaded lib and do some name-mangling of the exported symbols, and then either create a router symbol that resolves calls based on the lexical scope, or further mangle your deps' parents to call the dep by its mangled name rather than its original name. I think that’s what’s being proposed here, since Go has no such runtime support. (Effectively, in Go terms, you can simply think of it as all the references to git-http URL depspecs in import statements, being amended to qualify the repo name with the full git SHA, such that two different commits of the same project act as if they were two differently-named projects from Go's perspective.)

rpeden 8y ago
It's a relatively minor nitpick, but since version 3, npm flattens the package tree as much as it can and tries to bring everything possible to the top level node_modules directory. It'll still use nested node_modules where necessary to handle dependency version conflicts. A bit more info here: http://npm.github.io/how-npm-works-docs/npm3/how-npm3-works.html http://npm.github.io/how-npm-works-docs/npm3/how-npm3-works....

DougBTX 8y ago

> Is this to support dependencies which rely on the same package at different versions? Yes, that's right. See for example NPM's docs on how this works: http://npm.github.io/how-npm-works-docs/npm3/how-npm3-works.html http://npm.github.io/how-npm-works-docs/npm3/how-npm3-works....

aasasd 8y ago

A Java bro told me they have lots of pain with version conflicts, since Java world is so reliant both on tons of libraries and on strict interfaces. Apparently there's something called 'dependency shading' to overcome this issue, though seems like it has to be done by the dependency vendor, not the user. NPM notably doesn't have any problems with different versions (afaik), since imported modules are simply variables in the importing ones, and not in a global namespace. I think it's also possible to implement that in Python due to similar semantics, though it's not currently available.

Merovius 8y ago

> If so, why not just call it a version conflict and fail until it's resolved? Because resolving it, in general, doesn't scale. If the dependency graph is A -> B -> D A -> C -> D B and C have to coordinate their upgrade to Dv2, so as to not break A. But B and C might not know of each other and A might not even know about their dependencies on D. Worse, C might not be able or willing to upgrade to Dv2 in the foreseeable future, for lack of bandwidth, so B is kept from upgrading to Dv2 or has to break A. If you allow both Dv1 and Dv2 to coexist, B can upgrade to Dv2, A stays unbroken and at some point, C can upgrade to Dv2 too. No need to coordinate the timing or undue rush. Now multiply that graph with a hundred dependencies and a thousand projects A, and it becomes clear that avoiding the need of coordinated upgrades is a good thing.