4 ms·

For what it's worth, Dart does not let you have two versions of the same package in your application, even different major versions. This restriction does cause

by dkarl 9y ago

For what it's worth, Dart does not let you have two versions of the same package in your application, even different major versions. This restriction does cause real pain, but it doesn't appear to be insurmountable. Most of the pain seems to be in the performance issues over-constrained dependencies caused in our old version solver and not in the user's code itself. In almost all cases, I think there is a single version of a given package that would work in practice, and I think it's confusing for users to have an application that has multiple versions of what they think of as the "same" package inside it. This may be less of an issue in Go because it's structurally typed, but in Dart you could get weird errors like "Expected a Foo but got a Foo" because those "Foo"s are actually from different versions of "foo". Requiring a single version avoids that. I think this makes a strong case for not releasing major version upgrades that use the same package names. The very idea of two incompatible things having the same name should set off alarm bells. Instead of trying to make that work, we should be avoiding it. In the absence of this principle, the Java ecosystem has developed a compensatory mechanism of packaging "shadowed" versions of their dependencies alongside their own code. This is an ugly hack to accomplish the same thing after the fact, so we are already incurring even more complexity than would be imposed by following this rule.

XorNot 9y ago
I'm not really enthused by efforts to bring in the same packaging constructs as other languages, which all have the effect of making time-to-compile after git clone longer. Frankly I think we tend to conflate two separate but related tasks in these discussions: communicating updates, and distributing dependencies. vendor/ folders are a totally fine distribution system - optimal even. Time-to-compile is 0 because you get the dependencies with git clone. So really the problem we have is communicating updates (and some git server tooling smarts to deduplicate files but let github solve that).
- dkarl 9y ago
  According to the article, the solution I suggest has been proposed in the Go community, and I don't know of any language where it has actually been adopted, so Go might be the first. I just know that Java has been forced to work around its absence. As for the tasks that need to be solved here, the primary one I see is reconciling the needs of different libraries. What do you do when you depend on library A and library B and they need two incompatible versions of library C? As I see it, there's no clean way to answer that question if A and B expect the two incompatible versions to be present with the same name.
- ithkuil 9y ago
  Yeah, I'm also a bit concerned with vendoring going away. The proposed solution to preserving upstream is imho very elegant (caching proxies) and scales better than vendoring, but it requires a bit more infrastructure. Perhaps vgo could be taught to look in a local directory for an exploded content of what logically is the upstream archive (and that local directory could be checked in git, or preserved as a cache by your CI etc).
  - jacobvosmaer 9y ago
    I think vendoring is very useful and it would be a step back if it becomes harder. Caching proxies for zip downloads sounds nice, but it's more than just "a bit more infrastructure". I think it would be a huge burden to package publishers if each of them has to manage their own dependency zip mirror as a separate piece of infrastructure. You need version control anyway; checking your dependencies into that same version control does not require a new piece of infrastructure. Coming from Ruby, where rubygems.org is a very painful point of failure, in my eyes the fact that Go dependencies are not a separate download is a big plus. In fact without a single blessed dependency repository such as rubygems.org, in the Go case you have as many points of failure at build time as there are different code hosting sites in your dependency graph.
    - ithkuil 9y ago
      I don't think package publishers are the ones that need to manage the caching proxies: everyone who wants stable builds that don't break when some upstream deletes an old version of a package (or a networking error) needs a proxying cache. Vendoring turned your git repo as your poor man's proxying cache. It also made some people unhappy. In my current company we use phabricator for code reviews and it doesn't work well if the commit size is bigger than some threshold. I love to have the option of not checking in dependencies. I'm not sure this option has to be forced on everybody though.
munificent 9y ago
> I think this makes a strong case for not releasing major version upgrades that use the same package names. The very idea of two incompatible things having the same name should set off alarm bells. Instead of trying to make that work, we should be avoiding it. If you do that, I think you'll find in practice that one of two things happens (or more likely, both, in a confusing mixture): 1. People start releasing packages whose names include version numbers. "markdown2", etc. Then you get really confusing hallways conversations like, "Yeah, you need to use markdown2 1.0.0." 2. People start coming up with weird confusing names for the next major version of packages because the current nice name is taken. Then you get confusing conversations like, "Oh, yeah, you need to upgrade from flippitywidget to spongiform. It's almost exactly the same, but they removed that one deprecated method." Also don't forget to rename all of your imports.
- dkarl 9y ago
  I think you'll find in practice that one of two things happens I think the existence of those practices (like Java dependency shading) proves that people are struggling towards this solution on their own, without support from the language or the community. With official support, if major versions work the same way for everybody, it won't need to be so janky. In practice, I predict that people would start behaving better, doing what they should have been doing (and what many have been doing) all along: avoiding unnecessary breaking changes in non-0.x libraries, choosing function signatures carefully, and growing by accretion and living with their mistakes. Right now, I think some developers see major version bumps as a convenient way to erase their mistakes, without taking into account the cost imposed on users who end up juggling dependency conflicts.
- sitkack 9y ago
  The major version goes into the name, marldown2, but the version numbers should be monotonic, so when when viewed on a number line they are in-order. This also allows the programmer to import both, and have a smooth transition between the deps.
merb 9y ago
> In the absence of this principle, the Java ecosystem has developed a compensatory mechanism of packaging "shadowed" versions of their dependencies alongside their own code. well that is okish.. however... looking at the xml stuff.. that is probably bad. basically a lot of packages repackage xerces2 or the java.xml apis. besides that jaxp with streaming is present in java since 1.6+. But nobody removes this stuff.

by dkarl 9y ago

XorNot 9y ago

I'm not really enthused by efforts to bring in the same packaging constructs as other languages, which all have the effect of making time-to-compile after git clone longer. Frankly I think we tend to conflate two separate but related tasks in these discussions: communicating updates, and distributing dependencies. vendor/ folders are a totally fine distribution system - optimal even. Time-to-compile is 0 because you get the dependencies with git clone. So really the problem we have is communicating updates (and some git server tooling smarts to deduplicate files but let github solve that).

dkarl 9y ago
According to the article, the solution I suggest has been proposed in the Go community, and I don't know of any language where it has actually been adopted, so Go might be the first. I just know that Java has been forced to work around its absence. As for the tasks that need to be solved here, the primary one I see is reconciling the needs of different libraries. What do you do when you depend on library A and library B and they need two incompatible versions of library C? As I see it, there's no clean way to answer that question if A and B expect the two incompatible versions to be present with the same name.
ithkuil 9y ago
Yeah, I'm also a bit concerned with vendoring going away. The proposed solution to preserving upstream is imho very elegant (caching proxies) and scales better than vendoring, but it requires a bit more infrastructure. Perhaps vgo could be taught to look in a local directory for an exploded content of what logically is the upstream archive (and that local directory could be checked in git, or preserved as a cache by your CI etc).
- jacobvosmaer 9y ago
  I think vendoring is very useful and it would be a step back if it becomes harder. Caching proxies for zip downloads sounds nice, but it's more than just "a bit more infrastructure". I think it would be a huge burden to package publishers if each of them has to manage their own dependency zip mirror as a separate piece of infrastructure. You need version control anyway; checking your dependencies into that same version control does not require a new piece of infrastructure. Coming from Ruby, where rubygems.org is a very painful point of failure, in my eyes the fact that Go dependencies are not a separate download is a big plus. In fact without a single blessed dependency repository such as rubygems.org, in the Go case you have as many points of failure at build time as there are different code hosting sites in your dependency graph.
  - ithkuil 9y ago
    I don't think package publishers are the ones that need to manage the caching proxies: everyone who wants stable builds that don't break when some upstream deletes an old version of a package (or a networking error) needs a proxying cache. Vendoring turned your git repo as your poor man's proxying cache. It also made some people unhappy. In my current company we use phabricator for code reviews and it doesn't work well if the commit size is bigger than some threshold. I love to have the option of not checking in dependencies. I'm not sure this option has to be forced on everybody though.

munificent 9y ago

> I think this makes a strong case for not releasing major version upgrades that use the same package names. The very idea of two incompatible things having the same name should set off alarm bells. Instead of trying to make that work, we should be avoiding it. If you do that, I think you'll find in practice that one of two things happens (or more likely, both, in a confusing mixture): 1. People start releasing packages whose names include version numbers. "markdown2", etc. Then you get really confusing hallways conversations like, "Yeah, you need to use markdown2 1.0.0." 2. People start coming up with weird confusing names for the next major version of packages because the current nice name is taken. Then you get confusing conversations like, "Oh, yeah, you need to upgrade from flippitywidget to spongiform. It's almost exactly the same, but they removed that one deprecated method." Also don't forget to rename all of your imports.

dkarl 9y ago
I think you'll find in practice that one of two things happens I think the existence of those practices (like Java dependency shading) proves that people are struggling towards this solution on their own, without support from the language or the community. With official support, if major versions work the same way for everybody, it won't need to be so janky. In practice, I predict that people would start behaving better, doing what they should have been doing (and what many have been doing) all along: avoiding unnecessary breaking changes in non-0.x libraries, choosing function signatures carefully, and growing by accretion and living with their mistakes. Right now, I think some developers see major version bumps as a convenient way to erase their mistakes, without taking into account the cost imposed on users who end up juggling dependency conflicts.
sitkack 9y ago
The major version goes into the name, marldown2, but the version numbers should be monotonic, so when when viewed on a number line they are in-order. This also allows the programmer to import both, and have a smooth transition between the deps.

merb 9y ago

> In the absence of this principle, the Java ecosystem has developed a compensatory mechanism of packaging "shadowed" versions of their dependencies alongside their own code. well that is okish.. however... looking at the xml stuff.. that is probably bad. basically a lot of packages repackage xerces2 or the java.xml apis. besides that jaxp with streaming is present in java since 1.6+. But nobody removes this stuff.