7 ms·

Here’s the post’s core thesis: > At this point you could even get rid of the version altogether and just use the commit hash on the rare occasions when we need

by jitl 9y ago

Here’s the post’s core thesis: > At this point you could even get rid of the version altogether and just use the commit hash on the rare occasions when we need a version identifier. We're all on the internet, and we're all constantly running npm install or equivalent. Just say "Leftpad-17", it's cleaner. > And that's it. Package managers should provide no options for version pinning. > A package manager that followed such a proposal would foster an eco-system with greater care for introducing incompatibility. Packages that wantonly broke their consumers would "gain a reputation" and get out-competed by packages that didn't, rather than gaining a "temporary" pinning that serves only to perpetuate them. Pinning is absolutely crucial when building software from some sources out of your control, especially with larger teams or larger software. You cannot rely on the responsibility of others, be they 100s of upstream library authors, or 100s of other peer developers on your product. One among those people may cause a dependency to break your build, either by releasing a bad version, or foolishly adding a dependency without carefully checking its “reputation” for frequent breaking changes. In either case, without pinning, your build is non-deterministic and sometimes starts failing it’s tests without any diff. You can’t bisect that. Your only remediation is manual debugging and analysis. Work stops for N engineers because everyone’s branches are broken. I don’t think any level of community fostering is worth that kind of risk.

deleted 9y ago
[deleted]
0x445442 9y ago
Preach it brother. I work primarily in the enterprise Java world and I'd never just blindly compile in a dependency of LATEST and keep my fingers crossed. Any version bumps in my POM dependencies block are intentional. The transitive dependencies are hard enough to deal with already without having to worry if my dependent binary footprint changed significantly from the last build.
- chiefalchemist 9y ago
  Within a given team new dev always breeds bugs. Yet some persume that's somehow not true for upstream dependencies? I can't see making that assumption. I would think the best approch is "trust but verify" ANY update to a dependency. A dependency might save you some time but it's not a free pass to be irresponsible. There's not such thing as FOSS.
masklinn 9y ago
> Pinning is absolutely crucial when building software from some sources out of your control, especially with larger teams or larger software. You cannot rely on the responsibility of others, be they 100s of upstream library authors, or 100s of other peer developers on your product. In fact that's the entire reason why "semantics versioning" doesn't really work: * you can never be certain that the maintainer actually follows it * one user's breaking change is one maintainer's bugfix * while some packaging systems attempt to automate and enforce it (e.g. Elm's) even that only goes so far due to type system limitations (e.g. changing acceptable values from [0, 4] to [0, 4[ as a side-effect of some change deep in the bowels of the library's logic is pretty much guaranteed not to surface if you don't have a dependent type system, and may not do so even then)
- pothix 9y ago
  > you can never be certain that the maintainer actually follows it The same happens for the proposed model ¯\_(ツ)_/¯
  - masklinn 9y ago
    I think you and many other people misunderstood my criticism, it wasn't a criticism of semantic versioning per-se, but was rather than explanation of the issues with semver, which the proposed "scheme" indeed does not fix in any way, shape or form. The proposed author claims that "You cannot rely on the responsibility of others" but their scheme fails in the exact same way.
- kibwen 9y ago
  It only "doesn't really work" because this is ascribing too much responsibility to it. Semver is a social protocol designed to assist in solving--but by no means completely solve--the technical problem of how to resolve dependency versioning. It only gives us the barest protocol to allow us to begin describing the problem, and permits us to attempt to build technical solutions on top of it. Nobody's pretending it's perfect, but if anyone has a better solution, feel free to propose it (and neither the OP's, nor "romantic versioning", are any better at solving this problem than semver).
eikenberry 9y ago
Pinning isn't enough. People push history changes, rename things and just delete stuff all the time. I've had this happen to be more than once and would now never consider a application production ready until all dependencies are either vendored or forked.
- jitl 9y ago
  There are plenty of strategies for making your dependencies deterministic. At Airbnb, we use caching proxies in front of all third-party language package managers, so the first time someone in the org pulls “foo 1.2.3”, the source tarball is frozen forever. The package can’t be yanked upstream, force-pushed over, etc.
  - solipsism 9y ago
    This is complex vendoring, nothing more.
    - jitl 9y ago
      Yeap
  - greggman 9y ago
    why not just check in your dependencies like Firefox or chromium or WebKit. with your proxy only people using your proxy get those benefits. with checkin everyone gets those benefits.
smsm42 9y ago
> just use the commit hash Does the author honestly fail to realize that commit hashes are incomprehensible to most humans and look like noise, it takes extraordinary mental effort to compare them? While difference between 1.2.3 and 1.4.5 is instantly apparent to any human? > Pinning is absolutely crucial when building software from some sources out of your control Amen. I am surprised how the author does not recognize it. In fact, the only idea that does not come as outright false immediately after reading is "if you change behavior, rename". But thinking about it for a bit, it is wrong too. First, naming is hard. Finding good recognizable name for a package is hard enough, if each BC break would require a new one, we'd drown in names. Second, behavior breaks are not total. If RoR or ElasticSearch releases a new version with BC break, they do not stop being essentially the same package just with somewhat different behavior. Most of the knowledge you had about it is still relevant. Some pieces are broken, but not the whole concept. New name requires throwing out the whole concept and building a new one, essentially. It is not good for incremental gradual change.
- deleted 9y ago
  [deleted]
- kelnos 9y ago
  My reading of that was that "rename" in that context meant "rename your package from Foo-1 to Foo-2", not come up with a completely brand-new name.
  - jacalata 9y ago
    Mine too, eventually. I am surprised that the writer was unable to make even that very simple point clear.
    - devty 9y ago
      I thought the author did a decent job - he had two examples (Rails-5 and Leftpad-17) that made the point pretty clearly.
- nerdponx 9y ago
konschubert 9y ago
In the same vein, I do not understand the popularity of semantic versioning. Why would I trust hundreds of people that their non-breaking changes are really non-breaking to me? And suddenly, the auto-scaling fails in production because something has updated to include a new bug.
- civilitty 9y ago
  As with all things in life, semantic versioning is a heuristic for a largely stochastic process. You and only you are responsible for whether your code is broken or not. Semantic versioning is just this one standard most of us have agreed to follow as a community to make that responsibility a little easier to bear.
  - konschubert 9y ago
    I think I was being unclear. I didn't mean that semantic versioning is bad, I meant that specifying your dependencies in terms of semantic versions is bad.
mr_tristan 9y ago
Yep, I've always felt versioning is more or less a form of marketing for your software. It's handy that people are kind of using convention to say things like "this is a HUGE change" vs "oops, bugfix". But it's really just conventions of communication, which, in a nutshell, comes with zero guarantee . Semantic versioning and automatic "transitive" dependencies sure are handy, especially in that "naive" phase of development. But man, once you start having to start tracking performance and security, it's time to turn off any magic update. Or, in other words, when you've got people depending on you, you need to start putting your security hat on and assume that there can be a breach with every change, or your performance hat and assume the system will crash due to a performance problem. No way would I want any rebuild OF THE SAME SOURCE subject to change.
hinkley 9y ago
I have a crazy theory about versioning, except it's not the idea itself that's crazy, but what you have to do to get it. Software versioning is a case of the XY Problem. We don't know what we want, but we know something that might help us get it (and we are wrong). We don't care if you add behavior to your library (unless it fulfills a need we had or didn't know we had). What we really care about is if you take behavior away. And in some cases it's not even all the behavior. If I'm only using 20% of your library and you haven't changed it, I can keep upgrading whenever. What semantic versioning is trying and failing to achieve is to make libraries adhere to the Liskov Substitution Principle (LSP). If you had a library that had sufficient tests, then any change you make that causes you to add tests but not change an existing test means that it should satisfy the LSP (tests that correct ambiguities in the spec notwithstanding). People can feel pretty safe upgrading if all of the tests they depend on are intact. The difficult part of this 'solution' is that reusable code would have to have pretty exhaustive test coverage. I think we are already bifurcating into a world of library authors and library consumers, so I'm okay with this. We should feel comfortable demanding more than what we currently get from our library writers. In this world, what you version is your test harness, and not your code.
bluetwo 9y ago
I just use the date. Period.
solatic 9y ago
> Pinning is absolutely crucial Precisely this. The problem isn't with versioning, it's with non-deterministic builds. Once you have a large enough project that you can no longer keep manual track of version changes, you build tooling which checks for the latest version and opens a pull request (and subsequent CI build on the pre-merge commit) with the latest version. This keeps your project up to date and prevents the woes of non-deterministic building.
greggman 9y ago
pinning while more reliable is not actually fully reliable. If you want reliable you should be checking in copies of your dependencies either directly or as sub modules. then if you want to upgrade you check in the latest. Pinning still allows the person in control of the 3rdparty repo to mess you up. Your reproducibility is still at their mercy.
- viraptor 9y ago
  That depends on the repo. Some do not allow changes (short of the repo itself failing). Some allow only withdrawing versions, but no changes. But yes, you should either commit deps, or have your own repo/caching-proxy which will neither change nor drop old versions.
kakwa_ 9y ago
> Pinning is absolutely crucial when building software from some sources out of your control. IMHO it's a bad solution to a real problem. Too often, I've seen software with out of date libraries or frameworks, with versions set a the start of a project and never touched once since. In some cases it even lead to security flaws not being patched. Another side effect is that hard pinning complete versions makes for somewhat hard packaging (if you try to follow some distro guidelines (like the Debian ones), package libraries separately and not bundle everything together). In some extreme cases, avoiding the small and incremental breaks & fixes in API can lead to code bases so far from the modern APIs of their dependencies that throwing away the code and re-implementing from scratch is actually a better solution. I feel that was is missing in most languages is a way to properly maintain APIs (like tools native in the language that help ensure no breaks in the API contract) and language that have concepts of API versioning and API negotiation (kind of like rest API or protocol version negotiation) native to it. If this was also combined with a backward compatibility policy of "at least maintain version N + version (N-1)" and clear deprecation notifications, it would make for smoother updates and makes most systems more reliable. I would love to see a language with these items as core features. But we are far from this situation. What I generally do instead is to carefully chose my dependencies in projects that have a good track record of not breaking their APIs too often while being correctly maintained and also I tend to try to keep the number of dependencies in my projects to a minimum. I almost never hard pin dependency versions. It's no wonder that the MIL-STD-498 has a large focus on interfaces (Interface Requirements Specification (IRS) and Interface Design Description (IDD) specification documents). It's actually one of the harder parts to get right when designing complex systems (including complex pieces of software).

by jitl 9y ago

deleted 9y ago

[deleted]

0x445442 9y ago

Preach it brother. I work primarily in the enterprise Java world and I'd never just blindly compile in a dependency of LATEST and keep my fingers crossed. Any version bumps in my POM dependencies block are intentional. The transitive dependencies are hard enough to deal with already without having to worry if my dependent binary footprint changed significantly from the last build.

chiefalchemist 9y ago
Within a given team new dev always breeds bugs. Yet some persume that's somehow not true for upstream dependencies? I can't see making that assumption. I would think the best approch is "trust but verify" ANY update to a dependency. A dependency might save you some time but it's not a free pass to be irresponsible. There's not such thing as FOSS.

masklinn 9y ago

> Pinning is absolutely crucial when building software from some sources out of your control, especially with larger teams or larger software. You cannot rely on the responsibility of others, be they 100s of upstream library authors, or 100s of other peer developers on your product. In fact that's the entire reason why "semantics versioning" doesn't really work: * you can never be certain that the maintainer actually follows it * one user's breaking change is one maintainer's bugfix * while some packaging systems attempt to automate and enforce it (e.g. Elm's) even that only goes so far due to type system limitations (e.g. changing acceptable values from [0, 4] to [0, 4[ as a side-effect of some change deep in the bowels of the library's logic is pretty much guaranteed not to surface if you don't have a dependent type system, and may not do so even then)

pothix 9y ago
> you can never be certain that the maintainer actually follows it The same happens for the proposed model ¯\_(ツ)_/¯
- masklinn 9y ago
  I think you and many other people misunderstood my criticism, it wasn't a criticism of semantic versioning per-se, but was rather than explanation of the issues with semver, which the proposed "scheme" indeed does not fix in any way, shape or form. The proposed author claims that "You cannot rely on the responsibility of others" but their scheme fails in the exact same way.
kibwen 9y ago
It only "doesn't really work" because this is ascribing too much responsibility to it. Semver is a social protocol designed to assist in solving--but by no means completely solve--the technical problem of how to resolve dependency versioning. It only gives us the barest protocol to allow us to begin describing the problem, and permits us to attempt to build technical solutions on top of it. Nobody's pretending it's perfect, but if anyone has a better solution, feel free to propose it (and neither the OP's, nor "romantic versioning", are any better at solving this problem than semver).

eikenberry 9y ago

Pinning isn't enough. People push history changes, rename things and just delete stuff all the time. I've had this happen to be more than once and would now never consider a application production ready until all dependencies are either vendored or forked.

jitl 9y ago
There are plenty of strategies for making your dependencies deterministic. At Airbnb, we use caching proxies in front of all third-party language package managers, so the first time someone in the org pulls “foo 1.2.3”, the source tarball is frozen forever. The package can’t be yanked upstream, force-pushed over, etc.
- solipsism 9y ago
  This is complex vendoring, nothing more.
  - jitl 9y ago
    Yeap
- greggman 9y ago
  why not just check in your dependencies like Firefox or chromium or WebKit. with your proxy only people using your proxy get those benefits. with checkin everyone gets those benefits.

smsm42 9y ago

> just use the commit hash Does the author honestly fail to realize that commit hashes are incomprehensible to most humans and look like noise, it takes extraordinary mental effort to compare them? While difference between 1.2.3 and 1.4.5 is instantly apparent to any human? > Pinning is absolutely crucial when building software from some sources out of your control Amen. I am surprised how the author does not recognize it. In fact, the only idea that does not come as outright false immediately after reading is "if you change behavior, rename". But thinking about it for a bit, it is wrong too. First, naming is hard. Finding good recognizable name for a package is hard enough, if each BC break would require a new one, we'd drown in names. Second, behavior breaks are not total. If RoR or ElasticSearch releases a new version with BC break, they do not stop being essentially the same package just with somewhat different behavior. Most of the knowledge you had about it is still relevant. Some pieces are broken, but not the whole concept. New name requires throwing out the whole concept and building a new one, essentially. It is not good for incremental gradual change.

deleted 9y ago
[deleted]
kelnos 9y ago
My reading of that was that "rename" in that context meant "rename your package from Foo-1 to Foo-2", not come up with a completely brand-new name.
- jacalata 9y ago
  Mine too, eventually. I am surprised that the writer was unable to make even that very simple point clear.
  - devty 9y ago
    I thought the author did a decent job - he had two examples (Rails-5 and Leftpad-17) that made the point pretty clearly.
nerdponx 9y ago

konschubert 9y ago

In the same vein, I do not understand the popularity of semantic versioning. Why would I trust hundreds of people that their non-breaking changes are really non-breaking to me? And suddenly, the auto-scaling fails in production because something has updated to include a new bug.

civilitty 9y ago
As with all things in life, semantic versioning is a heuristic for a largely stochastic process. You and only you are responsible for whether your code is broken or not. Semantic versioning is just this one standard most of us have agreed to follow as a community to make that responsibility a little easier to bear.
- konschubert 9y ago
  I think I was being unclear. I didn't mean that semantic versioning is bad, I meant that specifying your dependencies in terms of semantic versions is bad.

mr_tristan 9y ago

Yep, I've always felt versioning is more or less a form of marketing for your software. It's handy that people are kind of using convention to say things like "this is a HUGE change" vs "oops, bugfix". But it's really just conventions of communication, which, in a nutshell, comes with zero guarantee . Semantic versioning and automatic "transitive" dependencies sure are handy, especially in that "naive" phase of development. But man, once you start having to start tracking performance and security, it's time to turn off any magic update. Or, in other words, when you've got people depending on you, you need to start putting your security hat on and assume that there can be a breach with every change, or your performance hat and assume the system will crash due to a performance problem. No way would I want any rebuild OF THE SAME SOURCE subject to change.

hinkley 9y ago

I have a crazy theory about versioning, except it's not the idea itself that's crazy, but what you have to do to get it. Software versioning is a case of the XY Problem. We don't know what we want, but we know something that might help us get it (and we are wrong). We don't care if you add behavior to your library (unless it fulfills a need we had or didn't know we had). What we really care about is if you take behavior away. And in some cases it's not even all the behavior. If I'm only using 20% of your library and you haven't changed it, I can keep upgrading whenever. What semantic versioning is trying and failing to achieve is to make libraries adhere to the Liskov Substitution Principle (LSP). If you had a library that had sufficient tests, then any change you make that causes you to add tests but not change an existing test means that it should satisfy the LSP (tests that correct ambiguities in the spec notwithstanding). People can feel pretty safe upgrading if all of the tests they depend on are intact. The difficult part of this 'solution' is that reusable code would have to have pretty exhaustive test coverage. I think we are already bifurcating into a world of library authors and library consumers, so I'm okay with this. We should feel comfortable demanding more than what we currently get from our library writers. In this world, what you version is your test harness, and not your code.

bluetwo 9y ago

I just use the date. Period.

solatic 9y ago

> Pinning is absolutely crucial Precisely this. The problem isn't with versioning, it's with non-deterministic builds. Once you have a large enough project that you can no longer keep manual track of version changes, you build tooling which checks for the latest version and opens a pull request (and subsequent CI build on the pre-merge commit) with the latest version. This keeps your project up to date and prevents the woes of non-deterministic building.

greggman 9y ago

pinning while more reliable is not actually fully reliable. If you want reliable you should be checking in copies of your dependencies either directly or as sub modules. then if you want to upgrade you check in the latest. Pinning still allows the person in control of the 3rdparty repo to mess you up. Your reproducibility is still at their mercy.

viraptor 9y ago
That depends on the repo. Some do not allow changes (short of the repo itself failing). Some allow only withdrawing versions, but no changes. But yes, you should either commit deps, or have your own repo/caching-proxy which will neither change nor drop old versions.

kakwa_ 9y ago

> Pinning is absolutely crucial when building software from some sources out of your control. IMHO it's a bad solution to a real problem. Too often, I've seen software with out of date libraries or frameworks, with versions set a the start of a project and never touched once since. In some cases it even lead to security flaws not being patched. Another side effect is that hard pinning complete versions makes for somewhat hard packaging (if you try to follow some distro guidelines (like the Debian ones), package libraries separately and not bundle everything together). In some extreme cases, avoiding the small and incremental breaks & fixes in API can lead to code bases so far from the modern APIs of their dependencies that throwing away the code and re-implementing from scratch is actually a better solution. I feel that was is missing in most languages is a way to properly maintain APIs (like tools native in the language that help ensure no breaks in the API contract) and language that have concepts of API versioning and API negotiation (kind of like rest API or protocol version negotiation) native to it. If this was also combined with a backward compatibility policy of "at least maintain version N + version (N-1)" and clear deprecation notifications, it would make for smoother updates and makes most systems more reliable. I would love to see a language with these items as core features. But we are far from this situation. What I generally do instead is to carefully chose my dependencies in projects that have a good track record of not breaking their APIs too often while being correctly maintained and also I tend to try to keep the number of dependencies in my projects to a minimum. I almost never hard pin dependency versions. It's no wonder that the MIL-STD-498 has a large focus on interfaces (Interface Requirements Specification (IRS) and Interface Design Description (IDD) specification documents). It's actually one of the harder parts to get right when designing complex systems (including complex pieces of software).