6 ms·

I never knew that the decision to not have versioning built into go was because of the diamond dependency problem. This makes a lot of sense now that I think ab

I never knew that the decision to not have versioning built into go was because of the diamond dependency problem. This makes a lot of sense now that I think about it. I wonder what alternatives to manually versioning things the go team is considering. Also, at the beginning the Go team wasn't all that enthusiastic about an IDE, but its great to see that Go will get an IntelliJ grade IDE soon, so that works for me :)

lostcolony 11y ago
I really am not a fan of the lack of versioning, even with that explanation. Of course diamond dependencies are a problem. But they haven't actually fixed that problem; they've just declared a method for picking one of the two dependencies that is ill defined ("whichever one you happen to have pulled in first"). At least with versioning I get told that there's an incompatibility in the expectations between two libraries. With Go's removal of version numbers, they haven't removed any of the complexity of resolving incompatibilities, they've just hidden the fact that there is one. I would have much preferred a strategy of maintaining version numbers, and resolving them via "either you specify what version you want pulled in, or we'll build with the most recent and throw a warning".
- skybrian 11y ago
  The fix for Google is to check in all third party libraries locally, so you get the same version as everyone else in the company until someone decides to check in an upgraded version. It works fine and anyone else is free to do the same. But that requires a monorepo that scales and people to test and check in the upgrades, so it doesn't work well for open source development, which is why people are working on alternatives.
  - lostcolony 11y ago
    Right, but you could do that with version numbers, too. The difference between updating from "whatever version was pulled in at the time we first set up the repo with this library" and "current" is no different than updating from "version X" to "version Y", except that you don't know what the differences are, and there's no way to determine that it would be easier (and possible) to update to version X.1, since that includes the fix/feature that triggered the desire for an update, and doesn't break as many things, so requires less work.
    - skybrian 11y ago
      Yes, if you're going to check stuff in locally, you need to document which version you pulled to make the next upgrade easier. So it's not like nobody is keeping track of versions (or better yet, git commits). But this can all be done using a separate system. The build system itself doesn't need to know anything about versions. If you look at Bazel, it doesn't let you specify versions for target deps. Only the developer actually upgrading a third-party library needs to deal with open source version numbers, and everyone else can build on their work.
      - lostcolony 11y ago
        "You'll need to document which version you pulled" - so now you're tasked with describing "I got this version" (be that git commit or whatever), rather than it being explicit as part of the package "This is (x) version". So you get to reimplement something yourself, in a non-portable, non-self-describing way. Such an improvement. Of course it could be done in a separate system. It could be done and tracked in a separate system even with version numbers. Go has just chosen to -require- you running your own package manager to make things stable, whereas for most languages it's optional and you can create a repeatable, portable build without it.
- mike_hearn 11y ago
  There are fixes for the diamond dependency problem out there, it's not like this is how to build a warp drive. For instance UNIXes have had symbol versioning since forever, they have RTLD_DEEPBIND in glibc, etc. Java is currently noodling around this problem with Jigsaw (though there's already a solution in OSGi) and it looks like for now they'll punt on solving it themselves, but provide the APIs needed for a simple container to solve it instead. It's fine to not ship a solution in version one. But to use it as a reason to not have version numbers at all seems like heading in the wrong direction (backwards).
- d_theorist 11y ago
  I have never really thought about this problem before, but would it not be possible to: 1. Treat each version of a package as a separate package. If there are two versions required, they are both included. 2. At compile time transform package names to include the desired version number. I may be completely misunderstanding or over-simplifying this problem.
  - lostcolony 11y ago
    For some things, yes. But there are a lot of potentially tricksy issues in doing that. Imagine a queueing library, for instance. You push something onto a queue created with library A, and then you want to read from library B (A and B are different version of the same library). Just about anything could happen (depending on how the library/language are implemented). You might get an error due to library B reading a data structure formatted for library A. Or you might get nothing back, because library B is checking for the queue in its own registry, while it exists in library A's. Etc. If there is no library-dependent state between calls, or the calls are not being mixed between libraries, you can totally do that, though. A JSON serialization library can totally be supported in that manner, for instance. The queueing library example, if it's only being used to support other libraries or separate functionalities, such that you never have to push onto a queue in library A, and expect to be able to pop it off from a queue in library B, would work, too. But in that latter case, that's not something the library writer can guarantee, it's all about your own usage. So there's a lot of potential issues.
    - pcwalton 11y ago
      Making the version part of the type generally fixes these issues. Anything that could go wrong will nearly always go wrong at compile time.
      - ustolemyname 11y ago
        Sure, that fixes some problems, but I don't understand how that would solve the first example listed above. Can you explain how the compiler would prevent this runtime error? // package a lib-v16.Queue(int32) // Stores in queue for version 16 of lib // package b int32 = lib-v18.DeQueue() // Loads from queue of version 18 of lib, which is empty And even if we accept that you shouldn't have state in the package and the queue should be passed around, now we just have two packages (a & b) that use incompatible versions of lib for potentially no reason.
- psadauskas 11y ago
  I think that the solution to this problem is one of the few things that nodejs got right. Instead of declaring a global dependency on `require("D")`, you assign it a variable name. Then, each package can declare its sub-dependencies, and even when the versions differ, the different versions are kept in a different scope. https://nodejs.org/api/modules.html https://nodejs.org/api/modules.html // Package A: import B from "B"; // A/node_modules/B/1.0.0 import C from "C"; // A/node_modules/C/1.0.0 // Package B: import D from "D"; // A/node_modules/B/1.0.0/node_modules/D/1.0.0 // Package C: import D from "D"; // A/node_modules/C/1.0.0/node_modules/D/2.0.0
  - jasonwatkinspdx 11y ago
    I'm convinced this is the right approach: dependencies should be purely local to a module. The tooling should make it easy to detect you have a diamond, but let each module get what it needs. As generally everyone moves towards more continuous update and integration, at the end of the day it's all gonna be sha's anyhow.
    - wtetzner 11y ago
      But what do you do in a typed language? For example, suppose you have modules A and B, which depend on module C. Let's also say that module A has a function foo that return a value of type C.t, and module B has a function bar that accepts a value of type C.t. Now let's say that module A is depending on version 1.0.0 of module C, and module B is depending on version 1.5.2 of module C. Does B.bar(A.foo()) still work? Is type t the same in C version 1.0.0 and 1.5.2?
      - eknkc 11y ago
        Good point. But if B.bar accepts an interface that C.t implements, then everything should work just fine and that should be the way to go on public APIs.
        wtetzner 11y ago
        But what if the interface is defined in C? The interface has to live somewhere, and wouldn't it be versioned too?
incepted 11y ago
No, it doesn't really make much sense. The diamond dependency problem only happens when two indirect dependencies are identical but on different versions and that these two versions are incompatible. That's two big if's. 99% of the time, it's not a problem. Notice first that this is a pretty rare problem since libraries tend to be backward compatible these days. And if you happen to come across a bad behaved one, you simply exclude the one you don't want from the graph, problem solved. This problem has been fixed in Maven for almost ten years now, the only reason why you would not implement this crucial feature in a version manager is laziness.
voidlogic 11y ago
Everyone seems to overlook this, but Go has had LiteIDE for a very long time and it keeps getting better.
- dawkins 11y ago
  I thought the same. I use it every day and its great. I dont understand why it doesn't get more traction.
  - vox_mollis 11y ago
    Because visually, it's reminiscent of Eclipse 1.0. For better or worse, a sizable proportion of developers choose IDEs on the basis of looks.
    - Vendan 11y ago
      Heh, I choose IDEs based on performance, not looks, and liteide is blazing fast compared to anything else I've used.
AsyncAwait 11y ago
I'm not sure if the Intellij Go Plugin is going to be made into a separate IDE, but it's already available for their other IDEs as a plugin and they have a team improving it constantly. Notably, it also works with their open-source IDEA Community Edition. https://github.com/go-lang-plugin-org/go-lang-idea-plugin https://github.com/go-lang-plugin-org/go-lang-idea-plugin So far the best "Go IDE", I've used.

lostcolony 11y ago

I really am not a fan of the lack of versioning, even with that explanation. Of course diamond dependencies are a problem. But they haven't actually fixed that problem; they've just declared a method for picking one of the two dependencies that is ill defined ("whichever one you happen to have pulled in first"). At least with versioning I get told that there's an incompatibility in the expectations between two libraries. With Go's removal of version numbers, they haven't removed any of the complexity of resolving incompatibilities, they've just hidden the fact that there is one. I would have much preferred a strategy of maintaining version numbers, and resolving them via "either you specify what version you want pulled in, or we'll build with the most recent and throw a warning".

skybrian 11y ago
The fix for Google is to check in all third party libraries locally, so you get the same version as everyone else in the company until someone decides to check in an upgraded version. It works fine and anyone else is free to do the same. But that requires a monorepo that scales and people to test and check in the upgrades, so it doesn't work well for open source development, which is why people are working on alternatives.
- lostcolony 11y ago
  Right, but you could do that with version numbers, too. The difference between updating from "whatever version was pulled in at the time we first set up the repo with this library" and "current" is no different than updating from "version X" to "version Y", except that you don't know what the differences are, and there's no way to determine that it would be easier (and possible) to update to version X.1, since that includes the fix/feature that triggered the desire for an update, and doesn't break as many things, so requires less work.
  - skybrian 11y ago
    Yes, if you're going to check stuff in locally, you need to document which version you pulled to make the next upgrade easier. So it's not like nobody is keeping track of versions (or better yet, git commits). But this can all be done using a separate system. The build system itself doesn't need to know anything about versions. If you look at Bazel, it doesn't let you specify versions for target deps. Only the developer actually upgrading a third-party library needs to deal with open source version numbers, and everyone else can build on their work.
    - lostcolony 11y ago
      "You'll need to document which version you pulled" - so now you're tasked with describing "I got this version" (be that git commit or whatever), rather than it being explicit as part of the package "This is (x) version". So you get to reimplement something yourself, in a non-portable, non-self-describing way. Such an improvement. Of course it could be done in a separate system. It could be done and tracked in a separate system even with version numbers. Go has just chosen to -require- you running your own package manager to make things stable, whereas for most languages it's optional and you can create a repeatable, portable build without it.
mike_hearn 11y ago
There are fixes for the diamond dependency problem out there, it's not like this is how to build a warp drive. For instance UNIXes have had symbol versioning since forever, they have RTLD_DEEPBIND in glibc, etc. Java is currently noodling around this problem with Jigsaw (though there's already a solution in OSGi) and it looks like for now they'll punt on solving it themselves, but provide the APIs needed for a simple container to solve it instead. It's fine to not ship a solution in version one. But to use it as a reason to not have version numbers at all seems like heading in the wrong direction (backwards).
d_theorist 11y ago
I have never really thought about this problem before, but would it not be possible to: 1. Treat each version of a package as a separate package. If there are two versions required, they are both included. 2. At compile time transform package names to include the desired version number. I may be completely misunderstanding or over-simplifying this problem.
- lostcolony 11y ago
  For some things, yes. But there are a lot of potentially tricksy issues in doing that. Imagine a queueing library, for instance. You push something onto a queue created with library A, and then you want to read from library B (A and B are different version of the same library). Just about anything could happen (depending on how the library/language are implemented). You might get an error due to library B reading a data structure formatted for library A. Or you might get nothing back, because library B is checking for the queue in its own registry, while it exists in library A's. Etc. If there is no library-dependent state between calls, or the calls are not being mixed between libraries, you can totally do that, though. A JSON serialization library can totally be supported in that manner, for instance. The queueing library example, if it's only being used to support other libraries or separate functionalities, such that you never have to push onto a queue in library A, and expect to be able to pop it off from a queue in library B, would work, too. But in that latter case, that's not something the library writer can guarantee, it's all about your own usage. So there's a lot of potential issues.
  - pcwalton 11y ago
    Making the version part of the type generally fixes these issues. Anything that could go wrong will nearly always go wrong at compile time.
    - ustolemyname 11y ago
      Sure, that fixes some problems, but I don't understand how that would solve the first example listed above. Can you explain how the compiler would prevent this runtime error? // package a lib-v16.Queue(int32) // Stores in queue for version 16 of lib // package b int32 = lib-v18.DeQueue() // Loads from queue of version 18 of lib, which is empty And even if we accept that you shouldn't have state in the package and the queue should be passed around, now we just have two packages (a & b) that use incompatible versions of lib for potentially no reason.
psadauskas 11y ago
I think that the solution to this problem is one of the few things that nodejs got right. Instead of declaring a global dependency on `require("D")`, you assign it a variable name. Then, each package can declare its sub-dependencies, and even when the versions differ, the different versions are kept in a different scope. https://nodejs.org/api/modules.html https://nodejs.org/api/modules.html // Package A: import B from "B"; // A/node_modules/B/1.0.0 import C from "C"; // A/node_modules/C/1.0.0 // Package B: import D from "D"; // A/node_modules/B/1.0.0/node_modules/D/1.0.0 // Package C: import D from "D"; // A/node_modules/C/1.0.0/node_modules/D/2.0.0
- jasonwatkinspdx 11y ago
  I'm convinced this is the right approach: dependencies should be purely local to a module. The tooling should make it easy to detect you have a diamond, but let each module get what it needs. As generally everyone moves towards more continuous update and integration, at the end of the day it's all gonna be sha's anyhow.
  - wtetzner 11y ago
    But what do you do in a typed language? For example, suppose you have modules A and B, which depend on module C. Let's also say that module A has a function foo that return a value of type C.t, and module B has a function bar that accepts a value of type C.t. Now let's say that module A is depending on version 1.0.0 of module C, and module B is depending on version 1.5.2 of module C. Does B.bar(A.foo()) still work? Is type t the same in C version 1.0.0 and 1.5.2?
    - eknkc 11y ago
      Good point. But if B.bar accepts an interface that C.t implements, then everything should work just fine and that should be the way to go on public APIs.
      - wtetzner 11y ago
        But what if the interface is defined in C? The interface has to live somewhere, and wouldn't it be versioned too?

incepted 11y ago

No, it doesn't really make much sense. The diamond dependency problem only happens when two indirect dependencies are identical but on different versions and that these two versions are incompatible. That's two big if's. 99% of the time, it's not a problem. Notice first that this is a pretty rare problem since libraries tend to be backward compatible these days. And if you happen to come across a bad behaved one, you simply exclude the one you don't want from the graph, problem solved. This problem has been fixed in Maven for almost ten years now, the only reason why you would not implement this crucial feature in a version manager is laziness.

voidlogic 11y ago

Everyone seems to overlook this, but Go has had LiteIDE for a very long time and it keeps getting better.

dawkins 11y ago
I thought the same. I use it every day and its great. I dont understand why it doesn't get more traction.
- vox_mollis 11y ago
  Because visually, it's reminiscent of Eclipse 1.0. For better or worse, a sizable proportion of developers choose IDEs on the basis of looks.
  - Vendan 11y ago
    Heh, I choose IDEs based on performance, not looks, and liteide is blazing fast compared to anything else I've used.

AsyncAwait 11y ago

I'm not sure if the Intellij Go Plugin is going to be made into a separate IDE, but it's already available for their other IDEs as a plugin and they have a team improving it constantly. Notably, it also works with their open-source IDEA Community Edition. https://github.com/go-lang-plugin-org/go-lang-idea-plugin https://github.com/go-lang-plugin-org/go-lang-idea-plugin So far the best "Go IDE", I've used.