6 ms·

Now everyone can know the joys and horrors of Bundler! ;) I kid. This announcement is very exciting for people tinkering with Rust like myself. I really like

Now everyone can know the joys and horrors of Bundler! ;) I kid. This announcement is very exciting for people tinkering with Rust like myself. I really like Go, but the lack of a solid package management solution that I like using is a downer. Before you rage-comment: I know how Go packages work. I know you think they're better than anything that's ever been invented. I know there are solutions out there for some portions of what I want. But nothing has been created that really works for me (yet). So, to see this development in another one of my "tinker with it but not build a ton of production quality code just yet" languages is exciting!

chimeracoder 13y ago
> I really like Go, but the lack of a solid package management solution that I like using is a downer. Before you rage-comment: I know how Go packages work. I know you think they're better than anything that's ever been invented. I know there are solutions out there for some portions of what I want. But nothing has been created that really works for me (yet). What is it that you do want in a Go "package manager", then? You say that you know how Go packages work, so I'm assuming it's not one of the typical misunderstandings of newcomers from the Python/Ruby/Node.js world (all 1.x code is forwards-compatible with no modifications, static linking means makes specifying versions of packages less relevant, the package/filesystem layout parallel means that vendoring project-specific forks of packages is relatively straightforward once you know how, etc.) I put "package manager" in scare quotation marks because one of the design goals of Go is essentially to render most of the functionality of such package managers irrelevant. I say this as someone who has been writing Go for work on a daily basis for over a year and half: while Go's package system isn't perfect, it's pretty damn good, and I haven't felt the need for a package manager at all ever since I learned the way things worked.
- pcwalton 13y ago
  Well, Rust also uses static linking by default and crates are also laid out based on the filesystem [1]. But we still feel the need for a package manager... [1]: You can nest subdirectories within the package directory if you wish, to provide more fine-grained namespacing, but all source files belonging to a Rust crate must be descendants of one directory (in contrast to Go, in which the source files must be children of one directory).
- frio 13y ago
  Version pinning. I don't want a package manager, but I do want to be able to say import ( "github.com/frio/mycoollibrary@1.2" ) ... or some such (where the tag above could be a plain old hash too). Vendoring in every library I use is not a great solution.
  - burntsushi 13y ago
    You can get something like this already with gopkg.in: http://godoc.org/gopkg.in/v1/docs#hdr-Supported_URLs http://godoc.org/gopkg.in/v1/docs#hdr-Supported_URLs (it uses tags or branch names). It should work with any GitHub repo that's using tags/branches with names conforming to semantic versioning.
    - bjeanes 13y ago
      I hadn't seen this but it seems like a really elegant solution compared to the alternatives. I look forward to trying it.
    - grey-area 13y ago
      That looks nice. Does it work with import statements though or is it only for go get?
      - burntsushi 13y ago
        It has to work with import statements---that's the whole point. :-)
        grey-area 13y ago
        Thanks for the link. Just tried it out, nice, and I love the use of semver. So you can do this to import a specific version of say yaml import "gopkg.in/v1/yaml" or for an arbitrary url on github https://gopkg.in/username/v1.0.0/pkg Weird that versions are before the pkg name though, that doesn't seem right... Here's hoping this sort of versioning scheme makes it into the go toolchain eventually. The only downside of this solution is it redirects only to github and guesses urls, I quite like that the go toolchain is indifferent as to where the sources are hosted. This would be very easy to add to the toolchain though; all they'd need to do is recognise imports like this (personally I prefer versions at the end of current urls, not in the middle): import "bitbucket.org/user/pkg/v1.1.2" in go get and download that tag to that path, no other changes in the toolchain would be required, and those who ignore versions could continue to do so, while people who wanted them use the version specific imports. I think the objection of the go team was that this doesn't solve any of the complex dependency issues that plague package managers, but it'd be nice just to be able to specify which version is expected for future users and have predictable builds when sharing code outside one org.
- kyrra 13y ago
  There are some people trying to remedy this issue. While the Golang designers don't feel the need to include it in the language, others may resolve this. Id recommend reading the goals doc, as they seem to have a good starting point. http://www.goinggo.net/2014/01/go-package-management-for-2014.html?m=1 http://www.goinggo.net/2014/01/go-package-management-for-201...
- stormbrew 13y ago
  > static linking means makes specifying versions of packages less relevant I'd really like to know why you think this is the case. I don't specify versions of dependencies because of dynamic linking, I specify versions of dependencies (wherever possible) so I know I can still build the thing later. An old binary of a go program is no use to me as a developer.
  - MikeAmelung 13y ago
    I think he's referring to the fact that you won't try to run some executable that depends on Ruby 1.9.3 and SomeGem 2.9, but you're on your rbenv that is Ruby 2.0 and SomeGem 3.0.1. I'm not a Ruby developer, but I know this scenario has caused me considerable trouble just trying to use things built with Ruby.
  - eikenberry 13y ago
    > I specify versions of dependencies (wherever possible) so I know I can still build the thing later. The only way to ensure this is to vendor the code or clone it into your own repo. Either way you control the dependency versions explicitly.
    - grey-area 13y ago
      This doesn't work when you have more complex dependencies, because they in turn have dependencies, which may clash. You have no way of knowing what pkg c version packages a and b expect when you include a and b.
      - eikenberry 13y ago
        If the dependencies clash, it doesn't matter what system you use. There isn't a package manager good enough to fix the issue of having conflicting dependencies. Vendoring or cloning (with explicit upstream merging) is again your best bet for dealing with this as it the only way you have complete control of the dependencies involved. If you are saying something like having a package X in your vendor lib that depends on packages Y and Z which you don't have vendored... then you haven't done it right. You vendor all dependencies.
- grey-area 13y ago
  I put "package manager" in scare quotation marks because one of the design goals of Go is essentially to render most of the functionality of such package managers irrelevant. Golang does have a de-facto package manager in go get (with its own conventions on file system layout etc), it just doesn't handle versioning. The Go packaging system is pretty good, but it isn't perfect, and I'm not convinced vendoring is a viable solution if you're going to be sharing code and collaborating on several open source projects - it's a recipe for lots of incompatible versions with security and other bugs unfixed floating around in different projects, and a lot of confusion from newcomers about the required version of a library because it is not specified anywhere. The current approach works perfectly if you are working in-house, willing to do your own dependency resolution, and are happy to vendor your own dependencies - it's completely reasonable within one company or org to do this. However it fails in more collaborative or open-source environments like libraries shared on github depending on other libraries shared on github. I do think they need version pinning for that (in fact they have version pinning for the language in go get, just not for individual packages), it wouldn't be a large change. For example, in the recent Stripe CTF, people ended up downloading and building different packages locally than those available on the server builds, because there was no clarity on which version was included. That was kind of annoying. It could be solved by vendoring, but then you have other issues in the long-term with relying on other people to keep their pkg dependencies up to date or you can be including two conflicting versions of a third pkg. They've taken an interesting approach in trying to eschew versions (which avoids some problems with dependency resolution), but I'm not sure it is sustainable long-term. Didn't they also try to make golang itself version free initially?
  - numbsafari 13y ago
    Check out this posting by James Graves regarding using your SCM to manage the versions[1]. There are two main downsides with this approach: (1) you have to import projects not managed in git and make a few changes (could be automated very easily) and (2) you have to deal with git submodules. (1) can be dealt with fairly easily with a little bit of perl or python (or... go). (2) is largely the approach he chose to take. You could also import those submodules directly into your tree and do subtree merges [2]. The "go doesn't have package management" meme is propagated by people unwilling to think outside the box of "what new tool and infrastructure can I force myself to depend on". At least in the world of go you are re-using existing code distribution systems and infrastructure that are, by their very nature, already versioned and distributed (git, mercurial) and a simple set of tools (go get) and you have some kind of compile-time checking about your dependencies. [1] http://goo.gl/vcbOmO http://goo.gl/vcbOmO [2] http://git-scm.com/book/ch6-7.html http://git-scm.com/book/ch6-7.html
stormbrew 13y ago
Handling packages in Go is always something that stops me from going any further when I look at it. Every time I realize I won't be able to properly specify dependencies with versions or pin them (like what Bundler does) it drives me away screaming. It's only the last few years dependency hell has been resolved in other environments (like Ruby and Node, to a lesser and sometimes weirder extent Python), I don't really want to go back in time any more.
cryptolect 13y ago
Coming from Ruby, when I last looked at Go about 5 months ago, I was dismayed at the package management state. My specific concern was around version pinning. I'll be excited to try out Cargo when it launches.
Argorak 13y ago
Have you tried GPM? https://github.com/pote/gpm https://github.com/pote/gpm
Matrixik 13y ago
Did you check: https://code.google.com/p/go-wiki/wiki/PackageManagementTools https://code.google.com/p/go-wiki/wiki/PackageManagementTool... ?

chimeracoder 13y ago

> I really like Go, but the lack of a solid package management solution that I like using is a downer. Before you rage-comment: I know how Go packages work. I know you think they're better than anything that's ever been invented. I know there are solutions out there for some portions of what I want. But nothing has been created that really works for me (yet). What is it that you do want in a Go "package manager", then? You say that you know how Go packages work, so I'm assuming it's not one of the typical misunderstandings of newcomers from the Python/Ruby/Node.js world (all 1.x code is forwards-compatible with no modifications, static linking means makes specifying versions of packages less relevant, the package/filesystem layout parallel means that vendoring project-specific forks of packages is relatively straightforward once you know how, etc.) I put "package manager" in scare quotation marks because one of the design goals of Go is essentially to render most of the functionality of such package managers irrelevant. I say this as someone who has been writing Go for work on a daily basis for over a year and half: while Go's package system isn't perfect, it's pretty damn good, and I haven't felt the need for a package manager at all ever since I learned the way things worked.

pcwalton 13y ago
Well, Rust also uses static linking by default and crates are also laid out based on the filesystem [1]. But we still feel the need for a package manager... [1]: You can nest subdirectories within the package directory if you wish, to provide more fine-grained namespacing, but all source files belonging to a Rust crate must be descendants of one directory (in contrast to Go, in which the source files must be children of one directory).
frio 13y ago
Version pinning. I don't want a package manager, but I do want to be able to say import ( "github.com/frio/mycoollibrary@1.2" ) ... or some such (where the tag above could be a plain old hash too). Vendoring in every library I use is not a great solution.
- burntsushi 13y ago
  You can get something like this already with gopkg.in: http://godoc.org/gopkg.in/v1/docs#hdr-Supported_URLs http://godoc.org/gopkg.in/v1/docs#hdr-Supported_URLs (it uses tags or branch names). It should work with any GitHub repo that's using tags/branches with names conforming to semantic versioning.
  - bjeanes 13y ago
    I hadn't seen this but it seems like a really elegant solution compared to the alternatives. I look forward to trying it.
  - grey-area 13y ago
    That looks nice. Does it work with import statements though or is it only for go get?
    - burntsushi 13y ago
      It has to work with import statements---that's the whole point. :-)
      - grey-area 13y ago
        Thanks for the link. Just tried it out, nice, and I love the use of semver. So you can do this to import a specific version of say yaml import "gopkg.in/v1/yaml" or for an arbitrary url on github https://gopkg.in/username/v1.0.0/pkg Weird that versions are before the pkg name though, that doesn't seem right... Here's hoping this sort of versioning scheme makes it into the go toolchain eventually. The only downside of this solution is it redirects only to github and guesses urls, I quite like that the go toolchain is indifferent as to where the sources are hosted. This would be very easy to add to the toolchain though; all they'd need to do is recognise imports like this (personally I prefer versions at the end of current urls, not in the middle): import "bitbucket.org/user/pkg/v1.1.2" in go get and download that tag to that path, no other changes in the toolchain would be required, and those who ignore versions could continue to do so, while people who wanted them use the version specific imports. I think the objection of the go team was that this doesn't solve any of the complex dependency issues that plague package managers, but it'd be nice just to be able to specify which version is expected for future users and have predictable builds when sharing code outside one org.
kyrra 13y ago
There are some people trying to remedy this issue. While the Golang designers don't feel the need to include it in the language, others may resolve this. Id recommend reading the goals doc, as they seem to have a good starting point. http://www.goinggo.net/2014/01/go-package-management-for-2014.html?m=1 http://www.goinggo.net/2014/01/go-package-management-for-201...
stormbrew 13y ago
> static linking means makes specifying versions of packages less relevant I'd really like to know why you think this is the case. I don't specify versions of dependencies because of dynamic linking, I specify versions of dependencies (wherever possible) so I know I can still build the thing later. An old binary of a go program is no use to me as a developer.
- MikeAmelung 13y ago
  I think he's referring to the fact that you won't try to run some executable that depends on Ruby 1.9.3 and SomeGem 2.9, but you're on your rbenv that is Ruby 2.0 and SomeGem 3.0.1. I'm not a Ruby developer, but I know this scenario has caused me considerable trouble just trying to use things built with Ruby.
- eikenberry 13y ago
  > I specify versions of dependencies (wherever possible) so I know I can still build the thing later. The only way to ensure this is to vendor the code or clone it into your own repo. Either way you control the dependency versions explicitly.
  - grey-area 13y ago
    This doesn't work when you have more complex dependencies, because they in turn have dependencies, which may clash. You have no way of knowing what pkg c version packages a and b expect when you include a and b.
    - eikenberry 13y ago
      If the dependencies clash, it doesn't matter what system you use. There isn't a package manager good enough to fix the issue of having conflicting dependencies. Vendoring or cloning (with explicit upstream merging) is again your best bet for dealing with this as it the only way you have complete control of the dependencies involved. If you are saying something like having a package X in your vendor lib that depends on packages Y and Z which you don't have vendored... then you haven't done it right. You vendor all dependencies.
grey-area 13y ago
I put "package manager" in scare quotation marks because one of the design goals of Go is essentially to render most of the functionality of such package managers irrelevant. Golang does have a de-facto package manager in go get (with its own conventions on file system layout etc), it just doesn't handle versioning. The Go packaging system is pretty good, but it isn't perfect, and I'm not convinced vendoring is a viable solution if you're going to be sharing code and collaborating on several open source projects - it's a recipe for lots of incompatible versions with security and other bugs unfixed floating around in different projects, and a lot of confusion from newcomers about the required version of a library because it is not specified anywhere. The current approach works perfectly if you are working in-house, willing to do your own dependency resolution, and are happy to vendor your own dependencies - it's completely reasonable within one company or org to do this. However it fails in more collaborative or open-source environments like libraries shared on github depending on other libraries shared on github. I do think they need version pinning for that (in fact they have version pinning for the language in go get, just not for individual packages), it wouldn't be a large change. For example, in the recent Stripe CTF, people ended up downloading and building different packages locally than those available on the server builds, because there was no clarity on which version was included. That was kind of annoying. It could be solved by vendoring, but then you have other issues in the long-term with relying on other people to keep their pkg dependencies up to date or you can be including two conflicting versions of a third pkg. They've taken an interesting approach in trying to eschew versions (which avoids some problems with dependency resolution), but I'm not sure it is sustainable long-term. Didn't they also try to make golang itself version free initially?
- numbsafari 13y ago
  Check out this posting by James Graves regarding using your SCM to manage the versions[1]. There are two main downsides with this approach: (1) you have to import projects not managed in git and make a few changes (could be automated very easily) and (2) you have to deal with git submodules. (1) can be dealt with fairly easily with a little bit of perl or python (or... go). (2) is largely the approach he chose to take. You could also import those submodules directly into your tree and do subtree merges [2]. The "go doesn't have package management" meme is propagated by people unwilling to think outside the box of "what new tool and infrastructure can I force myself to depend on". At least in the world of go you are re-using existing code distribution systems and infrastructure that are, by their very nature, already versioned and distributed (git, mercurial) and a simple set of tools (go get) and you have some kind of compile-time checking about your dependencies. [1] http://goo.gl/vcbOmO http://goo.gl/vcbOmO [2] http://git-scm.com/book/ch6-7.html http://git-scm.com/book/ch6-7.html

stormbrew 13y ago

Handling packages in Go is always something that stops me from going any further when I look at it. Every time I realize I won't be able to properly specify dependencies with versions or pin them (like what Bundler does) it drives me away screaming. It's only the last few years dependency hell has been resolved in other environments (like Ruby and Node, to a lesser and sometimes weirder extent Python), I don't really want to go back in time any more.

cryptolect 13y ago

Coming from Ruby, when I last looked at Go about 5 months ago, I was dismayed at the package management state. My specific concern was around version pinning. I'll be excited to try out Cargo when it launches.

Argorak 13y ago

Have you tried GPM? https://github.com/pote/gpm https://github.com/pote/gpm

Matrixik 13y ago

Did you check: https://code.google.com/p/go-wiki/wiki/PackageManagementTools https://code.google.com/p/go-wiki/wiki/PackageManagementTool... ?