9 ms·

The go vendoring stuff and GOPATH sounds like a complete nightmare to me. Also the fact that imports are full git URLS sounds totally crazy. Is `dep` fixing

The go vendoring stuff and GOPATH sounds like a complete nightmare to me. Also the fact that imports are full git URLS sounds totally crazy. Is `dep` fixing all that stuff? Last time I checked out go a few years ago, I didn't continue because I found the story of actually installing dependencies and keeping track of versions very confusing. You couldn't even `go get` a specific git tag or something. I'm happy that there seems to be some movement. But now we have godep, govendor, glide, dep. and every project uses one or the other. Are they all compatible? Or is it a minefield?

kbar13 9y ago
i believe dep and glide allow you to pin to specific versions. i actually really like that imports are URLs, makes it really obvious where packages come from, and it's a better system imo than a centralized package manager like pypi / rubygem where names need to be unique.
- saghm 9y ago
  > it's a better system imo than a centralized package manager like pypi / rubygem where names need to be unique I'll have to disagree there; while having namespaced packages gives some advantages, having a centralized repository gives you things like proper versioning, which is worth all the terrible package names in the world to me.
  - carussell 9y ago
    A central repository doesn't automatically give you versioning, and URLs as package IDs doesn't take it away. These are orthogonal issues. It just happens that Go has chosen an approach for now that results in a lot of pain.
    - saghm 9y ago
      I don't think they're completely orthogonal; I think it's much easier to enforce versioning with a central repository than URLs as package IDs. It's not a coincidence that the most common centralized package managers (e.g. Ruby, Python, NodeJS, Rust, Linux distro package managers, and brew) all enforce versioning and things like Go and Vim packaging don't.
      - weberc2 9y ago
        Go and vim don't have "distributed package managers"; they just punt on package management altogether. The next step up from that is a centralized package management system, and if you care that much you'll probably build in some notion of versions (even if they're nondeterministic or otherwise broken, as in the case of pip, npm, and many Linux package managers). Building a distributed package manager is quite a lot more work.
saghm 9y ago
> The go vendoring stuff and GOPATH sounds like a complete nightmare to me. Also the fact that imports are full git URLS sounds totally crazy. It is. > Is `dep` fixing all that stuff? Not from what I can tell; it lets you pin to git tags and branches, but the only versioning you'll get beyond that is if the maintainer of your dependency is gracious enough to use tags. It certainly isn't changing the way import paths work. Vendoring is still part of the workflow for projects I've seen using dep. I've heard there's work being done to get rid of the GOPATH nonsense, but from what I can tell people have been saying that for a while, so I'm not optimistic that it'll land any time soon.
- stubish 9y ago
  'dep' does have plans to address this, but yes, it is not going to happen soon. 'dep' seems great for Noddy projects where all the code you are working on is in a single package, and fails (like most of the vendoring tools) when you need to work on multiple packages. It is has unfortunately been focused on vendoring at the package level rather than at the project level, and has many assumptions buried deeply into the code. The recommendation for larger projects is the 'vg' tool, which wraps 'dep' with some GOPATH tricks to help manage your dependencies at the project level.
deleted 9y ago
[deleted]
weberc2 9y ago
GOPATH isn't a real problem; it's just a search directory like PATH or PYTHONPATH or CLASSPATH or NODEPATH, and it has a default value that makes it painless. Vendoring can be truly painful, but `dep` aims to fix that. Imports are not git URLs or indeed any kind of URLs; they're just directory paths into your GOPATH which the `go get` program can use if the directory path resembles the URL of a git (or hg/bzr/svn) repo. Don't think of `go get` as a package manager; think of it as a convenience utility to play around with a package before locking down the version via vendoring. Go's package management has a lot to improve on, but many of the common complaints are misinformed. > I'm happy that there seems to be some movement. But now we have godep, govendor, glide, dep. and every project uses one or the other. Are they all compatible? Or is it a minefield? They're not all compatible, but it's not really a problem because (last I checked) the prevailing philosophy was "libraries should not vendor--only binaries" which is to say "libraries shouldn't specify versions for their dependencies; the downstream binary projects should figure out what versions of each library should be used to build the binary". This means dependency tool compatibility isn't a problem because the dependency tooling punts on the dependency-resolution problem altogether, which seems like an even bigger problem. Remember that this is only painful if you're trying for deterministic builds, and in the absolute worst case, you forego Go's build tooling and use something like Nix. This is absolutely not a reason to change language.
- closeparen 9y ago
  The existence of the GOPATH environment variable is not a problem. The fact that the toolchain is violently opinionated about where you locate your working copies, and goes out of its way to fight you if you try to fake it with symlinks, is. The GOPATH issue will be solved when I can clone a project to anywhere in $HOME and build it without issue.
  - imsofuture 9y ago
    This is how I felt when I started using go, but I was unable to rationalize why beyond 'it's different'. What's the real difference between `cd ~/dev` and `cd $MYGO` (~/dev/go/src/github.com/myusername)?
imsofuture 9y ago
Go vendoring gets such a bad rap undeservedly. In practice it's the simplest and most straightforward dependency system (since 1.4/5 added vendoring) I've used. Step 1: Check out code that you want to use. That's actually the only step and you can do it by hand, or with one of a multitude of tools. All the tools differ slightly, and any metadata or manifests or things like that they do aren't compatible, but the code they put in your repository is, enabling anyone to clone a repository, and instantly build the exact version of all your dependencies. Imports also aren't full git URLs, they can be as a convenience that tells you where to get a package. You can't `go get` a specific tag, but that's fine, because `go get` isn't a package or dependency manager, it's just a convenience to grab code to your system, not to freeze a version as a dependency into your project. Frankly, the amount of people that put their nose in the air at go is just fine with me, just makes it more of a 'secret productivity tool' I guess. Edit: not to say it wouldn't have been nice if `dep` had been the one blessed path at version 1. Still, I have extremely little to complain about with the language and tooling since 1.4/1.5.
- alain_gilbert 9y ago
  The fact that the full url is used to import packages make it impossible to fork a repository. Once you forked it, you have to change all the imports across the repo, and then it's kinda either very hard to make Pull Requests, or pull the new commits from the original repo. This is one thing I really dislike about go dependencies. If you have a solution for this, let me know, I would be very interested to know more about it. But overall, I love go. I use it all the time anyway.
  - imsofuture 9y ago
    You don't have to change the package name just because it's forked. I threw together an example of doing this with `dep`. I forked Fatih's color package, and am using it as `github.com/fatih/color` no problem: https://github.com/sofuture/colortest https://github.com/sofuture/colortest (forked dep here: https://github.com/sofuture/color https://github.com/sofuture/color)
    - 9y ago
hasenj 9y ago
What's wrong with vendoring? You put the package you want inside the vendors/ directory. I think it's much better than a package manager that tries to download some version that satisfies some version specification in a file that declares dependencies.
- pjmlp 9y ago
  If the new version has moved from Git to new suppa-duppa source control, you will need to update all import statements after replacing it on the vendors directory. With proper package names via a package manager, usually the name stays the same.
- stubish 9y ago
  Managing vendoring yourself is impractical once you end up with more than a handful of dependencies, or once one of your dependencies pulls in more than a handful of transient dependencies. A larger project will have hundreds of dependencies and transient dependencies, and if you just pull them into your vendor directory and forget about them then you have a problem waiting to happen.
  - hasenj 9y ago
    I find the opposite to be true. If you casually add dependencies without understanding the dependency tree, you have a lot of problems waiting to happen. I prefer to be conservative about adding dependencies, and to always manually include them into the project.
    - stubish 9y ago
      You can be conservative about adding dependencies and still end up with hundreds. Database driver, database utilities, gRPC framework, monitoring & metrics, CLI & configuration tools, /x/ packages such as crypto or net, logging, linters, libraries to talk to other moving parts like Rabbit or Vault, and then all the transitive dependencies like yaml, toml, websockets, and the various 'standard' helpers the authors of your dependencies use... all for a microservice and administrative tool... and one part of the larger project.

kbar13 9y ago

i believe dep and glide allow you to pin to specific versions. i actually really like that imports are URLs, makes it really obvious where packages come from, and it's a better system imo than a centralized package manager like pypi / rubygem where names need to be unique.

saghm 9y ago
> it's a better system imo than a centralized package manager like pypi / rubygem where names need to be unique I'll have to disagree there; while having namespaced packages gives some advantages, having a centralized repository gives you things like proper versioning, which is worth all the terrible package names in the world to me.
- carussell 9y ago
  A central repository doesn't automatically give you versioning, and URLs as package IDs doesn't take it away. These are orthogonal issues. It just happens that Go has chosen an approach for now that results in a lot of pain.
  - saghm 9y ago
    I don't think they're completely orthogonal; I think it's much easier to enforce versioning with a central repository than URLs as package IDs. It's not a coincidence that the most common centralized package managers (e.g. Ruby, Python, NodeJS, Rust, Linux distro package managers, and brew) all enforce versioning and things like Go and Vim packaging don't.
    - weberc2 9y ago
      Go and vim don't have "distributed package managers"; they just punt on package management altogether. The next step up from that is a centralized package management system, and if you care that much you'll probably build in some notion of versions (even if they're nondeterministic or otherwise broken, as in the case of pip, npm, and many Linux package managers). Building a distributed package manager is quite a lot more work.

saghm 9y ago

> The go vendoring stuff and GOPATH sounds like a complete nightmare to me. Also the fact that imports are full git URLS sounds totally crazy. It is. > Is `dep` fixing all that stuff? Not from what I can tell; it lets you pin to git tags and branches, but the only versioning you'll get beyond that is if the maintainer of your dependency is gracious enough to use tags. It certainly isn't changing the way import paths work. Vendoring is still part of the workflow for projects I've seen using dep. I've heard there's work being done to get rid of the GOPATH nonsense, but from what I can tell people have been saying that for a while, so I'm not optimistic that it'll land any time soon.

stubish 9y ago
'dep' does have plans to address this, but yes, it is not going to happen soon. 'dep' seems great for Noddy projects where all the code you are working on is in a single package, and fails (like most of the vendoring tools) when you need to work on multiple packages. It is has unfortunately been focused on vendoring at the package level rather than at the project level, and has many assumptions buried deeply into the code. The recommendation for larger projects is the 'vg' tool, which wraps 'dep' with some GOPATH tricks to help manage your dependencies at the project level.

deleted 9y ago

[deleted]

weberc2 9y ago

GOPATH isn't a real problem; it's just a search directory like PATH or PYTHONPATH or CLASSPATH or NODEPATH, and it has a default value that makes it painless. Vendoring can be truly painful, but `dep` aims to fix that. Imports are not git URLs or indeed any kind of URLs; they're just directory paths into your GOPATH which the `go get` program can use if the directory path resembles the URL of a git (or hg/bzr/svn) repo. Don't think of `go get` as a package manager; think of it as a convenience utility to play around with a package before locking down the version via vendoring. Go's package management has a lot to improve on, but many of the common complaints are misinformed. > I'm happy that there seems to be some movement. But now we have godep, govendor, glide, dep. and every project uses one or the other. Are they all compatible? Or is it a minefield? They're not all compatible, but it's not really a problem because (last I checked) the prevailing philosophy was "libraries should not vendor--only binaries" which is to say "libraries shouldn't specify versions for their dependencies; the downstream binary projects should figure out what versions of each library should be used to build the binary". This means dependency tool compatibility isn't a problem because the dependency tooling punts on the dependency-resolution problem altogether, which seems like an even bigger problem. Remember that this is only painful if you're trying for deterministic builds, and in the absolute worst case, you forego Go's build tooling and use something like Nix. This is absolutely not a reason to change language.

closeparen 9y ago
The existence of the GOPATH environment variable is not a problem. The fact that the toolchain is violently opinionated about where you locate your working copies, and goes out of its way to fight you if you try to fake it with symlinks, is. The GOPATH issue will be solved when I can clone a project to anywhere in $HOME and build it without issue.
- imsofuture 9y ago
  This is how I felt when I started using go, but I was unable to rationalize why beyond 'it's different'. What's the real difference between `cd ~/dev` and `cd $MYGO` (~/dev/go/src/github.com/myusername)?

imsofuture 9y ago

Go vendoring gets such a bad rap undeservedly. In practice it's the simplest and most straightforward dependency system (since 1.4/5 added vendoring) I've used. Step 1: Check out code that you want to use. That's actually the only step and you can do it by hand, or with one of a multitude of tools. All the tools differ slightly, and any metadata or manifests or things like that they do aren't compatible, but the code they put in your repository is, enabling anyone to clone a repository, and instantly build the exact version of all your dependencies. Imports also aren't full git URLs, they can be as a convenience that tells you where to get a package. You can't `go get` a specific tag, but that's fine, because `go get` isn't a package or dependency manager, it's just a convenience to grab code to your system, not to freeze a version as a dependency into your project. Frankly, the amount of people that put their nose in the air at go is just fine with me, just makes it more of a 'secret productivity tool' I guess. Edit: not to say it wouldn't have been nice if `dep` had been the one blessed path at version 1. Still, I have extremely little to complain about with the language and tooling since 1.4/1.5.

alain_gilbert 9y ago
The fact that the full url is used to import packages make it impossible to fork a repository. Once you forked it, you have to change all the imports across the repo, and then it's kinda either very hard to make Pull Requests, or pull the new commits from the original repo. This is one thing I really dislike about go dependencies. If you have a solution for this, let me know, I would be very interested to know more about it. But overall, I love go. I use it all the time anyway.
- imsofuture 9y ago
  You don't have to change the package name just because it's forked. I threw together an example of doing this with `dep`. I forked Fatih's color package, and am using it as `github.com/fatih/color` no problem: https://github.com/sofuture/colortest https://github.com/sofuture/colortest (forked dep here: https://github.com/sofuture/color https://github.com/sofuture/color)
  - 9y ago

hasenj 9y ago

What's wrong with vendoring? You put the package you want inside the vendors/ directory. I think it's much better than a package manager that tries to download some version that satisfies some version specification in a file that declares dependencies.

pjmlp 9y ago
If the new version has moved from Git to new suppa-duppa source control, you will need to update all import statements after replacing it on the vendors directory. With proper package names via a package manager, usually the name stays the same.
stubish 9y ago
Managing vendoring yourself is impractical once you end up with more than a handful of dependencies, or once one of your dependencies pulls in more than a handful of transient dependencies. A larger project will have hundreds of dependencies and transient dependencies, and if you just pull them into your vendor directory and forget about them then you have a problem waiting to happen.
- hasenj 9y ago
  I find the opposite to be true. If you casually add dependencies without understanding the dependency tree, you have a lot of problems waiting to happen. I prefer to be conservative about adding dependencies, and to always manually include them into the project.
  - stubish 9y ago
    You can be conservative about adding dependencies and still end up with hundreds. Database driver, database utilities, gRPC framework, monitoring & metrics, CLI & configuration tools, /x/ packages such as crypto or net, logging, linters, libraries to talk to other moving parts like Rabbit or Vault, and then all the transitive dependencies like yaml, toml, websockets, and the various 'standard' helpers the authors of your dependencies use... all for a microservice and administrative tool... and one part of the larger project.