5 ms·

What’s the deal on package mangement? They did quite a few iterations here so I don’t follow. Don’t know about the Russ takeover either

pjmlp 4y ago
TL;DR version. Go package management was quite poor for many real use cases outside Google, so the community railed around many competing proposals, and when it appeared there was a clear winner, the Go team came out with their own solution instead. This after kind of supporting the ongoing community efforts.
- xena 4y ago
  Even more fun, Google doesn't even use Go modules internally! They use their own homegrown build system abomination that requires teams of people to maintain.
  - deleted 4y ago
    [deleted]
  - jen20 4y ago
    Is this something other than Bazel (or the progenitor thereof)? If so, calling it an abomination compared to the unfinished go build system is rather strong.
    - throwaway894345 4y ago
      Google's internal build system is called Blaze, and (as I understand it) it supports tons of use cases that the Go tool doesn't: * Distributing build jobs across clusters * Targeting any arbitrary platform, architecture combination * Supporting multiple languages and toolchains * Arbitrary build-time script execution (e.g., code generation)
      - mseepgood 4y ago
        Bazel is the open-source variant of Blaze: https://bazel.build/about/faq#what%E2%80%99s_up_with_the_word_%E2%80%9Cblaze%E2%80%9D_in_the_codebase https://bazel.build/about/faq#what%E2%80%99s_up_with_the_wor...
        jen20 4y ago
        Indeed - Bazel is a complete build system. “go build” is really not, and go modules are typically used for dependency management in Bazel for Go anyway.
  - throwaway894345 4y ago
    Pretty sure virtually every big tech company does the same thing, because they have to integrate a bunch of languages, target any arbitrary platform/architecture tuple, and support use cases like code generation and so on. There aren't any good solutions to this problem that don't require teams of people to maintain--the best in class open source solutions seem to be Bazel and Nix and these are beyond my skill to manage even for pretty basic use cases. Further, Google's internal build system predates Go modules by a decade or more; why would they pivot to Go modules?
    - saurik 4y ago
      Everyone ends up having to do this kind of stuff, and so the best tools are ones that take this in stride and make it easier to pull off even if it is at the expense of more pain for trivial project; and like, if you are going to try to build a baby version of the tool for beginners, you probably aren't the right person to even figure out what is required of that tool if you wouldn't normally use it.
      - throwaway894345 4y ago
        > Everyone ends up having to do this kind of stuff, and so the best tools are ones that take this in stride and make it easier to pull off even if it is at the expense of more pain for trivial project I disagree. Lots of projects will never need support for multiple languages, code generation, targeting a vast array of platforms and architectures, etc and for those that do, there are workarounds that are a lot less painful than Bazel or Nix. I can go a looong ways with `go build` and some CI jobs before the pain of Bazel or Nix pay off. Basically, I don't think the "trivial projects vs everyone else" is a very good taxonomy, but rather it's "'approaching FAANG scale' vs everyone else". If you're in the "approaching FAANG scale" group, then yeah, Bazel probably makes sense. For everyone else language-specific tools cobbled together with CI really is the least bad solution (and by a pretty big margin in my experience).
- Thaxll 4y ago
  And yet go mod is far superior to anything that was created before, having used dep which was honestly very slow and buggy, go mod works just fine and is one of the best pkg mgmt right now ( all languages included ). Package managment should not be created and maintained outside of the core team, maybe the process in which they decided the solution was not the best but the end result speaks for itself, go mod is very good.
  - leetrout 4y ago
    This is 100% correct. All your points are right. But the Go team ignored it for _years_ until dep got a lot of traction and we had this big community meeting about it and live on the call Russ said he was going to work on minimal version selection and build a tool. Sam Boyer had given a big talk about package management at GopherCon and on that call his only response when people asked what was going on was "no comment right now" or something like that. It was almost like it was his first time hearing of it. It just seemed to be handled very poorly. All parties involved are active on HN AFAIK so they can correct me. I am not trying to dig all the past back up just for funsies... maybe something will happen here and the Go core team will realize they need to work with Drew and sort it out. But history points to them doing what they want to do irrespective of the community.
- cp9 4y ago
  frankly, go mod is a much better solution than dep ever was
- alphazard 4y ago
  This is actually one of the reasons I trust the Go project is in good hands. The Go leadership has smart engineers, with good taste, willing to say no. The Go module implementation and version selection mechanism are better than what the community had converged on. Go has demonstrated that its governance model is capable of making better decisions than a pure democracy. This is not the case for most open source projects.
  - lanstin 4y ago
    And they, unlike almost any open source group outside of Linux and maybe libc, take compatibility seriously. Volunteers break so much stuff so wantonly, and it just isn’t necessary technically. I left a job and came a few years later, and the Go code all still worked under 1.19.
  - deleted 4y ago
    [deleted]
- Beltalowda 4y ago
  > when it appeared there was a clear winner Dep was very slow, tried to be very clever often resulting in it "cleverly" doing the wrong thing, and was generally a pain to deal with. At $dayjob we migrated to dep and then we had serious discussions if we should move back to glide (we didn't, as vgo was on the horizon by then). There were some plans to fix some of this IIRC, but they never really went beyond "plans". It's always difficult if people spend a lot of time on a particular solution and then it turns out that's actually not what's desired, or if someone else thinks of an even better solution. All option suck here: chucking out people's work sucks (especially in open source volunteer-effort context), but accepting a bad solution "because people spent time on it" and then being stuck with that for years or decades to come sucks even more. The communication could certainly have been better; a sort of "community committee" was given a repo on github.com/golang/dep and told "good luck". In hindsight the Go team should have watched development more closely and provided feedback sooner so that the direction could be adjusted. Actually, the entire setup probably wasn't a good idea in hindsight.
- throwaway894345 4y ago
  I was kind of miffed by this early on, but the Go team's solution has been surprisingly effective and it seems to have stood the test of time. Package management is hard, and Go modules are one of the best solutions in the entire programming ecosystem, which is to say they mostly do what you would expect with relatively little debugging. Some superficial stuff (e.g., error messages) could be improved and there are probably some niche use cases that aren't as intuitive as they could be (maybe something like nested modules?), but the overall approach generally works well.

pjmlp 4y ago

TL;DR version. Go package management was quite poor for many real use cases outside Google, so the community railed around many competing proposals, and when it appeared there was a clear winner, the Go team came out with their own solution instead. This after kind of supporting the ongoing community efforts.

xena 4y ago
Even more fun, Google doesn't even use Go modules internally! They use their own homegrown build system abomination that requires teams of people to maintain.
- deleted 4y ago
  [deleted]
- jen20 4y ago
  Is this something other than Bazel (or the progenitor thereof)? If so, calling it an abomination compared to the unfinished go build system is rather strong.
  - throwaway894345 4y ago
    Google's internal build system is called Blaze, and (as I understand it) it supports tons of use cases that the Go tool doesn't: * Distributing build jobs across clusters * Targeting any arbitrary platform, architecture combination * Supporting multiple languages and toolchains * Arbitrary build-time script execution (e.g., code generation)
    - mseepgood 4y ago
      Bazel is the open-source variant of Blaze: https://bazel.build/about/faq#what%E2%80%99s_up_with_the_word_%E2%80%9Cblaze%E2%80%9D_in_the_codebase https://bazel.build/about/faq#what%E2%80%99s_up_with_the_wor...
      - jen20 4y ago
        Indeed - Bazel is a complete build system. “go build” is really not, and go modules are typically used for dependency management in Bazel for Go anyway.
- throwaway894345 4y ago
  Pretty sure virtually every big tech company does the same thing, because they have to integrate a bunch of languages, target any arbitrary platform/architecture tuple, and support use cases like code generation and so on. There aren't any good solutions to this problem that don't require teams of people to maintain--the best in class open source solutions seem to be Bazel and Nix and these are beyond my skill to manage even for pretty basic use cases. Further, Google's internal build system predates Go modules by a decade or more; why would they pivot to Go modules?
  - saurik 4y ago
    Everyone ends up having to do this kind of stuff, and so the best tools are ones that take this in stride and make it easier to pull off even if it is at the expense of more pain for trivial project; and like, if you are going to try to build a baby version of the tool for beginners, you probably aren't the right person to even figure out what is required of that tool if you wouldn't normally use it.
    - throwaway894345 4y ago
      > Everyone ends up having to do this kind of stuff, and so the best tools are ones that take this in stride and make it easier to pull off even if it is at the expense of more pain for trivial project I disagree. Lots of projects will never need support for multiple languages, code generation, targeting a vast array of platforms and architectures, etc and for those that do, there are workarounds that are a lot less painful than Bazel or Nix. I can go a looong ways with `go build` and some CI jobs before the pain of Bazel or Nix pay off. Basically, I don't think the "trivial projects vs everyone else" is a very good taxonomy, but rather it's "'approaching FAANG scale' vs everyone else". If you're in the "approaching FAANG scale" group, then yeah, Bazel probably makes sense. For everyone else language-specific tools cobbled together with CI really is the least bad solution (and by a pretty big margin in my experience).
Thaxll 4y ago
And yet go mod is far superior to anything that was created before, having used dep which was honestly very slow and buggy, go mod works just fine and is one of the best pkg mgmt right now ( all languages included ). Package managment should not be created and maintained outside of the core team, maybe the process in which they decided the solution was not the best but the end result speaks for itself, go mod is very good.
- leetrout 4y ago
  This is 100% correct. All your points are right. But the Go team ignored it for _years_ until dep got a lot of traction and we had this big community meeting about it and live on the call Russ said he was going to work on minimal version selection and build a tool. Sam Boyer had given a big talk about package management at GopherCon and on that call his only response when people asked what was going on was "no comment right now" or something like that. It was almost like it was his first time hearing of it. It just seemed to be handled very poorly. All parties involved are active on HN AFAIK so they can correct me. I am not trying to dig all the past back up just for funsies... maybe something will happen here and the Go core team will realize they need to work with Drew and sort it out. But history points to them doing what they want to do irrespective of the community.
cp9 4y ago
frankly, go mod is a much better solution than dep ever was
alphazard 4y ago
This is actually one of the reasons I trust the Go project is in good hands. The Go leadership has smart engineers, with good taste, willing to say no. The Go module implementation and version selection mechanism are better than what the community had converged on. Go has demonstrated that its governance model is capable of making better decisions than a pure democracy. This is not the case for most open source projects.
- lanstin 4y ago
  And they, unlike almost any open source group outside of Linux and maybe libc, take compatibility seriously. Volunteers break so much stuff so wantonly, and it just isn’t necessary technically. I left a job and came a few years later, and the Go code all still worked under 1.19.
- deleted 4y ago
  [deleted]
Beltalowda 4y ago
> when it appeared there was a clear winner Dep was very slow, tried to be very clever often resulting in it "cleverly" doing the wrong thing, and was generally a pain to deal with. At $dayjob we migrated to dep and then we had serious discussions if we should move back to glide (we didn't, as vgo was on the horizon by then). There were some plans to fix some of this IIRC, but they never really went beyond "plans". It's always difficult if people spend a lot of time on a particular solution and then it turns out that's actually not what's desired, or if someone else thinks of an even better solution. All option suck here: chucking out people's work sucks (especially in open source volunteer-effort context), but accepting a bad solution "because people spent time on it" and then being stuck with that for years or decades to come sucks even more. The communication could certainly have been better; a sort of "community committee" was given a repo on github.com/golang/dep and told "good luck". In hindsight the Go team should have watched development more closely and provided feedback sooner so that the direction could be adjusted. Actually, the entire setup probably wasn't a good idea in hindsight.
throwaway894345 4y ago
I was kind of miffed by this early on, but the Go team's solution has been surprisingly effective and it seems to have stood the test of time. Package management is hard, and Go modules are one of the best solutions in the entire programming ecosystem, which is to say they mostly do what you would expect with relatively little debugging. Some superficial stuff (e.g., error messages) could be improved and there are probably some niche use cases that aren't as intuitive as they could be (maybe something like nested modules?), but the overall approach generally works well.