11 ms·

Russ Cox on Go dependency management

214 points by aberoham 8y ago

deleted 8y ago
[deleted]
ainar-g 8y ago
It's _Russ_ Cox, not Ross. Can the admins change the title?
- innagadadavida 8y ago
  Calling him out like this is pretty bad, why not make this about the Go dependency management instead of one individual?
  - coldtea 8y ago
    Because it's a long twitter of messages that he posted discussing Go's dependency. It's not a general discussion between various people, it's sorta like he wrote a blog post on the matter.
- dang 8y ago
  Fixed. Thanks!
hitekker 8y ago
One meta pattern I’ve noticed: If team members can’t talk straight with one another, the team can’t make good collective decisions. Mealy mouthed walls-of-texts are just as unhelpful as one sentence dismissals. Sooner or later, the silent miscommunication will manifest itself and impact the product directly. Needless to say, the health of a project relies upon the health of its politics.
- krageon 8y ago
  A lot of reasonably successful open-source projects (eg almost the whole npm namespace) is rife with some breed of toxicity and nevertheless does "fine" (for some value of fine - if you never look at it and ignore the security breaches) in terms of the technical merit.
  - rauhl 8y ago
    I don’t think it’s just open-source projects: a great deal of all projects end up rife with some toxicity or another, and normally they do alright in the end. Human beings are a problem: we have emotions, we have feelings, we have needs, we’re not hyper-rational automata. But we mostly muddle through alright. I don’t know if we can avoid this. What I think we can avoid are blow-ups, or at the least calm them back down again. We have to use our hearts, not just our heads, to say and mean, ‘yes, I acknowledge your feelings, which are valid.’ In the case of Go & dep, I think Sam Boyer feels let down because he’d been so proud of the idea that his tool dep would become the official tool. Rationally he acknowledged that it was the official experiment (maybe he even acknowledged that was an official experiment — I forget), but emotionally I believe that he thought that it’d become the official tool. To see it replaced is a colossal letdown. We can honour him and his investment of time & energy while at the same time acknowledging that the tool he created isn’t the right tool long-term. This is something the net is terrible for: it’d be far better to break the news, in person, over dinner or drinks. There’s no good way to do it, of course, but sharing physical space helps.
kevingadd 8y ago
A common theme in this thread is the relatively typical software development approach of 'well, I'll let them sort it out'. It's not lazy, and it sort of resembles delegating problems you can't handle to someone else - but the problem is if you do it enough on large scale projects you just end up with a huge stack of miscommunications, and a talented team that has put together a high quality product that completely fails to do what it needs to do. You can't successfully delegate without ensuring that everyone knows what their commitments are, and you need to pay attention on a regular basis to make sure that everyone still knows what the plan is. It's easy to end up in a position where you're making that mistake if you're a small team used to shipping smaller products or features. Suddenly you're on a bigger project, maybe a bunch of the developers work at another company, maybe you've got a few million users instead of a smaller set like a hundred thousand, and the same planning and team collaboration techniques you've used are subtly harmful and produce bad outcomes a year or two down the line. I observed the WebAssembly design process suffering from same of the same problems early on because it was a group of engineers from multiple companies who were generally used to just working in tiny silos on experiments or features of their own, not A Collaborative Effort For Millions Of Users, much like what Go has turned into now that it's scaled out and not just a prototype language made for Google internal use.
- rurban 8y ago
  For me the important point was that the implementer cannot delegate the community discussion to someone else. He needs to be part of it, and waste hours of important development time on bike shedding. Or just ignore it and apologize afterwards for rendering the bike shedding useless.
  - sgift 8y ago
    Yeah, it is bike shedding to discuss dependency management, because dependency management is an "insignificant or unimportant detail" of a programming language. /s
    - majewsky 8y ago
      You say "/s", but for some reason, many people still hold the sincere position that dependency management is something that can just be tacked on to a language as an afterthought.
kodablah 8y ago
Long live reasonable BDFLs. They may annoy the community here and there, but the cohesion and gatekeeping provided by a quality BDFL is hard to match with a community. It's very hard for a community to have collective foresight, but even when they do, often a gatekeeper emerges anyways during review. Of course, it requires thicker skin on both sides of the coin, but in hindsight the tech benefits often outweigh the politics. Sure, some communities (e.g. the Rust one) show that a sole person need not reside at the top. But that often comes with slow implementation/decision costs.
- lifthrasiir 8y ago
  > Of course, it requires thicker skin on both sides of the coin, but in hindsight the tech benefits often outweigh the politics. And this is exactly why the BDFL model is not sustainable in a longer term. I do agree that a consistent tie-breaker (a role of GvR, a stereotypical BDFL, was indeed much like this) is good to have, but a BDFL eventually becomes a venerable monarch whose everything become controversial enough to make one a political entity regardless of one's intention. As GvR shows this also puts a lot of burden to a single person.
- crtc 8y ago
  > But that often comes with slow implementation/decision costs. It also comes with feature bloat.
  - pcwalton 8y ago
    No, it doesn't. "Feature bloat" comes from wanting to appease everyone by taking the union of all proposals. That is ineffective community governance: it comes from allowing people to say yes but not no to features. Believe me, people love to say no to features in Rust.
- geodel 8y ago
  And even in Rust earlier module system written by someone else and was dumped wholesale without enough justification to author who finally left frustrated. Same with green threads which at one point they were hell bent on adding and then one fine day decided to dump. Also leading to ban someone in Rust community who was against green thread. I think what people do not want to talk about or do not get is unlike Oracle, Google, MS or Apple Mozilla can't just add few dozen highly paid people for language development. So this big community led approach is more of practical matter than some enlightened way to develop software.
deleted 8y ago
[deleted]
keypusher 8y ago
> Dep does not support using multiple major versions of a program in a single build. This alone is a complete showstopper. Go is meant for large-scale work, and in a large-scale program different parts will inevitably need different versions of some isolated dependency. This seems to be a major sticking point but I'm having trouble understanding why. I'm not familiar with any other package management system which supports simultaneous major versions in a single project. If two things require different versions of a package, those seem like separate projects to me. Is this to support dependencies which rely on the same package at different versions? If so, why not just call it a version conflict and fail until it's resolved?
- deleted 8y ago
  [deleted]
- MrBuddyCasino 8y ago
  > why not just call it a version conflict and fail until it's resolved Because its unnecessary and it doesn't scale. In Rust, versions become part of the symbol name, and the whole thing just becomes a non-issue. Go did the right thing here.
- majewsky 8y ago
  > If two things require different versions of a package, those seem like separate projects to me. You may be thinking of packages on the scale of frameworks. For example, if you have a Ruby on Rails app that mixes packages from Rails 3 and Rails 5, well sure, then you're in for some serious trouble. But a lot of packages are much more narrow in scope. Rust has a lot of small crates for basic data structures that have their APIs change betweeen versions. So you could end up with your logging package internally depending on some_data_structure=1.2, while your RabbitMQ client library internally depends on some_data_structure=2.3. As long as they do not expose some_data_structure in their API, you will never even notice (except maybe through increased binary size).
- derefr 8y ago
  > Is this to support dependencies which rely on the same package at different versions? Yes. > If two things require different versions of a package, those seem like separate projects to me Basically we’re talking about libs that have version conflicts, but also very slow release schedules, such that the version conflict isn’t going to be reconciled any time soon; or where one of the deps may even be effectively abandonware, but “works just fine” stand-alone so nobody’s going to update it. It just depends on extremely old versions of common shared deps. These are, effectively, irreconcilable version conflicts. The only thing to do, in most languages, is to fork one or the other dep to fix the conflict; or to split the project into microservices such that the conflicting components can live on opposite sides of a process memory boundary. Even these are sometimes impossible for office-political reasons. > I'm not familiar with any other package management system which supports simultaneous major versions in a single project. Node’s npm. Versions aren’t globally resolved; instead, each package effectively gets its own version resolution. Your root-level package gets a node_modules/ directory with your direct dependencies checked out into it; but your deps' deps are checked out into further node_modules/ directories that exist within their parent dep's checkout directory. It's somewhat as if each parent dep had vendored its deps, and had a vendor/ subdirectory; except that, in this case, the vendoring process is recursive. When a given package require()s a library by name, then, it’s actually importing its own npm-vendorified copy of the library from its own node_modules/ directory, rather than looking in your package's root node_modules/ directory. (In fact, there is no root level; you have "package isotropy", where a dep NPM checked out has the exact same NPM-populated subdirs as a project you git clone'd yourself.) Since these libs aren’t entered into a global namespace by require(), rather just have the module returned as a runtime object for the parent scope to bind to a local variable, there are no runtime conflicts between the separately-vendored versions, either. In a language without such runtime support, though, you would need your package-ecosystem tooling to reach into your downloaded lib and do some name-mangling of the exported symbols, and then either create a router symbol that resolves calls based on the lexical scope, or further mangle your deps' parents to call the dep by its mangled name rather than its original name. I think that’s what’s being proposed here, since Go has no such runtime support. (Effectively, in Go terms, you can simply think of it as all the references to git-http URL depspecs in import statements, being amended to qualify the repo name with the full git SHA, such that two different commits of the same project act as if they were two differently-named projects from Go's perspective.)
privateSFacct 8y ago
One issue I have is I can't really find the dep complaints about go's approach clearly. There was supposed to be a talk but I can't find the video. Yes - Russ should have been much clearer much earlier and not let dep get as far along as it did. But dep mentions the issues, particularly for small teams with go's new approach - I'd love to get a quick version of that.
- kaeshiwaza 8y ago
  It was a long series of "i will say you my concerns tomorrow" that just make stir up the sauce since month without any clear and obvious problem.
- Merovius 8y ago
  The closest to that is https://sdboyer.io/vgo/intro/ https://sdboyer.io/vgo/intro/ and https://sdboyer.io/vgo/failure-modes/ https://sdboyer.io/vgo/failure-modes/. IMO these posts are hard to digest and ultimately unconvincing. But YMMV.
ckastner 8y ago
Off-topic rant: I really hate how Twitter has popularized this devolved style of communicating by a flurry of fragments so much that we now have services like Thread reader (which I appreciate!) to undo that mess, and to make communication coherent again.
- nicpottier 8y ago
  Ya.. this is an especially funny case because I can't imagine Russ wrote this adhoc on Twitter, this looks like something very much prepared, then dribbled out, which makes me a bit sad for the world.
  - gant 8y ago
    Twitter has that feature where you can post multiple tweets in a chain. Honestly, why have a character limit at all at this point...
    - sigjuice 8y ago
      So it is accessible over SMS?
      - gant 8y ago
        Oh right, that's a thing (that was never available here in Germany).
      - greiskul 8y ago
        How many users actually use it over SMS though?
- swah 8y ago
  The great rants in Usenet and later Google Groups were awesome and searchable. I miss those.
- amdavidson 8y ago
  I really dislike reading the tweets in this bizarre thread reader. The page width is much to wide and the, line spacings are inappropriate for the size of the font, and that's not to mention the lack of contrast. Even worse, I could not find a single link back to Twitter, the canonical source. It's just as easy to read on the twitter website, it provides additional context, and allows for interaction. https://twitter.com/_rsc/status/1022588240501661696 https://twitter.com/_rsc/status/1022588240501661696
nicpottier 8y ago
I hope this apology and acknowledgment of the problems in the process appease Sam. By and large he has been the one to driving the last resistance to adoption of go modules and I can't help but think most of that is just investor's dilemma. At this point it would be great to no longer be talking about whether go modules should be the thing but working together towards making them work for everyone. (which I don't think is far off actually, Russ and co have done a good job post-reveal of addressing use cases that break with modules)
StevePerkins 8y ago
If you worked with Go quite a bit 3-4 years ago, but have been out of scene since then... what is the best resource(s) for getting back up to speed with its current state? I presume that the language spec itself hasn't change DRAMATICALLY over the past few years. But so much is unfamiliar in the project structure / build process area (e.g. "vendor directories", modules, dependency managament, etc). All the tours and tutorials focus on the language syntax and standard library. Looking for a refresher or intro to the "how do you actually get work done in the best manner?" plumbing aspects.
- swah 8y ago
  I'd say: assume nothing has changed. Take a look at the active projects in your area of interest. A few ones: https://echo.labstack.com https://echo.labstack.com https://github.com/go-kit/kit https://github.com/go-kit/kit Web glue frequently recommended around here: https://github.com/go-chi/chi https://github.com/go-chi/chi Github pages like https://github.com/avelino/awesome-go https://github.com/avelino/awesome-go have become a good starting point for finding interesting projects.
- carbocation 8y ago
  I use go as my daily driver... and I would also like to see this. Especially now that there is a blessed module system. (I suspect that since this is just in pre-release, we won’t get any battle-tested advice for a few months, though.)
- divan 8y ago
  You only need to pick up the whole `vendor/` thing (you can "vendor" deps by copying it manually if it works for you), get familiar with `dep` probably (as a lot of project use it nowadays by default), and that's basically it. Nothing that affects usual daily work has changed and that's what I insanely love in Go. Recently I had to pick up and work on project I did back in 2013, and it was so damn cool to see that it just works, compiles and there is almost zero problems with dependencies (the only one I had was due to removal of `code.google.com` and moving projects to github, so it was a matter of renaming import path and introducing vendoring).
- 8y ago
ThePhysicist 8y ago
I absolutely love writing programs in Go and at my current startup we're basing all our core backend infrastructure on it. One thing that I find extremely troubling about Go's dependency handling though (and which at least partly contributes to people's frustration I think) is the mixing of the concepts of a namespace and a resource identifier in Go's package names. It's absolutely reasonable that package names should be unique, making the package name double as a storage location is not always a good idea though (IMHO): For example, we recently switched from using gitlab.com to our own private Gitlab instance, which runs under a custom (non-publicly resolvable) domain. Since all our packages were named like "gitlab.com/.../..." (to make them "go get"-table) we would have to rename all of them to make them "go get"-table from the new location. In the end we solved it by defining a git URL rewriting rule that rewrites the old to the new URL when fetching the repository. This ensured that we could work primarily on our new Gitlab server while maintaining the possiblity to build packages on the gitlab.com server. While that works I think it would have been a better idea to keep package names and storage locations separate, or at least give users a way to override/modify the resolution of packages into locations using a more standardized approach: Such a mechanism could then allow for different name resolution strategies (like a package file, a list of standard repositories) while falling back to the standard way of resolving packages into URLs for "go get". Granted, this would add another layer of complexity, but as we already need a separate file for proper vendoring (to e.g. specify versions) it wouldn't hurt too much to also put (optional) information about storage locations into that file. Just my 2 cents, apart from this minor inconvenience I'm absolutely happy with Go and its ecosystem!
- aargh_aargh 8y ago
  Ah, it's a variation on the URL vs. URI vs. URN debate again.
- ithkuil 8y ago
  if you want to decouple the namespace from the repository location you can! https://golang.org/cmd/go/#hdr-Remote_import_paths https://golang.org/cmd/go/#hdr-Remote_import_paths (read until the end) or TL;DR: https://blog.bramp.net/post/2017/10/02/vanity-go-import-paths/ https://blog.bramp.net/post/2017/10/02/vanity-go-import-path...
shadowmint 8y ago
I think I’m more offended Russ said he was looking at cargo as ‘best of breed’ for inspiration.. but ended up ignoring it entirely to do his own thing. Just suck it up. You went off and did your own thing, and ignored everybody else, sticking pretty much to what you wanted to win the race in the first place (gopkg). Right, now thats out the way can we just get on with it and use modules? Decisions have been made, they’re here to stay. It doesn’t make everyone happy... but hey, we’re all massively relieved the mess has been sorted out, so thank you. Lets stop these deep soul searching what he said she said wastes of time and get get started using it~
- skybrian 8y ago
  It's possible to study another product and learn things from it while still deciding to do it your own way. Deciding to do things differently is not an insult. It's one way innovation happens.
- IshKebab 8y ago
  I think vgo's solution is better than Cargo's though.
jatins 8y ago
This is kinda unrelated. But I am _very_ new to Go. If one were to start a serious project with Go in 2018, what's the recommended way? Are people using Go modules? How to deal with 3rd party deps which dont have go.mod files? When I previously explored Go, having all code under GOPATH was recommended, but seems like things have changed with `vgo`.
- ben0x539 8y ago
  Modules are still in a beta pre-release kinda state, I think most people would stick to dep even for new projects right now.
  - kaeshiwaza 8y ago
    Dep has also issues, it's safer to continue to use Dep when you already use it but for new project it's recommended to switch now to go modules. Anyway, most of the time it doesn't change anythings to lock the version with dep or use minimal version selection.

214 points by aberoham 8y ago

deleted 8y ago

[deleted]

ainar-g 8y ago

It's _Russ_ Cox, not Ross. Can the admins change the title?

innagadadavida 8y ago
Calling him out like this is pretty bad, why not make this about the Go dependency management instead of one individual?
- coldtea 8y ago
  Because it's a long twitter of messages that he posted discussing Go's dependency. It's not a general discussion between various people, it's sorta like he wrote a blog post on the matter.
dang 8y ago
Fixed. Thanks!

hitekker 8y ago

One meta pattern I’ve noticed: If team members can’t talk straight with one another, the team can’t make good collective decisions. Mealy mouthed walls-of-texts are just as unhelpful as one sentence dismissals. Sooner or later, the silent miscommunication will manifest itself and impact the product directly. Needless to say, the health of a project relies upon the health of its politics.

krageon 8y ago
A lot of reasonably successful open-source projects (eg almost the whole npm namespace) is rife with some breed of toxicity and nevertheless does "fine" (for some value of fine - if you never look at it and ignore the security breaches) in terms of the technical merit.
- rauhl 8y ago
  I don’t think it’s just open-source projects: a great deal of all projects end up rife with some toxicity or another, and normally they do alright in the end. Human beings are a problem: we have emotions, we have feelings, we have needs, we’re not hyper-rational automata. But we mostly muddle through alright. I don’t know if we can avoid this. What I think we can avoid are blow-ups, or at the least calm them back down again. We have to use our hearts, not just our heads, to say and mean, ‘yes, I acknowledge your feelings, which are valid.’ In the case of Go & dep, I think Sam Boyer feels let down because he’d been so proud of the idea that his tool dep would become the official tool. Rationally he acknowledged that it was the official experiment (maybe he even acknowledged that was an official experiment — I forget), but emotionally I believe that he thought that it’d become the official tool. To see it replaced is a colossal letdown. We can honour him and his investment of time & energy while at the same time acknowledging that the tool he created isn’t the right tool long-term. This is something the net is terrible for: it’d be far better to break the news, in person, over dinner or drinks. There’s no good way to do it, of course, but sharing physical space helps.

kevingadd 8y ago

A common theme in this thread is the relatively typical software development approach of 'well, I'll let them sort it out'. It's not lazy, and it sort of resembles delegating problems you can't handle to someone else - but the problem is if you do it enough on large scale projects you just end up with a huge stack of miscommunications, and a talented team that has put together a high quality product that completely fails to do what it needs to do. You can't successfully delegate without ensuring that everyone knows what their commitments are, and you need to pay attention on a regular basis to make sure that everyone still knows what the plan is. It's easy to end up in a position where you're making that mistake if you're a small team used to shipping smaller products or features. Suddenly you're on a bigger project, maybe a bunch of the developers work at another company, maybe you've got a few million users instead of a smaller set like a hundred thousand, and the same planning and team collaboration techniques you've used are subtly harmful and produce bad outcomes a year or two down the line. I observed the WebAssembly design process suffering from same of the same problems early on because it was a group of engineers from multiple companies who were generally used to just working in tiny silos on experiments or features of their own, not A Collaborative Effort For Millions Of Users, much like what Go has turned into now that it's scaled out and not just a prototype language made for Google internal use.

rurban 8y ago
For me the important point was that the implementer cannot delegate the community discussion to someone else. He needs to be part of it, and waste hours of important development time on bike shedding. Or just ignore it and apologize afterwards for rendering the bike shedding useless.
- sgift 8y ago
  Yeah, it is bike shedding to discuss dependency management, because dependency management is an "insignificant or unimportant detail" of a programming language. /s
  - majewsky 8y ago
    You say "/s", but for some reason, many people still hold the sincere position that dependency management is something that can just be tacked on to a language as an afterthought.

kodablah 8y ago

Long live reasonable BDFLs. They may annoy the community here and there, but the cohesion and gatekeeping provided by a quality BDFL is hard to match with a community. It's very hard for a community to have collective foresight, but even when they do, often a gatekeeper emerges anyways during review. Of course, it requires thicker skin on both sides of the coin, but in hindsight the tech benefits often outweigh the politics. Sure, some communities (e.g. the Rust one) show that a sole person need not reside at the top. But that often comes with slow implementation/decision costs.

lifthrasiir 8y ago
> Of course, it requires thicker skin on both sides of the coin, but in hindsight the tech benefits often outweigh the politics. And this is exactly why the BDFL model is not sustainable in a longer term. I do agree that a consistent tie-breaker (a role of GvR, a stereotypical BDFL, was indeed much like this) is good to have, but a BDFL eventually becomes a venerable monarch whose everything become controversial enough to make one a political entity regardless of one's intention. As GvR shows this also puts a lot of burden to a single person.
crtc 8y ago
> But that often comes with slow implementation/decision costs. It also comes with feature bloat.
- pcwalton 8y ago
  No, it doesn't. "Feature bloat" comes from wanting to appease everyone by taking the union of all proposals. That is ineffective community governance: it comes from allowing people to say yes but not no to features. Believe me, people love to say no to features in Rust.
geodel 8y ago
And even in Rust earlier module system written by someone else and was dumped wholesale without enough justification to author who finally left frustrated. Same with green threads which at one point they were hell bent on adding and then one fine day decided to dump. Also leading to ban someone in Rust community who was against green thread. I think what people do not want to talk about or do not get is unlike Oracle, Google, MS or Apple Mozilla can't just add few dozen highly paid people for language development. So this big community led approach is more of practical matter than some enlightened way to develop software.

deleted 8y ago

[deleted]

keypusher 8y ago

> Dep does not support using multiple major versions of a program in a single build. This alone is a complete showstopper. Go is meant for large-scale work, and in a large-scale program different parts will inevitably need different versions of some isolated dependency. This seems to be a major sticking point but I'm having trouble understanding why. I'm not familiar with any other package management system which supports simultaneous major versions in a single project. If two things require different versions of a package, those seem like separate projects to me. Is this to support dependencies which rely on the same package at different versions? If so, why not just call it a version conflict and fail until it's resolved?

deleted 8y ago
[deleted]
MrBuddyCasino 8y ago
> why not just call it a version conflict and fail until it's resolved Because its unnecessary and it doesn't scale. In Rust, versions become part of the symbol name, and the whole thing just becomes a non-issue. Go did the right thing here.
majewsky 8y ago
> If two things require different versions of a package, those seem like separate projects to me. You may be thinking of packages on the scale of frameworks. For example, if you have a Ruby on Rails app that mixes packages from Rails 3 and Rails 5, well sure, then you're in for some serious trouble. But a lot of packages are much more narrow in scope. Rust has a lot of small crates for basic data structures that have their APIs change betweeen versions. So you could end up with your logging package internally depending on some_data_structure=1.2, while your RabbitMQ client library internally depends on some_data_structure=2.3. As long as they do not expose some_data_structure in their API, you will never even notice (except maybe through increased binary size).
derefr 8y ago
> Is this to support dependencies which rely on the same package at different versions? Yes. > If two things require different versions of a package, those seem like separate projects to me Basically we’re talking about libs that have version conflicts, but also very slow release schedules, such that the version conflict isn’t going to be reconciled any time soon; or where one of the deps may even be effectively abandonware, but “works just fine” stand-alone so nobody’s going to update it. It just depends on extremely old versions of common shared deps. These are, effectively, irreconcilable version conflicts. The only thing to do, in most languages, is to fork one or the other dep to fix the conflict; or to split the project into microservices such that the conflicting components can live on opposite sides of a process memory boundary. Even these are sometimes impossible for office-political reasons. > I'm not familiar with any other package management system which supports simultaneous major versions in a single project. Node’s npm. Versions aren’t globally resolved; instead, each package effectively gets its own version resolution. Your root-level package gets a node_modules/ directory with your direct dependencies checked out into it; but your deps' deps are checked out into further node_modules/ directories that exist within their parent dep's checkout directory. It's somewhat as if each parent dep had vendored its deps, and had a vendor/ subdirectory; except that, in this case, the vendoring process is recursive. When a given package require()s a library by name, then, it’s actually importing its own npm-vendorified copy of the library from its own node_modules/ directory, rather than looking in your package's root node_modules/ directory. (In fact, there is no root level; you have "package isotropy", where a dep NPM checked out has the exact same NPM-populated subdirs as a project you git clone'd yourself.) Since these libs aren’t entered into a global namespace by require(), rather just have the module returned as a runtime object for the parent scope to bind to a local variable, there are no runtime conflicts between the separately-vendored versions, either. In a language without such runtime support, though, you would need your package-ecosystem tooling to reach into your downloaded lib and do some name-mangling of the exported symbols, and then either create a router symbol that resolves calls based on the lexical scope, or further mangle your deps' parents to call the dep by its mangled name rather than its original name. I think that’s what’s being proposed here, since Go has no such runtime support. (Effectively, in Go terms, you can simply think of it as all the references to git-http URL depspecs in import statements, being amended to qualify the repo name with the full git SHA, such that two different commits of the same project act as if they were two differently-named projects from Go's perspective.)

privateSFacct 8y ago

One issue I have is I can't really find the dep complaints about go's approach clearly. There was supposed to be a talk but I can't find the video. Yes - Russ should have been much clearer much earlier and not let dep get as far along as it did. But dep mentions the issues, particularly for small teams with go's new approach - I'd love to get a quick version of that.

kaeshiwaza 8y ago
It was a long series of "i will say you my concerns tomorrow" that just make stir up the sauce since month without any clear and obvious problem.
Merovius 8y ago
The closest to that is https://sdboyer.io/vgo/intro/ https://sdboyer.io/vgo/intro/ and https://sdboyer.io/vgo/failure-modes/ https://sdboyer.io/vgo/failure-modes/. IMO these posts are hard to digest and ultimately unconvincing. But YMMV.

ckastner 8y ago

Off-topic rant: I really hate how Twitter has popularized this devolved style of communicating by a flurry of fragments so much that we now have services like Thread reader (which I appreciate!) to undo that mess, and to make communication coherent again.

nicpottier 8y ago
Ya.. this is an especially funny case because I can't imagine Russ wrote this adhoc on Twitter, this looks like something very much prepared, then dribbled out, which makes me a bit sad for the world.
- gant 8y ago
  Twitter has that feature where you can post multiple tweets in a chain. Honestly, why have a character limit at all at this point...
  - sigjuice 8y ago
    So it is accessible over SMS?
    - gant 8y ago
      Oh right, that's a thing (that was never available here in Germany).
    - greiskul 8y ago
      How many users actually use it over SMS though?
swah 8y ago
The great rants in Usenet and later Google Groups were awesome and searchable. I miss those.
amdavidson 8y ago
I really dislike reading the tweets in this bizarre thread reader. The page width is much to wide and the, line spacings are inappropriate for the size of the font, and that's not to mention the lack of contrast. Even worse, I could not find a single link back to Twitter, the canonical source. It's just as easy to read on the twitter website, it provides additional context, and allows for interaction. https://twitter.com/_rsc/status/1022588240501661696 https://twitter.com/_rsc/status/1022588240501661696

nicpottier 8y ago

I hope this apology and acknowledgment of the problems in the process appease Sam. By and large he has been the one to driving the last resistance to adoption of go modules and I can't help but think most of that is just investor's dilemma. At this point it would be great to no longer be talking about whether go modules should be the thing but working together towards making them work for everyone. (which I don't think is far off actually, Russ and co have done a good job post-reveal of addressing use cases that break with modules)

StevePerkins 8y ago

If you worked with Go quite a bit 3-4 years ago, but have been out of scene since then... what is the best resource(s) for getting back up to speed with its current state? I presume that the language spec itself hasn't change DRAMATICALLY over the past few years. But so much is unfamiliar in the project structure / build process area (e.g. "vendor directories", modules, dependency managament, etc). All the tours and tutorials focus on the language syntax and standard library. Looking for a refresher or intro to the "how do you actually get work done in the best manner?" plumbing aspects.

swah 8y ago
I'd say: assume nothing has changed. Take a look at the active projects in your area of interest. A few ones: https://echo.labstack.com https://echo.labstack.com https://github.com/go-kit/kit https://github.com/go-kit/kit Web glue frequently recommended around here: https://github.com/go-chi/chi https://github.com/go-chi/chi Github pages like https://github.com/avelino/awesome-go https://github.com/avelino/awesome-go have become a good starting point for finding interesting projects.
carbocation 8y ago
I use go as my daily driver... and I would also like to see this. Especially now that there is a blessed module system. (I suspect that since this is just in pre-release, we won’t get any battle-tested advice for a few months, though.)
divan 8y ago
You only need to pick up the whole `vendor/` thing (you can "vendor" deps by copying it manually if it works for you), get familiar with `dep` probably (as a lot of project use it nowadays by default), and that's basically it. Nothing that affects usual daily work has changed and that's what I insanely love in Go. Recently I had to pick up and work on project I did back in 2013, and it was so damn cool to see that it just works, compiles and there is almost zero problems with dependencies (the only one I had was due to removal of `code.google.com` and moving projects to github, so it was a matter of renaming import path and introducing vendoring).
8y ago

ThePhysicist 8y ago

I absolutely love writing programs in Go and at my current startup we're basing all our core backend infrastructure on it. One thing that I find extremely troubling about Go's dependency handling though (and which at least partly contributes to people's frustration I think) is the mixing of the concepts of a namespace and a resource identifier in Go's package names. It's absolutely reasonable that package names should be unique, making the package name double as a storage location is not always a good idea though (IMHO): For example, we recently switched from using gitlab.com to our own private Gitlab instance, which runs under a custom (non-publicly resolvable) domain. Since all our packages were named like "gitlab.com/.../..." (to make them "go get"-table) we would have to rename all of them to make them "go get"-table from the new location. In the end we solved it by defining a git URL rewriting rule that rewrites the old to the new URL when fetching the repository. This ensured that we could work primarily on our new Gitlab server while maintaining the possiblity to build packages on the gitlab.com server. While that works I think it would have been a better idea to keep package names and storage locations separate, or at least give users a way to override/modify the resolution of packages into locations using a more standardized approach: Such a mechanism could then allow for different name resolution strategies (like a package file, a list of standard repositories) while falling back to the standard way of resolving packages into URLs for "go get". Granted, this would add another layer of complexity, but as we already need a separate file for proper vendoring (to e.g. specify versions) it wouldn't hurt too much to also put (optional) information about storage locations into that file. Just my 2 cents, apart from this minor inconvenience I'm absolutely happy with Go and its ecosystem!

aargh_aargh 8y ago
Ah, it's a variation on the URL vs. URI vs. URN debate again.
ithkuil 8y ago
if you want to decouple the namespace from the repository location you can! https://golang.org/cmd/go/#hdr-Remote_import_paths https://golang.org/cmd/go/#hdr-Remote_import_paths (read until the end) or TL;DR: https://blog.bramp.net/post/2017/10/02/vanity-go-import-paths/ https://blog.bramp.net/post/2017/10/02/vanity-go-import-path...

shadowmint 8y ago

I think I’m more offended Russ said he was looking at cargo as ‘best of breed’ for inspiration.. but ended up ignoring it entirely to do his own thing. Just suck it up. You went off and did your own thing, and ignored everybody else, sticking pretty much to what you wanted to win the race in the first place (gopkg). Right, now thats out the way can we just get on with it and use modules? Decisions have been made, they’re here to stay. It doesn’t make everyone happy... but hey, we’re all massively relieved the mess has been sorted out, so thank you. Lets stop these deep soul searching what he said she said wastes of time and get get started using it~

skybrian 8y ago
It's possible to study another product and learn things from it while still deciding to do it your own way. Deciding to do things differently is not an insult. It's one way innovation happens.
IshKebab 8y ago
I think vgo's solution is better than Cargo's though.

jatins 8y ago

This is kinda unrelated. But I am _very_ new to Go. If one were to start a serious project with Go in 2018, what's the recommended way? Are people using Go modules? How to deal with 3rd party deps which dont have go.mod files? When I previously explored Go, having all code under GOPATH was recommended, but seems like things have changed with `vgo`.

ben0x539 8y ago
Modules are still in a beta pre-release kinda state, I think most people would stick to dep even for new projects right now.
- kaeshiwaza 8y ago
  Dep has also issues, it's safer to continue to use Dep when you already use it but for new project it's recommended to switch now to go modules. Anyway, most of the time it doesn't change anythings to lock the version with dep or use minimal version selection.