6 ms·

Go has a perfectly good package manager that works with sources hosted on GitHub and other sites -- there isn't any centralized place for people to publish sour

Go has a perfectly good package manager that works with sources hosted on GitHub and other sites -- there isn't any centralized place for people to publish sources, unlike the other package managers you mentioned. Go's package manager also came years after the language became widely used, and it is now very widely adopted according to the most recent survey[0]. I think C++ could have a good, unified package management story. It would just require the major stakeholders to all care enough to make it happen, which seems to be the missing piece here. [0]: https://blog.golang.org/survey2020-results#TOC_8 https://blog.golang.org/survey2020-results#TOC_8.

jasode 6y ago
>Go's package manager also came years after the language became widely used, and it is now very widely adopted according to the most recent survey[0]. Are you talking about "pkg.go.dev" and the "go get" command? Isn't there some path dependence in the history of events that's not comparable to C++? Consider: - Go language: created by Google Inc - "go get" syntax for package download designed and created by Google Inc - "pkg.go.dev" funded by Google Inc and highlighted on "golang.org" website that's also run by Google Inc. There is no business entity or institution in the C++ world that's analogous to Google's influence for Go + golang.org + "go get" + pkg.go.dev. >It would just require the major stakeholders to all _care_ enough to make it happen, But it's easier to care if there was an influential C++ behemoth that captured everyone's mindshare to move the entire ecosystem forward. C++ has no such "industry leader" that dictates (or heavily influences) technical direction from the top down.
- coder543 6y ago
  > Are you talking about "pkg.go.dev" and the "go get" command? No. I'm not talking about either of those. Your whole comment is, unfortunately, irrelevant. pkg.go.dev is not a package repo. It's just a place for documentation to be rendered. It renders documentation from third party hosted code, such as on GitHub or elsewhere. "go get" predates Go Modules, which is the current package management system. The whole original design of "go get" was to simply download code from somewhere on the internet, and place it in the right spot of the $GOPATH. This has nothing to do with a proper versioned package manager like Go Modules. AFAIK, "go get" was also never really designed for Google's internal use cases. They use a monorepo that was perfectly content with $GOPATH, and all their code was developed in the monorepo to begin with. There was nothing for them to "go get", except for the rare outside dependency that they were embedding into their monorepo, I would imagine. I've never worked for Google, these are just things I hear about. Go Modules was also not designed for Google. It was designed for the community, based on findings from community developed package managers for Go. Google has no real use for it — again, they use a monorepo. Nowadays, "go get" can be used with Go modules, but in practice, it feels like it almost never is. Maybe someone would use that command to upgrade an existing dependency, instead of editing the `go.mod` file to change the version there? So, your comment just shows that you haven't researched this enough. Yes, Go Modules was still guided by Googlers, who were even more in control of the language direction back then than they are now. Yes, change always causes some drama. But, I'm not really here to explain the history of Go package management... I'm just saying that C++ could have a nice, distributed package management system, it would just require the major stakeholders to all care and work together on it. The ISO C++ language committee is a finite number of people. They are the major stakeholders, as far as the language direction is concerned. If they didn't have the power to enact major language changes, we wouldn't be here talking about C++20. The stakeholders for Go were able to develop a package manager that is distributed (an idea compatible with how all C++ code is scattered across the web these days), and that achieved broad adoption, and this was some years after the language went into wide use. It’s an extremely relevant analogue for C++ to study, if the committee members wanted a package manager badly enough. > But it's easier to care if there was an influential C++ behemoth that captured everyone's mindshare to move the entire ecosystem forward. C++ has no such "industry leader" that dictates (or heavily influences) technical direction from the top down. You edited this in while I was replying, but I agree entirely. Getting the committee to agree to a package management solution would be much more difficult than having a single behemoth guide the decision. Does that mean it is impossible and therefore no one could do it? Everyone here talks like it is impossible, but it doesn't really seem to be.
  - jasode 6y ago
    >Yes, Go Modules was still guided by Googlers, who were even more in control of the language direction back then than they are now. Yes, change always causes some drama. But, I'm not really here to explain the history of Go package management... >I'm just saying that C++ could have a nice, distributed package management system, it would just require the major stakeholders to all care and work together on it. The ISO C++ language committee is a finite number of people. They are the major stakeholders, as far as the language direction is concerned. The ISO C++ committee can't learn from the history of Go modules community acceptance because they don't have the same power as Google. You seem to misunderstand what the C++ committee _is_. Yes, they have representatives from Microsoft/Google/Apple/Intel but the org is designed to review proposals from submitted papers. They are more like an ongoing academic conference rather than a devops team that runs websites. We seem to be discussing 2 different abstractions of making a "package manager". With your emphasis on Modules, you seem to be only focusing on the tool. To repeat my gp comment, I'm also focusing on the canonical package repository (or index, or discovery engine). Consider the following sentence from https://proxy.golang.org https://proxy.golang.org: >The Go team is providing the following services run by Google: a module mirror for accelerating Go module downloads, an index for discovering new modules, and a global go.sum database for authenticating module content. >As of Go 1.13, the go command by default downloads and authenticates modules using the Go module mirror and Go checksum database. You misunderstood my cite of "pkg.go.dev" run by Google Inc but this is the part of your survey that I was referring to: >The package discovery site pkg.go.dev is new to the list this year and was a top resource for 32% of respondents. Respondents who use pkg.go.dev are more likely to agree they are able to quickly find Go packages / libraries they need: 91% for pkg.go.dev users vs. 82% for everyone else. The ISO C++ committee is not set up to implement a new website to make the above Go-specific paragraphs be a similar reality for C++ with a search & replace "s/Go/C++/g". Think about _who_ funds and provides paid people to actually run the "proxy.golang.org". It's Google Inc. The C++ committee doesn't have an equivalent situation. Yes, the C++ committee can receive a proposal for new language syntax such as "std::unique" and after some back & forth commentary and debate, they say "approved" and then it's up to each C++ compiler vendor to then go and independently implement it on their own timeline. In contrast, if someone proposes "C++ should have a package manager", exactly _who_ will implement and maintain the canonical repo mirror? This is not independent lines of work that GCC, Clang, Microsoft, and Intel can do on their own. Even if we hypothetically extend the website "isocpp.org" to actually start hosting the canonical C++ repos instead of just blog posts about syntax proposals, _who_ is paying for it? Again, there is no single entity like Joyent/Mozilla/GoogleInc that raises their hand and says, "We'll set it up". I suppose we could imagine that the major players like MS+Google+Apple all contribute to a shared fund to pay for the repo mirror -- and the salaries for devops to remove malicious uploads -- but notice no other major language package manager Javascript/Rust/Go had to do it that way. So we have that friction of coordinating multiple corporations. Even if that website was set up, many existing C++ library writers (that existed for decades before a C++ package manager) wouldn't bother uploading their code to it. So that's another friction. E.g. Conan is supposedly the current winner of C++ package manager mindshare and ffmpeg is not on it. I think the disagreement is rooted in how we compare ISO C++ committee vs Google Inc. To me, releasing a C++20 language specification* does not say anything about implementing a canonical repo* so that a command line tool magically works the way people expect. EDIT to reply: >You can have a package manager without having a discovery tool or a central repo. This means your conversation is focusing on the tool which isn't the abstraction I'm emphasizing. >Package discovery tools are not very relevant to the discussion. It's relevant if the particular person wondering "why C++ doesn't have a package manager?!?" uses a mental of model of how npm and cargo work. They don't have to know if it's github vs gitlab vs somewhere else. The tool just works without thinking about the location. That's what a canonical repo as a default convention for the client tool provides.
dgellow 6y ago
Go has a small dedicated team that develops and designs the language. They take some input from the broader community but are still the one who decides how things evolve. They decided at some point that go modules was the way to go and everybody followed, because they are the authority who decides how Go evolves. C++ does not have an equivalent, it's completely decentralized which results in more messy situation. As a result you have an open market where different people try to build different tools and approaches for their own problems, then try to get others to use them (similar to what Go had before go modules, we had lot of package managers to chose from at the time). Instead of a top down decision it's a negotiation between the various actors. But the last thing we need is for the C++ standards committee to standardize a package manager. That would take forever to do, would result in a messy tool that tries to compromise with all the actors in some ways, make it very hard and slow to evolve over time and would likely result in a lot of pain, etc.
- coder543 6y ago
  > C++ does not have an equivalent, it's completely decentralized which results in more messy situation. That is the role of the ISO C++ committee, is it not? They are the major stakeholders. They would just have to care enough. They cared enough to release C++20, didn't they? It's not like they never get anything done, which seems to be the implication a lot of people make in this discussion. > Instead of a top down decision it's a negotiation between the various actors. My understanding is that the various committee members represent the disparate interests of the broader C++ community. I agree it would be very much like a negotiation. That doesn't mean that it can't be done. This whole thread is discussing things that have been done by the C++ committee: C++20. > But the last thing we need is for the C++ standards committee to standardize a package manager. That would take forever to do, would result in a messy tool that tries to compromise with all the actors in some ways, make it very hard and slow to evolve over time and would likely result in a lot of pain, etc. You just summarized my feelings about C++ in general. I would much rather people use Rust or Go or any number of other languages instead of C++, depending on project needs. Such opinions are rarely taken well in threads like this, though, so... I've been trying to be optimistic and point out that C++ could get package management. If the C++ committee process works well, then the package manager should also end up turning out well. I'll leave the reader to decide how well they think the long term direction and guidance of the C++ standard has been going and apply that to their feelings of a hypothetical future package manager.
humanrebar 6y ago
Do the Go sources on GitHub use waf, make, CMake, bazel, or something entirely bespoke? Or is a common build system assumed?
- coder543 6y ago
  Go just uses the Go build system, which is common to all Go projects, so... it assumes a common build system. Some people do occasionally use Bazel or other build systems on top of the Go build system for complicated monorepos.
  - humanrebar 6y ago
    Right. C++ has a chicken-and-the-egg problem in that neither its build not packaging ecosystem have even de facto standards. GitHub URLs don't solve either problem.
    - coder543 6y ago
      I think everyone agrees that a common build system is a necessary step if any of this is going to work. Thinking about how much or how little would be required beyond a common build system in order to get a working package management system is still a valid thing to do.
      - humanrebar 6y ago
        I expect it is necessary to keep build configuration portable with respect to packaging and environment configuration. In other words, I expect downloading from URLs in Makefiles and CMakeLists to be a local maximum. Keeping a parallel set of instructions or metadata that includes specific URLs and such might work, though. As long as you can skip all that when a system package, filesystem path, or git submodule is more appropriate.

jasode 6y ago

>Go's package manager also came years after the language became widely used, and it is now very widely adopted according to the most recent survey[0]. Are you talking about "pkg.go.dev" and the "go get" command? Isn't there some path dependence in the history of events that's not comparable to C++? Consider: - Go language: created by Google Inc - "go get" syntax for package download designed and created by Google Inc - "pkg.go.dev" funded by Google Inc and highlighted on "golang.org" website that's also run by Google Inc. There is no business entity or institution in the C++ world that's analogous to Google's influence for Go + golang.org + "go get" + pkg.go.dev. >It would just require the major stakeholders to all _care_ enough to make it happen, But it's easier to care if there was an influential C++ behemoth that captured everyone's mindshare to move the entire ecosystem forward. C++ has no such "industry leader" that dictates (or heavily influences) technical direction from the top down.

coder543 6y ago
> Are you talking about "pkg.go.dev" and the "go get" command? No. I'm not talking about either of those. Your whole comment is, unfortunately, irrelevant. pkg.go.dev is not a package repo. It's just a place for documentation to be rendered. It renders documentation from third party hosted code, such as on GitHub or elsewhere. "go get" predates Go Modules, which is the current package management system. The whole original design of "go get" was to simply download code from somewhere on the internet, and place it in the right spot of the $GOPATH. This has nothing to do with a proper versioned package manager like Go Modules. AFAIK, "go get" was also never really designed for Google's internal use cases. They use a monorepo that was perfectly content with $GOPATH, and all their code was developed in the monorepo to begin with. There was nothing for them to "go get", except for the rare outside dependency that they were embedding into their monorepo, I would imagine. I've never worked for Google, these are just things I hear about. Go Modules was also not designed for Google. It was designed for the community, based on findings from community developed package managers for Go. Google has no real use for it — again, they use a monorepo. Nowadays, "go get" can be used with Go modules, but in practice, it feels like it almost never is. Maybe someone would use that command to upgrade an existing dependency, instead of editing the `go.mod` file to change the version there? So, your comment just shows that you haven't researched this enough. Yes, Go Modules was still guided by Googlers, who were even more in control of the language direction back then than they are now. Yes, change always causes some drama. But, I'm not really here to explain the history of Go package management... I'm just saying that C++ could have a nice, distributed package management system, it would just require the major stakeholders to all care and work together on it. The ISO C++ language committee is a finite number of people. They are the major stakeholders, as far as the language direction is concerned. If they didn't have the power to enact major language changes, we wouldn't be here talking about C++20. The stakeholders for Go were able to develop a package manager that is distributed (an idea compatible with how all C++ code is scattered across the web these days), and that achieved broad adoption, and this was some years after the language went into wide use. It’s an extremely relevant analogue for C++ to study, if the committee members wanted a package manager badly enough. > But it's easier to care if there was an influential C++ behemoth that captured everyone's mindshare to move the entire ecosystem forward. C++ has no such "industry leader" that dictates (or heavily influences) technical direction from the top down. You edited this in while I was replying, but I agree entirely. Getting the committee to agree to a package management solution would be much more difficult than having a single behemoth guide the decision. Does that mean it is impossible and therefore no one could do it? Everyone here talks like it is impossible, but it doesn't really seem to be.
- jasode 6y ago
  >Yes, Go Modules was still guided by Googlers, who were even more in control of the language direction back then than they are now. Yes, change always causes some drama. But, I'm not really here to explain the history of Go package management... >I'm just saying that C++ could have a nice, distributed package management system, it would just require the major stakeholders to all care and work together on it. The ISO C++ language committee is a finite number of people. They are the major stakeholders, as far as the language direction is concerned. The ISO C++ committee can't learn from the history of Go modules community acceptance because they don't have the same power as Google. You seem to misunderstand what the C++ committee _is_. Yes, they have representatives from Microsoft/Google/Apple/Intel but the org is designed to review proposals from submitted papers. They are more like an ongoing academic conference rather than a devops team that runs websites. We seem to be discussing 2 different abstractions of making a "package manager". With your emphasis on Modules, you seem to be only focusing on the tool. To repeat my gp comment, I'm also focusing on the canonical package repository (or index, or discovery engine). Consider the following sentence from https://proxy.golang.org https://proxy.golang.org: >The Go team is providing the following services run by Google: a module mirror for accelerating Go module downloads, an index for discovering new modules, and a global go.sum database for authenticating module content. >As of Go 1.13, the go command by default downloads and authenticates modules using the Go module mirror and Go checksum database. You misunderstood my cite of "pkg.go.dev" run by Google Inc but this is the part of your survey that I was referring to: >The package discovery site pkg.go.dev is new to the list this year and was a top resource for 32% of respondents. Respondents who use pkg.go.dev are more likely to agree they are able to quickly find Go packages / libraries they need: 91% for pkg.go.dev users vs. 82% for everyone else. The ISO C++ committee is not set up to implement a new website to make the above Go-specific paragraphs be a similar reality for C++ with a search & replace "s/Go/C++/g". Think about _who_ funds and provides paid people to actually run the "proxy.golang.org". It's Google Inc. The C++ committee doesn't have an equivalent situation. Yes, the C++ committee can receive a proposal for new language syntax such as "std::unique" and after some back & forth commentary and debate, they say "approved" and then it's up to each C++ compiler vendor to then go and independently implement it on their own timeline. In contrast, if someone proposes "C++ should have a package manager", exactly _who_ will implement and maintain the canonical repo mirror? This is not independent lines of work that GCC, Clang, Microsoft, and Intel can do on their own. Even if we hypothetically extend the website "isocpp.org" to actually start hosting the canonical C++ repos instead of just blog posts about syntax proposals, _who_ is paying for it? Again, there is no single entity like Joyent/Mozilla/GoogleInc that raises their hand and says, "We'll set it up". I suppose we could imagine that the major players like MS+Google+Apple all contribute to a shared fund to pay for the repo mirror -- and the salaries for devops to remove malicious uploads -- but notice no other major language package manager Javascript/Rust/Go had to do it that way. So we have that friction of coordinating multiple corporations. Even if that website was set up, many existing C++ library writers (that existed for decades before a C++ package manager) wouldn't bother uploading their code to it. So that's another friction. E.g. Conan is supposedly the current winner of C++ package manager mindshare and ffmpeg is not on it. I think the disagreement is rooted in how we compare ISO C++ committee vs Google Inc. To me, releasing a C++20 language specification* does not say anything about implementing a canonical repo* so that a command line tool magically works the way people expect. EDIT to reply: >You can have a package manager without having a discovery tool or a central repo. This means your conversation is focusing on the tool which isn't the abstraction I'm emphasizing. >Package discovery tools are not very relevant to the discussion. It's relevant if the particular person wondering "why C++ doesn't have a package manager?!?" uses a mental of model of how npm and cargo work. They don't have to know if it's github vs gitlab vs somewhere else. The tool just works without thinking about the location. That's what a canonical repo as a default convention for the client tool provides.

dgellow 6y ago

Go has a small dedicated team that develops and designs the language. They take some input from the broader community but are still the one who decides how things evolve. They decided at some point that go modules was the way to go and everybody followed, because they are the authority who decides how Go evolves. C++ does not have an equivalent, it's completely decentralized which results in more messy situation. As a result you have an open market where different people try to build different tools and approaches for their own problems, then try to get others to use them (similar to what Go had before go modules, we had lot of package managers to chose from at the time). Instead of a top down decision it's a negotiation between the various actors. But the last thing we need is for the C++ standards committee to standardize a package manager. That would take forever to do, would result in a messy tool that tries to compromise with all the actors in some ways, make it very hard and slow to evolve over time and would likely result in a lot of pain, etc.

coder543 6y ago
> C++ does not have an equivalent, it's completely decentralized which results in more messy situation. That is the role of the ISO C++ committee, is it not? They are the major stakeholders. They would just have to care enough. They cared enough to release C++20, didn't they? It's not like they never get anything done, which seems to be the implication a lot of people make in this discussion. > Instead of a top down decision it's a negotiation between the various actors. My understanding is that the various committee members represent the disparate interests of the broader C++ community. I agree it would be very much like a negotiation. That doesn't mean that it can't be done. This whole thread is discussing things that have been done by the C++ committee: C++20. > But the last thing we need is for the C++ standards committee to standardize a package manager. That would take forever to do, would result in a messy tool that tries to compromise with all the actors in some ways, make it very hard and slow to evolve over time and would likely result in a lot of pain, etc. You just summarized my feelings about C++ in general. I would much rather people use Rust or Go or any number of other languages instead of C++, depending on project needs. Such opinions are rarely taken well in threads like this, though, so... I've been trying to be optimistic and point out that C++ could get package management. If the C++ committee process works well, then the package manager should also end up turning out well. I'll leave the reader to decide how well they think the long term direction and guidance of the C++ standard has been going and apply that to their feelings of a hypothetical future package manager.

humanrebar 6y ago

Do the Go sources on GitHub use waf, make, CMake, bazel, or something entirely bespoke? Or is a common build system assumed?

coder543 6y ago
Go just uses the Go build system, which is common to all Go projects, so... it assumes a common build system. Some people do occasionally use Bazel or other build systems on top of the Go build system for complicated monorepos.
- humanrebar 6y ago
  Right. C++ has a chicken-and-the-egg problem in that neither its build not packaging ecosystem have even de facto standards. GitHub URLs don't solve either problem.
  - coder543 6y ago
    I think everyone agrees that a common build system is a necessary step if any of this is going to work. Thinking about how much or how little would be required beyond a common build system in order to get a working package management system is still a valid thing to do.
    - humanrebar 6y ago
      I expect it is necessary to keep build configuration portable with respect to packaging and environment configuration. In other words, I expect downloading from URLs in Makefiles and CMakeLists to be a local maximum. Keeping a parallel set of instructions or metadata that includes specific URLs and such might work, though. As long as you can skip all that when a system package, filesystem path, or git submodule is more appropriate.