5 ms·

And what did Go do?

by lambda 5y ago

And what did Go do?

Laremere 5y ago
When you use `go get` to retrieve a version of a dependency, it adds a line to a `go.sum` while with a hash of the code at the version specified. You can distribute your code without a copy of the dependency. When someone else runs go get on your module, it will attempt to retrieve the same version of the source, checking the hash. If the hashes are different, an error is thrown.
- fnord123 5y ago
  That's what a Cargo.lock has. But it's recommended that you don't ship Cargo.lock that with libraries. I think the reason is because you can't have multiple versions of the same package. But I could be wrong. (If I'm wrong then you end up with horrific bloat with multiple trees of the same dep). What does go do when you have different deps depending on different versions of the same common dep?
  - estebank 5y ago
    Cargo.lock is ignored for libraries, it is only checked for binaries. https://doc.rust-lang.org/cargo/faq.html#why-do-binaries-have-cargolock-in-version-control-but-not-libraries https://doc.rust-lang.org/cargo/faq.html#why-do-binaries-hav... > If a library ends up being used transitively by several dependencies, it’s likely that just a single copy of the library is desired (based on semver compatibility). If Cargo used all of the dependencies' Cargo.lock files, then multiple copies of the library could be used, and perhaps even a version conflict. > In other words, libraries specify SemVer requirements for their dependencies but cannot see the full picture. Only end products like binaries have a full picture to decide what versions of dependencies should be used.
  - verdverm 5y ago
    Minimum Version Selection (https://research.swtch.com/vgo-mvs https://research.swtch.com/vgo-mvs) tl;dr, Go does a BFS of the dep tree, selecting the maximum version found. There are no ranges, so you can only use a version which has been listed. "minimum" comes for a minimal implementation which can create reproducible results without a lockfile
- TheDong 5y ago
  This is in the context of rust. > it adds a line to a `go.sum` while with a hash of the code at the version specified Cargo.lock also contains a checksum > You can distribute your code without a copy of the dependency Also true in rust, and the default way of using rust/cargo. > If the hashes are different, an error is thrown. Also true in rust. For an example of what this looks like: https://github.com/servo/webrender/blob/54b725be37f13b16694687ceab28b3fa6ac48921/Cargo.lock#L9-L16 https://github.com/servo/webrender/blob/54b725be37f13b166946... You haven't described anything different between go and rust in your comment since every feature you've pointed out applies equally to both.
TheDong 5y ago
A go module is just a git repo (or svn/hg/etc repo, but almost always git). Versions are git tags or commits. Dependencies are specified in a go.mod file, specified here: https://golang.org/ref/mod#go-mod-file https://golang.org/ref/mod#go-mod-file From the following go import path: import "mywebsite.com/foo/bar/baz/v2/fizz" We can determine that one of the following things is true: 1. mywebsite.com has a 'meta name="go-import"' html tag in it when accessed by the go tool that points to a git repo/svn repo/etc, and the contained package has the folder structure "foo/bar/baz/v2/fizz" 2. mywebsite.com/foo has a 'meta name="go-import"' html tag... (several obvious cases omitted) 3. mywebsite.com/foo/bar/baz is a git repo that has no version tag, but has a "v2" folder in it 4. mywebsite.com/foo/bar/baz is a git repo that has a version tag with v2.x.y 5. the go.mod has a replace or other directive for this import path The 4th case is the relevant one here. github.com/foo/bar/v2 means that the bar repo has a v2 tag usually. So, what version of that repo is used if there's v2.0.0 and v2.1.0? Well, the version is the smallest version that you or any module you depend on uses, so you simply have to recursively parse all go.mod files to understand the version used. Much simpler than Cargo.lock. How do you enable or disable features in a sub-go-module? Let's say I have a module that has a "+ui" or "-ui" build tag because you can optionally build an openGL ui or such. In rust, this would be a 'features = [ "ui" ]' in your Cargo.toml. In go, you would change your own compilation step to be "go build -tags ui"... And if you have two dependencies with those flags, and you wish to make them opposite values, that is simply impossible in go even though it's trivial in rust. There are other differences, but I feel like I've already made my point. My main point is that go modules make import paths way more opaque. It used to be that every path element was either part of the import path or a folder on the filesystem. Now, it's either part of the import path, or a git tag, or a folder, and none of those things are obvious. My secondary point is that go has a much simpler set of requirements than rust, mostly due to missing important features rust has.
verdverm 5y ago
Versions are only importable from tags (semver style)* There is no NPM or PyPi like host, sources originate from code hosts Modules are prefixed with a domain name (avoid dep confusion) Hashes are calculated on the module, it will fail if it is not correct GoProxy has a global transparency log (like a blockchain, but not) so that everyone is contributing and sharing hashes *there are special versions (v0.0.0 and branch names). They have special meaning, and get fixed with the access date and commit hash For the curious: https://golang.org/ref/mod https://golang.org/ref/mod and https://go.googlesource.com/proposal/+/master/design/25530-sumdb.md https://go.googlesource.com/proposal/+/master/design/25530-s...
- rat87 5y ago
  > Versions are only importable from tags (semver style)* really sucks > There is no NPM or PyPi like host, sources originate from code hosts Bad idea. Easy for packages to disapear. > Modules are prefixed with a domain name (avoid dep confusion) This seems like more trouble then its worth most of the time
- Groxx 5y ago
  So: >Versions are only importable from tags (semver style) Tags are mutable in git, which sometimes causes problems. Not that I think they had much choice here - git and tags are the de-facto community standard, they're just using it like everyone else. >There is no NPM or PyPi like host, sources originate from code hosts >Modules are prefixed with a domain name (avoid dep confusion) Agreed, I like this quite a bit... but it comes with the significant risk of domain name or hosting changes. Github probably won't add the metadata to redirect your old module name to your new non-github name, for instance. I've watched several modules fight through this sequence, and the end result is always "can't do anything, contact your users yourself / hope they read the readme". You win some, you lose some. In large part I think this part of Go modules is better than others'. >Hashes are calculated on the module, it will fail if it is not correct Nearly all lock-file-based dependency systems do this, Cargo included. >GoProxy has a global transparency log (like a blockchain, but not) so that everyone is contributing and sharing hashes Publishing a log now doesn't make it any safer tomorrow, it's purely about convincing you to trust them. This is precisely the same attack target as an NPM or PyPi. The same party hosts the tar'd source files + documentation + checksums: if it ever gets changed there, it's changed for everyone. Lock and go.sum files will only protect you from changes to your previously-known dependencies. They've done a pretty good job making goproxy/gosum easy to run yourself, and you can skip past the goproxy and download directly from the module's owner (which is a domain name, which works great here), which mitigates this specific risk at least. But no, they're absolutely running a pypi, they just don't let repo owners make changes directly (only by committing changes, e.g. module revocation).

And what did Go do?

by lambda 5y ago

And what did Go do?

When you use `go get` to retrieve a version of a dependency, it adds a line to a `go.sum` while with a hash of the code at the version specified. You can distribute your code without a copy of the dependency. When someone else runs go get on your module, it will attempt to retrieve the same version of the source, checking the hash. If the hashes are different, an error is thrown.

fnord123 5y ago
That's what a Cargo.lock has. But it's recommended that you don't ship Cargo.lock that with libraries. I think the reason is because you can't have multiple versions of the same package. But I could be wrong. (If I'm wrong then you end up with horrific bloat with multiple trees of the same dep). What does go do when you have different deps depending on different versions of the same common dep?
- estebank 5y ago
  Cargo.lock is ignored for libraries, it is only checked for binaries. https://doc.rust-lang.org/cargo/faq.html#why-do-binaries-have-cargolock-in-version-control-but-not-libraries https://doc.rust-lang.org/cargo/faq.html#why-do-binaries-hav... > If a library ends up being used transitively by several dependencies, it’s likely that just a single copy of the library is desired (based on semver compatibility). If Cargo used all of the dependencies' Cargo.lock files, then multiple copies of the library could be used, and perhaps even a version conflict. > In other words, libraries specify SemVer requirements for their dependencies but cannot see the full picture. Only end products like binaries have a full picture to decide what versions of dependencies should be used.
- verdverm 5y ago
  Minimum Version Selection (https://research.swtch.com/vgo-mvs https://research.swtch.com/vgo-mvs) tl;dr, Go does a BFS of the dep tree, selecting the maximum version found. There are no ranges, so you can only use a version which has been listed. "minimum" comes for a minimal implementation which can create reproducible results without a lockfile
TheDong 5y ago
This is in the context of rust. > it adds a line to a `go.sum` while with a hash of the code at the version specified Cargo.lock also contains a checksum > You can distribute your code without a copy of the dependency Also true in rust, and the default way of using rust/cargo. > If the hashes are different, an error is thrown. Also true in rust. For an example of what this looks like: https://github.com/servo/webrender/blob/54b725be37f13b16694687ceab28b3fa6ac48921/Cargo.lock#L9-L16 https://github.com/servo/webrender/blob/54b725be37f13b166946... You haven't described anything different between go and rust in your comment since every feature you've pointed out applies equally to both.

TheDong 5y ago

A go module is just a git repo (or svn/hg/etc repo, but almost always git). Versions are git tags or commits. Dependencies are specified in a go.mod file, specified here: https://golang.org/ref/mod#go-mod-file https://golang.org/ref/mod#go-mod-file From the following go import path: import "mywebsite.com/foo/bar/baz/v2/fizz" We can determine that one of the following things is true: 1. mywebsite.com has a 'meta name="go-import"' html tag in it when accessed by the go tool that points to a git repo/svn repo/etc, and the contained package has the folder structure "foo/bar/baz/v2/fizz" 2. mywebsite.com/foo has a 'meta name="go-import"' html tag... (several obvious cases omitted) 3. mywebsite.com/foo/bar/baz is a git repo that has no version tag, but has a "v2" folder in it 4. mywebsite.com/foo/bar/baz is a git repo that has a version tag with v2.x.y 5. the go.mod has a replace or other directive for this import path The 4th case is the relevant one here. github.com/foo/bar/v2 means that the bar repo has a v2 tag usually. So, what version of that repo is used if there's v2.0.0 and v2.1.0? Well, the version is the smallest version that you or any module you depend on uses, so you simply have to recursively parse all go.mod files to understand the version used. Much simpler than Cargo.lock. How do you enable or disable features in a sub-go-module? Let's say I have a module that has a "+ui" or "-ui" build tag because you can optionally build an openGL ui or such. In rust, this would be a 'features = [ "ui" ]' in your Cargo.toml. In go, you would change your own compilation step to be "go build -tags ui"... And if you have two dependencies with those flags, and you wish to make them opposite values, that is simply impossible in go even though it's trivial in rust. There are other differences, but I feel like I've already made my point. My main point is that go modules make import paths way more opaque. It used to be that every path element was either part of the import path or a folder on the filesystem. Now, it's either part of the import path, or a git tag, or a folder, and none of those things are obvious. My secondary point is that go has a much simpler set of requirements than rust, mostly due to missing important features rust has.

verdverm 5y ago

Versions are only importable from tags (semver style)* There is no NPM or PyPi like host, sources originate from code hosts Modules are prefixed with a domain name (avoid dep confusion) Hashes are calculated on the module, it will fail if it is not correct GoProxy has a global transparency log (like a blockchain, but not) so that everyone is contributing and sharing hashes *there are special versions (v0.0.0 and branch names). They have special meaning, and get fixed with the access date and commit hash For the curious: https://golang.org/ref/mod https://golang.org/ref/mod and https://go.googlesource.com/proposal/+/master/design/25530-sumdb.md https://go.googlesource.com/proposal/+/master/design/25530-s...

rat87 5y ago
> Versions are only importable from tags (semver style)* really sucks > There is no NPM or PyPi like host, sources originate from code hosts Bad idea. Easy for packages to disapear. > Modules are prefixed with a domain name (avoid dep confusion) This seems like more trouble then its worth most of the time
Groxx 5y ago
So: >Versions are only importable from tags (semver style) Tags are mutable in git, which sometimes causes problems. Not that I think they had much choice here - git and tags are the de-facto community standard, they're just using it like everyone else. >There is no NPM or PyPi like host, sources originate from code hosts >Modules are prefixed with a domain name (avoid dep confusion) Agreed, I like this quite a bit... but it comes with the significant risk of domain name or hosting changes. Github probably won't add the metadata to redirect your old module name to your new non-github name, for instance. I've watched several modules fight through this sequence, and the end result is always "can't do anything, contact your users yourself / hope they read the readme". You win some, you lose some. In large part I think this part of Go modules is better than others'. >Hashes are calculated on the module, it will fail if it is not correct Nearly all lock-file-based dependency systems do this, Cargo included. >GoProxy has a global transparency log (like a blockchain, but not) so that everyone is contributing and sharing hashes Publishing a log now doesn't make it any safer tomorrow, it's purely about convincing you to trust them. This is precisely the same attack target as an NPM or PyPi. The same party hosts the tar'd source files + documentation + checksums: if it ever gets changed there, it's changed for everyone. Lock and go.sum files will only protect you from changes to your previously-known dependencies. They've done a pretty good job making goproxy/gosum easy to run yourself, and you can skip past the goproxy and download directly from the module's owner (which is a domain name, which works great here), which mitigates this specific risk at least. But no, they're absolutely running a pypi, they just don't let repo owners make changes directly (only by committing changes, e.g. module revocation).