5 ms·

Inspecting the Source of Go Modules

42 points by todsacerdoti 8mo ago

delusional 8mo ago
> but it is better understood as exploiting the natural lack of verification in the GitHub web interface, which doesn’t show the authentic (and in this case malicious) source of a module version, as used by actual Go tooling. That's a load of crap. It can never become a github issue that the Go ecosystem has chosen to make it look like the packages you pull come from github, but are actually diverted to be served by Google from some strange pull-through cache. Calling this a "lack of verification in the GitHub web interfaces" completely inverts the abstraction layers and asks GitHub to implement specific features for your incorrect usecase. Then there's the misnomer of a hash database being anywhere close to analogous to PGP signed sources. This is amateur level stuff.
- direwolf20 8mo ago
  You are right. GitHub has nothing to do with this problem — it's entirely on Go.
- antonvs 8mo ago
  FTA: > “For example, there is no guarantee that the code displayed at https://github.com/example/mod/blob/v1.2.3/exp.go https://github.com/example/mod/blob/v1.2.3/exp.go is the actual contents of exp.go from v1.2.3 of module github.com/example/mod: GitHub allows force-pushing git tags and even built its recommended GitHub Actions workflows on top of mutable tags.” In a sense this is a git issue: the fact that git is mutable instead of append-only. Git wasn’t designed to serve as part of a code integrity system - you’d have to add that at some other level, such as Github. Or in the case of this example, recognize that tags can’t be secured and implement everything in terms of commit hashes, etc.
  - LtWorf 8mo ago
    > implement everything in terms of commit hashes And remember, git uses a vulnerable algorithm :)
  - embedding-shape 8mo ago
    > the fact that git is mutable instead of append-only Remind me again how I change a commit in the middle of a branch without affecting the commits that come later in the same branch?
    - arcologies1985 8mo ago
      git rebase can do that easily, I do it all the time when prepping feature branches for submission to the main branch
      - embedding-shape 8mo ago
        > git rebase can do that easily What exact parameters you use for not changing the latter commits? Must be a very new feature, git's whole idea used to be that commits are on top of the previous ones, rewriting one needs rewrite of all the latter ones.
        bmandale 8mo ago
        Instead of making a sarcastic response, just state plainly what you mean. This prevents you having to get into dumb arguments that don't mean anything to people who don't already know what you're trying to say.
        arcologies1985 8mo ago
        literally just 'git rebase origin/main' to insert new commits to main into my branch history prior to the commits I made. No rewrites of later commits required if there are no conflicts. You seem to have an extreme misunderstanding of how git works.
        embedding-shape 8mo ago
        Talking about extreme misunderstandings, rebase always rewrites commits, even when there are conflicts. Look at the commit sha's next time you're doing that.
aleksi 8mo ago
See also https://github.com/golang/go/issues/66653 https://github.com/golang/go/issues/66653 (that I don’t think is linked from the article)
philipwhiuk 8mo ago
I've always found it very weird that GitHub's tolerated being completely subverted to become a package manager for ecosystems that can't be bothered to manage it themselves. NPM doesn't just proxy to GitHub (even though both are owned by Microsoft). To see maintainers criticise GitHub for not being a perfect package manager is lunacy. See also https://github.com/CocoaPods/CocoaPods/issues/4989 https://github.com/CocoaPods/CocoaPods/issues/4989
- bspammer 8mo ago
  That’s fair, but GitHub themselves do it with GitHub Actions. Versioning of all official actions is done with git tags, which has always made me uncomfortable.
  - embedding-shape 8mo ago
    > GitHub themselves do it with GitHub Actions That makes sense, so does doing releases by using tags, why would that make you uncomfortable? What doesn't make sense, is creating a completely new language/framework/package manager and decide to place the package registry burden on someone else.
    - formerly_proven 8mo ago
      > That makes sense, so does doing releases by using tags, why would that make you uncomfortable? Tags are not immutable.
      - embedding-shape 8mo ago
        Are you confusing tags with branches? Git tags for all intents and purposes are immutable. If you have a tag in your local repository that was pulled down from another remote, and the remote changes the tag and you update your local repository, it'll reject the incoming tag. It's true they're both refs inside git, but git literally treats them as "shouldn't move", unlike branches. They're not immutable in the technical sense, so I guess you're technically right. But they're not used the same way as branches, and the tooling won't like that either.
pjmlp 8mo ago
> Go has indisputably the best package integrity story of any programming language ecosystem Nope, still catching up to Java and .NET.
- FiloSottile 8mo ago
  I would love to learn more. What's the package integrity story of Java and .NET? All I can find is documentation about artifacts on e.g. Maven Central being signed with any PGP key, which can freely change across package versions. If that's correct, it's no more than a convoluted checksum (without anything resembling the Checksum Database and its transparency log). If that's not correct, I am very curious what the workflow is when a package author loses a key. Or, more concretely: what's stopping Maven Central from serving a fake version of someone else's package to a targeted victim?

42 points by todsacerdoti 8mo ago

> but it is better understood as exploiting the natural lack of verification in the GitHub web interface, which doesn’t show the authentic (and in this case malicious) source of a module version, as used by actual Go tooling. That's a load of crap. It can never become a github issue that the Go ecosystem has chosen to make it look like the packages you pull come from github, but are actually diverted to be served by Google from some strange pull-through cache. Calling this a "lack of verification in the GitHub web interfaces" completely inverts the abstraction layers and asks GitHub to implement specific features for your incorrect usecase. Then there's the misnomer of a hash database being anywhere close to analogous to PGP signed sources. This is amateur level stuff.

direwolf20 8mo ago
You are right. GitHub has nothing to do with this problem — it's entirely on Go.
antonvs 8mo ago
FTA: > “For example, there is no guarantee that the code displayed at https://github.com/example/mod/blob/v1.2.3/exp.go https://github.com/example/mod/blob/v1.2.3/exp.go is the actual contents of exp.go from v1.2.3 of module github.com/example/mod: GitHub allows force-pushing git tags and even built its recommended GitHub Actions workflows on top of mutable tags.” In a sense this is a git issue: the fact that git is mutable instead of append-only. Git wasn’t designed to serve as part of a code integrity system - you’d have to add that at some other level, such as Github. Or in the case of this example, recognize that tags can’t be secured and implement everything in terms of commit hashes, etc.
- LtWorf 8mo ago
  > implement everything in terms of commit hashes And remember, git uses a vulnerable algorithm :)
- embedding-shape 8mo ago
  > the fact that git is mutable instead of append-only Remind me again how I change a commit in the middle of a branch without affecting the commits that come later in the same branch?
  - arcologies1985 8mo ago
    git rebase can do that easily, I do it all the time when prepping feature branches for submission to the main branch
    - embedding-shape 8mo ago
      > git rebase can do that easily What exact parameters you use for not changing the latter commits? Must be a very new feature, git's whole idea used to be that commits are on top of the previous ones, rewriting one needs rewrite of all the latter ones.
      - bmandale 8mo ago
        Instead of making a sarcastic response, just state plainly what you mean. This prevents you having to get into dumb arguments that don't mean anything to people who don't already know what you're trying to say.
      - arcologies1985 8mo ago
        literally just 'git rebase origin/main' to insert new commits to main into my branch history prior to the commits I made. No rewrites of later commits required if there are no conflicts. You seem to have an extreme misunderstanding of how git works.
        embedding-shape 8mo ago
        Talking about extreme misunderstandings, rebase always rewrites commits, even when there are conflicts. Look at the commit sha's next time you're doing that.

aleksi 8mo ago

See also https://github.com/golang/go/issues/66653 https://github.com/golang/go/issues/66653 (that I don’t think is linked from the article)

philipwhiuk 8mo ago

I've always found it very weird that GitHub's tolerated being completely subverted to become a package manager for ecosystems that can't be bothered to manage it themselves. NPM doesn't just proxy to GitHub (even though both are owned by Microsoft). To see maintainers criticise GitHub for not being a perfect package manager is lunacy. See also https://github.com/CocoaPods/CocoaPods/issues/4989 https://github.com/CocoaPods/CocoaPods/issues/4989

bspammer 8mo ago
That’s fair, but GitHub themselves do it with GitHub Actions. Versioning of all official actions is done with git tags, which has always made me uncomfortable.
- embedding-shape 8mo ago
  > GitHub themselves do it with GitHub Actions That makes sense, so does doing releases by using tags, why would that make you uncomfortable? What doesn't make sense, is creating a completely new language/framework/package manager and decide to place the package registry burden on someone else.
  - formerly_proven 8mo ago
    > That makes sense, so does doing releases by using tags, why would that make you uncomfortable? Tags are not immutable.
    - embedding-shape 8mo ago
      Are you confusing tags with branches? Git tags for all intents and purposes are immutable. If you have a tag in your local repository that was pulled down from another remote, and the remote changes the tag and you update your local repository, it'll reject the incoming tag. It's true they're both refs inside git, but git literally treats them as "shouldn't move", unlike branches. They're not immutable in the technical sense, so I guess you're technically right. But they're not used the same way as branches, and the tooling won't like that either.

pjmlp 8mo ago

> Go has indisputably the best package integrity story of any programming language ecosystem Nope, still catching up to Java and .NET.

FiloSottile 8mo ago
I would love to learn more. What's the package integrity story of Java and .NET? All I can find is documentation about artifacts on e.g. Maven Central being signed with any PGP key, which can freely change across package versions. If that's correct, it's no more than a convoluted checksum (without anything resembling the Checksum Database and its transparency log). If that's not correct, I am very curious what the workflow is when a package author loses a key. Or, more concretely: what's stopping Maven Central from serving a fake version of someone else's package to a targeted victim?