6 ms·

>> pkg.go.dev hard-codes a list of centralized git hosting services [that are the only allowed ones] So how in the world did git.lukeshu.com get to be on that

>> pkg.go.dev hard-codes a list of centralized git hosting services [that are the only allowed ones] So how in the world did git.lukeshu.com get to be on that list? https://pkg.go.dev/git.lukeshu.com/go/libfastimport https://pkg.go.dev/git.lukeshu.com/go/libfastimport

ddevault 6y ago
Note that some features are absent here. You can't click on any of the declarations to view their source, for example. This is the degraded view for unknown hosts.
- throwayws 6y ago
  That does not work in the old godoc.org either: https://godoc.org/git.lukeshu.com/go/libfastimport https://godoc.org/git.lukeshu.com/go/libfastimport So that's not against pkg.dev. Did somebody look into why? Anyway it seems that the repo was not added to pkg.dev automatically: https://github.com/golang/go/issues/38326 https://github.com/golang/go/issues/38326 and the automatic fetching should work now: https://github.com/golang/go/issues/37002 https://github.com/golang/go/issues/37002
  - arp242 6y ago
    > Anyway it seems that the repo was not added to pkg.dev automatically: > > https://github.com/golang/go/issues/38326 https://github.com/golang/go/issues/38326 The way this works (or rather, worked) is that you first need to "go get" a module and then it gets picked up by the pkg.go.dev system. This works like that for every site, and was kind weird and confusing, although as you mentioned this was improved on recently. Either way: not related to that module being on sr.ht; that's just coincidental.
  - ddevault 6y ago
    See my comment here which explains how "go-source" works: https://news.ycombinator.com/item?id=24024130 https://news.ycombinator.com/item?id=24024130 It doesn't work on godoc.org not because they don't recognize your domain, but because cgit doesn't have go-source meta tags.
    - throwayws 6y ago
      There was a proposal to add a new go-source-2 meta tag, but that was abandoned because a lot of code hosting sites would not have been supported yet: https://github.com/golang/go/issues/39559 https://github.com/golang/go/issues/39559 https://github.com/golang/go/issues/40477 https://github.com/golang/go/issues/40477 That is why they decided to turn it into an internal/source package inside pkgsite first to have test cases for a later go-source-2. If your site does not show source links on pkg.go.dev then add a comment to issue 40477 above and an exception will be added. - internal/source: https://github.com/golang/pkgsite/blob/master/internal/source/source.go https://github.com/golang/pkgsite/blob/master/internal/sourc... Note1: Maybe mention this on your website because that took a little to find. It does not fit into the narrative though. Note2: Requiring frontends to offer meta tags is the only way to make code discoverable with the url alone. cgit not offering this is on cgit and not on Go/pkgsite (haven't checked whether they do)
- arp242 6y ago
  Some features being absent is not the same as what you said in the article. "Any GitLab instance other than gitlab.com [..] none of these are going to work unless every host is added and the list is kept up-to-date manually." makes it sound like they don't work at all, but in fact it's just a few minor features that don't work. sr.ht has always worked on pkg.go.dev from day 1. Actually, pkg.go.dev needs to have some way of figuring out where to link to, so it's not necessarily easy to automatically make links that work for 100% of the sites. That's not "a blatant failure to comprehend the fundamentally decentralized nature of git hosting", that's having to deal with a large ecosystem spread out over thousands of domains. Go's model is a lot more decentralized than most other package systems (which also makes stuff like this harder).
  - ddevault 6y ago
    They do have a way, and godoc.org uses it: go-source meta tags. https://news.ycombinator.com/item?id=24024130 https://news.ycombinator.com/item?id=24024130
    - arp242 6y ago
      That doesn't address the main meat of the comment, which said that your claims in the article are highly misleading.
      - ddevault 6y ago
        I don't think that my claims are misleading.
        arp242 6y ago
        You suggested many code hosting sites don't work, while in fact they do work except for some (IMHO minor, but we can perhaps disagree on that) features. You don't think that's not a full and accurate representation of the state of things?
        6y ago
- enneff 6y ago
  How do you propose this problem be solved? I was partially responsible for the go-source meta tag thing in the first place. Is there some other mechanism to find a web-based view of the source of a git repo?
  - ddevault 6y ago
    Well, it's complicated, and we could brainstorm a few ideas. The problem seems to be that this brainstorming session never happened. One problem is that the import path doesn't really give you enough information to totally figure out how to fetch the package. It's treated like a URL, and then the go-source tag comes in to relate it to something VCS-controlled. If we encoded the VCS into the import, we could e.g. have "git+https://git.sr.ht/~sircmpwn/getopt" https://git.sr.ht/~sircmpwn/getopt" as the import path, or we could move the import details into a separate file (like go.mod). But if we assume that the import path and go.mod formats are fixed, then we still have more things we could do. For example, why this: <meta name="go-import" content="git.sr.ht/~sircmpwn/getopt git https://git.sr.ht/~sircmpwn/getopt" /> When we could have this: <link rel="alternate" type="application/x-git-http" "https://git.sr.ht/~sircmpwn/getopt" /> <link rel="alternate" type="application/x-git-ssh" "git@git.sr.ht:~sircmpwn/getopt" /> And why this: <meta name="go-source" content="git.sr.ht/~sircmpwn/getopt https://git.sr.ht/~sircmpwn/getopt https://git.sr.ht/~sircmpwn/getopt/tree/master{/dir} https://git.sr.ht/~sircmpwn/getopt/tree/master{/dir}/{file}#L{line}"> When the same tag could be "source-browser" or something similar? I'm sure many projects other than Go would be very happy to have features like this standardized and available for all of them to use, but the Go team sees no further than its own nose when designing this sort of thing. And furthermore, with the specific case of hardcoded software hosts on pkg.go.dev, why isn't it using the go-source meta tags to look up how to create links to files & line numbers? You guys forced this upon us and then don't even use it? There's no reason to hard-code regexes for various git domains when you could just fetch it like godoc.org does.
    - enneff 6y ago
      I agree that it would have been better to design these a little differently. In the first case the go-import meta tag is totally unnecessary. The go tool and module proxy can discover git repos from an import path just fine. It’s only required for “custom” import paths where the path is some domain but the code lives somewhere else, like github. In the latter case the decision to use go-source was actually made by the original author of godoc.org, not a google employee, and was done in coordination with another non-google initiative (gopkg.in) to make their source links work. So nothing to do with the go team really, sorry. https://github.com/golang/gddo/commit/864b1c0aba009e37d301365a2f6f6cf51a3caef3 https://github.com/golang/gddo/commit/864b1c0aba009e37d30136... If pkg.go.dev isn’t respecting go-source meta tags then that should probably be fixed. It would also imo be worth considering devising a more general, well-known mechanism for doing this. Worth proposing I think!
edoceo 6y ago
Likely cause Luke is providing some valuable code that other golang users depend on? Also: hi Luke! And thank you!

ddevault 6y ago

Note that some features are absent here. You can't click on any of the declarations to view their source, for example. This is the degraded view for unknown hosts.

throwayws 6y ago
That does not work in the old godoc.org either: https://godoc.org/git.lukeshu.com/go/libfastimport https://godoc.org/git.lukeshu.com/go/libfastimport So that's not against pkg.dev. Did somebody look into why? Anyway it seems that the repo was not added to pkg.dev automatically: https://github.com/golang/go/issues/38326 https://github.com/golang/go/issues/38326 and the automatic fetching should work now: https://github.com/golang/go/issues/37002 https://github.com/golang/go/issues/37002
- arp242 6y ago
  > Anyway it seems that the repo was not added to pkg.dev automatically: > > https://github.com/golang/go/issues/38326 https://github.com/golang/go/issues/38326 The way this works (or rather, worked) is that you first need to "go get" a module and then it gets picked up by the pkg.go.dev system. This works like that for every site, and was kind weird and confusing, although as you mentioned this was improved on recently. Either way: not related to that module being on sr.ht; that's just coincidental.
- ddevault 6y ago
  See my comment here which explains how "go-source" works: https://news.ycombinator.com/item?id=24024130 https://news.ycombinator.com/item?id=24024130 It doesn't work on godoc.org not because they don't recognize your domain, but because cgit doesn't have go-source meta tags.
  - throwayws 6y ago
    There was a proposal to add a new go-source-2 meta tag, but that was abandoned because a lot of code hosting sites would not have been supported yet: https://github.com/golang/go/issues/39559 https://github.com/golang/go/issues/39559 https://github.com/golang/go/issues/40477 https://github.com/golang/go/issues/40477 That is why they decided to turn it into an internal/source package inside pkgsite first to have test cases for a later go-source-2. If your site does not show source links on pkg.go.dev then add a comment to issue 40477 above and an exception will be added. - internal/source: https://github.com/golang/pkgsite/blob/master/internal/source/source.go https://github.com/golang/pkgsite/blob/master/internal/sourc... Note1: Maybe mention this on your website because that took a little to find. It does not fit into the narrative though. Note2: Requiring frontends to offer meta tags is the only way to make code discoverable with the url alone. cgit not offering this is on cgit and not on Go/pkgsite (haven't checked whether they do)
arp242 6y ago
Some features being absent is not the same as what you said in the article. "Any GitLab instance other than gitlab.com [..] none of these are going to work unless every host is added and the list is kept up-to-date manually." makes it sound like they don't work at all, but in fact it's just a few minor features that don't work. sr.ht has always worked on pkg.go.dev from day 1. Actually, pkg.go.dev needs to have some way of figuring out where to link to, so it's not necessarily easy to automatically make links that work for 100% of the sites. That's not "a blatant failure to comprehend the fundamentally decentralized nature of git hosting", that's having to deal with a large ecosystem spread out over thousands of domains. Go's model is a lot more decentralized than most other package systems (which also makes stuff like this harder).
- ddevault 6y ago
  They do have a way, and godoc.org uses it: go-source meta tags. https://news.ycombinator.com/item?id=24024130 https://news.ycombinator.com/item?id=24024130
  - arp242 6y ago
    That doesn't address the main meat of the comment, which said that your claims in the article are highly misleading.
    - ddevault 6y ago
      I don't think that my claims are misleading.
      - arp242 6y ago
        You suggested many code hosting sites don't work, while in fact they do work except for some (IMHO minor, but we can perhaps disagree on that) features. You don't think that's not a full and accurate representation of the state of things?
        6y ago
enneff 6y ago
How do you propose this problem be solved? I was partially responsible for the go-source meta tag thing in the first place. Is there some other mechanism to find a web-based view of the source of a git repo?
- ddevault 6y ago
  Well, it's complicated, and we could brainstorm a few ideas. The problem seems to be that this brainstorming session never happened. One problem is that the import path doesn't really give you enough information to totally figure out how to fetch the package. It's treated like a URL, and then the go-source tag comes in to relate it to something VCS-controlled. If we encoded the VCS into the import, we could e.g. have "git+https://git.sr.ht/~sircmpwn/getopt" https://git.sr.ht/~sircmpwn/getopt" as the import path, or we could move the import details into a separate file (like go.mod). But if we assume that the import path and go.mod formats are fixed, then we still have more things we could do. For example, why this: <meta name="go-import" content="git.sr.ht/~sircmpwn/getopt git https://git.sr.ht/~sircmpwn/getopt" /> When we could have this: <link rel="alternate" type="application/x-git-http" "https://git.sr.ht/~sircmpwn/getopt" /> <link rel="alternate" type="application/x-git-ssh" "git@git.sr.ht:~sircmpwn/getopt" /> And why this: <meta name="go-source" content="git.sr.ht/~sircmpwn/getopt https://git.sr.ht/~sircmpwn/getopt https://git.sr.ht/~sircmpwn/getopt/tree/master{/dir} https://git.sr.ht/~sircmpwn/getopt/tree/master{/dir}/{file}#L{line}"> When the same tag could be "source-browser" or something similar? I'm sure many projects other than Go would be very happy to have features like this standardized and available for all of them to use, but the Go team sees no further than its own nose when designing this sort of thing. And furthermore, with the specific case of hardcoded software hosts on pkg.go.dev, why isn't it using the go-source meta tags to look up how to create links to files & line numbers? You guys forced this upon us and then don't even use it? There's no reason to hard-code regexes for various git domains when you could just fetch it like godoc.org does.
  - enneff 6y ago
    I agree that it would have been better to design these a little differently. In the first case the go-import meta tag is totally unnecessary. The go tool and module proxy can discover git repos from an import path just fine. It’s only required for “custom” import paths where the path is some domain but the code lives somewhere else, like github. In the latter case the decision to use go-source was actually made by the original author of godoc.org, not a google employee, and was done in coordination with another non-google initiative (gopkg.in) to make their source links work. So nothing to do with the go team really, sorry. https://github.com/golang/gddo/commit/864b1c0aba009e37d301365a2f6f6cf51a3caef3 https://github.com/golang/gddo/commit/864b1c0aba009e37d30136... If pkg.go.dev isn’t respecting go-source meta tags then that should probably be fixed. It would also imo be worth considering devising a more general, well-known mechanism for doing this. Worth proposing I think!

edoceo 6y ago

Likely cause Luke is providing some valuable code that other golang users depend on? Also: hi Luke! And thank you!