4 ms·

Question: Why is go so integrated(?) with github? I mean, why can't I (or can I?) go get from my own personal gitlab server?

whateveracct 6y ago
I think you can? The urls in package names are just urls. So if you have a url to a git repo, it'll work.
- HumblyTossed 6y ago
  Okay, interesting. I tried looking that up, but I never once found an example and, well, I simply never thought to try it. Thank you.
q3k 6y ago
You can. If you have a `.git` in your import path, it should just work OOTB, using Git as expected. Otherwise, whatever code hosting service has to implement support for returning a proprietary meta tag when serving a ?go-get=1 query - and Gitlab already does that, IIRC. https://golang.org/cmd/go/#hdr-Remote_import_paths https://golang.org/cmd/go/#hdr-Remote_import_paths
- HumblyTossed 6y ago
  Thank you for the link. I have tried in the past to find this exact information and for some reason it just never came up. User error, I suppose.
- Steltek 6y ago
  For a truly private server, you'll need two needles of that mountainous haystack: set GOPRIVATE and tell git to use ssh first: git config --global url.git@HOSTNAME:.insteadOf https://HOSTNAME/
  - gbear0 6y ago
    Is there not a better solution for this? We have this baked into our docker builds and it irks me that we have to copy personal credentials into a docker build so we can use git to pull modules and build. Is everyone actually doing this, or do you setup read-only tokens per private module, or anything else?
    - morelisp 6y ago
      We set up an internal Athens[0] caching proxy with credentials to pull from our GitLab. So far it seems to be win/win/win - We don't need this dance in our Dockerfiles / build scripts / dev machines anymore. We have a baked GOPROXY / GONOSUMDB in our build image and developers configure the same proxy (via `go env -w`) locally. - We pull all packages over the proxy, so builds are faster / use less bandwidth / still mostly work when GitHub goes down. - SCAs get more difficult; the credentials the proxy has are more limited than any individual developer's GitLab tokens / credentials, and owning the proxy is going to be harder than a single developer's laptop. [0] https://docs.gomods.io/ https://docs.gomods.io/
    - ramonrue 6y ago
      At least for Gitlab, you can write a .netrc in the user‘s home dir like so: machine <your GitLab domain> (e.g. gitlab.com) login <your GitLab id> password <your GitLab personal access token> For the access token, you can also leverage Gitlab‘s CI Job Token. What we do is an „echo $(netrc contents with $CI_JOB_TOKEN) > ~/.netrc“ in the pre-script in CI. (https://stackoverflow.com/a/61257782 https://stackoverflow.com/a/61257782 and https://docs.gitlab.com/ee/security/token_overview.html https://docs.gitlab.com/ee/security/token_overview.html)
      - morelisp 6y ago
        If you do take this approach, writing your access token in plaintext into a well-known file in your home directory seems like a strictly worse idea than the (also more broadly-compatible) Git insteadof rule to pull via SSH and your local agent.
trianglesphere 6y ago
All you need to do is have a webpage that has a meta tag with name="go-import" and a content tag with content="URL src-control-method source-control link" See xyzc.dev/go/ppgen and inspect the source for an example.
tsimionescu 6y ago
It isn't, but it is limited to a few source control solutions. If you use something other than Git, Bazaar, SVN, Mercurial, or Fossil, then you're not going to be able to publish Go modules. Also, Go modules are dependent on DNS, so any domain name change for your repo will have to result in code (import path) changes.
- gary-kim 6y ago
  A change in your repo does not need to result in an import path change if you set up vanity URLs for your import path. For instance, all my go libraries use the import path of gomod.garykim.dev even though the repos are actually hosted on GitHub. That means if I ever change to self-hosting, for example, I just need to update some of the meta tags on the gomod.garykim.dev website to change the repo location.
  - tsimionescu 6y ago
    That still means that if you lose your domain name (gomod.garykim.dev) for whatever reason, you have to modify your code.
    - morelisp 6y ago
      You can use a replace directive to handle cases like this without updating the code. https://golang.org/ref/mod#go-mod-file-replace https://golang.org/ref/mod#go-mod-file-replace
      - tsimionescu 6y ago
        Only if you're the final consumer of that module - if you publish it so others use your module, the replace directive will be ignored and they will get a failed build (or have to add their own replace directive for your sub-dependency).
        morelisp 6y ago
        Yes, but this is roughly the same as any packaging system - the ones that don't use DNS as a default namespace require you to add explicit package repositories, which is no more or less onerous than the replace directive.
nzach 6y ago
As others have already pointed out you don't need to use Github. Some people call it vanity imports[0]. In my experience 'vanity imports' give better results when googling stuff about custom import paths. [0] - https://blog.bramp.net/post/2017/10/02/vanity-go-import-paths/ https://blog.bramp.net/post/2017/10/02/vanity-go-import-path...

whateveracct 6y ago

I think you can? The urls in package names are just urls. So if you have a url to a git repo, it'll work.

HumblyTossed 6y ago
Okay, interesting. I tried looking that up, but I never once found an example and, well, I simply never thought to try it. Thank you.

q3k 6y ago

You can. If you have a `.git` in your import path, it should just work OOTB, using Git as expected. Otherwise, whatever code hosting service has to implement support for returning a proprietary meta tag when serving a ?go-get=1 query - and Gitlab already does that, IIRC. https://golang.org/cmd/go/#hdr-Remote_import_paths https://golang.org/cmd/go/#hdr-Remote_import_paths

HumblyTossed 6y ago
Thank you for the link. I have tried in the past to find this exact information and for some reason it just never came up. User error, I suppose.
Steltek 6y ago
For a truly private server, you'll need two needles of that mountainous haystack: set GOPRIVATE and tell git to use ssh first: git config --global url.git@HOSTNAME:.insteadOf https://HOSTNAME/
- gbear0 6y ago
  Is there not a better solution for this? We have this baked into our docker builds and it irks me that we have to copy personal credentials into a docker build so we can use git to pull modules and build. Is everyone actually doing this, or do you setup read-only tokens per private module, or anything else?
  - morelisp 6y ago
    We set up an internal Athens[0] caching proxy with credentials to pull from our GitLab. So far it seems to be win/win/win - We don't need this dance in our Dockerfiles / build scripts / dev machines anymore. We have a baked GOPROXY / GONOSUMDB in our build image and developers configure the same proxy (via `go env -w`) locally. - We pull all packages over the proxy, so builds are faster / use less bandwidth / still mostly work when GitHub goes down. - SCAs get more difficult; the credentials the proxy has are more limited than any individual developer's GitLab tokens / credentials, and owning the proxy is going to be harder than a single developer's laptop. [0] https://docs.gomods.io/ https://docs.gomods.io/
  - ramonrue 6y ago
    At least for Gitlab, you can write a .netrc in the user‘s home dir like so: machine <your GitLab domain> (e.g. gitlab.com) login <your GitLab id> password <your GitLab personal access token> For the access token, you can also leverage Gitlab‘s CI Job Token. What we do is an „echo $(netrc contents with $CI_JOB_TOKEN) > ~/.netrc“ in the pre-script in CI. (https://stackoverflow.com/a/61257782 https://stackoverflow.com/a/61257782 and https://docs.gitlab.com/ee/security/token_overview.html https://docs.gitlab.com/ee/security/token_overview.html)
    - morelisp 6y ago
      If you do take this approach, writing your access token in plaintext into a well-known file in your home directory seems like a strictly worse idea than the (also more broadly-compatible) Git insteadof rule to pull via SSH and your local agent.

trianglesphere 6y ago

All you need to do is have a webpage that has a meta tag with name="go-import" and a content tag with content="URL src-control-method source-control link" See xyzc.dev/go/ppgen and inspect the source for an example.

tsimionescu 6y ago

It isn't, but it is limited to a few source control solutions. If you use something other than Git, Bazaar, SVN, Mercurial, or Fossil, then you're not going to be able to publish Go modules. Also, Go modules are dependent on DNS, so any domain name change for your repo will have to result in code (import path) changes.

gary-kim 6y ago
A change in your repo does not need to result in an import path change if you set up vanity URLs for your import path. For instance, all my go libraries use the import path of gomod.garykim.dev even though the repos are actually hosted on GitHub. That means if I ever change to self-hosting, for example, I just need to update some of the meta tags on the gomod.garykim.dev website to change the repo location.
- tsimionescu 6y ago
  That still means that if you lose your domain name (gomod.garykim.dev) for whatever reason, you have to modify your code.
  - morelisp 6y ago
    You can use a replace directive to handle cases like this without updating the code. https://golang.org/ref/mod#go-mod-file-replace https://golang.org/ref/mod#go-mod-file-replace
    - tsimionescu 6y ago
      Only if you're the final consumer of that module - if you publish it so others use your module, the replace directive will be ignored and they will get a failed build (or have to add their own replace directive for your sub-dependency).
      - morelisp 6y ago
        Yes, but this is roughly the same as any packaging system - the ones that don't use DNS as a default namespace require you to add explicit package repositories, which is no more or less onerous than the replace directive.

nzach 6y ago

As others have already pointed out you don't need to use Github. Some people call it vanity imports[0]. In my experience 'vanity imports' give better results when googling stuff about custom import paths. [0] - https://blog.bramp.net/post/2017/10/02/vanity-go-import-paths/ https://blog.bramp.net/post/2017/10/02/vanity-go-import-path...