3 ms·

This ain't great for Github Actions, as PRs might now fail as there are no secrets available, randomly depending on your runner's IP Also it still takes some g

This ain't great for Github Actions, as PRs might now fail as there are no secrets available, randomly depending on your runner's IP Also it still takes some gymnastics to optionally support docker creds in a workflow https://github.com/orgs/community/discussions/131321 https://github.com/orgs/community/discussions/131321

MrBuddyCasino 2y ago
OTOH, I don't understand by the big cloud platforms don't support caching, or at least make it easy. Azure pulling container dependencies on every build just feels rude.
- mmbleh 2y ago
  These platforms do cache quite a bit. It's just that there is a very high volume of traffic and a lot of it does update pretty frequently (or has to check for updates)
  - MrBuddyCasino 2y ago
    Are you saying they cache transparently? Because I haven't seen that mentioned in the docs.
- martypitt 2y ago
  Gitlab does this, and it works nicely
  - terminalbraid 2y ago
    Seconding, though it does require some setup at least for self-hosted. Gitlab also has a full container registry built in, so it's not difficult to pull the image you want, push it to gitlab, and reference that going forward.
- hahn-kev 2y ago
  Yeah I don't get why I have to setup caching myself for this kind of thing. Like wouldn't it be more efficient to do it lower down in their infra anyway?
- wkat4242 2y ago
  Yes rude and lazy on their part.
- InsomniacL 2y ago
  Is there any licence constraints? I.e Docker terms of service restrict distribution in this way? Is there any technical restraints? I.e Docker specify no-cache I expect Docker don't want their images cached and would want you to use their service and transform you in to a paying subscriber through limitations on free tier.
  - aragilar 2y ago
    Caching is called out on https://docs.docker.com/docker-hub/usage/manage/ https://docs.docker.com/docker-hub/usage/manage/. My feeling is the way the naming scheme was defined (and subsequent issues around modifying the default registry), docker wanted to try to lock people into using docker hub over allowing public mirrors to be set up easily. This failed, so they've needed to pivot somewhat to reduce their load.
    - InsomniacL 2y ago
      > This failed, so they've needed to pivot somewhat to reduce their load. Or it worked, and now through free tier limitations they get more people sign-up and more people taking paid subs
05 2y ago
Just mirror what you need to ghcr, I guess..
everfrustrated 2y ago
Unless something has changed, GitHub pays a license to Docker to allow their runners IP space to avoid the rate limit.
- baby_souffle 2y ago
  TIL! Can you link to a source for this? I am curious!
  - everfrustrated 2y ago
    May not be true anymore but was in the past when Docker first introduced the limits. Very hard to find anything definitive still left on the web. This is all I could find... https://github.com/actions/runner-images/issues/1445#issuecomment-713861495 https://github.com/actions/runner-images/issues/1445#issueco... https://github.com/actions/runner-images/issues/1445#issuecomment-714832439 https://github.com/actions/runner-images/issues/1445#issueco...
    - baby_souffle 2y ago
      Thanks! > Very hard to find anything definitive still left on the web Probably a lot happened behind closed doors so there probably wasn’t much to begin with.
- immibis 2y ago
  Which means GitHub is going subscription-only in the near or mid-term future. Will we repeat the same mistakes?
  - blitzar 2y ago
    Rate limit the number of lines of code changed or pulled per day for free accounts.
kylegalbraith 2y ago
As someone mentioned, GitHub has something to prevent this, but it's unclear (or at least undocumented) what. We at Depot [0] work around this by guaranteeing that a new runner brought online has a unique public IP address. Thus avoiding the need to login to Docker to pull anything. Subsequently, we also do the same unique public IP address for our Docker image build product as well. Which helps with doing image builds where you're pulling from base images, etc. [0] https://depot.dev https://depot.dev
Lammy 2y ago
I'm curious about this regarding GCP as well. I have a few Cloud Run Containers set up pulling their image directly from Docker Hub, then injecting my single config file from a Secrets-Manager-backed volume mount. That way I don't have to maintain my own package in GCP's Package Registry when the upstream project only publishes to Docker Hub

MrBuddyCasino 2y ago

OTOH, I don't understand by the big cloud platforms don't support caching, or at least make it easy. Azure pulling container dependencies on every build just feels rude.

mmbleh 2y ago
These platforms do cache quite a bit. It's just that there is a very high volume of traffic and a lot of it does update pretty frequently (or has to check for updates)
- MrBuddyCasino 2y ago
  Are you saying they cache transparently? Because I haven't seen that mentioned in the docs.
martypitt 2y ago
Gitlab does this, and it works nicely
- terminalbraid 2y ago
  Seconding, though it does require some setup at least for self-hosted. Gitlab also has a full container registry built in, so it's not difficult to pull the image you want, push it to gitlab, and reference that going forward.
hahn-kev 2y ago
Yeah I don't get why I have to setup caching myself for this kind of thing. Like wouldn't it be more efficient to do it lower down in their infra anyway?
wkat4242 2y ago
Yes rude and lazy on their part.
InsomniacL 2y ago
Is there any licence constraints? I.e Docker terms of service restrict distribution in this way? Is there any technical restraints? I.e Docker specify no-cache I expect Docker don't want their images cached and would want you to use their service and transform you in to a paying subscriber through limitations on free tier.
- aragilar 2y ago
  Caching is called out on https://docs.docker.com/docker-hub/usage/manage/ https://docs.docker.com/docker-hub/usage/manage/. My feeling is the way the naming scheme was defined (and subsequent issues around modifying the default registry), docker wanted to try to lock people into using docker hub over allowing public mirrors to be set up easily. This failed, so they've needed to pivot somewhat to reduce their load.
  - InsomniacL 2y ago
    > This failed, so they've needed to pivot somewhat to reduce their load. Or it worked, and now through free tier limitations they get more people sign-up and more people taking paid subs

05 2y ago

Just mirror what you need to ghcr, I guess..

everfrustrated 2y ago

Unless something has changed, GitHub pays a license to Docker to allow their runners IP space to avoid the rate limit.

baby_souffle 2y ago
TIL! Can you link to a source for this? I am curious!
- everfrustrated 2y ago
  May not be true anymore but was in the past when Docker first introduced the limits. Very hard to find anything definitive still left on the web. This is all I could find... https://github.com/actions/runner-images/issues/1445#issuecomment-713861495 https://github.com/actions/runner-images/issues/1445#issueco... https://github.com/actions/runner-images/issues/1445#issuecomment-714832439 https://github.com/actions/runner-images/issues/1445#issueco...
  - baby_souffle 2y ago
    Thanks! > Very hard to find anything definitive still left on the web Probably a lot happened behind closed doors so there probably wasn’t much to begin with.
immibis 2y ago
Which means GitHub is going subscription-only in the near or mid-term future. Will we repeat the same mistakes?
- blitzar 2y ago
  Rate limit the number of lines of code changed or pulled per day for free accounts.

kylegalbraith 2y ago

As someone mentioned, GitHub has something to prevent this, but it's unclear (or at least undocumented) what. We at Depot [0] work around this by guaranteeing that a new runner brought online has a unique public IP address. Thus avoiding the need to login to Docker to pull anything. Subsequently, we also do the same unique public IP address for our Docker image build product as well. Which helps with doing image builds where you're pulling from base images, etc. [0] https://depot.dev https://depot.dev

Lammy 2y ago

I'm curious about this regarding GCP as well. I have a few Cloud Run Containers set up pulling their image directly from Docker Hub, then injecting my single config file from a Secrets-Manager-backed volume mount. That way I don't have to maintain my own package in GCP's Package Registry when the upstream project only publishes to Docker Hub