4 ms·

I'm very conflicted about this. On the one hand, I recognize that there are potentially significant costs to be born to serve these repositories. On the other h

I'm very conflicted about this. On the one hand, I recognize that there are potentially significant costs to be born to serve these repositories. On the other hand, making docker part of your infrastructure requires a certain degree of availability. At some level this seems to me like using my IDE and after 6 hours it would stop working or finding that my CDNJS references to bootstrap stopped working after 6 hours of my site being up. I think it is exactly as if NPM or PIP were to stop working for the day if you included "too many" packages. I don't really have a good feeling for how these new limits might impact setting up new dev/test environments so I'll likely switch from using specific images to using generic images to limit my exposure. For example, at the moment, if I want a new dev container for a project in python, I'll FROM python:latest and supplement with pulls from support service containers like nginx:latest and postgres:latest. Moving forward, it would seem a safer approach will be to use pull a single 18.04 image and run the required installs into them. This super bums me out though as it seems to bypass some of the nicer aspects of getting up and running with docker.

geek_at 6y ago
Well to play devil's advocate if docker is part of your infrastructure and you need to make more than 200 pulls in 6 hours you should already host your own registry and maybe mirror a few dockerhub repos to your own
throwanem 6y ago
Do pulls for a "latest" tag bypass the cache?
- mikepurvis 6y ago
  No they don't, but if you do something silly like spin up a bare VM build node, and then pull your environment every time, then obviously you won't get any caching.
  - ArchOversight 6y ago
    Github Actions? Gitlab CI? Many of those don't do any caching because of the chance of poisoning a tag in the local registry, and don't have a good way to do caching per project. I can imagine that this affects those sorts of operations.
    - throwanem 6y ago
      Yeah, that's what I was getting at - in CI I can see it, but in local dev it would seem surprising to need more than the free-with-authentication quota. That said, if your CI needs more, it's probably time to invest in a paid account if you can, and certainly if you're commercial.
    - tr-gitlab 6y ago
      You can use the GitLab Dependency Proxy for caching images from DockerHub, it's pretty straightforward to pull cached images: https://docs.gitlab.com/ee/user/packages/dependency_proxy/ https://docs.gitlab.com/ee/user/packages/dependency_proxy/
  - atomi 6y ago
    Using `docker-machine` You can use an environment variable to pull from a local docker mirror from new bare VMs. `export ENGINE_REGISTRY_MIRROR=https://mirror.mysite.com` https://mirror.mysite.com`
- verdverm 6y ago
  No, latest is just a tag and is not handled differently. There is a CLI flag to ignore cached layers. This however does not effect the FROM line, so if you have the image, there will be no pull. Expect this to effect CI systems
- deleted 6y ago
  [deleted]
kzrdude 6y ago
It really sounds like a problem we just have to solve. Putting all the expectation on one central service is not reliable.. it should be a distributed network of content delivery nodes, of which you run a few yourself.
- pm90 6y ago
  I like this idea. I would think that the “default” hub should not be managed only by Docker, but eg by docker and other registries. I suspect that docker inc still wants full control over their registry though so I don’t see this happening. We might decide on another, non dockerhub open registry though, and use that instead.

geek_at 6y ago

Well to play devil's advocate if docker is part of your infrastructure and you need to make more than 200 pulls in 6 hours you should already host your own registry and maybe mirror a few dockerhub repos to your own

throwanem 6y ago

Do pulls for a "latest" tag bypass the cache?

mikepurvis 6y ago
No they don't, but if you do something silly like spin up a bare VM build node, and then pull your environment every time, then obviously you won't get any caching.
- ArchOversight 6y ago
  Github Actions? Gitlab CI? Many of those don't do any caching because of the chance of poisoning a tag in the local registry, and don't have a good way to do caching per project. I can imagine that this affects those sorts of operations.
  - throwanem 6y ago
    Yeah, that's what I was getting at - in CI I can see it, but in local dev it would seem surprising to need more than the free-with-authentication quota. That said, if your CI needs more, it's probably time to invest in a paid account if you can, and certainly if you're commercial.
  - tr-gitlab 6y ago
    You can use the GitLab Dependency Proxy for caching images from DockerHub, it's pretty straightforward to pull cached images: https://docs.gitlab.com/ee/user/packages/dependency_proxy/ https://docs.gitlab.com/ee/user/packages/dependency_proxy/
- atomi 6y ago
  Using `docker-machine` You can use an environment variable to pull from a local docker mirror from new bare VMs. `export ENGINE_REGISTRY_MIRROR=https://mirror.mysite.com` https://mirror.mysite.com`
verdverm 6y ago
No, latest is just a tag and is not handled differently. There is a CLI flag to ignore cached layers. This however does not effect the FROM line, so if you have the image, there will be no pull. Expect this to effect CI systems
deleted 6y ago
[deleted]

kzrdude 6y ago

It really sounds like a problem we just have to solve. Putting all the expectation on one central service is not reliable.. it should be a distributed network of content delivery nodes, of which you run a few yourself.

pm90 6y ago
I like this idea. I would think that the “default” hub should not be managed only by Docker, but eg by docker and other registries. I suspect that docker inc still wants full control over their registry though so I don’t see this happening. We might decide on another, non dockerhub open registry though, and use that instead.