3 ms·

As someone who maintains the registries we use globally at work, +1. I know people groan at running infrastructure, but the registry software is really well do

As someone who maintains the registries we use globally at work, +1. I know people groan at running infrastructure, but the registry software is really well documented and flexible. If you don't need to 'push', but only pull - configuring them as pull through caches is nice for availability and reliability -- while also saving from nickle/diming. They will get things from a configurable upstream, proxy.remoteurl. Contrary to what the documentation says, this can work with anything speaking the API. Not just Dockerhub. edit: My one criticism, it's not good from an HTTPS hardening perspective. It's functional, but audits find non-issues. You'll want nginx or something in front to ensure good HSTS header coverage for non-actionable requests, for example.

Yeroc 4y ago
All good points but while this saves you from the docker images disappearing it does nothing to solve the issue of those images no longer receiving important security updates and bug fixes going forward.
- bravetraveler 4y ago
  Indeed, buying time at most :) The situation just presented an opportunity for improvement, I don't intend to suggest it as a cure - but a good step! Edit: For anyone curious, our upstream is actually the same software somewhere else, utilized by CICD. That being the origin allows for pushes, with the pull-through caches being read-only by nature
WirelessGigabit 4y ago
The problem here is that the company I work at has started building these golden images, full of cruft and then no team gets allocated to maintain them.
tetha 4y ago
That's good to hear. So I'll just have to spend an hour or so tomorrow night ensuring our private pull-through registry is used on everything prod and the biggest explosion is averted. Images built by the company land in internal registries already, so that's fine as well. Means, it's mostly a question of, (a) checking for image squatting on the hub after orgs get deleted, which I don't know how to deal with just yet (could I just null-route the docker hub on my registry until evaluated and we just don't get new images?), and (b) ruffling through all of our container systems to see where people use what image to figure out which are verified, or paying, or obsoleted, and where they went, or what is going on. That'll be great fun.
- bravetraveler 4y ago
  The typical Docker registry software when configured as a 'pull through' doesn't allow for pushes, if memory serves. That may be an important consideration while handling the situation We run them in 'maintenance mode' just to be absolutely sure anything the upstream doesn't have (or had at one point) is permitted in! Though, I don't think they'll allow pushes anyway with 'proxy.remoteurl' defined. I'm not sure I followed your setup properly, but with the private registry defined as your 'proxy.remoteurl', you shouldn't have to worry about the Hub in particular - unless it's looking there, or people are pushing bad things into it
  - tetha 4y ago
    > I'm not sure I followed your setup properly, but with the private registry defined as your 'proxy.remoteurl', you shouldn't have to worry about the Hub in particular - unless it's looking there, or people are pushing bad things into it That is exactly the thing I am worried about, as we have a pull-through mirror for the docker hub. What happens if some goofus container from that chaotic team pulls in knownOSS/component, but knownOSS got deleted and - after 30 days of available recon by _all_ malicious teams on the planet - got squatted instantly afterwards with rather vile malware? Spend some pennies to make a dollar by getting into a lot of systems. Obviously, you can throw a million shoulds at me, shouldn't do that, should rename + vendor and such (though how would you validate the image you mirror.), but that's a messy thing to deal with and I am wondering about a centralized way to block it without needing anyone but the registry/mirror admins.
    - bravetraveler 4y ago
      Ah I see! I misunderstood, didn't realize that it's pointing to the Hub. I assumed the more strict sense of 'private' :) The Docker-provided registry software is limited in terms of "don't go here". You get all of upstream, essentially Quay or Harbor are more configurable in that regard, but I'm less familiar. We're privileged, being already very-offline and signature heavy... and that's someone else. I just run the systems/services!

Yeroc 4y ago

All good points but while this saves you from the docker images disappearing it does nothing to solve the issue of those images no longer receiving important security updates and bug fixes going forward.

bravetraveler 4y ago
Indeed, buying time at most :) The situation just presented an opportunity for improvement, I don't intend to suggest it as a cure - but a good step! Edit: For anyone curious, our upstream is actually the same software somewhere else, utilized by CICD. That being the origin allows for pushes, with the pull-through caches being read-only by nature

WirelessGigabit 4y ago

The problem here is that the company I work at has started building these golden images, full of cruft and then no team gets allocated to maintain them.

tetha 4y ago

That's good to hear. So I'll just have to spend an hour or so tomorrow night ensuring our private pull-through registry is used on everything prod and the biggest explosion is averted. Images built by the company land in internal registries already, so that's fine as well. Means, it's mostly a question of, (a) checking for image squatting on the hub after orgs get deleted, which I don't know how to deal with just yet (could I just null-route the docker hub on my registry until evaluated and we just don't get new images?), and (b) ruffling through all of our container systems to see where people use what image to figure out which are verified, or paying, or obsoleted, and where they went, or what is going on. That'll be great fun.

bravetraveler 4y ago
The typical Docker registry software when configured as a 'pull through' doesn't allow for pushes, if memory serves. That may be an important consideration while handling the situation We run them in 'maintenance mode' just to be absolutely sure anything the upstream doesn't have (or had at one point) is permitted in! Though, I don't think they'll allow pushes anyway with 'proxy.remoteurl' defined. I'm not sure I followed your setup properly, but with the private registry defined as your 'proxy.remoteurl', you shouldn't have to worry about the Hub in particular - unless it's looking there, or people are pushing bad things into it
- tetha 4y ago
  > I'm not sure I followed your setup properly, but with the private registry defined as your 'proxy.remoteurl', you shouldn't have to worry about the Hub in particular - unless it's looking there, or people are pushing bad things into it That is exactly the thing I am worried about, as we have a pull-through mirror for the docker hub. What happens if some goofus container from that chaotic team pulls in knownOSS/component, but knownOSS got deleted and - after 30 days of available recon by _all_ malicious teams on the planet - got squatted instantly afterwards with rather vile malware? Spend some pennies to make a dollar by getting into a lot of systems. Obviously, you can throw a million shoulds at me, shouldn't do that, should rename + vendor and such (though how would you validate the image you mirror.), but that's a messy thing to deal with and I am wondering about a centralized way to block it without needing anyone but the registry/mirror admins.
  - bravetraveler 4y ago
    Ah I see! I misunderstood, didn't realize that it's pointing to the Hub. I assumed the more strict sense of 'private' :) The Docker-provided registry software is limited in terms of "don't go here". You get all of upstream, essentially Quay or Harbor are more configurable in that regard, but I'm less familiar. We're privileged, being already very-offline and signature heavy... and that's someone else. I just run the systems/services!