3 ms·

What's the easiest way to cache registries like docker, pypi, and npm these days?

by esafak 1y ago

viraptor 1y ago
You pull the images you want to use, preferably with some automated process, then push them to your own repo. And anyways use your own repo when pulling for dev/production. It saves you from images disappearing as well.
- paulddraper 1y ago
  What do you like using for your own repo? Artifactory? Something else?
  - __turbobrew__ 1y ago
    Note, artifactory SaaS had downtime today as well.
  - GuinansEyebrows 1y ago
    I have experience with ECR. If you’re in the AWS ecosystem it does the job.
    - paulddraper 1y ago
      You've now replaced one service with another :) But maybe ECR is better than Docker Hub.
      - GuinansEyebrows 1y ago
        you're not wrong :) and i can't claim to have stress tested either.
  - KronisLV 1y ago
    There is Sonatype Nexus. A bit annoying to administer (automated cleanup works every time, 60% of the time), but supports most package formats (Maven, npm, NuGet and so on) alongside offering Docker registries, both hosted and proxy ones. Also can be deployed as a container itself.
lambda 1y ago
The images I use the most, we pull and push to our own internal registry, so we have full control. There are still some we pull from Docker Hub, especially in the build process of our own images. To work around that, on AWS, you can prefix the image with public.ecr.aws/docker/library/ for example public.ecr.aws/docker/library/python:3.12 and it will pull from AWS's mirror of Docker Hub.
- doctoboggan 1y ago
  > To work around that, on AWS, you can prefix the image with public.ecr.aws/docker/library/ I believe anyone can pull from the public ecr, not just clients in AWS
  - lambda 1y ago
    Yeah, anyone can, but I think you'll be rate limited if you're not on AWS. So fine for a one off, but not something to put in your CI.
holysoles 1y ago
Another reply had some good insight: https://news.ycombinator.com/item?id=45368092 https://news.ycombinator.com/item?id=45368092
pm90 1y ago
Someone mentioned Artifactory; but its honestly not needed. I would very highly recommend an architecture where you build everything into a docker image and push it to an internal container registry (like ecr; all public clouds have one) for all production deployments. This way, outages only affect your build/deploy pipeline.

What's the easiest way to cache registries like docker, pypi, and npm these days?

by esafak 1y ago

What's the easiest way to cache registries like docker, pypi, and npm these days?

You pull the images you want to use, preferably with some automated process, then push them to your own repo. And anyways use your own repo when pulling for dev/production. It saves you from images disappearing as well.

paulddraper 1y ago
What do you like using for your own repo? Artifactory? Something else?
- __turbobrew__ 1y ago
  Note, artifactory SaaS had downtime today as well.
- GuinansEyebrows 1y ago
  I have experience with ECR. If you’re in the AWS ecosystem it does the job.
  - paulddraper 1y ago
    You've now replaced one service with another :) But maybe ECR is better than Docker Hub.
    - GuinansEyebrows 1y ago
      you're not wrong :) and i can't claim to have stress tested either.
- KronisLV 1y ago
  There is Sonatype Nexus. A bit annoying to administer (automated cleanup works every time, 60% of the time), but supports most package formats (Maven, npm, NuGet and so on) alongside offering Docker registries, both hosted and proxy ones. Also can be deployed as a container itself.

lambda 1y ago

The images I use the most, we pull and push to our own internal registry, so we have full control. There are still some we pull from Docker Hub, especially in the build process of our own images. To work around that, on AWS, you can prefix the image with public.ecr.aws/docker/library/ for example public.ecr.aws/docker/library/python:3.12 and it will pull from AWS's mirror of Docker Hub.

doctoboggan 1y ago
> To work around that, on AWS, you can prefix the image with public.ecr.aws/docker/library/ I believe anyone can pull from the public ecr, not just clients in AWS
- lambda 1y ago
  Yeah, anyone can, but I think you'll be rate limited if you're not on AWS. So fine for a one off, but not something to put in your CI.

holysoles 1y ago

Another reply had some good insight: https://news.ycombinator.com/item?id=45368092 https://news.ycombinator.com/item?id=45368092

pm90 1y ago

Someone mentioned Artifactory; but its honestly not needed. I would very highly recommend an architecture where you build everything into a docker image and push it to an internal container registry (like ecr; all public clouds have one) for all production deployments. This way, outages only affect your build/deploy pipeline.