5 ms·

Does it start a unregistry container on the remote/receiving end or the local/sending end? I think that runs remotely. I wonder if you could go the other way in

by remram 1y ago

Does it start a unregistry container on the remote/receiving end or the local/sending end? I think that runs remotely. I wonder if you could go the other way instead?

selcuka 1y ago
You mean ssh'ing into the remote server, then pulling image from local? That would require your local host to be accessible from the remote host, or setting up some kind of ssh tunneling.
- mdaniel 1y ago
  `ssh -R` and `ssh -L` are amazing, and I just learned that -L and -R both support unix sockets on either end and also unix socket to tcp socket https://manpages.ubuntu.com/manpages/noble/man1/ssh.1.html#:~:text=specifies%20that%20connections%20to%20the%20given%20tcp%20port%20or%20unix%20socket%20on%20the%20local https://manpages.ubuntu.com/manpages/noble/man1/ssh.1.html#:... I would presume it's something akin to $(ssh -L /var/run/docker.sock:/tmp/d.sock sh -c 'docker -H unix:///tmp/d.sock save | docker load') type deal
- matt_kantor 1y ago
  This is what docker-pushmi-pullyu[1] does, using `ssh -R` as suggested by a sibling comment. [1]: https://github.com/mkantor/docker-pushmi-pullyu https://github.com/mkantor/docker-pushmi-pullyu
  - remram 1y ago
    That's also what the submitted tool does, I want to do the same thing just in the reverse direction. I just don't want to start extra containers on the prod machine.
    - selcuka 1y ago
      No, the second one (docker-pushmi-pullyu) runs the registry on the build host.
      - remram 1y ago
        I meant to reply to you, whoops. docker-pushmi-pullyu does an extra copy from build host to a registry, so it is just the standard workflow. I think Spegel does what I want (= serve images from the local cache as a registry), I might be able to build from that. It is meant to be integrated with Kubernetes though, making a simple transfer tool probably requires some adaptation.
  - psviderski 1y ago
    The problem with running a registry locally is that Docker doesn't provide an API to get individual image layers to be able to build a registry API on top. You have to hook into the containerd Docker uses under the hood. You can't do this locally in many cases, for example, on macOS the VM running Docker Desktop doesn't expose the containerd socket. I guess the workaround you implemented in docker-pushmi-pullyu is an extra copy to the registry which is a bummer.
    - matt_kantor 1y ago
      Yeah, a few years ago I remember looking into whether I could expose image layers from the engine as a volume to mount directly into the registry, but at least at the time it seemed complex, and when I write tools like this simplicity is a primary goal. As a mitigation docker-pushmi-pullyu caches pushed layers between runs[1]. More often than not I'm only changing upper layers of previously-pushed images, so this helps a lot. Also, since everything happens locally the push phase is typically quite fast even with cache misses (especially on an SSD), especially compared to the pull phase which is usually going over the internet (or another network). [1]: https://github.com/mkantor/docker-pushmi-pullyu/pull/19/files https://github.com/mkantor/docker-pushmi-pullyu/pull/19/file...
psviderski 1y ago
It starts an unregistry container on the remote side. I wonder, what's the use case on your mind for doing it the other way around?
- remram 1y ago
  I guess I feel a little dirty running the container on the prod server. My machine has all the dev tools, and it is also where I install and run this pussh tool, so I would rather have the container run there too.

by remram 1y ago

Does it start a unregistry container on the remote/receiving end or the local/sending end? I think that runs remotely. I wonder if you could go the other way instead?

selcuka 1y ago

You mean ssh'ing into the remote server, then pulling image from local? That would require your local host to be accessible from the remote host, or setting up some kind of ssh tunneling.

mdaniel 1y ago
`ssh -R` and `ssh -L` are amazing, and I just learned that -L and -R both support unix sockets on either end and also unix socket to tcp socket https://manpages.ubuntu.com/manpages/noble/man1/ssh.1.html#:~:text=specifies%20that%20connections%20to%20the%20given%20tcp%20port%20or%20unix%20socket%20on%20the%20local https://manpages.ubuntu.com/manpages/noble/man1/ssh.1.html#:... I would presume it's something akin to $(ssh -L /var/run/docker.sock:/tmp/d.sock sh -c 'docker -H unix:///tmp/d.sock save | docker load') type deal
matt_kantor 1y ago
This is what docker-pushmi-pullyu[1] does, using `ssh -R` as suggested by a sibling comment. [1]: https://github.com/mkantor/docker-pushmi-pullyu https://github.com/mkantor/docker-pushmi-pullyu
- remram 1y ago
  That's also what the submitted tool does, I want to do the same thing just in the reverse direction. I just don't want to start extra containers on the prod machine.
  - selcuka 1y ago
    No, the second one (docker-pushmi-pullyu) runs the registry on the build host.
    - remram 1y ago
      I meant to reply to you, whoops. docker-pushmi-pullyu does an extra copy from build host to a registry, so it is just the standard workflow. I think Spegel does what I want (= serve images from the local cache as a registry), I might be able to build from that. It is meant to be integrated with Kubernetes though, making a simple transfer tool probably requires some adaptation.
- psviderski 1y ago
  The problem with running a registry locally is that Docker doesn't provide an API to get individual image layers to be able to build a registry API on top. You have to hook into the containerd Docker uses under the hood. You can't do this locally in many cases, for example, on macOS the VM running Docker Desktop doesn't expose the containerd socket. I guess the workaround you implemented in docker-pushmi-pullyu is an extra copy to the registry which is a bummer.
  - matt_kantor 1y ago
    Yeah, a few years ago I remember looking into whether I could expose image layers from the engine as a volume to mount directly into the registry, but at least at the time it seemed complex, and when I write tools like this simplicity is a primary goal. As a mitigation docker-pushmi-pullyu caches pushed layers between runs[1]. More often than not I'm only changing upper layers of previously-pushed images, so this helps a lot. Also, since everything happens locally the push phase is typically quite fast even with cache misses (especially on an SSD), especially compared to the pull phase which is usually going over the internet (or another network). [1]: https://github.com/mkantor/docker-pushmi-pullyu/pull/19/files https://github.com/mkantor/docker-pushmi-pullyu/pull/19/file...

psviderski 1y ago

It starts an unregistry container on the remote side. I wonder, what's the use case on your mind for doing it the other way around?

remram 1y ago
I guess I feel a little dirty running the container on the prod server. My machine has all the dev tools, and it is also where I install and run this pussh tool, so I would rather have the container run there too.