6 ms·

Nice collection, although I don't really like binding volumes to host directories, because then you can't really use docker over SSH. I'm working on my own simi

Nice collection, although I don't really like binding volumes to host directories, because then you can't really use docker over SSH. I'm working on my own similar project here that exclusively uses docker named volumes: https://github.com/enigmaCurry/d.rymcg.tech https://github.com/enigmaCurry/d.rymcg.tech

sevg 4y ago
What do you mean by "can't really use docker over SSH"? I'm not sure what that means, or how it's related to bind mounts.
- EnigmaCurry 4y ago
  I mean using a remote Docker context over SSH, where I run `docker` commands on my laptop, but it runs the containers on a remote Docker server. My laptop does not run the docker daemon, so its just a client. If you tell me to run docker run -v ${HOME}/stuff:/stuff alpine It will mount /home/stuff on the server, not my own home directory on my laptop. I would have to run another process that rsync's my local /home/stuff to the server.
  - andix 4y ago
    Just run docker compose on the remote machine (and copy the compose file with scp). I don’t see a lot of benefits if you run docker compose on another machine than the docker daemon. You can even automate it with ansible, terraform, or similar, if you like it neat. Or just go with k8s if you need a more complex setup.
    - EnigmaCurry 4y ago
      That is an entirely valid approach, but this way it feels like the server requires more maintainance. I can destroy a docker named volume through the lifecycle of the container. But I cant delete a system directory. (edit: I mean using `docker compose down -v` it deletes a named volume, but not a system directory) Also, its really nice to be able to use `docker context use SERVER_NAME` so I can switch contexts (servers) very easily.
  - sevg 4y ago
    Thanks for the reply. I didn't know that was a thing. Though, I'm still confused by your example of having to rsync /home/stuff to the server. If you use a named Docker volume, is the remote Docker container somehow using a volume you have located on your laptop? Wouldn't you still have to transfer the volume from laptop to server?
    - EnigmaCurry 4y ago
      In my README I explain how to setup the Docker context over SSH. In my system all of the files get written to the volume from only three places: * From the docker image through VOLUME (fresh volumes copy the data from the image on start) * From a template container that writes config files. * From the container itself, writing files as it runs. What I don't do is create a directory someplace and manually edit files and mount them. When I run `docker build` on my laptop, this does copy files to the server (Docker designed build this way, and you have to set .dockerignore file to ignore files you dont want copied).
      - Terretta 4y ago
        What if the docker daemon is on a storage server and the host volume of /stuff contains, say, 10 terabytes of photo album content? > If you tell me to run > > docker run -v ${HOME}/stuff:/stuff alpine > > It will mount /home/stuff on the server, not my own > home directory on my laptop. That seems about right? And then you note: > What I don't do is create a directory someplace and manually edit files and mount them. But if /stuff is photos, and another container, say, runs ingestion tools, or some other photo collection processing, you don't let it touch the same data volume? Looking at your repo, I see your docker-compose volumes map e.g. data to data … volumes: - data:/data … which is what I do, so I guess I'm not following what you're saying to do differently. For instance, mounting a volume that can be edited by other containers lets me insta-move large files or sets of files between steps of containers, by container a doing a move not copy from its work path to its destination path watched as an incoming path by container b.
        4y ago
mythz 4y ago
> because then you can't really use docker over SSH Most of our App deployments are done with GHCR + SSH + Docker Compose with GitHub Actions on every commit [1] [1] https://docs.servicestack.net/ssh-github-action-deployment https://docs.servicestack.net/ssh-github-action-deployment
jacooper 4y ago
But names volumes kind of make backups harder + you will need bind mount for external config files anyway.
- EnigmaCurry 4y ago
  > kind of make backups harder How so? For me it makes it easier, just have to backup one directory: /var/lib/docker/volumes > you will need bind mount for external config files anyway I create all my config files from templates, generated by a container, with the config entirely driven by environment variables, and this runs before my main container (via `depends_on`) it writes the config to a named volume. So there are no "external" config files, only "internal" ones.

sevg 4y ago

What do you mean by "can't really use docker over SSH"? I'm not sure what that means, or how it's related to bind mounts.

EnigmaCurry 4y ago
I mean using a remote Docker context over SSH, where I run `docker` commands on my laptop, but it runs the containers on a remote Docker server. My laptop does not run the docker daemon, so its just a client. If you tell me to run docker run -v ${HOME}/stuff:/stuff alpine It will mount /home/stuff on the server, not my own home directory on my laptop. I would have to run another process that rsync's my local /home/stuff to the server.
- andix 4y ago
  Just run docker compose on the remote machine (and copy the compose file with scp). I don’t see a lot of benefits if you run docker compose on another machine than the docker daemon. You can even automate it with ansible, terraform, or similar, if you like it neat. Or just go with k8s if you need a more complex setup.
  - EnigmaCurry 4y ago
    That is an entirely valid approach, but this way it feels like the server requires more maintainance. I can destroy a docker named volume through the lifecycle of the container. But I cant delete a system directory. (edit: I mean using `docker compose down -v` it deletes a named volume, but not a system directory) Also, its really nice to be able to use `docker context use SERVER_NAME` so I can switch contexts (servers) very easily.
- sevg 4y ago
  Thanks for the reply. I didn't know that was a thing. Though, I'm still confused by your example of having to rsync /home/stuff to the server. If you use a named Docker volume, is the remote Docker container somehow using a volume you have located on your laptop? Wouldn't you still have to transfer the volume from laptop to server?
  - EnigmaCurry 4y ago
    In my README I explain how to setup the Docker context over SSH. In my system all of the files get written to the volume from only three places: * From the docker image through VOLUME (fresh volumes copy the data from the image on start) * From a template container that writes config files. * From the container itself, writing files as it runs. What I don't do is create a directory someplace and manually edit files and mount them. When I run `docker build` on my laptop, this does copy files to the server (Docker designed build this way, and you have to set .dockerignore file to ignore files you dont want copied).
    - Terretta 4y ago
      What if the docker daemon is on a storage server and the host volume of /stuff contains, say, 10 terabytes of photo album content? > If you tell me to run > > docker run -v ${HOME}/stuff:/stuff alpine > > It will mount /home/stuff on the server, not my own > home directory on my laptop. That seems about right? And then you note: > What I don't do is create a directory someplace and manually edit files and mount them. But if /stuff is photos, and another container, say, runs ingestion tools, or some other photo collection processing, you don't let it touch the same data volume? Looking at your repo, I see your docker-compose volumes map e.g. data to data … volumes: - data:/data … which is what I do, so I guess I'm not following what you're saying to do differently. For instance, mounting a volume that can be edited by other containers lets me insta-move large files or sets of files between steps of containers, by container a doing a move not copy from its work path to its destination path watched as an incoming path by container b.
      - 4y ago

mythz 4y ago

> because then you can't really use docker over SSH Most of our App deployments are done with GHCR + SSH + Docker Compose with GitHub Actions on every commit [1] [1] https://docs.servicestack.net/ssh-github-action-deployment https://docs.servicestack.net/ssh-github-action-deployment

jacooper 4y ago

But names volumes kind of make backups harder + you will need bind mount for external config files anyway.

EnigmaCurry 4y ago
> kind of make backups harder How so? For me it makes it easier, just have to backup one directory: /var/lib/docker/volumes > you will need bind mount for external config files anyway I create all my config files from templates, generated by a container, with the config entirely driven by environment variables, and this runs before my main container (via `depends_on`) it writes the config to a named volume. So there are no "external" config files, only "internal" ones.