3 ms·

In my README I explain how to setup the Docker context over SSH. In my system all of the files get written to the volume from only three places: * From th

In my README I explain how to setup the Docker context over SSH. In my system all of the files get written to the volume from only three places: * From the docker image through VOLUME (fresh volumes copy the data from the image on start) * From a template container that writes config files. * From the container itself, writing files as it runs. What I don't do is create a directory someplace and manually edit files and mount them. When I run `docker build` on my laptop, this does copy files to the server (Docker designed build this way, and you have to set .dockerignore file to ignore files you dont want copied).

Terretta 4y ago
What if the docker daemon is on a storage server and the host volume of /stuff contains, say, 10 terabytes of photo album content? > If you tell me to run > > docker run -v ${HOME}/stuff:/stuff alpine > > It will mount /home/stuff on the server, not my own > home directory on my laptop. That seems about right? And then you note: > What I don't do is create a directory someplace and manually edit files and mount them. But if /stuff is photos, and another container, say, runs ingestion tools, or some other photo collection processing, you don't let it touch the same data volume? Looking at your repo, I see your docker-compose volumes map e.g. data to data … volumes: - data:/data … which is what I do, so I guess I'm not following what you're saying to do differently. For instance, mounting a volume that can be edited by other containers lets me insta-move large files or sets of files between steps of containers, by container a doing a move not copy from its work path to its destination path watched as an incoming path by container b.
- EnigmaCurry 4y ago
  > What if the docker daemon is on a storage server and the host volume of /stuff contains, say, 10 terabytes of photo album content? In this extreme example I think probably a bind mount might make sense, especially if the files are already there. But the named volume would just be stored in /var/lib/docker/volumes/some-volume-name, so as long as that /var/lib has 10TB I don't see the problem. I can use my sftp container [1] to be able to sftp directly into a volume, but I've not yet transferred 10TB with it :) [1] https://github.com/EnigmaCurry/d.rymcg.tech/tree/master/sftp https://github.com/EnigmaCurry/d.rymcg.tech/tree/master/sftp
  - Terretta 4y ago
    docker etc is on a flash partition, data is on a few 16 drive arrays… … but this made me discover this: docker volume create \ --driver local \ --opt type=cifs \ --opt device=//uxxxxx.your-server.de/backup \ --opt o=addr=uxxxxx.your-server.de,\ username=uxxxxxxx,\ password=*****,\ file_mode=0777,dir_mode=0777 \ --name cif-volume So that simplifies a lot for my use cases. Thank you for all the replies!
    - EnigmaCurry 4y ago
      Thats really cool, I've only ever used the local driver with default settings, which stores to /var/lib/docker, but I guess theres different storage drivers that do different sorts of storage. nice find!
- EnigmaCurry 4y ago
  > Looking at your repo, I see your docker-compose volumes map e.g. data to data … volumes: - data:/data Theres two ways to mount a volume, one with a / and one without. /some/directory:data some-volume:data The first mounts a directory The second mounts a named volume. I suggest doing the second so that it can be maintained directly by docker through `docker volume create|rm` or `docker compose up|down [-v]`.
- EnigmaCurry 4y ago
  Replying in series as I see more of your edits: > For instance, mounting a volume that can be edited by other containers lets me insta-move large files or sets of files between steps of containers, As long as the containers are on the same docker host, many containers can mount the same volume.

Terretta 4y ago

What if the docker daemon is on a storage server and the host volume of /stuff contains, say, 10 terabytes of photo album content? > If you tell me to run > > docker run -v ${HOME}/stuff:/stuff alpine > > It will mount /home/stuff on the server, not my own > home directory on my laptop. That seems about right? And then you note: > What I don't do is create a directory someplace and manually edit files and mount them. But if /stuff is photos, and another container, say, runs ingestion tools, or some other photo collection processing, you don't let it touch the same data volume? Looking at your repo, I see your docker-compose volumes map e.g. data to data … volumes: - data:/data … which is what I do, so I guess I'm not following what you're saying to do differently. For instance, mounting a volume that can be edited by other containers lets me insta-move large files or sets of files between steps of containers, by container a doing a move not copy from its work path to its destination path watched as an incoming path by container b.

EnigmaCurry 4y ago
> What if the docker daemon is on a storage server and the host volume of /stuff contains, say, 10 terabytes of photo album content? In this extreme example I think probably a bind mount might make sense, especially if the files are already there. But the named volume would just be stored in /var/lib/docker/volumes/some-volume-name, so as long as that /var/lib has 10TB I don't see the problem. I can use my sftp container [1] to be able to sftp directly into a volume, but I've not yet transferred 10TB with it :) [1] https://github.com/EnigmaCurry/d.rymcg.tech/tree/master/sftp https://github.com/EnigmaCurry/d.rymcg.tech/tree/master/sftp
- Terretta 4y ago
  docker etc is on a flash partition, data is on a few 16 drive arrays… … but this made me discover this: docker volume create \ --driver local \ --opt type=cifs \ --opt device=//uxxxxx.your-server.de/backup \ --opt o=addr=uxxxxx.your-server.de,\ username=uxxxxxxx,\ password=*****,\ file_mode=0777,dir_mode=0777 \ --name cif-volume So that simplifies a lot for my use cases. Thank you for all the replies!
  - EnigmaCurry 4y ago
    Thats really cool, I've only ever used the local driver with default settings, which stores to /var/lib/docker, but I guess theres different storage drivers that do different sorts of storage. nice find!
EnigmaCurry 4y ago
> Looking at your repo, I see your docker-compose volumes map e.g. data to data … volumes: - data:/data Theres two ways to mount a volume, one with a / and one without. /some/directory:data some-volume:data The first mounts a directory The second mounts a named volume. I suggest doing the second so that it can be maintained directly by docker through `docker volume create|rm` or `docker compose up|down [-v]`.
EnigmaCurry 4y ago
Replying in series as I see more of your edits: > For instance, mounting a volume that can be edited by other containers lets me insta-move large files or sets of files between steps of containers, As long as the containers are on the same docker host, many containers can mount the same volume.