6 ms·

This has been a major Docker pain point, and not many people know about this trick. I didn't know you could have the variables in the Compose file directly, doe

This has been a major Docker pain point, and not many people know about this trick. I didn't know you could have the variables in the Compose file directly, does that really work? Our approach so far was to add yet another layer (a script to pass uid/gid to Compose), but if we don't need the script that would be fantastic. EDIT: Ah, I just saw the bashrc wrinkle you mention. Yeah, that's why we had the script, and it's a damn shame Docker can't do this natively. It has been a major hassle.

nickjj 5y ago
> I didn't know you could have the variables in the Compose file directly, does that really work? Yep, it's because the build args get read in from a .env file by default and then from there Docker Compose sends those build args to Docker when it builds the image. This was one of the topics from my talk at DockerCon last week (creating a production ready Docker Compose set up). The video and 6,000 word blog post for it will be coming out tomorrow. Both things will be added to the talk's reference links at https://github.com/nickjj/dockercon21-docker-best-practices https://github.com/nickjj/dockercon21-docker-best-practices.
- StavrosK 5y ago
  That's interesting, thanks! My shell sets the USER variable (but no USERID or GROUPID), which might be good enough for all our developers, but probably not reliable enough for a general audience.
  - nickjj 5y ago
    Honestly in practice everything tends to work fine without any hacks or extra scripts. I run all of my containers as a non-root user and create the user in the image with its default values of 1000:1000 for the uid:gid. I haven't bothered to expose the uid:gid as build arguments because it's pretty much never an issue in development or production. With a uid:gid of 1000:1000 built into the image any bind mounted files end up being correctly owned by the Docker host's user under the following conditions: - Docker Desktop on macOS - Docker Desktop on Windows using WSL 1 - Docker Desktop on Windows using WSL 2 and native Linux (as long as your dev box's user is set to 1000:1000) IMO it's really rare that your dev box's user wouldn't be 1000:1000 on native Linux or WSL 2. In production you also have full control over the uid:gid of your deploy user. The only time where it kind of stinks is CI, but it's super easy to get around this by simply not using volumes in CI. I have a bunch of examples of this pattern at: - https://github.com/nickjj/docker-flask-example - https://github.com/nickjj/docker-django-example - https://github.com/nickjj/docker-rails-example - https://github.com/nickjj/docker-phoenix-example - https://github.com/nickjj/docker-node-example - https://github.com/oleksandra-holovina/docker-play-example
    - q3k 5y ago
      > IMO it's really rare that your dev box's user wouldn't be 1000:1000 on native Linux or WSL 2. Any company-wide (GNU/)Linux deployment that uses LDAP or some other centralized user directory will not have devs with UID/GID 1000:1000. Hope is not a strategy.
      - nickjj 5y ago
        > Any company-wide (GNU/)Linux deployment that uses LDAP... You can go the extra mile and turn the UID:GID into build args like the original parent and you're good to go. No hacks necessary, and since it's all self contained into a .env file there's nothing extra you need to run since you're likely using an .env file already for other vars. Alternatively you could do this: https://news.ycombinator.com/item?id=27344491 https://news.ycombinator.com/item?id=27344491 In either case you can solve the problem without too much effort.
        q3k 5y ago
        > You can go the extra mile and turn the UID:GID into build args like the original parent and you're good to go. That doesn't help you if you're attempting to use pre-built/existing Docker images that are not built internally and make the assumption that “1000:1000 is good enough”. You then not only have to hack around Docker limitations, but also around someone else's broken assumption.
        nickjj 5y ago
        > That doesn't help you if you're attempting to use pre-built/existing Docker images that are not built internally Most pre-built images that I've come across don't require bind mounts to function. Images like PostgreSQL aren't affected by this because you can use a named volume, and most pre-built applications that are shipped as images tend to store their state in a database and don't require bind mounts to function.
    - mschuster91 5y ago
      > IMO it's really rare that your dev box's user wouldn't be 1000:1000 on native Linux or WSL 2. Any major company using LDAP/AD or other forms of centralized user management won't be able to make that guarantee. > In production you also have full control over the uid:gid of your deploy user. If you're running in an un-managed environment, yes - managed hosting of any kind generally doesn't provide these guarantees.

nickjj 5y ago

> I didn't know you could have the variables in the Compose file directly, does that really work? Yep, it's because the build args get read in from a .env file by default and then from there Docker Compose sends those build args to Docker when it builds the image. This was one of the topics from my talk at DockerCon last week (creating a production ready Docker Compose set up). The video and 6,000 word blog post for it will be coming out tomorrow. Both things will be added to the talk's reference links at https://github.com/nickjj/dockercon21-docker-best-practices https://github.com/nickjj/dockercon21-docker-best-practices.

StavrosK 5y ago
That's interesting, thanks! My shell sets the USER variable (but no USERID or GROUPID), which might be good enough for all our developers, but probably not reliable enough for a general audience.
- nickjj 5y ago
  Honestly in practice everything tends to work fine without any hacks or extra scripts. I run all of my containers as a non-root user and create the user in the image with its default values of 1000:1000 for the uid:gid. I haven't bothered to expose the uid:gid as build arguments because it's pretty much never an issue in development or production. With a uid:gid of 1000:1000 built into the image any bind mounted files end up being correctly owned by the Docker host's user under the following conditions: - Docker Desktop on macOS - Docker Desktop on Windows using WSL 1 - Docker Desktop on Windows using WSL 2 and native Linux (as long as your dev box's user is set to 1000:1000) IMO it's really rare that your dev box's user wouldn't be 1000:1000 on native Linux or WSL 2. In production you also have full control over the uid:gid of your deploy user. The only time where it kind of stinks is CI, but it's super easy to get around this by simply not using volumes in CI. I have a bunch of examples of this pattern at: - https://github.com/nickjj/docker-flask-example - https://github.com/nickjj/docker-django-example - https://github.com/nickjj/docker-rails-example - https://github.com/nickjj/docker-phoenix-example - https://github.com/nickjj/docker-node-example - https://github.com/oleksandra-holovina/docker-play-example
  - q3k 5y ago
    > IMO it's really rare that your dev box's user wouldn't be 1000:1000 on native Linux or WSL 2. Any company-wide (GNU/)Linux deployment that uses LDAP or some other centralized user directory will not have devs with UID/GID 1000:1000. Hope is not a strategy.
    - nickjj 5y ago
      > Any company-wide (GNU/)Linux deployment that uses LDAP... You can go the extra mile and turn the UID:GID into build args like the original parent and you're good to go. No hacks necessary, and since it's all self contained into a .env file there's nothing extra you need to run since you're likely using an .env file already for other vars. Alternatively you could do this: https://news.ycombinator.com/item?id=27344491 https://news.ycombinator.com/item?id=27344491 In either case you can solve the problem without too much effort.
      - q3k 5y ago
        > You can go the extra mile and turn the UID:GID into build args like the original parent and you're good to go. That doesn't help you if you're attempting to use pre-built/existing Docker images that are not built internally and make the assumption that “1000:1000 is good enough”. You then not only have to hack around Docker limitations, but also around someone else's broken assumption.
        nickjj 5y ago
        > That doesn't help you if you're attempting to use pre-built/existing Docker images that are not built internally Most pre-built images that I've come across don't require bind mounts to function. Images like PostgreSQL aren't affected by this because you can use a named volume, and most pre-built applications that are shipped as images tend to store their state in a database and don't require bind mounts to function.
  - mschuster91 5y ago
    > IMO it's really rare that your dev box's user wouldn't be 1000:1000 on native Linux or WSL 2. Any major company using LDAP/AD or other forms of centralized user management won't be able to make that guarantee. > In production you also have full control over the uid:gid of your deploy user. If you're running in an un-managed environment, yes - managed hosting of any kind generally doesn't provide these guarantees.