12 ms·

Docker 19.03: Rootless Mode (Experimental)

AdamGibbins 7y ago
Or we could just ditch Docker for one of the alternatives, like Podman that doesn't need root, nor a daemon.
- zufallsheld 7y ago
  Comparing the shortcomings of rootless podman (https://github.com/containers/libpod/blob/master/rootless.md https://github.com/containers/libpod/blob/master/rootless.md) and rootless docker, they seem almost the same. So this argument may not count, the daemon argument however applies.
  - rnhmjoj 7y ago
    I wonder if we will ever get rid of the ludicrous limitation of the privileged ports. It's a mechanism that only provided some sense of security in the 80s. The W3C[1] says "if you connect to a service on one of these ports you can be fairly sure that you have the real thing, and not a fake which some hacker has put up for you." Well, in 2019 computers aren't mainframes run by institutions and hackers can be root of their own system and run whatever they want on port 22. It's such an incovenience that I'm sure it caused countless services to be unnecessarily run as root. [1]: https://www.w3.org/Daemon/User/Installation/PrivilegedPorts.html https://www.w3.org/Daemon/User/Installation/PrivilegedPorts....
    - yjftsjthsd-h 7y ago
      Looks like starting with Linux 4.11 you can: sysctl net.ipv4.ip_unprivileged_port_start=443 ( https://stackoverflow.com/questions/413807/is-there-a-way-for-non-root-processes-to-bind-to-privileged-ports-on-linux/51439516#51439516 https://stackoverflow.com/questions/413807/is-there-a-way-fo... )
- coffekaesque 7y ago
  I don't like their idea of what a docker-compose replacement should be. And reading issues and limitations about podman pod commands is very discouraging. I would love to hear what others are using and their experiences though. I avoid anything Kubernetes because of a personal bias.
  - uponcoffee 7y ago
    Would you be willing to elaborate on the reason why you avoid kubernetes?
    - coffekaesque 7y ago
      Like I said it's a personal bias, mainly about Google. The only time I had to use it was with RH's cloud and if it wasn't for their good documentation I would have dropped the client. Everytime I looked under the hood it reminded why I hate being a developer around 35% of the time.
      - orf 7y ago
        That’s a rather bizarre reason to avoid a pretty solid system.
        yebyen 7y ago
        But also a pretty solid testament to the credit of RedHat's documentation, which I'll echo myself. My OpenShift experience is limited to 2017, but I've never heard anything but positive things about OpenShift's documentation, and I heard it has moved a lot closer to mainline Kubernetes since.
    - tonyarkles 7y ago
      I'm not the OP, but... Here's the scene: Most of the web projects I work on will never have a billion users. They might have 5, or 10. One or two have thousands. Several of them have 1 (me). Docker-compose works for me. I set up a container for my backend, a container for whatever's serving the static resources for the frontend, and a container for whatever databases are needed (Postgres, Redis, whatever). The databases get a filesystem volume mount that I can snapshot off the disk with a nightly cron job. I have a script that will transform a brand shiny new $5/mo DigitalOcean Ubuntu image into a machine with nginx+LetsEncrypt for SSL termination, and with Docker and docker-compose installed (and the Docker port firewalled off, natch). From there, I run "docker-compose up -d" and my project fires up and goes. Maybe I have to edit a line or two in the nginx.conf that my script put in place. To deploy, I do a local build on my laptop (or Jenkins for a few projects where it makes sense) via a script that pushes the built containers to Docker Hub, and runs docker-compose pull on the host. This has served me beautifully. I've looked at Kube more than once. It looks cool for things dramatically bigger than what I'm working on. For something that isn't massive scale, it's bloody complicated. If one of these projects ever gets to the point where a $40/mo DigitalOcean box can't handle the load, I'll probably look at it again. Until then, though, it feels like a very expensive (time-wise) premature optimization.
- paulddraper 7y ago
  Yep. I only half care about rootless. I definitely care about the daemon. It sucks. It flies in the face of traditional Linux process management where child processes are child processes. (Unless you want an init system, where you need a daemon. But docker is a sucky init system.) Docker breaks even the most basic things. $ time docker run some heavy computation Oh wait, that doesn't work.
  - the8472 7y ago
    For the trivial case a child process would work. But ultimately docker does try to be closer to an init system or maybe screen since you can detach/attach to processes. Since reparenting to arbitrary processes is not possible in linux it's also not possible to retain the parent-child relationship for spawned containers. If you want the fork-exec model then docker is indeed the wrong tool for the job.
    - jchw 7y ago
      The thing is, I don't even really like Docker as an init daemon. I have my gripes about Systemd but I see no downsides to not having a long-running daemon for a container engine. Really, whether you need root or not isn't even the most important issue; you can do sudo or suid or whatever with any container engine; Docker just has it be an implicit, unintuitive behavior. I used to use systemd+rkt for simple container setups when I didn't need all of Kubernetes. Never noticed any downsides versus using Docker, but on the flip side, I had far fewer issues with my containers not properly starting at boot.
      - paulddraper 7y ago
        I read an article (can't find it now) that said from a previous project the Docker authors concluded they wanted a daemon so they didn't have to do things like file locks, etc. around image management. Don't know if that accounts for the whole reason or not.
        jchw 7y ago
- meddlepal 7y ago
  Except that then you lose MacOS and Windows compatibility which is somewhat important to Docker.
- entropy1111 7y ago
  What are my options to replace Docker Compose? I dont want to introduce a chaotic mess by using kubernetes. Or to dedicate brain power to learn what they changed every week. Their readme really confuses me with podman play, kompose, k8s.
  - Conan_Kudo 7y ago
    There is an implementation of docker compose for podman in development: https://github.com/muayyad-alsadi/podman-compose https://github.com/muayyad-alsadi/podman-compose
    - entropy1111 7y ago
      >it's still underdevelopment and does not work yet. >https://github.com/muayyad-alsadi/podman-compose/issues/13 https://github.com/muayyad-alsadi/podman-compose/issues/13 Hard to trust a repo like that for production. Unless Red Hat provides an alternative I won't be able to use podman. IIRC they said it was too hard to keep up with K8S changes so even an abstraction for that would be too costly.
- Gondolin 7y ago
  Now that systemd-nspawn also has oci support, I wonder if podman/cri-o are going to switch to nspawn rather than runc.
ohiovr 7y ago
Docker has supported namespaces for a while now so that even if the user in the container is root it could be a subordinate id on the host with no administrative authority. What is new though?
- techntoke 7y ago
  That still required the daemon run as root. This runs the daemon rootless as well.
- cyphar 7y ago
  The daemon is running as an unprivileged user. Docker with userns-remap is still running as root (and recent vulnerabilities like CVE-2018-15664 are still a significant worry even if you ran with user namespaces enabled).
a-ve 7y ago
To the container wizards: Is it possible to orchestrate lxc containers using kubernetes? I've been looking at lxc containers for a while and really would not like to run Docker as root.
- cyphar 7y ago
  LXD has orchestration support natively, though it's not at all like Kubernetes (you are manually moving containers around and so on). I have heard that some folks have looked into using LXC under Kubernetes (and theoretically the OCI templates for LXC could possibly make this somewhat work) but there isn't an obvious way to do that today AFAIK. And I'm not convinced (given CNI which touches some deep bits of runc's particular behaviour) it would work with everything you'd want it to.
  - bboreham 7y ago
    > given CNI which touches some deep bits of runc's particular behaviour Could you explain? I’m a maintainer of CNI and I know almost nothing about runc, so I’m not clear where they touch. The first CNI implementation came out of rkt, pre-dating runc by a year or two.
    - cyphar 7y ago
      I mean that it likely makes certain assumptions about how runc sets up containers which aren't true of LXC. I'm sure you could get CNI to work with LXC (in fact, someone might've already done that -- I'm not sure tbh) but it wouldn't be something you'd be able to drop-in without at least a bit of extra work. For instance, LXC's hooks run in different contexts to OCI hooks (though we recently discovered that runc runs hooks in different contexts to the OCI spec, but that's another topic). And historically, yes it might predate the runc binary but the libcontainer code that is now part of runc predated CNI -- and all of our fun idiosyncrasies are in libcontainer.
      - bboreham 7y ago
        > likely makes certain assumptions about how runc sets up containers Nope. CNI takes as parameters a “container ID” (any string) and a network namespace path. No knowledge is needed or implied about how those things fit with actual containers.

Or we could just ditch Docker for one of the alternatives, like Podman that doesn't need root, nor a daemon.

zufallsheld 7y ago
Comparing the shortcomings of rootless podman (https://github.com/containers/libpod/blob/master/rootless.md https://github.com/containers/libpod/blob/master/rootless.md) and rootless docker, they seem almost the same. So this argument may not count, the daemon argument however applies.
- rnhmjoj 7y ago
  I wonder if we will ever get rid of the ludicrous limitation of the privileged ports. It's a mechanism that only provided some sense of security in the 80s. The W3C[1] says "if you connect to a service on one of these ports you can be fairly sure that you have the real thing, and not a fake which some hacker has put up for you." Well, in 2019 computers aren't mainframes run by institutions and hackers can be root of their own system and run whatever they want on port 22. It's such an incovenience that I'm sure it caused countless services to be unnecessarily run as root. [1]: https://www.w3.org/Daemon/User/Installation/PrivilegedPorts.html https://www.w3.org/Daemon/User/Installation/PrivilegedPorts....
  - yjftsjthsd-h 7y ago
    Looks like starting with Linux 4.11 you can: sysctl net.ipv4.ip_unprivileged_port_start=443 ( https://stackoverflow.com/questions/413807/is-there-a-way-for-non-root-processes-to-bind-to-privileged-ports-on-linux/51439516#51439516 https://stackoverflow.com/questions/413807/is-there-a-way-fo... )
coffekaesque 7y ago
I don't like their idea of what a docker-compose replacement should be. And reading issues and limitations about podman pod commands is very discouraging. I would love to hear what others are using and their experiences though. I avoid anything Kubernetes because of a personal bias.
- uponcoffee 7y ago
  Would you be willing to elaborate on the reason why you avoid kubernetes?
  - coffekaesque 7y ago
    Like I said it's a personal bias, mainly about Google. The only time I had to use it was with RH's cloud and if it wasn't for their good documentation I would have dropped the client. Everytime I looked under the hood it reminded why I hate being a developer around 35% of the time.
    - orf 7y ago
      That’s a rather bizarre reason to avoid a pretty solid system.
      - yebyen 7y ago
        But also a pretty solid testament to the credit of RedHat's documentation, which I'll echo myself. My OpenShift experience is limited to 2017, but I've never heard anything but positive things about OpenShift's documentation, and I heard it has moved a lot closer to mainline Kubernetes since.
  - tonyarkles 7y ago
    I'm not the OP, but... Here's the scene: Most of the web projects I work on will never have a billion users. They might have 5, or 10. One or two have thousands. Several of them have 1 (me). Docker-compose works for me. I set up a container for my backend, a container for whatever's serving the static resources for the frontend, and a container for whatever databases are needed (Postgres, Redis, whatever). The databases get a filesystem volume mount that I can snapshot off the disk with a nightly cron job. I have a script that will transform a brand shiny new $5/mo DigitalOcean Ubuntu image into a machine with nginx+LetsEncrypt for SSL termination, and with Docker and docker-compose installed (and the Docker port firewalled off, natch). From there, I run "docker-compose up -d" and my project fires up and goes. Maybe I have to edit a line or two in the nginx.conf that my script put in place. To deploy, I do a local build on my laptop (or Jenkins for a few projects where it makes sense) via a script that pushes the built containers to Docker Hub, and runs docker-compose pull on the host. This has served me beautifully. I've looked at Kube more than once. It looks cool for things dramatically bigger than what I'm working on. For something that isn't massive scale, it's bloody complicated. If one of these projects ever gets to the point where a $40/mo DigitalOcean box can't handle the load, I'll probably look at it again. Until then, though, it feels like a very expensive (time-wise) premature optimization.
paulddraper 7y ago
Yep. I only half care about rootless. I definitely care about the daemon. It sucks. It flies in the face of traditional Linux process management where child processes are child processes. (Unless you want an init system, where you need a daemon. But docker is a sucky init system.) Docker breaks even the most basic things. $ time docker run some heavy computation Oh wait, that doesn't work.
- the8472 7y ago
  For the trivial case a child process would work. But ultimately docker does try to be closer to an init system or maybe screen since you can detach/attach to processes. Since reparenting to arbitrary processes is not possible in linux it's also not possible to retain the parent-child relationship for spawned containers. If you want the fork-exec model then docker is indeed the wrong tool for the job.
  - jchw 7y ago
    The thing is, I don't even really like Docker as an init daemon. I have my gripes about Systemd but I see no downsides to not having a long-running daemon for a container engine. Really, whether you need root or not isn't even the most important issue; you can do sudo or suid or whatever with any container engine; Docker just has it be an implicit, unintuitive behavior. I used to use systemd+rkt for simple container setups when I didn't need all of Kubernetes. Never noticed any downsides versus using Docker, but on the flip side, I had far fewer issues with my containers not properly starting at boot.
    - paulddraper 7y ago
      I read an article (can't find it now) that said from a previous project the Docker authors concluded they wanted a daemon so they didn't have to do things like file locks, etc. around image management. Don't know if that accounts for the whole reason or not.
      - jchw 7y ago
meddlepal 7y ago
Except that then you lose MacOS and Windows compatibility which is somewhat important to Docker.
entropy1111 7y ago
What are my options to replace Docker Compose? I dont want to introduce a chaotic mess by using kubernetes. Or to dedicate brain power to learn what they changed every week. Their readme really confuses me with podman play, kompose, k8s.
- Conan_Kudo 7y ago
  There is an implementation of docker compose for podman in development: https://github.com/muayyad-alsadi/podman-compose https://github.com/muayyad-alsadi/podman-compose
  - entropy1111 7y ago
    >it's still underdevelopment and does not work yet. >https://github.com/muayyad-alsadi/podman-compose/issues/13 https://github.com/muayyad-alsadi/podman-compose/issues/13 Hard to trust a repo like that for production. Unless Red Hat provides an alternative I won't be able to use podman. IIRC they said it was too hard to keep up with K8S changes so even an abstraction for that would be too costly.
Gondolin 7y ago
Now that systemd-nspawn also has oci support, I wonder if podman/cri-o are going to switch to nspawn rather than runc.

Docker has supported namespaces for a while now so that even if the user in the container is root it could be a subordinate id on the host with no administrative authority. What is new though?

techntoke 7y ago
That still required the daemon run as root. This runs the daemon rootless as well.
cyphar 7y ago
The daemon is running as an unprivileged user. Docker with userns-remap is still running as root (and recent vulnerabilities like CVE-2018-15664 are still a significant worry even if you ran with user namespaces enabled).

a-ve 7y ago

To the container wizards: Is it possible to orchestrate lxc containers using kubernetes? I've been looking at lxc containers for a while and really would not like to run Docker as root.

cyphar 7y ago
LXD has orchestration support natively, though it's not at all like Kubernetes (you are manually moving containers around and so on). I have heard that some folks have looked into using LXC under Kubernetes (and theoretically the OCI templates for LXC could possibly make this somewhat work) but there isn't an obvious way to do that today AFAIK. And I'm not convinced (given CNI which touches some deep bits of runc's particular behaviour) it would work with everything you'd want it to.
- bboreham 7y ago
  > given CNI which touches some deep bits of runc's particular behaviour Could you explain? I’m a maintainer of CNI and I know almost nothing about runc, so I’m not clear where they touch. The first CNI implementation came out of rkt, pre-dating runc by a year or two.
  - cyphar 7y ago
    I mean that it likely makes certain assumptions about how runc sets up containers which aren't true of LXC. I'm sure you could get CNI to work with LXC (in fact, someone might've already done that -- I'm not sure tbh) but it wouldn't be something you'd be able to drop-in without at least a bit of extra work. For instance, LXC's hooks run in different contexts to OCI hooks (though we recently discovered that runc runs hooks in different contexts to the OCI spec, but that's another topic). And historically, yes it might predate the runc binary but the libcontainer code that is now part of runc predated CNI -- and all of our fun idiosyncrasies are in libcontainer.
    - bboreham 7y ago
      > likely makes certain assumptions about how runc sets up containers Nope. CNI takes as parameters a “container ID” (any string) and a network namespace path. No knowledge is needed or implied about how those things fit with actual containers.