6 ms·

Show HN: Plash – Build and run containers

66 points by ihucos 8y ago

gravypod 8y ago
How would you use this to build a docker container within a docker container? This would be an interesting use case for continuous integration where you might have ephemeral building workers.
- dlor 8y ago
  Check out kaniko, we built it for that exact use case: github.com/GoogleContainerTools/kaniko
- ihucos 8y ago
  I think the simplest would be like this: ihucos@macbook:~$ docker run --privileged --volume /var/lib/plash -ti python bash root@d2a59ae96bc2:/# pip3 install plash Collecting plash Downloading https://files.pythonhosted.org/packages/41/9a/a3a7c6fd104ff197b7656a51e1b396eee42bde3a578d70eb1c5d524e8e0f/plash-1.1359-py3-none-any.whl (749kB) 100% || 757kB 2.0MB/s Installing collected packages: plash Successfully installed plash-1.1359 root@d2a59ae96bc2:/# plash init root@d2a59ae96bc2:/# plash -A plash: fetching 100% plash: extracting... d2a59ae96bc2:/# type apk apk is /sbin/apk d2a59ae96bc2:/# printf hello from alpine hellod2a59ae96bc2:/# root@d2a59ae96bc2:/# exit exit ohh yeah, you would need to save to builded container somewhere or pipe it out from docker (but docker does not differentiate stderr and stdout). In any case to import the container builded with plash back to the docker: ihucos@macbook:~$ plash export-tar --eval-file /tmp/plashscript | docker import - + touch /hello --: sha256:8e26d9d222bb4840feb3aeb475522e76aa8fa835049eac08121961694203e99e
projectramo 8y ago
Will this run alongside virtual box? (In other words, does it need hyper-V turned on on a windows box?
- ihucos 8y ago
  Hey, Yeah, that should work. It usually uses linux namespaces/overlay/unionfs/mount/chroot there are no dependencies on hyper-V. In the worst case scenario plash would also be happy with not cleaning up mountpoints (PLASH_NO_UNSHARE=1) and using unionfs-fuse. During development I run it on FreeBSD and macOS. So its quite portable. Feel free to send me an mail (mail at irae me) for support. EDIT: Running it natively on a non Linux-System is probably not what you think, since you can't run a linux root file system for example on top of the macOS kernel. Experimenting with the FreeBSD linux abi emulation layer did not yield usable results for me.
axtn 8y ago
Are there any plans to be backwards-compatible with Dockerfiles and ‘docker-build’?
- ihucos 8y ago
  Good morning, it certainly would be very nice. I explored this a little bit but on the end had to focus on stability of the core components. It wouldn't be anything too hard, but quite a lot of work to implement all the syntax quirks, keeping up with new commands, etc. So right now at least I personally don't have concrete plans to implement it. What you can do right now is use images from a local docker installation: $ plash --from-docker busybox -- printf hi
bthornbury 8y ago
Some comparisons between the cli of plash and equivalent commands in docker (only because I'm already familiar with it) would be pretty helpful.
- ihucos 8y ago
  Yep true. I am still trying to figure out how to write good documentation/tutorials. I will definitely write some general recipes in the README for common tasks, some also doable in other container software like mounting containers, importing from docker, exporting to docker, using an image published in github etc. I'll shortly ping you via PM afterwards.
- ihucos 8y ago
  check it out, there are some recipes now.
anarcat 8y ago
I find the first item in the README a bit misleading: Security: plash containers run completely unprivileged. What this actually means is that plash containers run as the current user, so not as root. But they provide no isolation whatsoever, which I would expect from containers (and which Docker provides, to a certain extent). The README does mention this, but only much later: - Plash processes have the same operating system access rights than the process that started it. There is no security relevant isolation feature. Exactly as with running programs "normally", don't run programs you do not trust with plash and try to avoid using plash with the root user. rootless containers are still a little ways off it seems...
- bthornbury 8y ago
  https://github.com/projectatomic/bubblewrap https://github.com/projectatomic/bubblewrap This is the most promising one I've seen.
  - austinjp 8y ago
    I'm unclear on their readme. Right at the bottom, they seem to suggest that once runC implements rootless containers, it will be a superior solution since it conforms to the Open Containers Initiative. Checking runC's GitHub, they merged a rootless container branch into their master branch a year ago. A related blog post says it's now inside Docker so very well used. So does Docker now provide containers that can guarantee immunity from (certain) privilege escalation attacks? Would you move to Docker/runC?
    - ihucos 8y ago
      > So does Docker now provide containers that can guarantee immunity from (certain) privilege escalation attacks? In my opinion the isolation mechanism of major container software like docker is flawed and thinking about it gives me headaches. One problem is that you need a daemon running as root. It's better if you don't have to leave the user space at all. To be fair plash relies on fuse and newuidmap/newgidmap which are suid, so there is at least a little bit happening with root access rights. But it's comparably very little and there is nothing self-baked running as root. There are many details that for me are unclear in docker, one out of the of the tip of my head, is that the usage of a overlay mount is disabled by non-root users (also inside user maps) because the kernel does not want to guarantee that a bogus setup can cause a security issue. > Would you move to Docker/runC? Never :-) The whole point of this is to not use Docker/runC. One of the core aspects of plash is that containers should just be normal processes. Processes is an interface that existed since decades, for containers I need to learn a new tool just to kill one or list all that are running (use ps/kill in plash)
etaioinshrdlu 8y ago
How is this different from gVisor? Not complaining, this looks very awesome. I am just interested. This is going to be a great world when we have arbitrary nesting of containers, finally we will have closure :) I think "docker compose" is a big lie.
- ihucos 8y ago
  gVisor is primarly focused on isolation, plash actually only wants to containerize processes and leaves optional isolation to other tools if required (potentially gVisor). To quote gVisor: Containers are not a sandbox. While containers have revolutionized how we develop, package, and deploy applications, running untrusted or potentially malicious code without additional isolation is not a good idea. I think the approach gVisor uses is basically the same that is used to run python on Googles web hosting platform. As I understand it, with gVisor a process does not directly talk to the kernel but actually to another process. > This is going to be a great world when we have arbitrary nesting of containers, finally we will have closure :) Yep. With plash you can do that (In plash unprivileged containers inside unprivileged containers also works, but may need some looking into it). > I think "docker compose" is a big lie. I also have strong feelings about docker compose. It's really reinventing everything but now for containers. With plash you can just use supervisord, the init of your operating system and other established means of managing background processes.
miduil 8y ago
Really cool, I like the documentation on the website - would be nice if the README was actual HTML though. Does that mean I can build container images within a running container (for example the GitLab-CI) without doing a stunt like docker-in-docker or fakechroot/fakeroot? What are you using instead of fakeroot? I've seen people like [zwischenzugs][0] using User Namespaces - which makes it impossible to be run within a container. Author also points out that kaniko still requires you to use root, not sure if the situation has changed with gVisor support, at least a related issue [1] hasn't been marked as resolved. Have you investigated into creating reproducible containers? Thank you in advance! [0]: https://zwischenzugs.com/2018/04/23/unprivileged-docker-builds-a-proof-of-concept/ https://zwischenzugs.com/2018/04/23/unprivileged-docker-buil... [1]: https://github.com/GoogleContainerTools/kaniko/issues/105 https://github.com/GoogleContainerTools/kaniko/issues/105 Edit: Why the downvote? Am I misunderstanding something?
- ihucos 8y ago
  Thank you :-) > Does that mean I can build container images within a running container (for example the GitLab-CI) without doing a stunt like docker-in-docker or fakechroot/fakeroot? Sure, plash's unit tests involve building and they run inside a plash instance at travis CI. I'll have lunch now and then continue with the reply after.
  - miduil 8y ago
    I think you can already build Docker images via Docker within travis CI, because they somehow provide you (semi?-)privileged access to the required Linux cgroup and namespace features. If you are running within a regular Docker container commands such as sysctl -w kernel.unprivileged_userns_clone=1 are not possible, due to /sys being read-only. Am I doing something wrong?
    - ihucos 8y ago
      > I think you can already build Docker images via Docker within travis CI, [...] The nice think about plash is that you don't have to worry about "official" support. > [...] due to /sys being read-only. Dammit, no, so yeah, apparently unfortunately for regular docker containers (not privileged ones) you need to disable unsharing in plash (export PLASH_NO_UNSHARE=1). In that case the only requirements are chroot, a rudimentary mount binary and overlay OR unionfs-fuse and root access. So it should even run in BSD/Minix/macOS and stuff like that, BUT the mountpoints will not get cleaned up. For an instance running for a while that would be a problem, for a temporary docker container not.
naasking 8y ago
Unfortunate name choice. There's already a Plash [1,2] project for the Principle Least Authority Shell. [1] http://www.cs.jhu.edu/~seaborn/plash/html/ http://www.cs.jhu.edu/~seaborn/plash/html/ [2] http://plash.beasts.org/index.html http://plash.beasts.org/index.html
0n34n7 8y ago
Any reason for not using markdown in your README? Not rendering well on my phone at all.
- ihucos 8y ago
  Not particular strong ones. I actually imagined that .txt is more accessible, maybe the wrapping must be tighter. I like it because its very WYSIWYG. The UTF-8 art looks better there than in a code block. And it's one little dependency less, if for example at some point development should migrate to another platform.

66 points by ihucos 8y ago

How would you use this to build a docker container within a docker container? This would be an interesting use case for continuous integration where you might have ephemeral building workers.

dlor 8y ago
Check out kaniko, we built it for that exact use case: github.com/GoogleContainerTools/kaniko
ihucos 8y ago
I think the simplest would be like this: ihucos@macbook:~$ docker run --privileged --volume /var/lib/plash -ti python bash root@d2a59ae96bc2:/# pip3 install plash Collecting plash Downloading https://files.pythonhosted.org/packages/41/9a/a3a7c6fd104ff197b7656a51e1b396eee42bde3a578d70eb1c5d524e8e0f/plash-1.1359-py3-none-any.whl (749kB) 100% || 757kB 2.0MB/s Installing collected packages: plash Successfully installed plash-1.1359 root@d2a59ae96bc2:/# plash init root@d2a59ae96bc2:/# plash -A plash: fetching 100% plash: extracting... d2a59ae96bc2:/# type apk apk is /sbin/apk d2a59ae96bc2:/# printf hello from alpine hellod2a59ae96bc2:/# root@d2a59ae96bc2:/# exit exit ohh yeah, you would need to save to builded container somewhere or pipe it out from docker (but docker does not differentiate stderr and stdout). In any case to import the container builded with plash back to the docker: ihucos@macbook:~$ plash export-tar --eval-file /tmp/plashscript | docker import - + touch /hello --: sha256:8e26d9d222bb4840feb3aeb475522e76aa8fa835049eac08121961694203e99e

projectramo 8y ago

Will this run alongside virtual box? (In other words, does it need hyper-V turned on on a windows box?

ihucos 8y ago
Hey, Yeah, that should work. It usually uses linux namespaces/overlay/unionfs/mount/chroot there are no dependencies on hyper-V. In the worst case scenario plash would also be happy with not cleaning up mountpoints (PLASH_NO_UNSHARE=1) and using unionfs-fuse. During development I run it on FreeBSD and macOS. So its quite portable. Feel free to send me an mail (mail at irae me) for support. EDIT: Running it natively on a non Linux-System is probably not what you think, since you can't run a linux root file system for example on top of the macOS kernel. Experimenting with the FreeBSD linux abi emulation layer did not yield usable results for me.

axtn 8y ago

Are there any plans to be backwards-compatible with Dockerfiles and ‘docker-build’?

ihucos 8y ago
Good morning, it certainly would be very nice. I explored this a little bit but on the end had to focus on stability of the core components. It wouldn't be anything too hard, but quite a lot of work to implement all the syntax quirks, keeping up with new commands, etc. So right now at least I personally don't have concrete plans to implement it. What you can do right now is use images from a local docker installation: $ plash --from-docker busybox -- printf hi

bthornbury 8y ago

Some comparisons between the cli of plash and equivalent commands in docker (only because I'm already familiar with it) would be pretty helpful.

ihucos 8y ago
Yep true. I am still trying to figure out how to write good documentation/tutorials. I will definitely write some general recipes in the README for common tasks, some also doable in other container software like mounting containers, importing from docker, exporting to docker, using an image published in github etc. I'll shortly ping you via PM afterwards.
ihucos 8y ago
check it out, there are some recipes now.

anarcat 8y ago

I find the first item in the README a bit misleading: Security: plash containers run completely unprivileged. What this actually means is that plash containers run as the current user, so not as root. But they provide no isolation whatsoever, which I would expect from containers (and which Docker provides, to a certain extent). The README does mention this, but only much later: - Plash processes have the same operating system access rights than the process that started it. There is no security relevant isolation feature. Exactly as with running programs "normally", don't run programs you do not trust with plash and try to avoid using plash with the root user. rootless containers are still a little ways off it seems...

bthornbury 8y ago
https://github.com/projectatomic/bubblewrap https://github.com/projectatomic/bubblewrap This is the most promising one I've seen.
- austinjp 8y ago
  I'm unclear on their readme. Right at the bottom, they seem to suggest that once runC implements rootless containers, it will be a superior solution since it conforms to the Open Containers Initiative. Checking runC's GitHub, they merged a rootless container branch into their master branch a year ago. A related blog post says it's now inside Docker so very well used. So does Docker now provide containers that can guarantee immunity from (certain) privilege escalation attacks? Would you move to Docker/runC?
  - ihucos 8y ago
    > So does Docker now provide containers that can guarantee immunity from (certain) privilege escalation attacks? In my opinion the isolation mechanism of major container software like docker is flawed and thinking about it gives me headaches. One problem is that you need a daemon running as root. It's better if you don't have to leave the user space at all. To be fair plash relies on fuse and newuidmap/newgidmap which are suid, so there is at least a little bit happening with root access rights. But it's comparably very little and there is nothing self-baked running as root. There are many details that for me are unclear in docker, one out of the of the tip of my head, is that the usage of a overlay mount is disabled by non-root users (also inside user maps) because the kernel does not want to guarantee that a bogus setup can cause a security issue. > Would you move to Docker/runC? Never :-) The whole point of this is to not use Docker/runC. One of the core aspects of plash is that containers should just be normal processes. Processes is an interface that existed since decades, for containers I need to learn a new tool just to kill one or list all that are running (use ps/kill in plash)

etaioinshrdlu 8y ago

How is this different from gVisor? Not complaining, this looks very awesome. I am just interested. This is going to be a great world when we have arbitrary nesting of containers, finally we will have closure :) I think "docker compose" is a big lie.

ihucos 8y ago
gVisor is primarly focused on isolation, plash actually only wants to containerize processes and leaves optional isolation to other tools if required (potentially gVisor). To quote gVisor: Containers are not a sandbox. While containers have revolutionized how we develop, package, and deploy applications, running untrusted or potentially malicious code without additional isolation is not a good idea. I think the approach gVisor uses is basically the same that is used to run python on Googles web hosting platform. As I understand it, with gVisor a process does not directly talk to the kernel but actually to another process. > This is going to be a great world when we have arbitrary nesting of containers, finally we will have closure :) Yep. With plash you can do that (In plash unprivileged containers inside unprivileged containers also works, but may need some looking into it). > I think "docker compose" is a big lie. I also have strong feelings about docker compose. It's really reinventing everything but now for containers. With plash you can just use supervisord, the init of your operating system and other established means of managing background processes.

miduil 8y ago

Really cool, I like the documentation on the website - would be nice if the README was actual HTML though. Does that mean I can build container images within a running container (for example the GitLab-CI) without doing a stunt like docker-in-docker or fakechroot/fakeroot? What are you using instead of fakeroot? I've seen people like [zwischenzugs][0] using User Namespaces - which makes it impossible to be run within a container. Author also points out that kaniko still requires you to use root, not sure if the situation has changed with gVisor support, at least a related issue [1] hasn't been marked as resolved. Have you investigated into creating reproducible containers? Thank you in advance! [0]: https://zwischenzugs.com/2018/04/23/unprivileged-docker-builds-a-proof-of-concept/ https://zwischenzugs.com/2018/04/23/unprivileged-docker-buil... [1]: https://github.com/GoogleContainerTools/kaniko/issues/105 https://github.com/GoogleContainerTools/kaniko/issues/105 Edit: Why the downvote? Am I misunderstanding something?

ihucos 8y ago
Thank you :-) > Does that mean I can build container images within a running container (for example the GitLab-CI) without doing a stunt like docker-in-docker or fakechroot/fakeroot? Sure, plash's unit tests involve building and they run inside a plash instance at travis CI. I'll have lunch now and then continue with the reply after.
- miduil 8y ago
  I think you can already build Docker images via Docker within travis CI, because they somehow provide you (semi?-)privileged access to the required Linux cgroup and namespace features. If you are running within a regular Docker container commands such as sysctl -w kernel.unprivileged_userns_clone=1 are not possible, due to /sys being read-only. Am I doing something wrong?
  - ihucos 8y ago
    > I think you can already build Docker images via Docker within travis CI, [...] The nice think about plash is that you don't have to worry about "official" support. > [...] due to /sys being read-only. Dammit, no, so yeah, apparently unfortunately for regular docker containers (not privileged ones) you need to disable unsharing in plash (export PLASH_NO_UNSHARE=1). In that case the only requirements are chroot, a rudimentary mount binary and overlay OR unionfs-fuse and root access. So it should even run in BSD/Minix/macOS and stuff like that, BUT the mountpoints will not get cleaned up. For an instance running for a while that would be a problem, for a temporary docker container not.

naasking 8y ago

Unfortunate name choice. There's already a Plash [1,2] project for the Principle Least Authority Shell. [1] http://www.cs.jhu.edu/~seaborn/plash/html/ http://www.cs.jhu.edu/~seaborn/plash/html/ [2] http://plash.beasts.org/index.html http://plash.beasts.org/index.html

0n34n7 8y ago

Any reason for not using markdown in your README? Not rendering well on my phone at all.

ihucos 8y ago
Not particular strong ones. I actually imagined that .txt is more accessible, maybe the wrapping must be tighter. I like it because its very WYSIWYG. The UTF-8 art looks better there than in a code block. And it's one little dependency less, if for example at some point development should migrate to another platform.