9 ms·

I actually wish that instead of docker & etc we had just gotten a better chroot... Or maybe just a new kernel syscall that is chroot()++.

gnuser 1y ago
Pids and cgroups all the way down (also why the wise greybeards rejected docker)
zoobab 1y ago
Working on proot-docker, a bash script on top of skopeo and proot: https://github.com/mtseet/proot-docker https://github.com/mtseet/proot-docker We need more people to improve it!
nesarkvechnep 1y ago
Come to FreeBSD, we have just that - jails.
- VWWHFSfQ 1y ago
  Or DragonFlyBSD with vkernels https://www.dragonflybsd.org/docs/handbook/vkernel/ https://www.dragonflybsd.org/docs/handbook/vkernel/
- masom 1y ago
  yup! FreeBSD jails are essentially what OP wants with chroot++. I was pretty puzzled when Docker and LXC came around as this whole new thing believed to have "never been done before"; FreeBSD had supported a very similar concept for years before security groups were added in Linux. Jails and ezjail were stellar to make mini no-overhead containers when running various services on a server. Being able to archive them and expand them on a new machine was also pretty cool (as long as the BSD version was the same.)
  - microtonal 1y ago
    this whole new thing believed to have "never been done before"; Nobody with knowledge of sandboxing believed this, Virtuozzo and later OpenVZ had been on Linux for a long time after all. Virtuozzo was even from a similar time frame as FreeBSD jails (2000-ish). The key innovation of Docker was to provide a standardized way to build, distribute, and run container images.
    - alfiedotwtf 1y ago
      Virsh had worked for a long time before docker came around, but yeah… you essentially had to build your own Docker-like infrastructure that only you were using
  - tonyarkles 1y ago
    Solaris Zones too. Absolute magic, many years before Docker and friends.
mystified5016 1y ago
Isn't LXC more or less an unsupervised chroot in an isolated process?
- goku12 1y ago
  Yes. And so is bubblewrap - if security through isolation is a priority.
- deleted 1y ago
  [deleted]
SuperNinKenDo 1y ago
systemd-nspawn is probably what you want.
duped 1y ago
what would "better chroot" do?
- goku12 1y ago
  I'm not GP, but if I were to hazard a guess, they want something more than just mount space isolation. Something akin to BSD jails, without the bells and whistles of OCI containers like overlay filesystem, network virtualization, resource management, etc. That requirement is pretty legitimate, since its easier and suitable enough for many applications for which we currently use OCI containers. For example, isolated builds, development environments, sandboxes etc. (I have an isolated build tool for Gentoo). But Linux already has multiple solutions that fit the bill, like systemd-nspawn, LXC, bubblewrap, etc. Too bad, they aren't as widely known as chroot.
  - duped 1y ago
    None of those things do what chroot does but many of them involve chroot - so I'm still not grasping what "better chroot" is, other than "not chroot, but something completely different." It sounds like people want "better exec"
    - aidanhs 1y ago
      One annoying part of using chroot if you're creating them on the fly is teardown - you have to manually invoke umount, and also take care to get this right for partially created chroots (maybe you detected an error after mounting proc, in the process of getting other files in place). This was my original motivation in creating machroot (mentioned elsewhere in this thread) and having it use namespaces.
- NexRebular 1y ago
  what Solaris and now illumos zones do[1] [1] https://www.usenix.org/legacy/event/lisa04/tech/full_papers/price/price.pdf https://www.usenix.org/legacy/event/lisa04/tech/full_papers/...
paulddraper 1y ago
Apples and oranges. Among many other things, Docker (and Podman etc) has 1. Images and OverlayFS 2. Networking 3. User namespace mappings 4. Resource management --- If all you want is file system isolation, then docker (and postman, etc) is massive overkill chroot is correct.
- paulddraper 1y ago
  *podman, etc
devrandoom 1y ago
We kind of did but its all put in the context of containers. Check out the unshare command. unshare --mount Most examples you'll find put it in the context of containers, like https://www.redhat.com/en/blog/mount-namespaces https://www.redhat.com/en/blog/mount-namespaces
alphazard 1y ago
There seems to be a fundamental mismatch between how sane people think about sandboxing, and how linux manages namespaces. A linux-naive developer would expect to spawn a new process from a payload with access to nothing. It can't see other processes, it has a read only root with nothing in it, there are no network devices, no users, etc. Then they would expect to read documentation to learn how to add things to the sandbox. They want to pass in a directory, or a network interface, or some users. The effort goes into adding resources to the sandbox, not taking them away. Instead there is this elaborate ceremony where the principal process basically spawns another version of itself endowed with all the same privileges and then gives them up, hopefully leaving itself with only the stuff it wants the sandboxed process to have. Make sure you don't forget to revoke anything.
- mytailorisrich 1y ago
  I believe this is because on POSIX systems the only way to create a new process is fork().
  - adrian_b 1y ago
    There is the later added posix_spawn, which could be implemented with a system call, even if on Linux it is emulated with clone + exec. posix_spawn can do much, but not all, of what is possible with clone + exec. Presumably the standard editors have been scared to add too complex function parameters for its invocation, though that should not have been a problem if all parameters had reasonable default values.
- rootnod3 1y ago
  That is pretty much what jails are in FreeBSD, especially thin jails.
  - vacuity 1y ago
    Or capabilities. Additive security has been known for decades; Linux really dropped the ball here. Linux file descriptors (open file descriptions, whatever) are close to a genuine capability model, except there's plenty of leakage where you can get at the insecure base.
udev4096 1y ago
Docker is not using chroot in any way. Escaping chroot is only a matter of calling chdir('..') and poof, you are out of the "sandbox"
aidanhs 1y ago
As a number of comments have noted, there are a bunch of different axes that chroot could be 'better' on - e.g. security and sandboxing. I wrote https://github.com/aidanhs/machroot https://github.com/aidanhs/machroot (initially forked from bubble wrap) a while ago to lean into the pure "pretend I see another filesystem" aspect of chroot with additional conveniences (so no security focus). For example, it allows setting up overlay filesystems, allows mounting squashfs filesystems with an overlay on top...and because it uses a mount namespace, means you don't need to tear down the mount points - just exit the command and you're done. The codebase is pretty small so I just tweaked it with whatever features I needed at the time, rather than try and make it a fully fledged tool. (honestly you can probably replicate most of it with a shell script that invokes unshare and appropriate mount commands)
jbverschoor 1y ago
I built https://github.com/jrz/container-shell https://github.com/jrz/container-shell as a simple chroot on docker. I use it for both development and sandboxing. Besides that I have a simple script that starts an ephemeral docker with debian-full + tools. I also have some scripts that leverage macOS's 'sandbox-exec'
IshKebab 1y ago
Plan9 had a proper solution for this. New processes don't get access to any files by default - you have to explicitly mount directories for them, capability style. Shame Plan9 blew its weirdness budget.
- anthk 1y ago
  rfork it's easy :D
- teddyh 1y ago
  It has nothing to do with weirdness; Unix itself was plenty weird for its time. The relevant difference between Unix™ and Plan 9 is that Unix source code was given away (or cheaply licensed) to hardware companies which all wrote their own operating systems on top (SunOS, Ultrix, HP-UX, etc. etc.). This made Unix the common factor of very many commercial workstation environments. Plan 9? It was sold directly as a commercial product, for no hardware platform in particular. Nobody wanted to buy it. People liked Unix because it was free – either really free, via BSD, or as a Unix derivative provided at no cost when people bought their workstations. A new revolutionary operating system had absolutely no reason for anybody to buy it: No commercial developers wanted to develop to a platform without users, and no users wanted a platform without software. Plan 9 only changed their license many years later, when it was too late for anybody to care, and Unix had become the established standard.
ajross 1y ago
How do you imagine a "better chroot" would differ from a Docker container? Seems to me like that's exactly what Docker is.
deleted 1y ago
[deleted]
1oooqooq 1y ago
ironically docker never gave you true network isolation because there's no way to make it user friendly. plus the many exploits on the all powerful daemon. but most professional world use systemd to bootstrap isolated processes nowadays, which is kinda if what you are hinting at. cgroups2 and namespaces are what you want.
amy214 1y ago
We should get a better change root yes, and the LD_LIBRARY_PATH gets autoupdated with respect to the new root. A few flags here and there to set permissions of the child process and we're off too the races. Oh wow, wow. That all sounded so intensely complex, incomprehensible. What we are going to need to do is build a program to handle all that, highly formalized. Let's make it so formalized it's one of those things like taxes or AWS where people can just make a living from understanding the beast. It can be like systemd meets multics meets java. have it's own various complicated commands, complicated file formats, and so on. The chroot() is only historically understood by everyone, so let's steal a page from the java playbook and just rename everything with our own terminology. The product will be so outstanding, wow, I call it "Shocker"

gnuser 1y ago

Pids and cgroups all the way down (also why the wise greybeards rejected docker)

zoobab 1y ago

Working on proot-docker, a bash script on top of skopeo and proot: https://github.com/mtseet/proot-docker https://github.com/mtseet/proot-docker We need more people to improve it!

nesarkvechnep 1y ago

Come to FreeBSD, we have just that - jails.

VWWHFSfQ 1y ago
Or DragonFlyBSD with vkernels https://www.dragonflybsd.org/docs/handbook/vkernel/ https://www.dragonflybsd.org/docs/handbook/vkernel/
masom 1y ago
yup! FreeBSD jails are essentially what OP wants with chroot++. I was pretty puzzled when Docker and LXC came around as this whole new thing believed to have "never been done before"; FreeBSD had supported a very similar concept for years before security groups were added in Linux. Jails and ezjail were stellar to make mini no-overhead containers when running various services on a server. Being able to archive them and expand them on a new machine was also pretty cool (as long as the BSD version was the same.)
- microtonal 1y ago
  this whole new thing believed to have "never been done before"; Nobody with knowledge of sandboxing believed this, Virtuozzo and later OpenVZ had been on Linux for a long time after all. Virtuozzo was even from a similar time frame as FreeBSD jails (2000-ish). The key innovation of Docker was to provide a standardized way to build, distribute, and run container images.
  - alfiedotwtf 1y ago
    Virsh had worked for a long time before docker came around, but yeah… you essentially had to build your own Docker-like infrastructure that only you were using
- tonyarkles 1y ago
  Solaris Zones too. Absolute magic, many years before Docker and friends.

mystified5016 1y ago

Isn't LXC more or less an unsupervised chroot in an isolated process?

goku12 1y ago
Yes. And so is bubblewrap - if security through isolation is a priority.
deleted 1y ago
[deleted]

SuperNinKenDo 1y ago

systemd-nspawn is probably what you want.

duped 1y ago

what would "better chroot" do?

goku12 1y ago
I'm not GP, but if I were to hazard a guess, they want something more than just mount space isolation. Something akin to BSD jails, without the bells and whistles of OCI containers like overlay filesystem, network virtualization, resource management, etc. That requirement is pretty legitimate, since its easier and suitable enough for many applications for which we currently use OCI containers. For example, isolated builds, development environments, sandboxes etc. (I have an isolated build tool for Gentoo). But Linux already has multiple solutions that fit the bill, like systemd-nspawn, LXC, bubblewrap, etc. Too bad, they aren't as widely known as chroot.
- duped 1y ago
  None of those things do what chroot does but many of them involve chroot - so I'm still not grasping what "better chroot" is, other than "not chroot, but something completely different." It sounds like people want "better exec"
  - aidanhs 1y ago
    One annoying part of using chroot if you're creating them on the fly is teardown - you have to manually invoke umount, and also take care to get this right for partially created chroots (maybe you detected an error after mounting proc, in the process of getting other files in place). This was my original motivation in creating machroot (mentioned elsewhere in this thread) and having it use namespaces.
NexRebular 1y ago
what Solaris and now illumos zones do[1] [1] https://www.usenix.org/legacy/event/lisa04/tech/full_papers/price/price.pdf https://www.usenix.org/legacy/event/lisa04/tech/full_papers/...

paulddraper 1y ago

Apples and oranges. Among many other things, Docker (and Podman etc) has 1. Images and OverlayFS 2. Networking 3. User namespace mappings 4. Resource management --- If all you want is file system isolation, then docker (and postman, etc) is massive overkill chroot is correct.

paulddraper 1y ago
*podman, etc

devrandoom 1y ago

We kind of did but its all put in the context of containers. Check out the unshare command. unshare --mount Most examples you'll find put it in the context of containers, like https://www.redhat.com/en/blog/mount-namespaces https://www.redhat.com/en/blog/mount-namespaces

alphazard 1y ago

There seems to be a fundamental mismatch between how sane people think about sandboxing, and how linux manages namespaces. A linux-naive developer would expect to spawn a new process from a payload with access to nothing. It can't see other processes, it has a read only root with nothing in it, there are no network devices, no users, etc. Then they would expect to read documentation to learn how to add things to the sandbox. They want to pass in a directory, or a network interface, or some users. The effort goes into adding resources to the sandbox, not taking them away. Instead there is this elaborate ceremony where the principal process basically spawns another version of itself endowed with all the same privileges and then gives them up, hopefully leaving itself with only the stuff it wants the sandboxed process to have. Make sure you don't forget to revoke anything.

mytailorisrich 1y ago
I believe this is because on POSIX systems the only way to create a new process is fork().
- adrian_b 1y ago
  There is the later added posix_spawn, which could be implemented with a system call, even if on Linux it is emulated with clone + exec. posix_spawn can do much, but not all, of what is possible with clone + exec. Presumably the standard editors have been scared to add too complex function parameters for its invocation, though that should not have been a problem if all parameters had reasonable default values.
rootnod3 1y ago
That is pretty much what jails are in FreeBSD, especially thin jails.
- vacuity 1y ago
  Or capabilities. Additive security has been known for decades; Linux really dropped the ball here. Linux file descriptors (open file descriptions, whatever) are close to a genuine capability model, except there's plenty of leakage where you can get at the insecure base.

udev4096 1y ago

Docker is not using chroot in any way. Escaping chroot is only a matter of calling chdir('..') and poof, you are out of the "sandbox"

aidanhs 1y ago

As a number of comments have noted, there are a bunch of different axes that chroot could be 'better' on - e.g. security and sandboxing. I wrote https://github.com/aidanhs/machroot https://github.com/aidanhs/machroot (initially forked from bubble wrap) a while ago to lean into the pure "pretend I see another filesystem" aspect of chroot with additional conveniences (so no security focus). For example, it allows setting up overlay filesystems, allows mounting squashfs filesystems with an overlay on top...and because it uses a mount namespace, means you don't need to tear down the mount points - just exit the command and you're done. The codebase is pretty small so I just tweaked it with whatever features I needed at the time, rather than try and make it a fully fledged tool. (honestly you can probably replicate most of it with a shell script that invokes unshare and appropriate mount commands)

jbverschoor 1y ago

I built https://github.com/jrz/container-shell https://github.com/jrz/container-shell as a simple chroot on docker. I use it for both development and sandboxing. Besides that I have a simple script that starts an ephemeral docker with debian-full + tools. I also have some scripts that leverage macOS's 'sandbox-exec'

IshKebab 1y ago

Plan9 had a proper solution for this. New processes don't get access to any files by default - you have to explicitly mount directories for them, capability style. Shame Plan9 blew its weirdness budget.

anthk 1y ago
rfork it's easy :D
teddyh 1y ago
It has nothing to do with weirdness; Unix itself was plenty weird for its time. The relevant difference between Unix™ and Plan 9 is that Unix source code was given away (or cheaply licensed) to hardware companies which all wrote their own operating systems on top (SunOS, Ultrix, HP-UX, etc. etc.). This made Unix the common factor of very many commercial workstation environments. Plan 9? It was sold directly as a commercial product, for no hardware platform in particular. Nobody wanted to buy it. People liked Unix because it was free – either really free, via BSD, or as a Unix derivative provided at no cost when people bought their workstations. A new revolutionary operating system had absolutely no reason for anybody to buy it: No commercial developers wanted to develop to a platform without users, and no users wanted a platform without software. Plan 9 only changed their license many years later, when it was too late for anybody to care, and Unix had become the established standard.

ajross 1y ago

How do you imagine a "better chroot" would differ from a Docker container? Seems to me like that's exactly what Docker is.

deleted 1y ago

[deleted]

1oooqooq 1y ago

ironically docker never gave you true network isolation because there's no way to make it user friendly. plus the many exploits on the all powerful daemon. but most professional world use systemd to bootstrap isolated processes nowadays, which is kinda if what you are hinting at. cgroups2 and namespaces are what you want.

amy214 1y ago

We should get a better change root yes, and the LD_LIBRARY_PATH gets autoupdated with respect to the new root. A few flags here and there to set permissions of the child process and we're off too the races. Oh wow, wow. That all sounded so intensely complex, incomprehensible. What we are going to need to do is build a program to handle all that, highly formalized. Let's make it so formalized it's one of those things like taxes or AWS where people can just make a living from understanding the beast. It can be like systemd meets multics meets java. have it's own various complicated commands, complicated file formats, and so on. The chroot() is only historically understood by everyone, so let's steal a page from the java playbook and just rename everything with our own terminology. The product will be so outstanding, wow, I call it "Shocker"