3 ms·

This... is actually pretty great to have. Very cool. That said... It really just makes me bummed about the lack of good support for container and sandboxing te

by jchw 2y ago

This... is actually pretty great to have. Very cool. That said... It really just makes me bummed about the lack of good support for container and sandboxing technology on macOS. I know, totally different thing. But still, we'll probably never have a "native" Docker/Podman/etc. on macOS, and things like the Bazel or Nix sandbox are significantly less effective on macOS due to there just being less kernel APIs for isolating resources. e.g. There are no network namespaces. I get why, but it's surprising how limiting not having network namespaces is once you have used a platform that has an equivalent to this. For Nix it's really tricky since a lot of things need at least a working localhost inside the sandbox. I am sure increasing the surface area of XNU is not a serious consideration, for good reasons, but it's a bit of a shame.

vbezhenar 2y ago
There are Windows containers and I don't think that most developers using Docker on Windows are even aware of them. IMO it's kind of good to have Linux as a unified container platform. It's easy to run Linux application in Windows or macOS VM. It's hard form the licensing perspective to run Windows VM and it's very hard to run macOS VM.
- pjmlp 2y ago
  We are very much aware, they are commonly used on deployments used in Windows Server. Typical examples, Sitecore, Sharepoint, Dynamics, Optimizelly, COM services, SQL Server.
- jchw 2y ago
  I think it's just that Windows Containers are a lot unlike Linux Docker containers; they're not useless, but they're very different. I don't think Darwin containers have to be so different from Linux containers, not the least of which because Darwin is UNIX. It would be a lot more like if you ported Docker to run natively on BSD than Windows, since it basically is that. An easy solution to the licensing issue is just having an open source userland that you can use, which Darwin sorta does. That said, since XNU syscalls are not stable, there is some awkwardness here. Either they would need to be made stable somehow (e.g. by providing something like personalities, where different processes get different syscall behavior) or the runtime would need to drop libc/base libraries in from the sky and containers would just need to assume they're available at a specific location. Either thing seems quite viable if Apple wanted to do it, IMO.
- jayd16 2y ago
  Its hard to miss Windows containers when using docker but its easy to gloss over. I've been trying them out recently and while the edges are sharper than linux containers (for now?) its actually still very cool.
jbverschoor 2y ago
sandbox the cli util included with macOS allows you to conure basically anything Then, using docker/orbstack, I’ve created an easy chroot shell on steroids. It mounts the current dir to a container. Great for development or sandboxing https://github.com/jrz/container-shell https://github.com/jrz/container-shell
- jchw 2y ago
  Not trying to be condescending but simply put the sandbox-exec functionality is a toy compared to what is offered in many other UNIX-likes. It's not that it's useless (both Nix and Bazel use macOS's native sandboxing to great effect) but it is just limited compared to what's out there. Like yes, you can do relatively fine-grained limiting of resource use, but you're limited on how you can isolate resource use. On Linux, network namespaces allow you to get an entire separate networking environment, including a separate loopback, separate IP tables rules, etc. This is very powerful. An example of why it matters: Let's say you want to run some fully hermetic, isolated build. Sometimes it is very useful to have a network stack even if there is no actual access to a network. An example from Nix is that you can shim central package repo protocols and provide vendored packages over localhost. On macOS, the sandbox can't give you an isolated localhost, so instead Nix has an escape hatch that lets you just give specific derivations access to the regular localhost, only in the Darwin sandbox. (This obviously has the downsides you'd expect; you might connect to other processes on the machine by mistake, and you might run into port conflicts trying to listen on ports.) Can this be resolved without more macOS features? Honestly, I bet you could do it. You'd probably need to inject a usermode network stack somehow, though, and that might just be a bit tricky to do on Darwin. Ultimately, I fully understand why Apple won't implement something like network namespaces in the kernel (it's complicated and it introduces a large surface area for bugs) but not having them is a bummer. They're awesome. I use them for a variety of things. An end user use case that I think is really cool is isolating processes into a network jail where the only route out is a Wireguard VPN tunnel. (And to be clear, I'm not saying if Darwin had network namespaces it'd suddenly be on the same level as everything else, but it would be a great start IMO.)
  - jbverschoor 2y ago

by jchw 2y ago

vbezhenar 2y ago

There are Windows containers and I don't think that most developers using Docker on Windows are even aware of them. IMO it's kind of good to have Linux as a unified container platform. It's easy to run Linux application in Windows or macOS VM. It's hard form the licensing perspective to run Windows VM and it's very hard to run macOS VM.

pjmlp 2y ago
We are very much aware, they are commonly used on deployments used in Windows Server. Typical examples, Sitecore, Sharepoint, Dynamics, Optimizelly, COM services, SQL Server.
jchw 2y ago
I think it's just that Windows Containers are a lot unlike Linux Docker containers; they're not useless, but they're very different. I don't think Darwin containers have to be so different from Linux containers, not the least of which because Darwin is UNIX. It would be a lot more like if you ported Docker to run natively on BSD than Windows, since it basically is that. An easy solution to the licensing issue is just having an open source userland that you can use, which Darwin sorta does. That said, since XNU syscalls are not stable, there is some awkwardness here. Either they would need to be made stable somehow (e.g. by providing something like personalities, where different processes get different syscall behavior) or the runtime would need to drop libc/base libraries in from the sky and containers would just need to assume they're available at a specific location. Either thing seems quite viable if Apple wanted to do it, IMO.
jayd16 2y ago
Its hard to miss Windows containers when using docker but its easy to gloss over. I've been trying them out recently and while the edges are sharper than linux containers (for now?) its actually still very cool.

jbverschoor 2y ago

sandbox the cli util included with macOS allows you to conure basically anything Then, using docker/orbstack, I’ve created an easy chroot shell on steroids. It mounts the current dir to a container. Great for development or sandboxing https://github.com/jrz/container-shell https://github.com/jrz/container-shell

jchw 2y ago
Not trying to be condescending but simply put the sandbox-exec functionality is a toy compared to what is offered in many other UNIX-likes. It's not that it's useless (both Nix and Bazel use macOS's native sandboxing to great effect) but it is just limited compared to what's out there. Like yes, you can do relatively fine-grained limiting of resource use, but you're limited on how you can isolate resource use. On Linux, network namespaces allow you to get an entire separate networking environment, including a separate loopback, separate IP tables rules, etc. This is very powerful. An example of why it matters: Let's say you want to run some fully hermetic, isolated build. Sometimes it is very useful to have a network stack even if there is no actual access to a network. An example from Nix is that you can shim central package repo protocols and provide vendored packages over localhost. On macOS, the sandbox can't give you an isolated localhost, so instead Nix has an escape hatch that lets you just give specific derivations access to the regular localhost, only in the Darwin sandbox. (This obviously has the downsides you'd expect; you might connect to other processes on the machine by mistake, and you might run into port conflicts trying to listen on ports.) Can this be resolved without more macOS features? Honestly, I bet you could do it. You'd probably need to inject a usermode network stack somehow, though, and that might just be a bit tricky to do on Darwin. Ultimately, I fully understand why Apple won't implement something like network namespaces in the kernel (it's complicated and it introduces a large surface area for bugs) but not having them is a bummer. They're awesome. I use them for a variety of things. An end user use case that I think is really cool is isolating processes into a network jail where the only route out is a Wireguard VPN tunnel. (And to be clear, I'm not saying if Darwin had network namespaces it'd suddenly be on the same level as everything else, but it would be a great start IMO.)
- jbverschoor 2y ago