3 ms·

Hmm, what kind of VM? By default, my M1 Mac will only run arm64 software under Docker. If an arm64 build is not available for something (ie x64 only), I can cha

Hmm, what kind of VM? By default, my M1 Mac will only run arm64 software under Docker. If an arm64 build is not available for something (ie x64 only), I can change the architecture setting, and only then will Docker attempt to run the process using Qemo. If Docker on Mac is already a VM, why does it need Qemo? Why does the Docker VM not emulate across architectures?

dboreham 4y ago
VM != cpu emulation, but fwiw I hear the opposite: M1 mac runs aarch64 containers natively, and emulates x64 containers. Mostly this works until it becomes confused and tries to run an x64 container without emulation.
- xhrpost 4y ago
  Yes I got the architectures messed up, meant what you said.
- pxc 4y ago
  > M1 mac runs aarch64 containers natively, and emulates x64 containers Macs don't run any containers natively, because macOS doesn't support containers and doesn't have a Linux ABI translation layer (some operating systems, like Illumos and NetBSD, do, and you can run Linux binaries on them almost like you can run Windows binaries on Linux via WINE). On M1, when you run aarch64 containers under Docker, you are 'merely' emulating an environment for a Linux kernel to run in and you can pass certain aspects of your CPU through. But when you do the same with x86_64 containers, you are additionally 'emulating' (via a translation layer) x86_64. Hopefully some day macOS will have its own truly native containers, like Linux and Windows do. That could be a serious savings for containerized macOS development environments, because they will be way more efficient than Docker Desktop on Mac currently can be. For now, the only way to get that kind of efficiency is Nix— which is great and better suited to that use case than Docker, but underutilized at many organizations. Probably even then people will still use virtualized Linux, since that's a closer environment to where apps will usually run in prod. But at such an org I'd argue for using macOS containers for local development, should they ever come to be available. The efficiency gains are well worth it IME.
detaro 4y ago
I'd guess so they get away with one VM (and have all the containers in one VM and thus under one kernel, for networking them etc?)? One arm64 VM, using qemu to run x86 when needed, achieves that.
pxc 4y ago
> Why does the Docker VM not emulate across architectures? Because it's easier to use a CPU emulator on just one kernel a single VM than manage multiple VMs. So you can use QEMU just like Macs already use Rosetta for x86_64 macOS binaries on aarch64 macOS. Linux further has a built-in feature for running 'foreign' (different CPU architecture, but still Linux) binaries (and more!) in a transparent way, so if you set this up in the VM you can invoke those x86_64 binaries as if they were native, without really thinking about it: https://www.kernel.org/doc/html/latest/admin-guide/binfmt-misc.html https://www.kernel.org/doc/html/latest/admin-guide/binfmt-mi... You can also use it with DOS and Windows binaries, or via Apple's Rosetta for Linux instead of QEMU for CPU emulation (and there are some other open-source competitors in that space as well). Pretty neat!

dboreham 4y ago

VM != cpu emulation, but fwiw I hear the opposite: M1 mac runs aarch64 containers natively, and emulates x64 containers. Mostly this works until it becomes confused and tries to run an x64 container without emulation.

xhrpost 4y ago
Yes I got the architectures messed up, meant what you said.
pxc 4y ago
> M1 mac runs aarch64 containers natively, and emulates x64 containers Macs don't run any containers natively, because macOS doesn't support containers and doesn't have a Linux ABI translation layer (some operating systems, like Illumos and NetBSD, do, and you can run Linux binaries on them almost like you can run Windows binaries on Linux via WINE). On M1, when you run aarch64 containers under Docker, you are 'merely' emulating an environment for a Linux kernel to run in and you can pass certain aspects of your CPU through. But when you do the same with x86_64 containers, you are additionally 'emulating' (via a translation layer) x86_64. Hopefully some day macOS will have its own truly native containers, like Linux and Windows do. That could be a serious savings for containerized macOS development environments, because they will be way more efficient than Docker Desktop on Mac currently can be. For now, the only way to get that kind of efficiency is Nix— which is great and better suited to that use case than Docker, but underutilized at many organizations. Probably even then people will still use virtualized Linux, since that's a closer environment to where apps will usually run in prod. But at such an org I'd argue for using macOS containers for local development, should they ever come to be available. The efficiency gains are well worth it IME.

detaro 4y ago

I'd guess so they get away with one VM (and have all the containers in one VM and thus under one kernel, for networking them etc?)? One arm64 VM, using qemu to run x86 when needed, achieves that.

pxc 4y ago

> Why does the Docker VM not emulate across architectures? Because it's easier to use a CPU emulator on just one kernel a single VM than manage multiple VMs. So you can use QEMU just like Macs already use Rosetta for x86_64 macOS binaries on aarch64 macOS. Linux further has a built-in feature for running 'foreign' (different CPU architecture, but still Linux) binaries (and more!) in a transparent way, so if you set this up in the VM you can invoke those x86_64 binaries as if they were native, without really thinking about it: https://www.kernel.org/doc/html/latest/admin-guide/binfmt-misc.html https://www.kernel.org/doc/html/latest/admin-guide/binfmt-mi... You can also use it with DOS and Windows binaries, or via Apple's Rosetta for Linux instead of QEMU for CPU emulation (and there are some other open-source competitors in that space as well). Pretty neat!