6 ms·

OpenBSD-current now runs as guest under Apple Hypervisor

408 points by gpi 9mo ago

my123 9mo ago
Note that this is about Virtualization.framework (Apple's first party VMM). OpenBSD worked on Hypervisor.framework + qemu since a very long time.
- cpach 9mo ago
  Good point. The naming of those frameworks is sooo confusing. IMHO, nearly impossible to not mix them up.
  - carlm42 9mo ago
    My mental model is that each of these covers a different layer of the stack, from lowest to highest: * hypervisor-framework handles the hypervisor bits, like creating virtual machines, virtualising hardware resources, basically a C API on top of Apple's hypervisor * virtualization-framework is a higher-level API, meant to make it easy to run a full-blown VM with an OS and hardware integration, without having to reinvent the integration with lower-level primitives that hypervisor-framework provides * containerization-framework uses virtualization-framework to run Linux containers on macOS in microVMs. By analogy to not mix them up, it's a bit like KVM > QEMU > containerd. Hope this helps!
    - Angostura 9mo ago
      Well, it help me. So thanks!
- tannhaeuser 9mo ago
  Out of my depth here. Is that the one Tahoe was introducing? What did it solve that was impossible before?
  - m132 9mo ago
    Virtualization.framework was introduced in Big Sur. It builds on top of Hypervisor.framework and is essentially Apple's QEMU (in some ways quite literally, it implements QEMU's pvpanic protocol for example). Before QEMU and other VMMs gained ARM64 Hypervisor.framework support, it was the only way to run virtual machines on ARM Macs and still is the only official way to virtualize ARM macOS. The new Tahoe framework you're probably thinking of is Containerization, which is a WSL2-esque wrapper around Virtualization.framework allowing for easy installation of Linux containers.
    - archy_ 9mo ago
      >a WSL2-esque wrapper around Virtualization.framework allowing for easy installation of Linux containers. So Linux is now a first class citizen on both Windows and Mac? I guess it really is true that 'if you can't beat em, join em.' Jobs must be rolling in his grave.
      - trentnelson 9mo ago
        I mean to be fair, WSL1 and WSL2 are extremely successful engineering efforts by Microsoft. I can’t imagine having to go back to the Cygwin days.
        freedomben 9mo ago
        I'm one of the few I think who really liked Cygwin. Far from perfect of course, but I even still prefer it to WSL depending on what I'm doing.
      - thebiss 9mo ago
        It's well supported by the architecture. You may be interested in: - Lima - wsl2-like access to a virtual machine https://github.com/lima-vm/lima/blob/master/README.md https://github.com/lima-vm/lima/blob/master/README.md - vfkit - CLI creation and management of applehv VMs https://github.com/crc-org/vfkit https://github.com/crc-org/vfkit - podman machine - easily run x86 containers in CoreOs, via the podman CLI https://docs.podman.io/en/latest/markdown/podman-machine.1.html https://docs.podman.io/en/latest/markdown/podman-machine.1.h...
- atmosx 9mo ago
  Oh good point. I mixed it up, UTM is using qemu under hood, but as someone mentioned now OpenBSD snapshot boots with qemu seemlesly. It's still virtualised though.
  - irusensei 9mo ago
    It can also use the apple native hypervisor.
    - atmosx 9mo ago
      Tried it earlier using UTM and the Apple hypervisor but didn’t boot.
SomaticPirate 9mo ago
Is there a guide on how to do this? I haven’t ever used the raw hypervisor.
- eschaton 9mo ago
  It should just be a matter of producing a kernel and, if necessary, RAM disk that can be booted the same way as Linux.
  - cpach 9mo ago
    Then one needs to launch it. Not sure if there are any lancher UIs out there, or if one has to write custom code for that.
    - eschaton 9mo ago
      My point is that as long as OpenBSD can boot like Linux, you just have to tell whatever VM front-end you’re using that you’re booting a Linux but give it an OpenBSD kernel and RAM disk. Traditionally BSD has booted very differently than Linux, because Linus adopted the same boot process as MINIX when he first developed it (since he was actually using the MINIX boot blocks at first). BSD has historically used a bootstrap that understands V7FS/FFS and can load a kernel from a path on it. MINIX takes the actual kernel and RAM disk images as parameters so it doesn’t need to know about filesystems, and that tradition continued with Linux bootstraps once it was standalone.
      - Rediscover 9mo ago
        Who else was rdev'ing the Linux kernel to tell it where the root ext2(?) partition was long before they were using RAM disks? Like with SLS or MCC?
        fragmede 9mo ago
        Originally Linux had Minix FS, followed by ext. Ext2 wouldn't make an appearance until 1993 by Rémy Card, so it depends on when you were using it.
patjensen 9mo ago
The bigger news is that this also fixes the QEMU compatibility bug that makes OpenBSD hang out of the box on arm64 when starting X. It started in 7.3 with the frame buffer changes and the only workaround was to disable the kernel driver. Maybe more people will get to try out OpenBSD successfully now.
- danwills 9mo ago
  [flagged]
  - fleshmonad 9mo ago
    Maybe not posting such comments would increase the value of this site.
- someguyiguess 9mo ago
  I am one of them! Been wanting to try it out for a while but my only available machine is an MBP
  - patjensen 9mo ago
    I can confirm it is fixed in the latest OpenBSD snapshot build. Works great.
- archy_ 9mo ago
  Why does QEMU need to start X? Shouldn't that be OpenBSD's responsibility?
  - deleted 9mo ago
    [deleted]
  - codys 9mo ago
    OpenBSD does start X. And subsequently OpenBSD apparently hangs (or did so previously) when OpenBSD was running under Qemu. The subject in the parent comment changed to OpenBSD when they mentioned it, and it appears you may have overlooked the subject change.
Fiveplus 9mo ago
A good update. The VIRTIO_NET_F_MTU negotiation has been a roadblock for many guest OS implementations on apple's virtualization stack. The spec is vague enough that linux just does it while openbsd had to explicitly patch in support to handle the hypervisor's hardmtu limit. This is a big deal for local development imho. With the raw single-thread performance of the M4/M5 chips, an openbsd guest is arguably the best environment for testing pf configurations or running isolated mail servers (for example). Being able to rely on viogpu without the black-screen-of-death means we can slowly move away from serial console-only installs for quick VMs. Big kudos to Helg and Stefan!
- eru 9mo ago
  > With the raw single-thread performance of the M4/M5 chips, an openbsd guest is arguably the best environment for testing pf configurations or running isolated mail servers (for example). A unikernel would probably be even better? (But then you need a mail server that's set up for running as a unikernel, without an underlying OS.)
- wang_li 9mo ago
  >Being able to rely on viogpu without the black-screen-of-death means we can slowly move away from serial console-only installs for quick VMs. No, thanks. My IaC doesn't want or need any interaction when spinning up a quick or slow VM.
iberator 9mo ago
No X and networking. What's the point then? Useless imo
- mghackerlady 9mo ago
  Networking is a disappointment but OpenBSD kinda expects you to use the command line. Fire up tmux (included because they invented it) and read the god tier manpages and play some of the games that are included
  - cyberpunk 9mo ago
    OpenBSD did not “invent” tmux afaik it was imported later.
    - brynet 9mo ago
      tmux was my first OpenBSD port, way back in 2008, albeit it was rather short-lived. https://marc.info/?l=openbsd-ports-cvs&m=121226747005033&w=2 https://marc.info/?l=openbsd-ports-cvs&m=121226747005033&w=2 I had discovered it searching on SourceForge originally, but the tmux creator Nicholas Marriott was already an OpenBSD user and he took MAINTAINER for the port. A year later, tmux was imported by nicm@ to the OpenBSD base system, where it has remained upstream for last 16 years (GitHub sync's from OpenBSD). https://marc.info/?l=openbsd-cvs&m=124389728412353&w=2 https://marc.info/?l=openbsd-cvs&m=124389728412353&w=2
hindustanuday 9mo ago
[dead]
MillionOClock 9mo ago
Maybe I am missing something but the last few times I tested VMs it seemed to end up never shrinking in RAM size once it had grown, is this a real issue and if so is there any improvement coming on that front?
- TacticalCoder 9mo ago
  > Maybe I am missing something but the last few times I tested VMs ... Tested VMs on what? For VMs are used daily and there are, what, hundreds of millions of VMs running as we speak? Billions?
- daneel_w 9mo ago
  You're missing the complexity of making the guest inform the host that it has fully freed this and that slab of memory and that the host may reclaim it until further notice. It's a bit more complicated than the other way around, where the guest believes it has e.g. 4 GiB of RAM available but the host doesn't allocate all of it for the guest until it tries to read/write there. A virtual machine is something entirely else than a containerized piece of software.
ggm 9mo ago
Well done! FreeBSD 15 is a complete no-go for X right now on utm, rdp/vnc is the only way. Hopefully somebody will work out how to get a frame buffer working there, from this.
singularity2001 9mo ago
so does redox at least this fork: https://github.com/pannous/redox https://github.com/pannous/redox completely Rust-based without any Makefiles.
infi_v12 9mo ago
This is a significant milestone for OpenBSD on Apple hardware. The improved support for Virtualization.framework will definitely make local development and testing much smoother for many users. Kudos to the developers!
deleted 9mo ago
[deleted]
maximgeorge 9mo ago
[dead]
6r17 9mo ago
I wonder if openbsd is secure running as a guest ? it it able to isolate it-self sufficiently so that the host cannot mathematically breach it ? (which makes openbsd very suitable for keyholding)
- muricula 9mo ago
  The host kernel and probably the host VMM can see guest memory, so I wouldn't use it for that.
  - daneel_w 9mo ago
    https://www.amd.com/en/developer/sev.html https://www.amd.com/en/developer/sev.html
- ikmckenz 9mo ago
  As of 2025 OpenBSD has support for AMD SEV and SEV-ES, with support for SEV-SNP work-in-progress, so with the right hardware yes it's able to isolate itself sufficiently https://www.bsdcan.org/2025/timetable/timetable-Confidential-Computing-with.html https://www.bsdcan.org/2025/timetable/timetable-Confidential...
irusensei 9mo ago
On a slightly related note UTM remote is such a nice remote client for VMs that I wish they would make it compatible with other hypervisor protocols such as libvirtd and bhyve.

408 points by gpi 9mo ago

my123 9mo ago

Note that this is about Virtualization.framework (Apple's first party VMM). OpenBSD worked on Hypervisor.framework + qemu since a very long time.

cpach 9mo ago
Good point. The naming of those frameworks is sooo confusing. IMHO, nearly impossible to not mix them up.
- carlm42 9mo ago
  My mental model is that each of these covers a different layer of the stack, from lowest to highest: * hypervisor-framework handles the hypervisor bits, like creating virtual machines, virtualising hardware resources, basically a C API on top of Apple's hypervisor * virtualization-framework is a higher-level API, meant to make it easy to run a full-blown VM with an OS and hardware integration, without having to reinvent the integration with lower-level primitives that hypervisor-framework provides * containerization-framework uses virtualization-framework to run Linux containers on macOS in microVMs. By analogy to not mix them up, it's a bit like KVM > QEMU > containerd. Hope this helps!
  - Angostura 9mo ago
    Well, it help me. So thanks!
tannhaeuser 9mo ago
Out of my depth here. Is that the one Tahoe was introducing? What did it solve that was impossible before?
- m132 9mo ago
  Virtualization.framework was introduced in Big Sur. It builds on top of Hypervisor.framework and is essentially Apple's QEMU (in some ways quite literally, it implements QEMU's pvpanic protocol for example). Before QEMU and other VMMs gained ARM64 Hypervisor.framework support, it was the only way to run virtual machines on ARM Macs and still is the only official way to virtualize ARM macOS. The new Tahoe framework you're probably thinking of is Containerization, which is a WSL2-esque wrapper around Virtualization.framework allowing for easy installation of Linux containers.
  - archy_ 9mo ago
    >a WSL2-esque wrapper around Virtualization.framework allowing for easy installation of Linux containers. So Linux is now a first class citizen on both Windows and Mac? I guess it really is true that 'if you can't beat em, join em.' Jobs must be rolling in his grave.
    - trentnelson 9mo ago
      I mean to be fair, WSL1 and WSL2 are extremely successful engineering efforts by Microsoft. I can’t imagine having to go back to the Cygwin days.
      - freedomben 9mo ago
        I'm one of the few I think who really liked Cygwin. Far from perfect of course, but I even still prefer it to WSL depending on what I'm doing.
    - thebiss 9mo ago
      It's well supported by the architecture. You may be interested in: - Lima - wsl2-like access to a virtual machine https://github.com/lima-vm/lima/blob/master/README.md https://github.com/lima-vm/lima/blob/master/README.md - vfkit - CLI creation and management of applehv VMs https://github.com/crc-org/vfkit https://github.com/crc-org/vfkit - podman machine - easily run x86 containers in CoreOs, via the podman CLI https://docs.podman.io/en/latest/markdown/podman-machine.1.html https://docs.podman.io/en/latest/markdown/podman-machine.1.h...
atmosx 9mo ago
Oh good point. I mixed it up, UTM is using qemu under hood, but as someone mentioned now OpenBSD snapshot boots with qemu seemlesly. It's still virtualised though.
- irusensei 9mo ago
  It can also use the apple native hypervisor.
  - atmosx 9mo ago
    Tried it earlier using UTM and the Apple hypervisor but didn’t boot.

SomaticPirate 9mo ago

Is there a guide on how to do this? I haven’t ever used the raw hypervisor.

eschaton 9mo ago
It should just be a matter of producing a kernel and, if necessary, RAM disk that can be booted the same way as Linux.
- cpach 9mo ago
  Then one needs to launch it. Not sure if there are any lancher UIs out there, or if one has to write custom code for that.
  - eschaton 9mo ago
    My point is that as long as OpenBSD can boot like Linux, you just have to tell whatever VM front-end you’re using that you’re booting a Linux but give it an OpenBSD kernel and RAM disk. Traditionally BSD has booted very differently than Linux, because Linus adopted the same boot process as MINIX when he first developed it (since he was actually using the MINIX boot blocks at first). BSD has historically used a bootstrap that understands V7FS/FFS and can load a kernel from a path on it. MINIX takes the actual kernel and RAM disk images as parameters so it doesn’t need to know about filesystems, and that tradition continued with Linux bootstraps once it was standalone.
    - Rediscover 9mo ago
      Who else was rdev'ing the Linux kernel to tell it where the root ext2(?) partition was long before they were using RAM disks? Like with SLS or MCC?
      - fragmede 9mo ago
        Originally Linux had Minix FS, followed by ext. Ext2 wouldn't make an appearance until 1993 by Rémy Card, so it depends on when you were using it.

patjensen 9mo ago

The bigger news is that this also fixes the QEMU compatibility bug that makes OpenBSD hang out of the box on arm64 when starting X. It started in 7.3 with the frame buffer changes and the only workaround was to disable the kernel driver. Maybe more people will get to try out OpenBSD successfully now.

danwills 9mo ago
[flagged]
- fleshmonad 9mo ago
  Maybe not posting such comments would increase the value of this site.
someguyiguess 9mo ago
I am one of them! Been wanting to try it out for a while but my only available machine is an MBP
- patjensen 9mo ago
  I can confirm it is fixed in the latest OpenBSD snapshot build. Works great.
archy_ 9mo ago
Why does QEMU need to start X? Shouldn't that be OpenBSD's responsibility?
- deleted 9mo ago
  [deleted]
- codys 9mo ago
  OpenBSD does start X. And subsequently OpenBSD apparently hangs (or did so previously) when OpenBSD was running under Qemu. The subject in the parent comment changed to OpenBSD when they mentioned it, and it appears you may have overlooked the subject change.

Fiveplus 9mo ago

A good update. The VIRTIO_NET_F_MTU negotiation has been a roadblock for many guest OS implementations on apple's virtualization stack. The spec is vague enough that linux just does it while openbsd had to explicitly patch in support to handle the hypervisor's hardmtu limit. This is a big deal for local development imho. With the raw single-thread performance of the M4/M5 chips, an openbsd guest is arguably the best environment for testing pf configurations or running isolated mail servers (for example). Being able to rely on viogpu without the black-screen-of-death means we can slowly move away from serial console-only installs for quick VMs. Big kudos to Helg and Stefan!

eru 9mo ago
> With the raw single-thread performance of the M4/M5 chips, an openbsd guest is arguably the best environment for testing pf configurations or running isolated mail servers (for example). A unikernel would probably be even better? (But then you need a mail server that's set up for running as a unikernel, without an underlying OS.)
wang_li 9mo ago
>Being able to rely on viogpu without the black-screen-of-death means we can slowly move away from serial console-only installs for quick VMs. No, thanks. My IaC doesn't want or need any interaction when spinning up a quick or slow VM.

iberator 9mo ago

No X and networking. What's the point then? Useless imo

mghackerlady 9mo ago
Networking is a disappointment but OpenBSD kinda expects you to use the command line. Fire up tmux (included because they invented it) and read the god tier manpages and play some of the games that are included
- cyberpunk 9mo ago
  OpenBSD did not “invent” tmux afaik it was imported later.
  - brynet 9mo ago
    tmux was my first OpenBSD port, way back in 2008, albeit it was rather short-lived. https://marc.info/?l=openbsd-ports-cvs&m=121226747005033&w=2 https://marc.info/?l=openbsd-ports-cvs&m=121226747005033&w=2 I had discovered it searching on SourceForge originally, but the tmux creator Nicholas Marriott was already an OpenBSD user and he took MAINTAINER for the port. A year later, tmux was imported by nicm@ to the OpenBSD base system, where it has remained upstream for last 16 years (GitHub sync's from OpenBSD). https://marc.info/?l=openbsd-cvs&m=124389728412353&w=2 https://marc.info/?l=openbsd-cvs&m=124389728412353&w=2

hindustanuday 9mo ago

[dead]

MillionOClock 9mo ago

Maybe I am missing something but the last few times I tested VMs it seemed to end up never shrinking in RAM size once it had grown, is this a real issue and if so is there any improvement coming on that front?

TacticalCoder 9mo ago
> Maybe I am missing something but the last few times I tested VMs ... Tested VMs on what? For VMs are used daily and there are, what, hundreds of millions of VMs running as we speak? Billions?
daneel_w 9mo ago
You're missing the complexity of making the guest inform the host that it has fully freed this and that slab of memory and that the host may reclaim it until further notice. It's a bit more complicated than the other way around, where the guest believes it has e.g. 4 GiB of RAM available but the host doesn't allocate all of it for the guest until it tries to read/write there. A virtual machine is something entirely else than a containerized piece of software.

ggm 9mo ago

Well done! FreeBSD 15 is a complete no-go for X right now on utm, rdp/vnc is the only way. Hopefully somebody will work out how to get a frame buffer working there, from this.

singularity2001 9mo ago

so does redox at least this fork: https://github.com/pannous/redox https://github.com/pannous/redox completely Rust-based without any Makefiles.

infi_v12 9mo ago

This is a significant milestone for OpenBSD on Apple hardware. The improved support for Virtualization.framework will definitely make local development and testing much smoother for many users. Kudos to the developers!

deleted 9mo ago

[deleted]

maximgeorge 9mo ago

[dead]

6r17 9mo ago

I wonder if openbsd is secure running as a guest ? it it able to isolate it-self sufficiently so that the host cannot mathematically breach it ? (which makes openbsd very suitable for keyholding)

muricula 9mo ago
The host kernel and probably the host VMM can see guest memory, so I wouldn't use it for that.
- daneel_w 9mo ago
  https://www.amd.com/en/developer/sev.html https://www.amd.com/en/developer/sev.html
ikmckenz 9mo ago
As of 2025 OpenBSD has support for AMD SEV and SEV-ES, with support for SEV-SNP work-in-progress, so with the right hardware yes it's able to isolate itself sufficiently https://www.bsdcan.org/2025/timetable/timetable-Confidential-Computing-with.html https://www.bsdcan.org/2025/timetable/timetable-Confidential...

irusensei 9mo ago

On a slightly related note UTM remote is such a nice remote client for VMs that I wish they would make it compatible with other hypervisor protocols such as libvirtd and bhyve.