6 ms·

Interesting. Would this still allow the virtualized machines to manage their own page tables as though native? I can't say I understand how it's implemented, b

Interesting. Would this still allow the virtualized machines to manage their own page tables as though native? I can't say I understand how it's implemented, but I'm concerned it might leave the host system with the same page thrashing the choice to use virtualization was intended to avoid.

Hello71 3y ago
that depends on your setup. ballooning is just a function to move memory from the guest to the host; if you move too much memory you'll get thrashing. the bigger problem is I/O indirection through the host; for I/O heavy workloads you'll want to use PCI passthrough for best performance, which (assuming it's working properly) should be identical to native performance.
shrubble 3y ago
You could examine LXC, which is a container setup, for isolation. Adding memory doesn't require a restart of the container. You may wish to examine ZFS or, going farther afield, even ZFS with FreeBSD with or without jails/bhyve.
- jeroenhd 3y ago
  They're trying to use VMs to run multiple page table managers on one single host. Containerization will use the same kernel's page management, so I don't think that'll solve the author's problem.
magicalhippo 3y ago
I don't know how VirtIO does it but VMWare has a kernel driver which allocates the pages from the guest OS. IIRC it locks these pages so they are unavailable until the driver releases them back.
jeroenhd 3y ago
I don't know if this has any impact on the page tables, but the way it works is relatively simple: the hypervisor communicates with a driver inside the VM and tells it to "allocate" large amounts of memory. How much depends on the configuration; if you tell the hypervisor to configure a minimum of 16GB and a maximum of 48GB, then the driver will "reserve" 32GB of RAM. To the VM, it's as if some kind of driver is taking up 20GB of RAM, while I reality the hypervisor just doesn't allocate the RAM to the VM. Then, when the VM runs out of memory, the virtio driver starts releasing pages for as long as the hypervisor lets it, slowly increasing the amount of RAM allocated to the virtual machine. This works until the hypervisor runs out of memory to allocate, after which the driver inside the VM will refuse to release pages and the VM's OOM will kick in. The default is grow only (start with the minimum amount of RAM and only release more to the VM as time goes by), but it's possible to ask the driver to start "allocating" memory again with enhancements, like the ones proxmox do or by using scripting. With the guest tools installed, the hypervisor knows exactly how much free RAM each VM has. This shouldn't cause problems for page tables on the client side as far as I know. However, certain workloads may not work well; if the VM allocates memory faster than the balloon driver canrrelease it, you'll get out of memory errors. Similarly, quick allocate-free cycles will prevent any auto shrink mechanism from working right. You could try and run an experiment if you wish; the balloon driver works just as well regardless of whether you're allocating 16GB of RAM or 512MB. You could run a bunch of VMs with a model of the memory allocation you expect on your desktop and see if you end up page trashing like you fear.

Hello71 3y ago

that depends on your setup. ballooning is just a function to move memory from the guest to the host; if you move too much memory you'll get thrashing. the bigger problem is I/O indirection through the host; for I/O heavy workloads you'll want to use PCI passthrough for best performance, which (assuming it's working properly) should be identical to native performance.

shrubble 3y ago

You could examine LXC, which is a container setup, for isolation. Adding memory doesn't require a restart of the container. You may wish to examine ZFS or, going farther afield, even ZFS with FreeBSD with or without jails/bhyve.

jeroenhd 3y ago
They're trying to use VMs to run multiple page table managers on one single host. Containerization will use the same kernel's page management, so I don't think that'll solve the author's problem.

magicalhippo 3y ago

I don't know how VirtIO does it but VMWare has a kernel driver which allocates the pages from the guest OS. IIRC it locks these pages so they are unavailable until the driver releases them back.

jeroenhd 3y ago

I don't know if this has any impact on the page tables, but the way it works is relatively simple: the hypervisor communicates with a driver inside the VM and tells it to "allocate" large amounts of memory. How much depends on the configuration; if you tell the hypervisor to configure a minimum of 16GB and a maximum of 48GB, then the driver will "reserve" 32GB of RAM. To the VM, it's as if some kind of driver is taking up 20GB of RAM, while I reality the hypervisor just doesn't allocate the RAM to the VM. Then, when the VM runs out of memory, the virtio driver starts releasing pages for as long as the hypervisor lets it, slowly increasing the amount of RAM allocated to the virtual machine. This works until the hypervisor runs out of memory to allocate, after which the driver inside the VM will refuse to release pages and the VM's OOM will kick in. The default is grow only (start with the minimum amount of RAM and only release more to the VM as time goes by), but it's possible to ask the driver to start "allocating" memory again with enhancements, like the ones proxmox do or by using scripting. With the guest tools installed, the hypervisor knows exactly how much free RAM each VM has. This shouldn't cause problems for page tables on the client side as far as I know. However, certain workloads may not work well; if the VM allocates memory faster than the balloon driver canrrelease it, you'll get out of memory errors. Similarly, quick allocate-free cycles will prevent any auto shrink mechanism from working right. You could try and run an experiment if you wish; the balloon driver works just as well regardless of whether you're allocating 16GB of RAM or 512MB. You could run a bunch of VMs with a model of the memory allocation you expect on your desktop and see if you end up page trashing like you fear.