3 ms·

Its page cache[0]. A similar thing happens with wsl2 on Windows[1]. The issue is the linux kernel running in the lima-vm thinks all the ram you give the VM is f

Its page cache[0]. A similar thing happens with wsl2 on Windows[1]. The issue is the linux kernel running in the lima-vm thinks all the ram you give the VM is free to cache files on, it doesn't know there is a host OS that could also make use of that ram. So after loading up a ton of containers where each has its own OS dependencies that ends up being a lot of files the kernel keeps cached so long as nothing else needs the memory. There isn't really a fix besides right sizing the amount of ram you give the VM. Or I guess you could disable file caching but I wouldn't recommend it[2]. [0] https://www.linuxatemyram.com/ https://www.linuxatemyram.com/ [1] https://news.ycombinator.com/item?id=26897095 https://news.ycombinator.com/item?id=26897095 [2] https://github.com/Feh/nocache https://github.com/Feh/nocache

3np 4y ago
One coukd also put their VM root filesystem in ZFS, switching the Linux page cache for more tunable ARC.
deleted 4y ago
[deleted]
MuffinFlavored 4y ago
When you're right, you're right. https://github.com/lima-vm/lima/blob/master/examples/default.yaml#L41 https://github.com/lima-vm/lima/blob/master/examples/default... 4gig default ~$ free -m total used free shared buff/cache available Mem: 3907 304 1446 1 2156 3431 Swap: 0 Thank you.
- ecnahc515 4y ago
  To add to this, the hypervisor can do more to integrate with the guest so that this issue is mitigated somewhat. For example, with KVM and a Linux guest there's a lot of optimizations to share memory pages between the two. For reference: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/virtualization_tuning_and_optimization_guide/chap-ksm https://access.redhat.com/documentation/en-us/red_hat_enterp....
FooBarWidget 4y ago
Does HyperV have a solution where the hypervisor integrates with the guest kernel, telling it to evict page caches when the host needs more memory?
- pzmarzly 4y ago
  The mechanism is called balooning and technically doesn't need cooperation on guest kernel side (but it makes everything much smoother). Not idea about HyperV, but it is implemented e.g. in virtio https://pmhahn.github.io/virtio-balloon/ https://pmhahn.github.io/virtio-balloon/ . You can google some failed experiments to add it to qemu
bloopernova 4y ago
Interesting! It looks like Fedora CoreOS inside podman maybe doesn't do this? [root@localhost ~]# free -h total used free shared buff/cache available Mem: 7.8Gi 212Mi 7.2Gi 8.0Mi 333Mi 7.3Gi Swap: 0B 0B 0B And Activity Monitor says that qemu-system-x86_64 is using 1.27GB of real memory.
saynay 4y ago
I remember VMWare solved this with a balloon driver, that basically asks the guest for a lot of memory at a very low priority, so it has precedence over the page cache, but under any program that actually needs it.

3np 4y ago

One coukd also put their VM root filesystem in ZFS, switching the Linux page cache for more tunable ARC.

deleted 4y ago

[deleted]

MuffinFlavored 4y ago

When you're right, you're right. https://github.com/lima-vm/lima/blob/master/examples/default.yaml#L41 https://github.com/lima-vm/lima/blob/master/examples/default... 4gig default ~$ free -m total used free shared buff/cache available Mem: 3907 304 1446 1 2156 3431 Swap: 0 Thank you.

ecnahc515 4y ago
To add to this, the hypervisor can do more to integrate with the guest so that this issue is mitigated somewhat. For example, with KVM and a Linux guest there's a lot of optimizations to share memory pages between the two. For reference: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/virtualization_tuning_and_optimization_guide/chap-ksm https://access.redhat.com/documentation/en-us/red_hat_enterp....

FooBarWidget 4y ago

Does HyperV have a solution where the hypervisor integrates with the guest kernel, telling it to evict page caches when the host needs more memory?

pzmarzly 4y ago
The mechanism is called balooning and technically doesn't need cooperation on guest kernel side (but it makes everything much smoother). Not idea about HyperV, but it is implemented e.g. in virtio https://pmhahn.github.io/virtio-balloon/ https://pmhahn.github.io/virtio-balloon/ . You can google some failed experiments to add it to qemu

bloopernova 4y ago

Interesting! It looks like Fedora CoreOS inside podman maybe doesn't do this? [root@localhost ~]# free -h total used free shared buff/cache available Mem: 7.8Gi 212Mi 7.2Gi 8.0Mi 333Mi 7.3Gi Swap: 0B 0B 0B And Activity Monitor says that qemu-system-x86_64 is using 1.27GB of real memory.

saynay 4y ago

I remember VMWare solved this with a balloon driver, that basically asks the guest for a lot of memory at a very low priority, so it has precedence over the page cache, but under any program that actually needs it.