8 ms·

How are they accomplishing this? I glanced at the documentation but couldn’t find anything in this regard.

• rwmj 3y ago
  By running the containers in VMs, and attempting to make the VMs boot very quickly by carefully tuning qemu (or Firecracker) and the guest kernel. The main problem with this approach is surprisingly not the time overhead - Kubernetes is fscking slow at scheduling regular containers - but the memory overhead, since you need to allocate sufficient memory up front for the largest possible memory usage of the container. Most containers expect to get more memory from the system simply by doing sbrk/mmap, and VMs simply don't work this way.
  • suprjami 3y ago
    Doesn't KVM work that way? You can give a VM "up to" a certain amount of memory but it doesn't allocate all that memory at first boot. iiuc that's because each VM is just a process which can increase its own heap. It's explained in more detail here: https://serverfault.com/questions/773581/virt-manager-partioning-the-available-memory https://serverfault.com/questions/773581/virt-manager-partio...
    • rwmj 3y ago
      Not really, no. The kernel has a certain overhead tracking pages and page tables, so adding extra memory and waiting for it to be swapped in isn't free. Plus you still have to account for the memory being used somehow. If you use cgroups - as is done most commonly - that will track the full memory allocated. Not to mention this would only solve half of the problem, you also have to think about what happens with munmap.
      • suprjami 3y ago
        Good points, thank you for the explanation!