4 ms·
Firecracker keeps a bitmap of which pages have been dirtied (it's a flag you can turn on), so you can make incremental snapshots of only the changed pages (more
by CompuIves 4y ago
Firecracker keeps a bitmap of which pages have been dirtied (it's a flag you can turn on), so you can make incremental snapshots of only the changed pages (more here: https://github.com/firecracker-microvm/firecracker/blob/main/docs/snapshotting/snapshot-support.md https://github.com/firecracker-microvm/firecracker/blob/main...).
In our case we changed Firecracker to use a shared mmap instead of an private mmap, so in our case the dirtied pages were synced back automatically to the backing memory file. The main reason for this was to reduce IO on snapshot time. I'm also looking at other ways we can do this, because using a shared mmap fragments the underlying xfs fs pretty fast. Maybe we can batch writes more instead of writing single pages.
- liuliu 4y agoCan these just be shared memory all the way through or in your case, the persistence to disk at fork time is important?
- CompuIves 4y agoIt could be shared memory all the way through, but the memory of the original VM should become read-only once a cloned VM starts reading from it. So then both VMs (the original VM and the new VM) should put their writes in a new CoW layer. Using XFS with CoW has been the easiest way to enable this, but if there's a way that we can do this purely in-memory, that would be even faster. That said, for hibernation we would still have to persist to disk, but timing is less important there.