6 ms·

I'm curious as to why this isn't 0ms for a VM. The entire state of the system can be known ahead of time. Why does the kernel need to do any kind of dynamic ini

I'm curious as to why this isn't 0ms for a VM. The entire state of the system can be known ahead of time. Why does the kernel need to do any kind of dynamic initialization? Why aren't all data structures and variables statically assigned to proper values for the given VM. So at VM start up time it simply enters the main dispatch loop.

tristor 3y ago
Mostly because that's not how the kernel is designed. While what you're saying is true, it would require a fundamentally different kernel design. The kernel as it is today is designed to work on a variety of devices, not just VMs, so it behaves in a way in which it doesn't have specific knowledge of the device prior to initialization. To do what you're suggesting, the kernel would need to have a special mode where it supports injection of this information from the VM host in advance of initialization, and there's probably a bunch of possible security issues involved as well.
- anthk 3y ago
  NetBSD can just trim the kernel down to Virtio support and not much more.
- deleted 3y ago
  [deleted]
- alexey-salmin 3y ago
  You don't need kernel support, just freeze it's state right after the boot and clone it afterwards. Boot time will be the setup of shadow page tables and other hypervisor structures at new offsets. Plus a few tricks to avoid reproducible RNGs.
  - inkyoto 3y ago
    > […] freeze it's state right after the boot and clone it afterwards. You can't do that easily as devices can be attached, detached or reattached dynamically whilst the VM is running and when it is shut down. In lieu of the hardware / VM support for device trees, with the existing design, the kernel has to probe each device upon every boot.
    - alexey-salmin 3y ago
      Well it's not an ordinary boot of course, it's a clone and the state on both side of the fence should be faithfully cloned including virtual devices. The guest kernel will have no way of knowing whether it was resumed once or twice. Applicability is limited, but works well to spin up serverless compute on-demand.
andai 3y ago
I wonder the same thing for my regular OS too. Sure the first time it boots on this hardware, it needs to see what's what. But the subsequent hundred/thousand boots, at least for my usage, will be identical.
- vel0city 3y ago
  That's kind of the point of Windows' Fast Boot option. The base OS pretty much just hibernates at a normal shutdown. No point in re-initializing and restarting all the base services.
  - walteweiss 3y ago
    Is it Windows or BIOS/UEFI option? Will it work on Linux, *BSDs?
    - yjftsjthsd-h 3y ago
      That particular feature is a Windows feature, though there's no reason you theoretically couldn't do something similar on another OS.
    - vel0city 3y ago
      There is also often a UEFI option for a "fast boot" as well, where it skips some steps in its initialization, doesn't wait for any user input or logging to the local console if its coming back from a successful boot/power off cycle. But that's separate from the Windows feature which is related to hibernation. You could have Windows Fast Boot enabled while having your UEFI set to a full boot cycle or vice versa. The UEFI option should work fine with pretty much any OS the hardware should be able to boot.
- goneri 3y ago
  Before you can reload some previous known state, you need to initialize the hardware, load the firmwares and probe the devices (USB, SATA, PCI), discover the storage volumes and the partition tables. All these steps take time.
mixmastamyk 3y ago
Sounds like resuming from a snapshot.
- actionfromafar 3y ago
  Also sounds like unikernels.
- marwis 3y ago
  Exactly, you just need to separate things that need to happen before snapshot from things that must be run every time. Then you just snapshot as part of the build. This is how GraalVM and OpenJ9 achieve instant startup of Java programs.
  - pjmlp 3y ago
    Or Smalltalk and Common Lisp images.
shaky-carrousel 3y ago
Wouldn't that mess with RNGs?
- nielsole 3y ago
  maybe one could use hardware RNGs? not sure if OSes can be made to not cache/buffer entropy, or be forced to reset RNGs? EDIT: at least linux seems to be capable of doing that: https://www.systutorials.com/docs/linux/man/9-crypto_rng_reset/ https://www.systutorials.com/docs/linux/man/9-crypto_rng_res...

tristor 3y ago

Mostly because that's not how the kernel is designed. While what you're saying is true, it would require a fundamentally different kernel design. The kernel as it is today is designed to work on a variety of devices, not just VMs, so it behaves in a way in which it doesn't have specific knowledge of the device prior to initialization. To do what you're suggesting, the kernel would need to have a special mode where it supports injection of this information from the VM host in advance of initialization, and there's probably a bunch of possible security issues involved as well.

anthk 3y ago
NetBSD can just trim the kernel down to Virtio support and not much more.
deleted 3y ago
[deleted]
alexey-salmin 3y ago
You don't need kernel support, just freeze it's state right after the boot and clone it afterwards. Boot time will be the setup of shadow page tables and other hypervisor structures at new offsets. Plus a few tricks to avoid reproducible RNGs.
- inkyoto 3y ago
  > […] freeze it's state right after the boot and clone it afterwards. You can't do that easily as devices can be attached, detached or reattached dynamically whilst the VM is running and when it is shut down. In lieu of the hardware / VM support for device trees, with the existing design, the kernel has to probe each device upon every boot.
  - alexey-salmin 3y ago
    Well it's not an ordinary boot of course, it's a clone and the state on both side of the fence should be faithfully cloned including virtual devices. The guest kernel will have no way of knowing whether it was resumed once or twice. Applicability is limited, but works well to spin up serverless compute on-demand.

andai 3y ago

I wonder the same thing for my regular OS too. Sure the first time it boots on this hardware, it needs to see what's what. But the subsequent hundred/thousand boots, at least for my usage, will be identical.

vel0city 3y ago
That's kind of the point of Windows' Fast Boot option. The base OS pretty much just hibernates at a normal shutdown. No point in re-initializing and restarting all the base services.
- walteweiss 3y ago
  Is it Windows or BIOS/UEFI option? Will it work on Linux, *BSDs?
  - yjftsjthsd-h 3y ago
    That particular feature is a Windows feature, though there's no reason you theoretically couldn't do something similar on another OS.
  - vel0city 3y ago
    There is also often a UEFI option for a "fast boot" as well, where it skips some steps in its initialization, doesn't wait for any user input or logging to the local console if its coming back from a successful boot/power off cycle. But that's separate from the Windows feature which is related to hibernation. You could have Windows Fast Boot enabled while having your UEFI set to a full boot cycle or vice versa. The UEFI option should work fine with pretty much any OS the hardware should be able to boot.
goneri 3y ago
Before you can reload some previous known state, you need to initialize the hardware, load the firmwares and probe the devices (USB, SATA, PCI), discover the storage volumes and the partition tables. All these steps take time.

mixmastamyk 3y ago

Sounds like resuming from a snapshot.

actionfromafar 3y ago
Also sounds like unikernels.
marwis 3y ago
Exactly, you just need to separate things that need to happen before snapshot from things that must be run every time. Then you just snapshot as part of the build. This is how GraalVM and OpenJ9 achieve instant startup of Java programs.
- pjmlp 3y ago
  Or Smalltalk and Common Lisp images.

shaky-carrousel 3y ago

Wouldn't that mess with RNGs?

nielsole 3y ago
maybe one could use hardware RNGs? not sure if OSes can be made to not cache/buffer entropy, or be forced to reset RNGs? EDIT: at least linux seems to be capable of doing that: https://www.systutorials.com/docs/linux/man/9-crypto_rng_reset/ https://www.systutorials.com/docs/linux/man/9-crypto_rng_res...