3 ms·

Very naiive question... But why can't you "just" memcopy and exec an image of an already booted system?

throwawaylinux 3y ago
Simplistic answer is because hardware and firmware environment gets configured during boot, even for virtual machines. Some virtual machine environments do have facilities to snapshot the entire state of a machine and resume it (potentially on another machine), but if you look at what is involved with snapshot metadata it's more than just the state of memory, you have CPU and device state as well.
- PMunch 3y ago
  Wouldn't hibernating the machine and then copying the hibernation data to another machine do the trick? Sure it wouldn't be as instant, but it would skip most of the boot configuration.
  - Angostura 3y ago
    Possibly a bit fragile if there is any situation where one day the other machine isn't entirely identical? (just hand-waving here)
  - j16sdiz 3y ago
    The usual culprit is CPU feature flags on different CPU generations.
    - LeifCarrotson 3y ago
      Well then wouldn't you just have one image file for each CPU generation in your datacenter. Or better yet, have one image file on the hypervisor for each individual machine! That machine is probably dedicated to Lambda instances and boots them 1000x per day, if you're spending valuable time that's actually customer-facing latency on each call to reboot the system then switch that memory<->time tradeoff back towards memory!
  - throwawaylinux 3y ago
    Conceptually at a high level yes (if the hardware was ~identical). Resuming from hibernation on the same machine resuming from hibernation is basically loading an image into memory and executing it that OP asked about too. But in either the resume or hypothetical hypernation migration cases, you can't just start running where the previous boot left off, you have to go through hibernation resume functions which basically do the same thing as boot in terms of configuring hardware. On virtual machines there probably isn't nearly so much real hardware to set up, and hypervisors can optimize virtual device setup. The machine image could also be provided via COW snapshot of memory. So I don't know, possibly at some corners it could be faster to do a resume type operation than a full boot. I haven't played with hypervisor guest boot at these kinds of speeds before.
tinus_hn 3y ago
There’s things like encryption keys and tokens that need to be unique so it’s not easy, if you do it like that you would run into new, difficult to predict vulnerabilities. So it’s difficult and dangerous for not a lot of gain. Perhaps an easier approach would be to boot a few spare systems in advance and hibernate them so they can be booted quickly.
crest 3y ago
You sort of can, but drivers have to be in sync with the (emulated) devices they're controlling. The fastest way to go about this would be to snapshot an already booted virtual system. It could be as simple as instructing the restored snapshot to reseed its RNG and (re-)attach to the network, but you'll probably in into a few more assumption broken by cloning a multiple VMs from the same snapshot. If pre-booting and suspending/resuming each individual VM is good enough it could look similar to a pre-forking HTTP server like old Apache versions.
- Shorel 3y ago
  Yeah, for real hardware it is important to be flexible. But these virtual machines can all be exactly the same, except for the network MAC and IP. It could even be possible to compile executables that bypass the need for a kernel, just have some libraries with the required stuff, and boot to the program, MS-DOS style.
  - Terretta 3y ago
    > could even be possible to compile executables that bypass the need for a kernel, just have some libraries with the required stuff, and boot to the program Or consider: https://github.com/uniqernel/awesome-unikernels https://github.com/uniqernel/awesome-unikernels

Simplistic answer is because hardware and firmware environment gets configured during boot, even for virtual machines. Some virtual machine environments do have facilities to snapshot the entire state of a machine and resume it (potentially on another machine), but if you look at what is involved with snapshot metadata it's more than just the state of memory, you have CPU and device state as well.

PMunch 3y ago
Wouldn't hibernating the machine and then copying the hibernation data to another machine do the trick? Sure it wouldn't be as instant, but it would skip most of the boot configuration.
- Angostura 3y ago
  Possibly a bit fragile if there is any situation where one day the other machine isn't entirely identical? (just hand-waving here)
- j16sdiz 3y ago
  The usual culprit is CPU feature flags on different CPU generations.
  - LeifCarrotson 3y ago
    Well then wouldn't you just have one image file for each CPU generation in your datacenter. Or better yet, have one image file on the hypervisor for each individual machine! That machine is probably dedicated to Lambda instances and boots them 1000x per day, if you're spending valuable time that's actually customer-facing latency on each call to reboot the system then switch that memory<->time tradeoff back towards memory!
- throwawaylinux 3y ago
  Conceptually at a high level yes (if the hardware was ~identical). Resuming from hibernation on the same machine resuming from hibernation is basically loading an image into memory and executing it that OP asked about too. But in either the resume or hypothetical hypernation migration cases, you can't just start running where the previous boot left off, you have to go through hibernation resume functions which basically do the same thing as boot in terms of configuring hardware. On virtual machines there probably isn't nearly so much real hardware to set up, and hypervisors can optimize virtual device setup. The machine image could also be provided via COW snapshot of memory. So I don't know, possibly at some corners it could be faster to do a resume type operation than a full boot. I haven't played with hypervisor guest boot at these kinds of speeds before.

tinus_hn 3y ago

There’s things like encryption keys and tokens that need to be unique so it’s not easy, if you do it like that you would run into new, difficult to predict vulnerabilities. So it’s difficult and dangerous for not a lot of gain. Perhaps an easier approach would be to boot a few spare systems in advance and hibernate them so they can be booted quickly.

crest 3y ago

You sort of can, but drivers have to be in sync with the (emulated) devices they're controlling. The fastest way to go about this would be to snapshot an already booted virtual system. It could be as simple as instructing the restored snapshot to reseed its RNG and (re-)attach to the network, but you'll probably in into a few more assumption broken by cloning a multiple VMs from the same snapshot. If pre-booting and suspending/resuming each individual VM is good enough it could look similar to a pre-forking HTTP server like old Apache versions.

Shorel 3y ago
Yeah, for real hardware it is important to be flexible. But these virtual machines can all be exactly the same, except for the network MAC and IP. It could even be possible to compile executables that bypass the need for a kernel, just have some libraries with the required stuff, and boot to the program, MS-DOS style.
- Terretta 3y ago
  > could even be possible to compile executables that bypass the need for a kernel, just have some libraries with the required stuff, and boot to the program Or consider: https://github.com/uniqernel/awesome-unikernels https://github.com/uniqernel/awesome-unikernels