13 ms·
This API is only used by one project to my knowledge... CRIU is a project to save an application or containers complete running state to a file, and then resto
by londons_explore 4y ago
This API is only used by one project to my knowledge...
CRIU is a project to save an application or containers complete running state to a file, and then restore it elsewhere.
It comes with lots of caveats.
- boucher 4y agoCRIU is a really impressive tool, which can do some really cool things. It's pretty difficult to use, given the nature of what it's doing and all the moving parts to orchestrate. I added experimental support to Docker for migrating containers, but for lots of reasons it never made it as a full fledged feature.
- xuhu 4y agoSince the ELF interpreter and glibc initialize differently depending on cpu flags, it's probably unlikely that migrating between arbitrary hosts can be expected to always work.
- mmis1000 4y agoProbably that's why hyperv have a option to force any guest only able to access to a common cpu instrument set. So they will always have the same flag and migration is possible.
- geofft 4y agohttps://github.com/twosigma/libvirtcpuid https://github.com/twosigma/libvirtcpuid works around this. (But in practice, we mostly rely on the fact that if you are in Google Cloud and specify a minimum CPU version, Google masks CPUID in that VM to exactly match that CPU, even if the underlying hardware is newer. They do this so live-migration of the whole VM works, but it also helps for process migration. I'm not sure where this is specifically documented but see e.g. https://stackoverflow.com/a/44507857 https://stackoverflow.com/a/44507857 .)
- touisteur 4y agoI use it a lot (without criu or libsoccr) for high-availability shenanigans, to avoid the reconnection delay. Machine A is 'main' and had established a connection to distant-machine 1. A crashes (the hardware stops), machine B takes over its MAC, IP and all its established TCP connections. Nothing can be seen from the distant-machine side (maybe a gratuitous ARP slips out... to no avail). And yes, for <1 millisecond takeover this is necessary and I thank the criu project with all my engineering heart for not going the 'just put a module in there and be done with it' but actually making sockets checkpointable and restartable, and saving me from the pains of a userland network stack. The actual details are far more funny and interesting (we could talk about checkpoint not being an atomic operation for the kernel, how you need to do some magic with "plug" qdiscs and qdiscs being applicable on egress only you'll look into IFBs and I love Linux it is so versatile and full of amazing little features). Don't forget to hot-update conntrack too... And since libsoccr is GPL you might need to do this yourself, and you'll want to do it anyway, because it's interesting and you'll learn so many things. My only gripe is the checkpoint still being a bit slow and maybe if I keep annoying Jens Axboe on twitter maybe soon it'll be a io_uring chain <3.
- dj_gitmo 4y agoIf you don't mind me asking, what kind of workload requires this kind of "high-availability shenanigans". Sounds fascinating.
- hamandcheese 4y agoIf RDS could fail over in 1ms I would be extremely happy. In practice, a few minutes of downtime for things like DB upgrades is usually acceptable in the business I am in, however this is enough time to cause quite a lot of alerting noise. If the window of unavailability was instead 1ms, there would be dramatically less noise, potentially none.
- dinosaurdynasty 4y ago9-1-1 callcenters often do, some use specialized server hardware that can in <1ms switch from one motherboard/CPU to another when it fails. Apparently there are some workloads in finance that use similar hardware.