4 ms·

In the late 90s I attended a tour at Holland Signaal, a dutch defense company producing radar and anti-missile systems. I remember vividly how they demonstrate

by fleddr 4y ago

In the late 90s I attended a tour at Holland Signaal, a dutch defense company producing radar and anti-missile systems. I remember vividly how they demonstrated an unbreakable process. They had a computer running a process and no matter what happened to that computer, the next one would flawlessly continue the process down to the cycle, with no change or corruption or skipping a beat. It may very well be that this is actually not very difficult, but it seemed difficult and impressive. Perhaps more shocking were ultra high resolution radar screens, some 3 generations ahead of anything I had seen in the consumer space, showing an incredible visualization of the air space, live. Showing exactly which plane is where, the model/type, age, fuel on board, hostile/friendly, all of it. They even had a "situation room" with a holodeck chair in the middle, full of controls. The entire room was covered in wall-size screens basically showing the air space of the entire country, being live analyzed. Sounds very 2022, not 1998.

zamadatix 4y ago
Of all of the amazing things I've learned/seen in IT over the years the one that had the greatest "jaw to floor" effect was when I first learned of VMware's Fault Tolerance feature (shortly after I learned about VMs and live VM migration, which was the previous "jaw to floor" winner). Fault Tolerance allows for a very similar cycle accurate continuation concept but trades off specialized synchronous design and near instant failover for a ~1 second hitch to gain the ability to do this with a non-specialized VM and software stack. For even more fun if you have more than 2 hosts when 1 is lost it'll rebuild a fault tolerant state automatically with another host.
- fleddr 4y ago
  That's so cool, I'm only a casual VMWare user and had no idea. With things like this possible, and possible for a long time now, it honestly makes me cringe at how absolutely terrible (slow, inefficient, unreliable, complex, not robust) the typical modern software stack is.
  - xwolfi 4y ago
    It's not true ! The typical software stack is optimized with need as it arises You wouldnt want to write ultrafast software for your idea to deliver cats back from the vets with a network of teenagers om bicycle. Imagine, years to write your website in C++/ASM to end up failing the first week. What s embarrassing is how much software cost to build in the 80s.
- hangonhn 4y ago
  I came here to say the same thing: “telefork” is just a much more limited version of VMWare’s VMotion. FT is, IIRC, just an extension of the same technique. You’re basically just continuously monitoring and copying state over to the other host instead of shutting down the original VM after you’re done copying. Essentially ESX is an OS that can already a full on telefork with its “process” being a VM. You can practically get to that state if your VM is made super lightweight. (Source: I used to work at VMWare and one of the implementers of FT was a cycling buddy.)
- nolist_policy 4y ago
  QEMU can do that too nowadays with COLO (COarsegrained LOckstepping) [1]. It's quite fun to randomly yank a Node, wait for resync and repeat and see how the VM stays alive all the time. :) [1] https://wiki.qemu.org/Features/COLO https://wiki.qemu.org/Features/COLO
  - dividuum 4y ago
    Is there any cloud provider that offers such VMs? Seems incredibly simpler to fail over like that instead of implementing a userland procedure.
    - electroly 4y ago
      You can run VMware Cloud in AWS to get this, but it's not as useful as it sounds: you can't have the pair of fault tolerant instances across regions or even AZs. You still need another plan for recovering from region or AZ failures, and if you need that, why not just also use that plan for intra-AZ failures instead of shelling out for VMware?
    - nolist_policy 4y ago
      The COLO effort is driven mainly by Intel China and they are currently deploying it at some Chinese cloud provider AFAIK. Unfortunately, as cool as VMWare FT and QEMU COLO are, you need high-bandwith and low-latency links between the nodes and still they increase latency and reduce performance a lot. And in 99.99% of cases just replicating the disk and restarting the VMs in case of a node failure is good enough.
nijave 4y ago
https://en.wikipedia.org/wiki/IBM_Parallel_Sysplex https://en.wikipedia.org/wiki/IBM_Parallel_Sysplex https://en.wikipedia.org/wiki/Tandem_Computers https://en.wikipedia.org/wiki/Tandem_Computers

by fleddr 4y ago

zamadatix 4y ago

Of all of the amazing things I've learned/seen in IT over the years the one that had the greatest "jaw to floor" effect was when I first learned of VMware's Fault Tolerance feature (shortly after I learned about VMs and live VM migration, which was the previous "jaw to floor" winner). Fault Tolerance allows for a very similar cycle accurate continuation concept but trades off specialized synchronous design and near instant failover for a ~1 second hitch to gain the ability to do this with a non-specialized VM and software stack. For even more fun if you have more than 2 hosts when 1 is lost it'll rebuild a fault tolerant state automatically with another host.

fleddr 4y ago
That's so cool, I'm only a casual VMWare user and had no idea. With things like this possible, and possible for a long time now, it honestly makes me cringe at how absolutely terrible (slow, inefficient, unreliable, complex, not robust) the typical modern software stack is.
- xwolfi 4y ago
  It's not true ! The typical software stack is optimized with need as it arises You wouldnt want to write ultrafast software for your idea to deliver cats back from the vets with a network of teenagers om bicycle. Imagine, years to write your website in C++/ASM to end up failing the first week. What s embarrassing is how much software cost to build in the 80s.
hangonhn 4y ago
I came here to say the same thing: “telefork” is just a much more limited version of VMWare’s VMotion. FT is, IIRC, just an extension of the same technique. You’re basically just continuously monitoring and copying state over to the other host instead of shutting down the original VM after you’re done copying. Essentially ESX is an OS that can already a full on telefork with its “process” being a VM. You can practically get to that state if your VM is made super lightweight. (Source: I used to work at VMWare and one of the implementers of FT was a cycling buddy.)
nolist_policy 4y ago
QEMU can do that too nowadays with COLO (COarsegrained LOckstepping) [1]. It's quite fun to randomly yank a Node, wait for resync and repeat and see how the VM stays alive all the time. :) [1] https://wiki.qemu.org/Features/COLO https://wiki.qemu.org/Features/COLO
- dividuum 4y ago
  Is there any cloud provider that offers such VMs? Seems incredibly simpler to fail over like that instead of implementing a userland procedure.
  - electroly 4y ago
    You can run VMware Cloud in AWS to get this, but it's not as useful as it sounds: you can't have the pair of fault tolerant instances across regions or even AZs. You still need another plan for recovering from region or AZ failures, and if you need that, why not just also use that plan for intra-AZ failures instead of shelling out for VMware?
  - nolist_policy 4y ago
    The COLO effort is driven mainly by Intel China and they are currently deploying it at some Chinese cloud provider AFAIK. Unfortunately, as cool as VMWare FT and QEMU COLO are, you need high-bandwith and low-latency links between the nodes and still they increase latency and reduce performance a lot. And in 99.99% of cases just replicating the disk and restarting the VMs in case of a node failure is good enough.

nijave 4y ago

https://en.wikipedia.org/wiki/IBM_Parallel_Sysplex https://en.wikipedia.org/wiki/IBM_Parallel_Sysplex https://en.wikipedia.org/wiki/Tandem_Computers https://en.wikipedia.org/wiki/Tandem_Computers