3 ms·

I say the same performance with absolute confidence. I expected a drop in performance when a particular contract I worked with asked us to see if we couldn't g

by shakna 6y ago

I say the same performance with absolute confidence. I expected a drop in performance when a particular contract I worked with asked us to see if we couldn't get everything running on a VM instead of the IBM iron. Like you suspect, that was about not paying for the hardware anymore because just the price of electricity was a noticeable impact on the bank's overall budget. It was an R&D project to see if we could avoid that. What we put together was a QEMU instance, running atop 3 very very cheap commodity servers in duplication, so that if one went down it would switch over without downtime. We chose the cheap servers, because we already had them hanging around. (Probably around $1000 in hardware all up, today.) We did not see a drop in performance or reliance. But we did see an increase in performance. As in, ~30%. I avoided suggesting this is always the case, but it is a significant possibility when changing. As far as I know, the bank in question is now running a similar setup everywhere they used to have a mainframe.

gigatexal 6y ago
Fascinating. If the main mainframe use case is more database ops for banking or airline scheduling that doesn’t seem very compute heavy in my head so throwing hardware at the problem makes sense especially if you can save millions to invest in development and modernization.
- shakna 6y ago
  I agree with you there. Most mainframe tasks today aren't that heavy on the compute - most relied on the amazing I/O stack, and that makes it easier to replace them. Heavily vectorised code with modern instruction sets can also be more performant than a lot of the older compute chips, but that requires more rewriting of code, and someone who intricately understands both the code and the math. Which makes modernising much more expensive. The VM approach is a simpler way to get you most of the way there, but replacing heavy compute stacks is usually going to require a decent bit of investment.
exikyut 6y ago
While reading the parent comment ("stay online for ever") I was reminded of VMware lockstep VMs, which nowadays support up to 4 cores. I wouldn't be surprised if a few places do OpenVMS on lockstep. I am very curious what you mean by "in duplication". Do you mean "3 copies of production and a router" (probably not) - or are you saying you did something like lockstep for QEMU? Incidentally I just found https://wiki.qemu.org/Features/COLO https://wiki.qemu.org/Features/COLO, which looks like it may be being successfully used privately in one or two places (looking at the email addresses).
- shakna 6y ago
  It was a little while ago, but it was 3 production servers running with the mirror backend, with a dirty-bitmap [0], and a complicated routing setup (which could hold packets instead of dropping them, depending on their importance). I didn't really have anything to do with the routing though, so I can't say much. I'd probably use COLO if I was to do it again today. [0] https://kashyapc.fedorapeople.org/QEMU-Docs/_build/html/docs/interop/live-block-operations.html#live-disk-synchronization-drive-mirror-and-blockdev-mirror https://kashyapc.fedorapeople.org/QEMU-Docs/_build/html/docs...
  - exikyut 6y ago
    Oooh, disk synchronization. Live migration but just for the block layer! Cool. I wonder what sorts of scenarios would benefit from using this instead of any of the dozen distributed database architectures it seems are out there. Incidentally while poking through https://www.qemu.org/docs/master/system/invocation.html https://www.qemu.org/docs/master/system/invocation.html I noticed that there are some COLO-related options in there, which is a bit exciting.
- my123 6y ago
  Some Arm processors that you can buy today also support dual-core lock step in hardware, such as the Jetson AGX Xavier from Nvidia. (however, not enabled by default for perf reasons, because you halve your cores by design in that mode) It's there for safety reasons for automotive. (see https://blogs.nvidia.com/blog/2020/05/20/xavier-achieves-industry-first-safety-assessment/ https://blogs.nvidia.com/blog/2020/05/20/xavier-achieves-ind... )

by shakna 6y ago

gigatexal 6y ago

Fascinating. If the main mainframe use case is more database ops for banking or airline scheduling that doesn’t seem very compute heavy in my head so throwing hardware at the problem makes sense especially if you can save millions to invest in development and modernization.

shakna 6y ago
I agree with you there. Most mainframe tasks today aren't that heavy on the compute - most relied on the amazing I/O stack, and that makes it easier to replace them. Heavily vectorised code with modern instruction sets can also be more performant than a lot of the older compute chips, but that requires more rewriting of code, and someone who intricately understands both the code and the math. Which makes modernising much more expensive. The VM approach is a simpler way to get you most of the way there, but replacing heavy compute stacks is usually going to require a decent bit of investment.

exikyut 6y ago

While reading the parent comment ("stay online for ever") I was reminded of VMware lockstep VMs, which nowadays support up to 4 cores. I wouldn't be surprised if a few places do OpenVMS on lockstep. I am very curious what you mean by "in duplication". Do you mean "3 copies of production and a router" (probably not) - or are you saying you did something like lockstep for QEMU? Incidentally I just found https://wiki.qemu.org/Features/COLO https://wiki.qemu.org/Features/COLO, which looks like it may be being successfully used privately in one or two places (looking at the email addresses).

shakna 6y ago
It was a little while ago, but it was 3 production servers running with the mirror backend, with a dirty-bitmap [0], and a complicated routing setup (which could hold packets instead of dropping them, depending on their importance). I didn't really have anything to do with the routing though, so I can't say much. I'd probably use COLO if I was to do it again today. [0] https://kashyapc.fedorapeople.org/QEMU-Docs/_build/html/docs/interop/live-block-operations.html#live-disk-synchronization-drive-mirror-and-blockdev-mirror https://kashyapc.fedorapeople.org/QEMU-Docs/_build/html/docs...
- exikyut 6y ago
  Oooh, disk synchronization. Live migration but just for the block layer! Cool. I wonder what sorts of scenarios would benefit from using this instead of any of the dozen distributed database architectures it seems are out there. Incidentally while poking through https://www.qemu.org/docs/master/system/invocation.html https://www.qemu.org/docs/master/system/invocation.html I noticed that there are some COLO-related options in there, which is a bit exciting.
my123 6y ago
Some Arm processors that you can buy today also support dual-core lock step in hardware, such as the Jetson AGX Xavier from Nvidia. (however, not enabled by default for perf reasons, because you halve your cores by design in that mode) It's there for safety reasons for automotive. (see https://blogs.nvidia.com/blog/2020/05/20/xavier-achieves-industry-first-safety-assessment/ https://blogs.nvidia.com/blog/2020/05/20/xavier-achieves-ind... )