9 ms·

A mainframe is not a supercomputer. Science is done on clusters of x86 chips and Nvidia gpus with custom interconnect and Linux. A system that runs a bank or a

A mainframe is not a supercomputer. Science is done on clusters of x86 chips and Nvidia gpus with custom interconnect and Linux. A system that runs a bank or an insurance company and wants good availability can be built in one of two ways: you either spend money on software that deals with the hardware being unreliable and save money on software (pioneered by Google) or you spend money on hardware that promises to be highly reliable and save on software. No new player believes mainframe (extremely expensive hardware) is cost effective, they all use commodity hardware. A bank that needs to run binaries from the 70s for which they don't have the source code can keep paying IBM and not investing in reverse engineering the binary and implementing it in Java. A bank that has a billion lines of code in Cobol can compile it to run on jvm on commodity hardware and run it in parallel with the mainframe for a year to validate and then switch over to the new system and stop overpaying for hardware, but that sounds risky, so they keep paying a million dollars for a system with the same performance as a 50 thousand dollar server.

pmart123 7y ago
My guess is a mainframe would be much better at handling a high volume of transactions much better than a $50k server? I would also think it would better handle conditions where "eventually consistent" doesn't work? I'm sure most companies are running batch processes and data-intensive calculations that could be easily shifted to cloud workloads in practice, but it does seem like there are some use cases where a mainframe could be better?
- marcosdumay 7y ago
  Yes if you have a huge volume of transactions and a huge amount of constraints that can not be partitioned. But the real world constraints can almost always be partitioned. For read-only batch computations you can always add some extra redundancy and partition the problem. So, I don't think it is likely that a mainframe would be useful here.
- fhars 7y ago
  And it is especially quite a bit better in handling a high volume of transactions while physically replacing one of the CPUs and half of the RAM in the system than many $50k servers.
  - wolf550e 7y ago
    You are paying about 100x on hardware because your software is limited to a single address space. Even if the mainframe never goes down, the entire site will go down (eventually the rack power supply, HVAC, fiber, natural disaster, backhoe, etc. will get you even if your CPUs and RAM are redundant and replaced before they fail), and then either your entire business stops or at least processing for that region stops, or your system is resilient to site failure because you built a distributed system anyway. If you could rewrite your software to be distributed and handle a node/site going down, you could run a single site on 5 servers that together outperform the mainframe (by a lot) and can be serviced on a whole server basis (though of course, expensive x86 servers also have reliability features), or use really cheap hardware without even redundant power supplies, but have enough of them to not care. The modern solutions are better than the mainframe, and the only reason to use them is management risk aversion and unwillingness to learn new things.
    - chx 7y ago
      A single mainframe can easily be located in multiple datacenters. The Hungarian state owned electricity distributor company has one, one half is at Budapest the other half -- if memory serves -- is at Miskolc, a bit more than a hundred miles away.
      - wolf550e 7y ago
        Is that a single address space system or merely two systems with db2 databases and disk volumes on a SAN in replication? I think it's the latter. 100 miles will add 5ms (round trip) to your disk flush on commit. So a system like this has the sequential and random IO latencies of a RAID of SSDs but the flush (database commit) times of a 15K RPM spinning rust disk. People lived with mechanical disks, it's ok. Sync disk replication (in one direction) over a fiber line is not an exclusive feature. Having both sides be active, instead of active and hot standby requires some smarts from the software, but modern distributed databases do that, and if you're careful you can get far with batch sync jobs.
- wolf550e 7y ago
  No. If you need reliability and have money, separate commodity servers running a consistent (paxos or raft, not eventually consistent) in-memory database (the memory is a few terabytes) with logs to flash and to disaster recovery site will outperform a mainframe. A single computer with 4 sockets of xeons will outperform a mainframe but will have more downtime. The mainframe has best possible single thread performance and as much cache as possible, redundancy and parts can be replaced while it's online, but not that many cores. The cost is very high - when I looked, 1 million per year is the baby version with only 1 CPU enabled and no license to run the cryptographic accelerator and limits on software, etc. Commodity hardware you buy and use for 5 years, so the amount of good hardware you can buy for the price of owning a mainframe for 5 years is a lot. For the money, you can buy a lot more CPU, ram, network, storage, etc and hire Kyle Kingsbury to audit your distributed database.
  - ch_123 7y ago
    >The cost is very high - when I looked, 1 million per year is the baby version with only 1 CPU enabled and no license to run the cryptographic accelerator and limits on software, etc. At one point, IBM was selling base-level mainframes for $75,000 (see https://arstechnica.com/information-technology/2013/07/ibm-unveils-new-mainframe-for-the-rest-of-us/ https://arstechnica.com/information-technology/2013/07/ibm-u...) It is true though that a 'realistic' configuration is likely to cost north of $1 million, and that none of these numbers include the price of the software.
    - StrangeDoctor 7y ago
      It’s the support contract that’s the real cost.
    - rodgerd 7y ago
      A standard IBM "gotcha" is to offer deep discounts and charge the 20% p.a. maint cost on the list price - e.g. "We'll give you this hardware for $1." then a year later "So sorry the maint is 3 x $250k x 0.2 = $150k p.a.". Another is that the software and hardware designed to turn a baby system into anything useful is astonishingly expensive to people not used to dealing with this end of the market. Enjoy finding that your hypervisor is licensed at 5 figures per core, and that your cores are six figures a pop.
thekyle 7y ago
> you either spend money on software that deals with the hardware being unreliable and save money on software (pioneered by Google) or you spend money on hardware that promises to be highly reliable and save on software. Seems like the first "save money on software" shouldn't be there.
- silvestrov 7y ago
  sounds like it should be "spend money on software" as the software becomes much more complex when hardware being unreliable (and distributed rather than one big box)
  - thekyle 7y ago
    Yeah, I think maybe they meant to say "save money on hardware" since they already said "spend money on software" earlier in the sentence.
- wolf550e 7y ago
  Should have been "... and save money on hardware (pioneered by Google) ..." but too late to edit now.
  - greglindahl 7y ago
    Google pioneered something done before Google existed?
    - wolf550e 7y ago
      Change it to "made famous by Google". Do you have an example of a famous large system built out of many cheap computers (cheaper than normal servers) that predates Google?
      - dragonwriter 7y ago
        > Do you have an example of a famous large system built out of many cheap computers (cheaper than normal servers) that predates Google? That approach is, in scientific computing, at least as old as Beowulf (1994, four years before Google was founded), the prototypical system from which we get the term “beowulf cluster”.
rjeli 7y ago
Is it really common for banks to run legacy applications without sources? How would they know or trust the behavior?
- jtulkki 7y ago
  Because it's been running a certain way for decades?
  - stjohnswarts 7y ago
    Yep at a known fixed cost. Corps like knowns and hate unknowns, especially stalwarts like the various big banks. It's kind of like buying stocks rather than shorting stocks. Can you handle the unknown of potentially unlimited losses many times your original "investment" in the venture?
- mathattack 7y ago
  Not just banks. Any long standing company has this problem.
  - lanstin 7y ago
    I have worked with a dot com that linked to commercial C libraries for which they never had the source. Did finally reimplement but not till 2016 or something.
- tyingq 7y ago
  I can't speak for banks specifically, but for sure lots of old stodgy Fortune 500 companies have critical software running for which they have lost the source code. Similarly, critical closed source software for which the vendor no longer exists. And things like servers running that they are afraid to shut down...they have no idea if the output is used by anyone. It's a mess out there :)
  - mdorazio 7y ago
    Can confirm. The last two clients I worked with (both Fortune 500) used mainframes for business critical operations. The most recent one didn't actually have source for some components of their business ops software stack - they've just been working as-is for about 30 years and no one will touch them. I was told they did an assessment a year or two ago to price out what it would take to move the business completely off mainframe and onto an x86 stack. It was in the hundreds of millions of dollars to do so because so much other software has been built to interact with and rely on the mainframe over the years that switching off it would be a multi-year effort across every department in the company. So of course that ROI calculation was pretty damn easy and the mainframe isn't going anywhere.
nabla9 7y ago
Do you know any big bank not using mainframes for their core operations? 90% of big banks use mainframes, so there must be some big banks that don't use them. Interested to know who they are. Chinese? It's OK to do most things with chunk of normal servers, but when you need to handle very large number of transactions with fast commits, and stuff like eventual consistency is not allowed, it becomes expensive to handle them no matter what.
- solresol 7y ago
  Commonwealth Bank of Australia -- they were on mainframes, but spent a few hundred million dollars to get off. One of the other big Australian banks (Westpac) has an "exit IBM" project as well, but it isn't complete yet.
  - nabla9 7y ago
    I googled it. Apparently they moved their home loan systems etc. from "windows-based mainframe" to UNIX. What is windows-based mainframe? (HP Superdome??) They also moved into SAP analytics. There is no information anywhere saying that they moved their critical banking systems and transaction processing.
    - rightbyte 7y ago
      Going from IBM to SAP seems like no less lock in?
      - dagw 7y ago
        But only on the software side. Dozens of companies will compete to sell you hardware solutions to run your SAP stuff on.
        blaser-waffle 7y ago
        SuSE Linux on Lenovo, Dell, HP, whatever
lonelappde 7y ago
Mainframe hardware isn't reliable though. It maybe more reliable, but no hardware is reliable. What you call reliable is implemented by an architecture that lets the system route around hardware damage. The difference is that a mainframe is a mostly closed box where you preselect what hardware components are wired together, while a cloud computing type system is an open system where you can dynamically rearchitect it.
dehrmann 7y ago
What you're saying is we've been 737-maxing it for the last few decades?
- wolf550e 7y ago
  Banks work because processes performed by humans that were developed before digital computers were invented, not because technology. They have logs of all transactions and are willing to reconcile them manually and are insured/lawyered against loss when they can't reverse an erroneous transfer. Money can take two business days to move instead of milliseconds, and their customers are mostly fine with this. Programmers are taught that database transactions exist so that when you move money from one account to another and crash in the middle, no money is ever created or destroyed. Well, cat picture websites might do that, but banks don't. They reconcile logs at end of day.
  - dehrmann 7y ago
    I was commenting on this: > you either spend money on software that deals with the hardware being unreliable...or you spend money on hardware that promises to be highly reliable and save on software.
    - wolf550e 7y ago
      Then I don't understand. Which strategy is like the 737-max ? Banks generally don't build systems that crash and lose your money.
derefr 7y ago
Re: binaries from the 70s, if the re-engineering costs are the only reason to stay on mainframe (rather than e.g. microcomputers not being up to the task IO-wise), then why not just run the workload in a z/OS emulator on commodity hardware? I mean, the modern z/OS machines are just running 70s-era workloads on an "older z/OS uarch" emulator anyways, so...
- snorkel 7y ago
  Technically that would mostly work but IT leaders at banks and insurance companies would rather pay high premiums to IBM then stick their neck out endorsing anything risky. Renegotiating with IBM is the safest career move at such places. Cloud is steadily edging out mainframes at banks but a lot of old guard IT leads are circling their wagons around old blue because learning how to do high resiliency on cloud is new and scary, but old blue is safe and well understood.
  - owl57 7y ago
    Using safe and well understood tech to handle my money sounds like a good thing.
  - ska 7y ago
    It's hard to understate the value of safe and well understood for core systems. After all it's not like a massive migration to a new technology stack has ever blown up in anyone's face, right? (</s> if not obvious)
- macspoofing 7y ago
  >then why not just run the workload in a z/OS emulator on commodity hardware? Who are the vendors that provide a migration path from mainframe to a set of commodity hardware running an emulator, and subsequently provide continual maintenance and support?
  - mike22 7y ago
    The big cloud vendors have partners who specialize in such work. Random one from the first page of Google results: https://aws.amazon.com/blogs/apn/migrating-a-mainframe-to-aws-in-5-steps/ https://aws.amazon.com/blogs/apn/migrating-a-mainframe-to-aw...
dangxiaopin 7y ago
The newest trend in banks is to migrate to AWS. See recent Capital One hack. They view it as a combination of commodity hardware with cheap software.
- MrApathy 7y ago
  Yes and no. I work for a bulge bracket bank and we're moving a significant number of applications to the cloud, including AWS, GCP, and our internal cloud offering. But we have literally thousands of internally developed applications. We can move thousands of apps the cloud and still have a need to keep thousands on virtual/physical machines. My own apps are stuck on commodity physical hardware for at least the next few years. The type of applications that can historically been run on mainframes is not really moving to AWS/cloud. Most of what's going to the cloud is what I would consider to be "supporting" applications, not core applications. My own experience, that of others may differ.

pmart123 7y ago

My guess is a mainframe would be much better at handling a high volume of transactions much better than a $50k server? I would also think it would better handle conditions where "eventually consistent" doesn't work? I'm sure most companies are running batch processes and data-intensive calculations that could be easily shifted to cloud workloads in practice, but it does seem like there are some use cases where a mainframe could be better?

marcosdumay 7y ago
Yes if you have a huge volume of transactions and a huge amount of constraints that can not be partitioned. But the real world constraints can almost always be partitioned. For read-only batch computations you can always add some extra redundancy and partition the problem. So, I don't think it is likely that a mainframe would be useful here.
fhars 7y ago
And it is especially quite a bit better in handling a high volume of transactions while physically replacing one of the CPUs and half of the RAM in the system than many $50k servers.
- wolf550e 7y ago
  You are paying about 100x on hardware because your software is limited to a single address space. Even if the mainframe never goes down, the entire site will go down (eventually the rack power supply, HVAC, fiber, natural disaster, backhoe, etc. will get you even if your CPUs and RAM are redundant and replaced before they fail), and then either your entire business stops or at least processing for that region stops, or your system is resilient to site failure because you built a distributed system anyway. If you could rewrite your software to be distributed and handle a node/site going down, you could run a single site on 5 servers that together outperform the mainframe (by a lot) and can be serviced on a whole server basis (though of course, expensive x86 servers also have reliability features), or use really cheap hardware without even redundant power supplies, but have enough of them to not care. The modern solutions are better than the mainframe, and the only reason to use them is management risk aversion and unwillingness to learn new things.
  - chx 7y ago
    A single mainframe can easily be located in multiple datacenters. The Hungarian state owned electricity distributor company has one, one half is at Budapest the other half -- if memory serves -- is at Miskolc, a bit more than a hundred miles away.
    - wolf550e 7y ago
      Is that a single address space system or merely two systems with db2 databases and disk volumes on a SAN in replication? I think it's the latter. 100 miles will add 5ms (round trip) to your disk flush on commit. So a system like this has the sequential and random IO latencies of a RAID of SSDs but the flush (database commit) times of a 15K RPM spinning rust disk. People lived with mechanical disks, it's ok. Sync disk replication (in one direction) over a fiber line is not an exclusive feature. Having both sides be active, instead of active and hot standby requires some smarts from the software, but modern distributed databases do that, and if you're careful you can get far with batch sync jobs.
wolf550e 7y ago
No. If you need reliability and have money, separate commodity servers running a consistent (paxos or raft, not eventually consistent) in-memory database (the memory is a few terabytes) with logs to flash and to disaster recovery site will outperform a mainframe. A single computer with 4 sockets of xeons will outperform a mainframe but will have more downtime. The mainframe has best possible single thread performance and as much cache as possible, redundancy and parts can be replaced while it's online, but not that many cores. The cost is very high - when I looked, 1 million per year is the baby version with only 1 CPU enabled and no license to run the cryptographic accelerator and limits on software, etc. Commodity hardware you buy and use for 5 years, so the amount of good hardware you can buy for the price of owning a mainframe for 5 years is a lot. For the money, you can buy a lot more CPU, ram, network, storage, etc and hire Kyle Kingsbury to audit your distributed database.
- ch_123 7y ago
  >The cost is very high - when I looked, 1 million per year is the baby version with only 1 CPU enabled and no license to run the cryptographic accelerator and limits on software, etc. At one point, IBM was selling base-level mainframes for $75,000 (see https://arstechnica.com/information-technology/2013/07/ibm-unveils-new-mainframe-for-the-rest-of-us/ https://arstechnica.com/information-technology/2013/07/ibm-u...) It is true though that a 'realistic' configuration is likely to cost north of $1 million, and that none of these numbers include the price of the software.
  - StrangeDoctor 7y ago
    It’s the support contract that’s the real cost.
  - rodgerd 7y ago
    A standard IBM "gotcha" is to offer deep discounts and charge the 20% p.a. maint cost on the list price - e.g. "We'll give you this hardware for $1." then a year later "So sorry the maint is 3 x $250k x 0.2 = $150k p.a.". Another is that the software and hardware designed to turn a baby system into anything useful is astonishingly expensive to people not used to dealing with this end of the market. Enjoy finding that your hypervisor is licensed at 5 figures per core, and that your cores are six figures a pop.

thekyle 7y ago

> you either spend money on software that deals with the hardware being unreliable and save money on software (pioneered by Google) or you spend money on hardware that promises to be highly reliable and save on software. Seems like the first "save money on software" shouldn't be there.

silvestrov 7y ago
sounds like it should be "spend money on software" as the software becomes much more complex when hardware being unreliable (and distributed rather than one big box)
- thekyle 7y ago
  Yeah, I think maybe they meant to say "save money on hardware" since they already said "spend money on software" earlier in the sentence.
wolf550e 7y ago
Should have been "... and save money on hardware (pioneered by Google) ..." but too late to edit now.
- greglindahl 7y ago
  Google pioneered something done before Google existed?
  - wolf550e 7y ago
    Change it to "made famous by Google". Do you have an example of a famous large system built out of many cheap computers (cheaper than normal servers) that predates Google?
    - dragonwriter 7y ago
      > Do you have an example of a famous large system built out of many cheap computers (cheaper than normal servers) that predates Google? That approach is, in scientific computing, at least as old as Beowulf (1994, four years before Google was founded), the prototypical system from which we get the term “beowulf cluster”.

rjeli 7y ago

Is it really common for banks to run legacy applications without sources? How would they know or trust the behavior?

jtulkki 7y ago
Because it's been running a certain way for decades?
- stjohnswarts 7y ago
  Yep at a known fixed cost. Corps like knowns and hate unknowns, especially stalwarts like the various big banks. It's kind of like buying stocks rather than shorting stocks. Can you handle the unknown of potentially unlimited losses many times your original "investment" in the venture?
mathattack 7y ago
Not just banks. Any long standing company has this problem.
- lanstin 7y ago
  I have worked with a dot com that linked to commercial C libraries for which they never had the source. Did finally reimplement but not till 2016 or something.
tyingq 7y ago
I can't speak for banks specifically, but for sure lots of old stodgy Fortune 500 companies have critical software running for which they have lost the source code. Similarly, critical closed source software for which the vendor no longer exists. And things like servers running that they are afraid to shut down...they have no idea if the output is used by anyone. It's a mess out there :)
- mdorazio 7y ago
  Can confirm. The last two clients I worked with (both Fortune 500) used mainframes for business critical operations. The most recent one didn't actually have source for some components of their business ops software stack - they've just been working as-is for about 30 years and no one will touch them. I was told they did an assessment a year or two ago to price out what it would take to move the business completely off mainframe and onto an x86 stack. It was in the hundreds of millions of dollars to do so because so much other software has been built to interact with and rely on the mainframe over the years that switching off it would be a multi-year effort across every department in the company. So of course that ROI calculation was pretty damn easy and the mainframe isn't going anywhere.

nabla9 7y ago

Do you know any big bank not using mainframes for their core operations? 90% of big banks use mainframes, so there must be some big banks that don't use them. Interested to know who they are. Chinese? It's OK to do most things with chunk of normal servers, but when you need to handle very large number of transactions with fast commits, and stuff like eventual consistency is not allowed, it becomes expensive to handle them no matter what.

solresol 7y ago
Commonwealth Bank of Australia -- they were on mainframes, but spent a few hundred million dollars to get off. One of the other big Australian banks (Westpac) has an "exit IBM" project as well, but it isn't complete yet.
- nabla9 7y ago
  I googled it. Apparently they moved their home loan systems etc. from "windows-based mainframe" to UNIX. What is windows-based mainframe? (HP Superdome??) They also moved into SAP analytics. There is no information anywhere saying that they moved their critical banking systems and transaction processing.
  - rightbyte 7y ago
    Going from IBM to SAP seems like no less lock in?
    - dagw 7y ago
      But only on the software side. Dozens of companies will compete to sell you hardware solutions to run your SAP stuff on.
      - blaser-waffle 7y ago
        SuSE Linux on Lenovo, Dell, HP, whatever

lonelappde 7y ago

Mainframe hardware isn't reliable though. It maybe more reliable, but no hardware is reliable. What you call reliable is implemented by an architecture that lets the system route around hardware damage. The difference is that a mainframe is a mostly closed box where you preselect what hardware components are wired together, while a cloud computing type system is an open system where you can dynamically rearchitect it.

dehrmann 7y ago

What you're saying is we've been 737-maxing it for the last few decades?

wolf550e 7y ago
Banks work because processes performed by humans that were developed before digital computers were invented, not because technology. They have logs of all transactions and are willing to reconcile them manually and are insured/lawyered against loss when they can't reverse an erroneous transfer. Money can take two business days to move instead of milliseconds, and their customers are mostly fine with this. Programmers are taught that database transactions exist so that when you move money from one account to another and crash in the middle, no money is ever created or destroyed. Well, cat picture websites might do that, but banks don't. They reconcile logs at end of day.
- dehrmann 7y ago
  I was commenting on this: > you either spend money on software that deals with the hardware being unreliable...or you spend money on hardware that promises to be highly reliable and save on software.
  - wolf550e 7y ago
    Then I don't understand. Which strategy is like the 737-max ? Banks generally don't build systems that crash and lose your money.

derefr 7y ago

Re: binaries from the 70s, if the re-engineering costs are the only reason to stay on mainframe (rather than e.g. microcomputers not being up to the task IO-wise), then why not just run the workload in a z/OS emulator on commodity hardware? I mean, the modern z/OS machines are just running 70s-era workloads on an "older z/OS uarch" emulator anyways, so...

snorkel 7y ago
Technically that would mostly work but IT leaders at banks and insurance companies would rather pay high premiums to IBM then stick their neck out endorsing anything risky. Renegotiating with IBM is the safest career move at such places. Cloud is steadily edging out mainframes at banks but a lot of old guard IT leads are circling their wagons around old blue because learning how to do high resiliency on cloud is new and scary, but old blue is safe and well understood.
- owl57 7y ago
  Using safe and well understood tech to handle my money sounds like a good thing.
- ska 7y ago
  It's hard to understate the value of safe and well understood for core systems. After all it's not like a massive migration to a new technology stack has ever blown up in anyone's face, right? (</s> if not obvious)
macspoofing 7y ago
>then why not just run the workload in a z/OS emulator on commodity hardware? Who are the vendors that provide a migration path from mainframe to a set of commodity hardware running an emulator, and subsequently provide continual maintenance and support?
- mike22 7y ago
  The big cloud vendors have partners who specialize in such work. Random one from the first page of Google results: https://aws.amazon.com/blogs/apn/migrating-a-mainframe-to-aws-in-5-steps/ https://aws.amazon.com/blogs/apn/migrating-a-mainframe-to-aw...

dangxiaopin 7y ago

The newest trend in banks is to migrate to AWS. See recent Capital One hack. They view it as a combination of commodity hardware with cheap software.

MrApathy 7y ago
Yes and no. I work for a bulge bracket bank and we're moving a significant number of applications to the cloud, including AWS, GCP, and our internal cloud offering. But we have literally thousands of internally developed applications. We can move thousands of apps the cloud and still have a need to keep thousands on virtual/physical machines. My own apps are stuck on commodity physical hardware for at least the next few years. The type of applications that can historically been run on mainframes is not really moving to AWS/cloud. Most of what's going to the cloud is what I would consider to be "supporting" applications, not core applications. My own experience, that of others may differ.