3 ms·

So we know that these large companies use mainframes, in part, because they've always used mainframes. But it also seems like there are compelling reasons now i

So we know that these large companies use mainframes, in part, because they've always used mainframes. But it also seems like there are compelling reasons now in terms of large amounts of memory & compute & I/O & speciality co-processors. So how come it's not a sexy thing now? Having a ton of compute + memory + I/O in AWS/GCP/Azure can be really, really expensive... I'd love to know a cost comparison. Is it even possible for an ordinary person to rent a VM on one of these machines and do some compute? If I have a startup that needs to crunch a bunch of models or would have an easier time running some system centrally versus having to have the skills & time to create a purely distributed system, why aren't mainframes a good/better/easier option? How come AWS doesn't have any? Does IBM have quick/easy on-ramps to hosted mainframes via Softlayer (I can't find them if they do)?

tyingq 6y ago
Pretty sure commodity servers have now eclipsed any compute, IO, etc, advantage mainframes ever had. With perhaps a few exceptions for some really niche things...like VISA using TPF (which is an unusual OS on a mainframe) because it's basically a huge, high transaction rate, distributed NoSql database. The only real remaining advantage is probably the overall reliability, redundancy, engineering, etc. Like being able to swap out a CPU while the system is running.
- azinman2 6y ago
  Surely you can get more memory/IO/CPU on a mainframe than a commodity server, no?
  - wolf550e 6y ago
    No. A mainframe is not a supercomputer. You can get more cores, more dram, more iops for less money if you don't need high reliability in a single machine (because your software handles nodes failing) and if you don't need compatibility with binaries from the 60s. The chip is impressive, because they don't skimp on costs, because they sell them for a million a pop, but consider how many ryzen on xeon cores you can get for that price, with all the cool DRAM and IO tech you want. https://en.wikichip.org/wiki/ibm/microarchitectures/z15 https://en.wikichip.org/wiki/ibm/microarchitectures/z15 compare with: https://en.wikichip.org/wiki/intel/xeon_platinum https://en.wikichip.org/wiki/intel/xeon_platinum https://en.wikichip.org/wiki/intel/microarchitectures/cascade_lake https://en.wikichip.org/wiki/intel/microarchitectures/cascad...
  - tyingq 6y ago
    Max cores is 190, so no on CPU. Memory and IO, perhaps, but it's unusual to put more into a single partition than you would into a typical large X86-64 server. And it's a lot more pricey.
zozbot234 6y ago
A true "cost comparison" would have to include existing on-prem platforms, and account for new players like Oxide Computer Systems who are working to bring newer, cloud-native technologies to the on-prem datacenter. The mainframe may be competitive with "cloud" solutions, but that's not where the most obvious comparison is.
greedo 6y ago
Hah no. When my company was looking to justify buying a new mainframe, they tried to leverage our server team by saying "You can run Linux in LPARs just like you do on VMware etc." So we met with the sales guys. They asked the size of our average Oracle VM (RAM mostly), and when they heard 256GB, they smiled. Then the mainframe manager asked how much it would cost to provision that much RAM for 10-12 LPARs. I wish I had taken a photo of his expression. Needless to say the meeting ended shortly after, and we didn't pursue spending almost 100x the amount we were spending on our ESX hosts...
- neverartful 6y ago
  While you can run Linux in their own LPARs, I think the much more common scenario is to run them as guests under z/VM. Why is this? Because a single LPAR typically represents a sizable chunk of the overall resources (and cost). On the other hand, you can create an LPAR for z/VM (say with 2-4 IFLs) and have those hardware resources spread out across many Linux z/VM guests.
  - greedo 6y ago
    You're still going to pay through the nose for 384GB of RAM (times ten VMs).
    - neverartful 6y ago
      No disagreement there.
rossdavidh 6y ago
If you have a lot of hardware in the same facility, configured in a way such that you can hotswap and load balance and do all of the other things you might want, you only call it a "mainframe" if it's from IBM, who is I believe the only player left in that business. Otherwise, you call it a "server farm", and pretend that it's an entirely different beast. A mainframe (in recent decades) is essentially a proprietary, high cost, tricked out server farm in a box (which is often big enough to be a small building, btw), which might make sense in a few cases but mostly is not as good as a much larger, non-proprietary mainframe, called a "server farm", which is supported by several competing vendors.
- zozbot234 6y ago
  > Otherwise, you call it a "server farm", and pretend that it's an entirely different beast. Well, server farms don't usually expose a single system image. You can't set the OS to "migrate" workloads seamlessly across hosts the way it can migrate them across CPU cores. Linux is now getting some of the underlying features that are needed to enable this use case, but integrating them into a coherent whole will take some time.
  - philsnow 6y ago
    What's the difference between the system image in a huge zOS machine (which, from what I understand supports all kinds of NUMA and hot-pluggable everything, including CPUs, memory, tapes, and disks because they're designed to never, ever need to be shut down) and the cluster "image" provided by a high-availability k8s cluster (which also supports all kinds of hot-pluggable everything)? In a way, the kubelet is the kernel of a k8s cluster. You wouldn't run masters on every node in a cluster, because consensus gets harder quadratically. Maybe workload migration doesn't have to live in the (linux) kernel.
    - zozbot234 6y ago
      Kubernetes is just middleware, it's not a full distributed operating system like, e.g. Amoeba or Inferno. It was never designed from the ground up to expose a meaningful view of the whole cluster to application developers or to those managing the system. As a result of this history, it also features a significant amount of incidental complexity.

tyingq 6y ago

Pretty sure commodity servers have now eclipsed any compute, IO, etc, advantage mainframes ever had. With perhaps a few exceptions for some really niche things...like VISA using TPF (which is an unusual OS on a mainframe) because it's basically a huge, high transaction rate, distributed NoSql database. The only real remaining advantage is probably the overall reliability, redundancy, engineering, etc. Like being able to swap out a CPU while the system is running.

azinman2 6y ago
Surely you can get more memory/IO/CPU on a mainframe than a commodity server, no?
- wolf550e 6y ago
  No. A mainframe is not a supercomputer. You can get more cores, more dram, more iops for less money if you don't need high reliability in a single machine (because your software handles nodes failing) and if you don't need compatibility with binaries from the 60s. The chip is impressive, because they don't skimp on costs, because they sell them for a million a pop, but consider how many ryzen on xeon cores you can get for that price, with all the cool DRAM and IO tech you want. https://en.wikichip.org/wiki/ibm/microarchitectures/z15 https://en.wikichip.org/wiki/ibm/microarchitectures/z15 compare with: https://en.wikichip.org/wiki/intel/xeon_platinum https://en.wikichip.org/wiki/intel/xeon_platinum https://en.wikichip.org/wiki/intel/microarchitectures/cascade_lake https://en.wikichip.org/wiki/intel/microarchitectures/cascad...
- tyingq 6y ago
  Max cores is 190, so no on CPU. Memory and IO, perhaps, but it's unusual to put more into a single partition than you would into a typical large X86-64 server. And it's a lot more pricey.

zozbot234 6y ago

A true "cost comparison" would have to include existing on-prem platforms, and account for new players like Oxide Computer Systems who are working to bring newer, cloud-native technologies to the on-prem datacenter. The mainframe may be competitive with "cloud" solutions, but that's not where the most obvious comparison is.

greedo 6y ago

Hah no. When my company was looking to justify buying a new mainframe, they tried to leverage our server team by saying "You can run Linux in LPARs just like you do on VMware etc." So we met with the sales guys. They asked the size of our average Oracle VM (RAM mostly), and when they heard 256GB, they smiled. Then the mainframe manager asked how much it would cost to provision that much RAM for 10-12 LPARs. I wish I had taken a photo of his expression. Needless to say the meeting ended shortly after, and we didn't pursue spending almost 100x the amount we were spending on our ESX hosts...

neverartful 6y ago
While you can run Linux in their own LPARs, I think the much more common scenario is to run them as guests under z/VM. Why is this? Because a single LPAR typically represents a sizable chunk of the overall resources (and cost). On the other hand, you can create an LPAR for z/VM (say with 2-4 IFLs) and have those hardware resources spread out across many Linux z/VM guests.
- greedo 6y ago
  You're still going to pay through the nose for 384GB of RAM (times ten VMs).
  - neverartful 6y ago
    No disagreement there.

rossdavidh 6y ago

If you have a lot of hardware in the same facility, configured in a way such that you can hotswap and load balance and do all of the other things you might want, you only call it a "mainframe" if it's from IBM, who is I believe the only player left in that business. Otherwise, you call it a "server farm", and pretend that it's an entirely different beast. A mainframe (in recent decades) is essentially a proprietary, high cost, tricked out server farm in a box (which is often big enough to be a small building, btw), which might make sense in a few cases but mostly is not as good as a much larger, non-proprietary mainframe, called a "server farm", which is supported by several competing vendors.

zozbot234 6y ago
> Otherwise, you call it a "server farm", and pretend that it's an entirely different beast. Well, server farms don't usually expose a single system image. You can't set the OS to "migrate" workloads seamlessly across hosts the way it can migrate them across CPU cores. Linux is now getting some of the underlying features that are needed to enable this use case, but integrating them into a coherent whole will take some time.
- philsnow 6y ago
  What's the difference between the system image in a huge zOS machine (which, from what I understand supports all kinds of NUMA and hot-pluggable everything, including CPUs, memory, tapes, and disks because they're designed to never, ever need to be shut down) and the cluster "image" provided by a high-availability k8s cluster (which also supports all kinds of hot-pluggable everything)? In a way, the kubelet is the kernel of a k8s cluster. You wouldn't run masters on every node in a cluster, because consensus gets harder quadratically. Maybe workload migration doesn't have to live in the (linux) kernel.
  - zozbot234 6y ago
    Kubernetes is just middleware, it's not a full distributed operating system like, e.g. Amoeba or Inferno. It was never designed from the ground up to expose a meaningful view of the whole cluster to application developers or to those managing the system. As a result of this history, it also features a significant amount of incidental complexity.