4 ms·

Sorry to say that your previous "evangelism" shows. It would be very interesting to hear what "unbelievably powerful" means (as in facts and numbers) compared

Sorry to say that your previous "evangelism" shows. It would be very interesting to hear what "unbelievably powerful" means (as in facts and numbers) compared to what a hyperscaler can provide. E.g. SPEC for CPU, GFLOPs, Tensor-ops for inference, latency and so on. Last I checked the consolidation story that claimed factors of savings took underutilized ancient Intel boxes as baseline. "feature-full" is also quite a stretch when it comes to software when most modern software needs to be ported or run in a Linux LPAR. In addition I'd question that not getting access easily is the main reason for the lack of adoption. It might be part of the problem but the other problem is that the thing is a niche product and IBM doesn't seem to be utterly successful in either explaining what the niche is nor how to extend it.

cbsmith 3y ago
Last I checked, the Teliums had some pretty impressive numbers for memory bandwidth even compared to NVidia.
- wmf 3y ago
  It doesn't. Mainframe terminology is so obfuscated that it's impossible to tell for sure but it sounds like less memory bandwidth than Genoa.
  - rbanffy 3y ago
    They are tailored to the traditional mainframe workloads (they do a lot of hardware/software co-design in their mainframe lineup), so I wouldn't expect a mainframe designed for the generic cloud hyperscale workloads. In any case, I have played with their LinuxONE Community Cloud service (running on the previous-gen z15) and it's very fast. The impression I get is that it doesn't need to wait for IO. There is a ton of very clever engineering on those machines and the z16 is a technological wonder. https://www.anandtech.com/show/16924/did-ibm-just-preview-the-future-of-caches https://www.anandtech.com/show/16924/did-ibm-just-preview-th...
- sonnyOrullivan 3y ago
  Telum is attached to the L2/L3 so it's that bandwidth as long as the model fits into it. Afterwards you go to memory and you really don't want to compare DDR4 modules with RAIM against HBM3 or the likes that a current compute GPU uses. Latency might be closer but you mentioned bandwidth.
  - rbanffy 3y ago
    IIRC, with 32 MB per core (8 per die - a Telum package has two of them), one socket has 256MB of cache for 16 cores. L1 and L2 are in-core, but L3 is on-die unused L2 from other core and L4 is unused L3 on the other die. I am not sure if that continues off-package, but if it does, a drawer with 4 chips of 16 cores each will have 2 GB of off-chip cache and a full-sized Z with 5 drawers would have 10 GB of off-drawer cache (at this level it's probably not that much faster than same-drawer memory). As for the RAIM, I think it's safe to assume it has a very wide path (n-1 modules) to the sockets and that aggregate bandwidth will not leave the 4 sockets starved (even if latencies suffer because of the redundancy, but you can replace a defective memory module with a running computer without it having to pause.
    - zwaps 3y ago
      That's not a lot compared to 8xA100
      - cbsmith 3y ago
        A100's are definitely still the best, but IBM's offering isn't a joke. Depending on what you're trying to do, it might be a good way to go.
        justinclift 3y ago
        We'd probably need to see pricing for IBMs offering, because it's possible it'll be eye watering-ly high compared to buying even A100's.
        jabl 3y ago
bayindirh 3y ago
Being able to spare a couple of cores as a new partition and moving a partition to it to replace the memory and even the processor of the faulty partition with zero downtime is great. Consider that partitions are completely isolated from each other. Not some pesky soft isolation either, it's all done in hardware. In practice, every partition in a mainframe is a different logical yet isolated mainframe. Mainframes are built for different kinds of workloads. They are not cloud machines. They are batch machines with 6 or more nines of uptime. In my current job, a mainframe would be useless. However, fore mission critical core services which needs predictable latencies (bank transaction engines, central big databases etc.) I'll take a mainframe all day, every day.
- sonnyOrullivan 3y ago
  > Consider that partitions are completely isolated from each other. Not some pesky soft isolation either, it's all done in hardware. In practice, every partition in a mainframe is a different logical yet isolated mainframe. "Completely isolated in hardware" as in isolated by a software hypervisor (PR/SM) that doesn't have a whole lot of hardware support? > Mainframes are built for different kinds of workloads. They are not cloud machines. They are batch machines with 6 or more nines of uptime. That's kind of the point. What exactly is the niche, i.e. which new customer with exactly what software and latency requirements would switch from their current system to a z? IBM won't tell you apart from buzzword bingo.
  - fuzztester 3y ago
    >buzzword bingo I'm guessing that many of the terms in this thread are buzzwords for many of us reading it :), since relatively few people work on mainframes. I had not heard of the term buzzword bingo before, so I googled it: https://en.m.wikipedia.org/wiki/Buzzword_bingo https://en.m.wikipedia.org/wiki/Buzzword_bingo Interesting. It aligns with my corporate experience in a few cases.
    - 20after4 3y ago
      https://www.youtube.com/watch?v=RXJKdh1KZ0w https://www.youtube.com/watch?v=RXJKdh1KZ0w
Tor3 3y ago
Check out this youtube video: https://www.youtube.com/watch?v=ouAG4vXFORc&pp=ygUMbWFpbmZyYW1lIDQw https://www.youtube.com/watch?v=ouAG4vXFORc&pp=ygUMbWFpbmZyY... It does a reasonably good job at comparing mainframes to a regular server setup. It's not about SPEC or GFLOPs etc.

cbsmith 3y ago

Last I checked, the Teliums had some pretty impressive numbers for memory bandwidth even compared to NVidia.

wmf 3y ago
It doesn't. Mainframe terminology is so obfuscated that it's impossible to tell for sure but it sounds like less memory bandwidth than Genoa.
- rbanffy 3y ago
  They are tailored to the traditional mainframe workloads (they do a lot of hardware/software co-design in their mainframe lineup), so I wouldn't expect a mainframe designed for the generic cloud hyperscale workloads. In any case, I have played with their LinuxONE Community Cloud service (running on the previous-gen z15) and it's very fast. The impression I get is that it doesn't need to wait for IO. There is a ton of very clever engineering on those machines and the z16 is a technological wonder. https://www.anandtech.com/show/16924/did-ibm-just-preview-the-future-of-caches https://www.anandtech.com/show/16924/did-ibm-just-preview-th...
sonnyOrullivan 3y ago
Telum is attached to the L2/L3 so it's that bandwidth as long as the model fits into it. Afterwards you go to memory and you really don't want to compare DDR4 modules with RAIM against HBM3 or the likes that a current compute GPU uses. Latency might be closer but you mentioned bandwidth.
- rbanffy 3y ago
  IIRC, with 32 MB per core (8 per die - a Telum package has two of them), one socket has 256MB of cache for 16 cores. L1 and L2 are in-core, but L3 is on-die unused L2 from other core and L4 is unused L3 on the other die. I am not sure if that continues off-package, but if it does, a drawer with 4 chips of 16 cores each will have 2 GB of off-chip cache and a full-sized Z with 5 drawers would have 10 GB of off-drawer cache (at this level it's probably not that much faster than same-drawer memory). As for the RAIM, I think it's safe to assume it has a very wide path (n-1 modules) to the sockets and that aggregate bandwidth will not leave the 4 sockets starved (even if latencies suffer because of the redundancy, but you can replace a defective memory module with a running computer without it having to pause.
  - zwaps 3y ago
    That's not a lot compared to 8xA100
    - cbsmith 3y ago
      A100's are definitely still the best, but IBM's offering isn't a joke. Depending on what you're trying to do, it might be a good way to go.
      - justinclift 3y ago
        We'd probably need to see pricing for IBMs offering, because it's possible it'll be eye watering-ly high compared to buying even A100's.
      - jabl 3y ago

bayindirh 3y ago

Being able to spare a couple of cores as a new partition and moving a partition to it to replace the memory and even the processor of the faulty partition with zero downtime is great. Consider that partitions are completely isolated from each other. Not some pesky soft isolation either, it's all done in hardware. In practice, every partition in a mainframe is a different logical yet isolated mainframe. Mainframes are built for different kinds of workloads. They are not cloud machines. They are batch machines with 6 or more nines of uptime. In my current job, a mainframe would be useless. However, fore mission critical core services which needs predictable latencies (bank transaction engines, central big databases etc.) I'll take a mainframe all day, every day.

sonnyOrullivan 3y ago
> Consider that partitions are completely isolated from each other. Not some pesky soft isolation either, it's all done in hardware. In practice, every partition in a mainframe is a different logical yet isolated mainframe. "Completely isolated in hardware" as in isolated by a software hypervisor (PR/SM) that doesn't have a whole lot of hardware support? > Mainframes are built for different kinds of workloads. They are not cloud machines. They are batch machines with 6 or more nines of uptime. That's kind of the point. What exactly is the niche, i.e. which new customer with exactly what software and latency requirements would switch from their current system to a z? IBM won't tell you apart from buzzword bingo.
- fuzztester 3y ago
  >buzzword bingo I'm guessing that many of the terms in this thread are buzzwords for many of us reading it :), since relatively few people work on mainframes. I had not heard of the term buzzword bingo before, so I googled it: https://en.m.wikipedia.org/wiki/Buzzword_bingo https://en.m.wikipedia.org/wiki/Buzzword_bingo Interesting. It aligns with my corporate experience in a few cases.
  - 20after4 3y ago
    https://www.youtube.com/watch?v=RXJKdh1KZ0w https://www.youtube.com/watch?v=RXJKdh1KZ0w

Tor3 3y ago

Check out this youtube video: https://www.youtube.com/watch?v=ouAG4vXFORc&pp=ygUMbWFpbmZyYW1lIDQw https://www.youtube.com/watch?v=ouAG4vXFORc&pp=ygUMbWFpbmZyY... It does a reasonably good job at comparing mainframes to a regular server setup. It's not about SPEC or GFLOPs etc.