5 ms·

Telum II at Hot Chips 2024: Mainframe with a Unique Caching Strategy

78 points by mfiguiere 2y ago

_a_a_a_ 2y ago
Seems very complex therefore very expensive (and possibly slow where it matters, at L2). Or it might just work.
- jauntywundrkind 2y ago
  On the contrary! Yes there's a lot of cache. But rather than try to have a bunch of cores reading each cache (sharing 96MB L3 for AMD's consumer cores), now there's a lot of separate 36MB L2 caches. (And yes, then again, some fancy protocols to create a virtual L3 cache from these L2 caches. But less cache heirarchy & more like networking. It still seems beautifully simpler in many ways to me!)
  - buildbot 2y ago
    L3 caches are already basically distributed and networked through a ring bus or other NoC on many x86 chips, for example sapphire rapids has 1.875MB of L3 per core, which is pooled into a single coherent L3. Fun fact, this is smaller than each cores L2 (2MB). From https://chipsandcheese.com/2023/03/12/a-peek-at-sapphire-rapids/ https://chipsandcheese.com/2023/03/12/a-peek-at-sapphire-rap... “ the chip appears to be set up to expose all four chiplets as a monolithic entity, with a single large L3 instance. Interconnect optimization gets harder when you have to connect more nodes, and SPR is a showcase of this. Intel’s mesh has to connect 56 cores with 56 L3 slices.”
jmclnx 2y ago
To bad the mainframe business will not be spun off from IBM. Then you may see innovation, but IBM see it as a cash cow.
- wmf 2y ago
  Do the customers want innovation?
  - kev009 2y ago
    The CPUs has been a tour de force from the S/360, they have never relented, so empirically yes the customers care a lot or they wouldn't keep doing this. The software side seems to be more a tale of dichotomy. The MVS lineage is technically impressive but undoubtedly bizarre and old feeling. The TPF lineage seems like eventually somewhere the cloud movement will dip for certain cases so it is ahead of time. Linux is neither stale nor avant-garde, I guess that is their strategy to remain "contemporary". VM was always the most delightful one but internally forever the odd one out.
- rbanffy 2y ago
  I fail to understand where you think IBM has been lacking in innovation - Telum and Telum II (as is POWER10) are very impressive designs the likes of which you won't see anytime soon on x86 or ARM space. They target a relatively small segment of the market where people will pay whatever it takes to reach 99.99999% uptime or the most transactions per second. If mainframes were not competitive at that, they would have ceased to exist a long time ago.
  - nxobject 2y ago
    Or, at the very least, IBM would have sold a product that required far less investment in unique technology, e.g. software emulation on commodity hardware.
    - rbanffy 2y ago
      Everything from the ground up is designed for business transaction performance. That includes the OS, which is somewhat limited for a lot of other uses.
mikewarot 2y ago
That's a huge amount of effort to let most of the transistors in a computer (in the RAM) sit idle most of the time. Surely there are viable non-Von Neuman alternatives that could be spun out into general purpose computing.
- MichaelZuo 2y ago
  The vast majority of transistors in any modern CPU are ‘idle’ at any given moment.
  - kev009 2y ago
    With the cascade of different clock domains on a core and package, the control loops can spend that thermal budget effectively elsewhere; idling is one of the benefits of CMOS.
  - rbanffy 2y ago
    I believe GP was counting the transistors in DRAM, not only those on the CPU. If someone is really into high performance, it's ideal to never have to wait for DRAM, either with predictive fetches or explicit cache warming. For that, the more cache you have, the better.
  - yencabulator 2y ago
    Perhaps most weirdly, we've reached the point where that is actually desirable. Power+heat is the limit now, and slapping on some extra circuitry that is only used for some operations makes the chip perform better. https://en.wikipedia.org/wiki/Dark_silicon https://en.wikipedia.org/wiki/Dark_silicon
- creer 2y ago
  There have been quite a number of attempts at computing in memory kinds of chip design through computing architectures. So far with very limited success. It's not impossible but in search of good applications.
tedunangst 2y ago
I wonder what workloads would benefit from having an L4 victim cache on another CPU, but that other CPU doesn't need its own L2 cache.
- _a_a_a_ 2y ago
  At a guess, a single thread which benefits from as much cache as it can get.
  - tedunangst 2y ago
    Sure, but having to buy entire CPUs filled with idle cores to scale up cache seems very expensive.
    - _a_a_a_ 2y ago
      I said this is where it would help (AFAICS), not that it was the best solution.
    - kev009 2y ago
      These cores are typically licensed with class/restrictions so in absolute terms yes but in the financial engineering of how the system is delivered with excess and restricted hardware no (see core types on the prior/shipping generation here https://www.ibm.com/downloads/cas/6NW3RPQV https://www.ibm.com/downloads/cas/6NW3RPQV) There are probably design reuse and RAS considerations that make it not currently worthwhile to i.e. have a distinct physical design for SAP or whatever cores.
      - sillywalk 2y ago
        I don't know if it's still the case, but in terms of RAS, the Z/Series CPUs from ~2004 had duplicated/compared instruction-fetch/decode and execution units. https://pages.cs.wisc.edu/~remzi/Classes/838/Fall2001/Papers/tale-of-two.pdf https://pages.cs.wisc.edu/~remzi/Classes/838/Fall2001/Papers...
    - nxobject 2y ago
virgulino 2y ago
"Why Do Mainframes Still Exist? What's Inside One? 40TB, 200+ Cores, AI, and more! - Dave explores the IBM z16 mainframe from design to assembly and testing. What's inside a modern IBM z16 mainframe that makes it relevant today?" - by Dave Plummer. This is an amazing 23-minute video by the Microsoft programmer who developed the Windows NT Task Manager, among other things. He visits IBM and talks to engineers about the Telum chip architecture (Hot Chips 2023), used in the z16 mainframe. Special attention is paid to the cache. https://youtu.be/ouAG4vXFORc https://youtu.be/ouAG4vXFORc
- pclmulqdq 2y ago
  Dave Plummer seems to be a bit careless with facts in his videos, and I wouldn't generally trust him as a source of information. In an episode on hard drives, he talked about how drivers for hard drives still report a constant number of sectors per track, so they must have a physical layout that matches that. Hard drive manufacturers are open about the actual layout of their drives and that they virtualize the hard drive for the OS so that it behaves well. A few Microsoft engineers also dispute a lot of the facts of his stories about the development of the start menu. Caveat emptor.
  - froh 2y ago
    do you happen to have factual.insights into false reporting on the Mainframe in this specific video? afaict (and I've worked with mainframes for a couple of years) this is spot on. poor signal/noise ratio but the facts are right.
nxobject 2y ago
Another good article by Chips and Cheese on Telum II @ Hot Chips – an interview with IBM directors of development at the processor and system levels. [1] [1] https://chipsandcheese.com/2024/09/05/an-interview-with-susan-eickoff-and-christian-jacobi-from-ibm-at-hot-chips-2024/ https://chipsandcheese.com/2024/09/05/an-interview-with-susa...
chihwei 2y ago
Mainframe sounds like a good idea to solve many of today's problems. Why don't people start thinking about making a RISC-V or x86 Mainframe?
- rbanffy 2y ago
  A mainframe is just a very large server, with lots of reliability features (RAID-like memory, fault detection and mitigation, redundant components, etc) and lots of intelligent peripherals that offload work from the CPU so that it can spend as much time as possible running application code (and don't waste time handling interrupts, assembling network packets, dealing with IO, etc). A lot of these offload functions are baked into the ISA, making it a VERY CISC machine. I believe Unisys still makes x86-based mainframes running MCP.
  - neverartful 2y ago
    And loads of IP that IBM will defend vigorously with any infringement.
    - rbanffy 2y ago
      They invest a colossal amount of money creating those patents. There are lots of bullshit patents in this space, but IBM is not playing that kind of game.
- beAbU 2y ago
  Modern cloud environments are basically virtualised infinitely scalable mainframes.
  - yencabulator 2y ago
    Modern cloud environments tend to be aimed at running multiple independent workloads well on a huge server. Mainframes are generally aimed at running a smaller number of large workloads well on a huge server. Sort of analogous to multithreaded vs singlethreaded performance in CPU benchmarks. My personal take: The typical x86[1] is a sports car. Gets going fast, reaches most destinations fast, not great for driving for several hours, and not great at moving lots of cargo. A mainframe is a freight train. Somewhat slow to get going, but can haul large amounts of cargo without breaks for a long time. Mainframes weren't built for an interactive, highly variable, query-response workload; they were built for the classic overnight/monthly batch job that streams through a large amount of data. [1]: It's not about the CPU, it's about the architecture around it, like this article talks about cache, expanded to I/O etc concerns.
nxobject 2y ago
Random question to anyone who might know anything about this – is the uarch internally POWER, like System i (AS/400)?
- rbanffy 2y ago
  No. It's completely different. Unlike System i, it's also fully documented (in the rather impenetrable "Principles of Operation" red book).
  - nxobject 2y ago
    Ah, thank you! I would have never thought to search under that name. It makes sense for such a non-mainstream architecture, but I wish there were (even reverse-engineered) resources as well on "this is what the execution engine looks like". There are instructions there that clearly scream "very extensive microcoding is going on here", e.g. vintage EBCDIC/BCD conversion, string/stream instructions, control channel supervision, etc.
    - ruslan 2y ago
      It seems to me the mentioned "Principles of Operation" document describes a virtal machine compatible with good old System/370 what's now known as IBM z/Architecture. But the Telum CPU itself runs on Power10 cores or alike RISCs.
      - rbanffy 2y ago
        No. The cores are not POWER10 at all. While there is a ton of microcode and interception magic happening (and nothing but the hypervisor, PR/SM IIRC, runs on metal - the hypervisor exposes partitions to the zVM environment) the cores are still very different from POWER and run their own s390x ISA. You might be confusing it with the AS/400 CISC ISA, which exists as an emulation layer on top of POWER, since all IBMi machines are almost identical to their POWER counterparts.
        2y ago
MobiusHorizons 2y ago
These mainframe COUs are really cool. But I still don’t feel like I understand where they make sense. I’ve heard they are used in finance, but what does that mean? High frequency trading? Processing credit card payments? Managing bank balances? I’ve always thought of mainframes as batch / offline systems, but this sounds much more online and low latency.
- sillywalk 2y ago
  Transaction processing was basically invented on mainframes. High-throughput, high-volume, low-latency. I'm not sure about high-frequency trading, but in article nxobject linked above they say ~70% of financial transactions are processed on mainframes. So credit card transactions, bank transactions. Historically, things like airline reservations were also done on mainframes. https://en.wikipedia.org/wiki/Transaction_Processing_Facility https://en.wikipedia.org/wiki/Transaction_Processing_Facilit...

78 points by mfiguiere 2y ago

_a_a_a_ 2y ago

Seems very complex therefore very expensive (and possibly slow where it matters, at L2). Or it might just work.

jauntywundrkind 2y ago
On the contrary! Yes there's a lot of cache. But rather than try to have a bunch of cores reading each cache (sharing 96MB L3 for AMD's consumer cores), now there's a lot of separate 36MB L2 caches. (And yes, then again, some fancy protocols to create a virtual L3 cache from these L2 caches. But less cache heirarchy & more like networking. It still seems beautifully simpler in many ways to me!)
- buildbot 2y ago
  L3 caches are already basically distributed and networked through a ring bus or other NoC on many x86 chips, for example sapphire rapids has 1.875MB of L3 per core, which is pooled into a single coherent L3. Fun fact, this is smaller than each cores L2 (2MB). From https://chipsandcheese.com/2023/03/12/a-peek-at-sapphire-rapids/ https://chipsandcheese.com/2023/03/12/a-peek-at-sapphire-rap... “ the chip appears to be set up to expose all four chiplets as a monolithic entity, with a single large L3 instance. Interconnect optimization gets harder when you have to connect more nodes, and SPR is a showcase of this. Intel’s mesh has to connect 56 cores with 56 L3 slices.”

jmclnx 2y ago

To bad the mainframe business will not be spun off from IBM. Then you may see innovation, but IBM see it as a cash cow.

wmf 2y ago
Do the customers want innovation?
- kev009 2y ago
  The CPUs has been a tour de force from the S/360, they have never relented, so empirically yes the customers care a lot or they wouldn't keep doing this. The software side seems to be more a tale of dichotomy. The MVS lineage is technically impressive but undoubtedly bizarre and old feeling. The TPF lineage seems like eventually somewhere the cloud movement will dip for certain cases so it is ahead of time. Linux is neither stale nor avant-garde, I guess that is their strategy to remain "contemporary". VM was always the most delightful one but internally forever the odd one out.
rbanffy 2y ago
I fail to understand where you think IBM has been lacking in innovation - Telum and Telum II (as is POWER10) are very impressive designs the likes of which you won't see anytime soon on x86 or ARM space. They target a relatively small segment of the market where people will pay whatever it takes to reach 99.99999% uptime or the most transactions per second. If mainframes were not competitive at that, they would have ceased to exist a long time ago.
- nxobject 2y ago
  Or, at the very least, IBM would have sold a product that required far less investment in unique technology, e.g. software emulation on commodity hardware.
  - rbanffy 2y ago
    Everything from the ground up is designed for business transaction performance. That includes the OS, which is somewhat limited for a lot of other uses.

mikewarot 2y ago

That's a huge amount of effort to let most of the transistors in a computer (in the RAM) sit idle most of the time. Surely there are viable non-Von Neuman alternatives that could be spun out into general purpose computing.

MichaelZuo 2y ago
The vast majority of transistors in any modern CPU are ‘idle’ at any given moment.
- kev009 2y ago
  With the cascade of different clock domains on a core and package, the control loops can spend that thermal budget effectively elsewhere; idling is one of the benefits of CMOS.
- rbanffy 2y ago
  I believe GP was counting the transistors in DRAM, not only those on the CPU. If someone is really into high performance, it's ideal to never have to wait for DRAM, either with predictive fetches or explicit cache warming. For that, the more cache you have, the better.
- yencabulator 2y ago
  Perhaps most weirdly, we've reached the point where that is actually desirable. Power+heat is the limit now, and slapping on some extra circuitry that is only used for some operations makes the chip perform better. https://en.wikipedia.org/wiki/Dark_silicon https://en.wikipedia.org/wiki/Dark_silicon
creer 2y ago
There have been quite a number of attempts at computing in memory kinds of chip design through computing architectures. So far with very limited success. It's not impossible but in search of good applications.

tedunangst 2y ago

I wonder what workloads would benefit from having an L4 victim cache on another CPU, but that other CPU doesn't need its own L2 cache.

_a_a_a_ 2y ago
At a guess, a single thread which benefits from as much cache as it can get.
- tedunangst 2y ago
  Sure, but having to buy entire CPUs filled with idle cores to scale up cache seems very expensive.
  - _a_a_a_ 2y ago
    I said this is where it would help (AFAICS), not that it was the best solution.
  - kev009 2y ago
    These cores are typically licensed with class/restrictions so in absolute terms yes but in the financial engineering of how the system is delivered with excess and restricted hardware no (see core types on the prior/shipping generation here https://www.ibm.com/downloads/cas/6NW3RPQV https://www.ibm.com/downloads/cas/6NW3RPQV) There are probably design reuse and RAS considerations that make it not currently worthwhile to i.e. have a distinct physical design for SAP or whatever cores.
    - sillywalk 2y ago
      I don't know if it's still the case, but in terms of RAS, the Z/Series CPUs from ~2004 had duplicated/compared instruction-fetch/decode and execution units. https://pages.cs.wisc.edu/~remzi/Classes/838/Fall2001/Papers/tale-of-two.pdf https://pages.cs.wisc.edu/~remzi/Classes/838/Fall2001/Papers...
  - nxobject 2y ago

virgulino 2y ago

"Why Do Mainframes Still Exist? What's Inside One? 40TB, 200+ Cores, AI, and more! - Dave explores the IBM z16 mainframe from design to assembly and testing. What's inside a modern IBM z16 mainframe that makes it relevant today?" - by Dave Plummer. This is an amazing 23-minute video by the Microsoft programmer who developed the Windows NT Task Manager, among other things. He visits IBM and talks to engineers about the Telum chip architecture (Hot Chips 2023), used in the z16 mainframe. Special attention is paid to the cache. https://youtu.be/ouAG4vXFORc https://youtu.be/ouAG4vXFORc

pclmulqdq 2y ago
Dave Plummer seems to be a bit careless with facts in his videos, and I wouldn't generally trust him as a source of information. In an episode on hard drives, he talked about how drivers for hard drives still report a constant number of sectors per track, so they must have a physical layout that matches that. Hard drive manufacturers are open about the actual layout of their drives and that they virtualize the hard drive for the OS so that it behaves well. A few Microsoft engineers also dispute a lot of the facts of his stories about the development of the start menu. Caveat emptor.
- froh 2y ago
  do you happen to have factual.insights into false reporting on the Mainframe in this specific video? afaict (and I've worked with mainframes for a couple of years) this is spot on. poor signal/noise ratio but the facts are right.

nxobject 2y ago

Another good article by Chips and Cheese on Telum II @ Hot Chips – an interview with IBM directors of development at the processor and system levels. [1] [1] https://chipsandcheese.com/2024/09/05/an-interview-with-susan-eickoff-and-christian-jacobi-from-ibm-at-hot-chips-2024/ https://chipsandcheese.com/2024/09/05/an-interview-with-susa...

chihwei 2y ago

Mainframe sounds like a good idea to solve many of today's problems. Why don't people start thinking about making a RISC-V or x86 Mainframe?

rbanffy 2y ago
A mainframe is just a very large server, with lots of reliability features (RAID-like memory, fault detection and mitigation, redundant components, etc) and lots of intelligent peripherals that offload work from the CPU so that it can spend as much time as possible running application code (and don't waste time handling interrupts, assembling network packets, dealing with IO, etc). A lot of these offload functions are baked into the ISA, making it a VERY CISC machine. I believe Unisys still makes x86-based mainframes running MCP.
- neverartful 2y ago
  And loads of IP that IBM will defend vigorously with any infringement.
  - rbanffy 2y ago
    They invest a colossal amount of money creating those patents. There are lots of bullshit patents in this space, but IBM is not playing that kind of game.
beAbU 2y ago
Modern cloud environments are basically virtualised infinitely scalable mainframes.
- yencabulator 2y ago
  Modern cloud environments tend to be aimed at running multiple independent workloads well on a huge server. Mainframes are generally aimed at running a smaller number of large workloads well on a huge server. Sort of analogous to multithreaded vs singlethreaded performance in CPU benchmarks. My personal take: The typical x86[1] is a sports car. Gets going fast, reaches most destinations fast, not great for driving for several hours, and not great at moving lots of cargo. A mainframe is a freight train. Somewhat slow to get going, but can haul large amounts of cargo without breaks for a long time. Mainframes weren't built for an interactive, highly variable, query-response workload; they were built for the classic overnight/monthly batch job that streams through a large amount of data. [1]: It's not about the CPU, it's about the architecture around it, like this article talks about cache, expanded to I/O etc concerns.

nxobject 2y ago

Random question to anyone who might know anything about this – is the uarch internally POWER, like System i (AS/400)?

rbanffy 2y ago
No. It's completely different. Unlike System i, it's also fully documented (in the rather impenetrable "Principles of Operation" red book).
- nxobject 2y ago
  Ah, thank you! I would have never thought to search under that name. It makes sense for such a non-mainstream architecture, but I wish there were (even reverse-engineered) resources as well on "this is what the execution engine looks like". There are instructions there that clearly scream "very extensive microcoding is going on here", e.g. vintage EBCDIC/BCD conversion, string/stream instructions, control channel supervision, etc.
  - ruslan 2y ago
    It seems to me the mentioned "Principles of Operation" document describes a virtal machine compatible with good old System/370 what's now known as IBM z/Architecture. But the Telum CPU itself runs on Power10 cores or alike RISCs.
    - rbanffy 2y ago
      No. The cores are not POWER10 at all. While there is a ton of microcode and interception magic happening (and nothing but the hypervisor, PR/SM IIRC, runs on metal - the hypervisor exposes partitions to the zVM environment) the cores are still very different from POWER and run their own s390x ISA. You might be confusing it with the AS/400 CISC ISA, which exists as an emulation layer on top of POWER, since all IBMi machines are almost identical to their POWER counterparts.
      - 2y ago

MobiusHorizons 2y ago

These mainframe COUs are really cool. But I still don’t feel like I understand where they make sense. I’ve heard they are used in finance, but what does that mean? High frequency trading? Processing credit card payments? Managing bank balances? I’ve always thought of mainframes as batch / offline systems, but this sounds much more online and low latency.

sillywalk 2y ago
Transaction processing was basically invented on mainframes. High-throughput, high-volume, low-latency. I'm not sure about high-frequency trading, but in article nxobject linked above they say ~70% of financial transactions are processed on mainframes. So credit card transactions, bank transactions. Historically, things like airline reservations were also done on mainframes. https://en.wikipedia.org/wiki/Transaction_Processing_Facility https://en.wikipedia.org/wiki/Transaction_Processing_Facilit...