3 ms·
For those who wonder where this fits relative to platforms we are usually more accustomed to, they sell these to banks and financial services that have a long h
by llm_nerd 2y ago
For those who wonder where this fits relative to platforms we are usually more accustomed to, they sell these to banks and financial services that have a long history of mainframes and are basically just upgrading in place within minimal change. With recent iterations they've added some AI processing on the silicon, offering baby steps to imbuing solutions like fraud detection with neural nets on chip.
But to put it in context, the 24 TOPS that they advertise -- the inference performance of their AI module on their Telum II -- doesn't even match an M4's neural engine (40 TOPS). And of course compared to a dedicated device like an H100 SXM that can hit 4000 TOPS (yes, 166x more). Of course for both the M4 and H100 chip I'm giving quantized numbers, but presumably the Telum II is as it boasts about its quantized support.
Massive caches. Tonnes of memory support. Neat device. A "you won't get fired for leasing this" solution for a few of the Fortune 500.
But you can almost certainly build a magnitudes faster device in just about every dimension using more traditional hardware stacks and for a fraction of the price.
- claudex 2y agoIt's 24 TOPS per CPU, so 192 TOPS for a full mainframe. And you can add Spyre accelerator which have 32 AI unit per card (so 32x24 TOPS) and up to 48 cards per mainframe (so 48x32x24 = 36 864 TOPS). But yeah, you could buy a lot of H100 for the price of a mainframe with such a configuration.
- rswail 2y ago> But you can almost certainly build a magnitudes faster device in just about every dimension using more traditional hardware stacks and for a fraction of the price. Using the sentence "more traditional hardware stacks" when comparing to mainframes is sorta funny.
- UltraSane 2y agoIBM strictly forbids anyone from publishing performance benchmarks of their mainframes which should tell you that they are pretty slow for the money.
- hulitu 2y ago> which should tell you that they are pretty slow for the money. It depends. Some years ago, the memory was working at half the bus speed.
- UltraSane 2y agoIf IBM mainframes had amazing MIPS/$ IBM would make sure the world knows it.
- rbanffy 2y agoCPU MIPS is just a small part of the equation here. A mainframe has lots of specialized hardware dealing with IO that allow the CPU to be running application code close to 100% of the time. A pure CPU intensive benchmark would be misleading and the restriction on publishing benchmarks is not really aimed towards the users, but towards the competition - who'd be more than happy to compare machines with different designs and different performance characteristics to IBM's designs. MIPS per MIPS, IBM itself offers solutions that have higher processing power than mainframes in their POWER and IBMi series of machines. And you can surely assemble an x86 or ARM monster with a lot of cores, but it won't match the performance of the Z for the workloads it is designed for - on-line transaction processing.
- UltraSane 2y agoI'm sure the absolute performance of the Telus II CPU is very good since it runs at 5.5Ghz and has 36MB of L2 cache per core. But I'm talking about price per dollar and this is where IBM mainframes fail miserably because IBM charges through the nose for them. I would not be surprised if a IBM mainframe costs 10x of what a x86 or ARM server of equal performance does.
- rbanffy 2y agoPerformance is not a single number. Mainframes are terrible choices for HPC, AI training, data analytics, etc. Where they excel is that narrow niche of high-volume, real-time transaction processing. In order to get there, the whole design of the machine is carefully balanced - the right number of cores per socket, the cache sizes (and the dynamic allocation of L3 and L4 according to pressures, trying to minimize eviction) all the way to CICS, z/OS and the way they allocate memory and processing power.