9 ms·
A Tour Inside the IBM Z17
- mrweasel 2y agoI'm completely fascinated by the diagram. In a four rack system, 2.5 rack is dedicated to I/O, half a rack is just empty and the remaining is the actual processing and memory. The I/O probably isn't endless networking adaptors, so what is it?
- stonogo 2y agoCould be storage, networking, crypto HSM, or cluster interconnect. See page 28 on https://www.redbooks.ibm.com/redbooks/pdfs/sg248580.pdf https://www.redbooks.ibm.com/redbooks/pdfs/sg248580.pdf
- bitwize 2y agoProbably channels. In an IBM mainframe, each I/O device is connected on its own channel, which is actually a separate computer that handles the transfer to/from the main CPU. This has been the case going back to the System/360, which is why mainframes are legendary for their transaction throughput and reliability. There's probably a lot of redundancy in the I/O hardware, as they have to be rock solid and hot swappable while the system is running.
- bigbuppo 2y agoThere's also the problem in that they need to take into account floor loading. They're not going to tell a customer upgrading from an older machine to a new one that, "oh, by the way, the rack weighs twice what it used to, so you'll need to upgrade your floor while you're at it." Especially important for raised floors.
- philjohn 2y agoand power density. They want you to be able to switch out the old mainframe racks for z17 racks and still stay within the same power budget.
- Someone 2y agohttps://www.redbooks.ibm.com/abstracts/sg248579.html https://www.redbooks.ibm.com/abstracts/sg248579.html: “The IBM z17 supports a PCIe I/O infrastructure. PCIe features are installed in PCIe+ I/O drawers. Up to 12 I/O drawers per IBM z17 can be ordered, which allows for up to 192 PCIe I/O and special purpose features. For a four CPC drawer system, up to 48 PCIe+ fan-out slots can be populated with fan-out cards for data communications between the CPC drawers and the I/O infrastructure, and for coupling. The multiple channel subsystem (CSS) architecture allows up to six CSSs, each with 256 channels. The IBM z17 implements PCIe Generation 5 (PCIe+ Gen5), which is used to connect the PCIe Generation 4 (PCIe+ Gen4) dual port fan-out features in the CPC drawers. The I/O infrastructure is designed to reduce processor usage and I/O latency, and provide increased throughput and availability.”
- deleted 2y ago[deleted]
- gtirloni 2y agoz Systems have always been amazing engineering feats. Too bad adopting it comes with a gargantuan amount of... IBM.
- gosub100 2y agoYeah a few years ago there was a Talos (I think) desktop motherboard that had a POWER 8 cpu in it. It was expensive due to low production runs but I wish it had taken off. I think IBM is up to power 9 now, but I doubt if there are any personal motherboards for it.
- 9front 2y agoThat's Raptor Computing Systems (www.raptorcs.com) now selling Talos II workstations.
- wmf 2y agoBeing somewhat pedantic, Power and z are completely different architectures.
- Pet_Ant 2y agoCompletely? Isn't Z just microcoded on top of POWER under the hood?
- formerly_proven 2y agoI dunno, but the z-processors and the POWER processors look a lot different even from a floor plan / die shot perspective. The former also clock much higher. Doesn't smell like microcode to me.
- wmf 2y agoNo, Power and Telum processors are very different internally.
- classichasclass 2y agoNo. The microarchitectures have some notable similarities and cross-pollinate each other, but they are distinct. You may be thinking of IBM i (formerly known as AS/400 and i5), which has a completely abstracted instruction set that on modern systems is internally recompiled to Power.
- charlieo88 2y agoHow much does it cost? I'm just curious. No, I don't want to book a meeting to "discuss" it.
- zcdziura 2y agoIf you have to ask, you can't afford it!
- tw04 2y agoYou don't buy a mainframe, it's consumption based pricing. They aren't just going to list a price, because they need to size the hardware to what they think the workload will be. Could they just list prices? Sure. Will they ever do it? No.
- pickle-wizard 2y agoI haven't worked with mainframes since the z10, but back then you could get into an entry model for about $100k. Though the sky is the limit. The typical machine I would order had a list price of about 1 million. Of course no one pays list. Discounts can be pretty substantial depending on how much business you do with IBM or how bad they want to get your business.
- gosub100 2y agoIt's probably impossible to say because of the service contracts that come with it. Nobody would buy one brand new and not pay for support and consulting too.
- rbanffy 2y agoAccounting is a key aspect as well. A lot of would-be capex that turns into opex that way.
- NelsonMinar 2y agoYou can get a software emulator for free and run it on a PC! It's quite robust and used by IBM's own developers. https://en.wikipedia.org/wiki/Hercules_(emulator) https://en.wikipedia.org/wiki/Hercules_(emulator)
- pkaye 2y agoI always enjoy reading those IBM Redbooks and learning about the technical details of these mainframe systems.
- mrandish 2y agoI recently learned that the IBM Z series mainframes are generally compatible with software written for the legendary IBM 360 launched in 1964. While I'm sure there are caveats, maintaining any backward compatibility with a platform from over 60 years ago is impressive. Having started in 8-bit microcomputers and progressing to various desktop platforms and servers, mainframes were esoteric hulking beasts that were fascinating but remained mysterious to me. In recent years I've started expanding my appreciation of classic mainframes and minis through reading blogs and retro history. This IEEE retrospective on the creation of the IBM 360 was eye-opening. https://spectrum.ieee.org/building-the-system360-mainframe-nearly-destroyed-ibm https://spectrum.ieee.org/building-the-system360-mainframe-n... Having read pretty deeply on the evolution of early computers from the ENIAC era through Whirlwind, CDC, early Cray and DEC, I was familiar with the broad strokes but I never fully appreciated how much the IBM 360 was a major step change in both software and hardware architecture. It's also a dramatic story because it's rare for a decades-old company as successful and massive as IBM to take such a huge "bet the company" risk. The sheer size and breadth of the 360 effort as well as its long-term success profoundly impacted the future of computing. It's interesting seeing how architectural concepts from the 360 (as well as DEC's popular PDP-8, 10 and 11) went on to influence the design of early CPUs and microcomputers. The engineers and hobbyists creating early micros had learned computers in the late 60s and early 70s mostly on the 360s and PDPs which were ubiquitous in universities.
- KerrAvon 2y agoBook recommendation: in-depth on the people, processes, and technology. Incredible detail on all aspects. https://direct.mit.edu/books/monograph/4262/IBM-s-360-and-Early-370-Systems https://direct.mit.edu/books/monograph/4262/IBM-s-360-and-Ea...
- mrandish 2y agoThanks for the recommendation! I've ordered it. After reading the IEEE article I linked above, I got the book the article was based on ("IBM: The Rise and Fall and Reinvention of a Global Icon"). While it's a thorough recounting of IBM's storied history, it wasn't what I was looking for. The author specifically says his focus was not the technical details as he felt too much had been written from that perspective. Instead that book was a more traditional historian's analysis which I found kind of dry.
- strongpigeon 2y agoI don't mean this in a condescending way at all, but really out of sheer curiosity: Who uses mainframes nowadays and what for? I find mainframes fascinating, but I'm so unfamiliar with them that I don't know what or why I'd ever use one for (as opposed to "traditional" hardware or cloud services).
- deleted 2y ago[deleted]
- tw04 2y agoPretty much every fortune 500 that's been around for more than 30 years. Batch processing primarily - from closing their books to order processing, differs by company. But if you ask the right person, they'll tell you where it's at.
- tgv 2y agoFortune 500? More like Fortune 50000 (ok, exaggeration). But there are so many banks in the world, and their automation can run back to the 1950s. They are only slowly moving away from mainframes, if only because a rewrite of a complex system that nobody understands is tough, and possibly very costly if it is the key to billions of euros/dollars/...
- justin66 2y agoThat's all true, but these machines often run java code. That's something to contemplate.
- rbanffy 2y agoIBM prices processors differently for running COBOL and Java - if you run mostly Java code, your monthly licensing fees will be vastly different. On boot, the service elements (their BMCs - on modern machines they are x86 boxes running Linux) loads microcode in accordance to a saved configuration - some CPUs will run z/OS, some will run z/OS and COBOL apps, some will run Java, some will run z/VM, some will run Linux. This is all configured on the computer whose job is to bring up the big metal (first the power, then the cooling, and only then, the compute). Under everything on the mainframe side is the PR/SM hypervisor, which is, IIRC, what manages LPARS (logical partitions, completely isolated environments sharing the same hardware). The cheapest licensing is Linux under a custom z/VM (they aren't called z but LinuxONE), and the most expensive is the ability to run z/OS and COBOL code. Running Java under z/OS is somewhat cheaper. Last time I saw it, it was very complicated.
- ThinkBeat 2y agoDoes IBM mainframes still have the pricing model where you "buy" hardware and then still pay IBM for main processing? (Where you can save money buying Linux or Java accelerators to run things on for free
- bob1029 2y agoI think what you are referring to is the "sub capacity" pricing model wherein a rolling average of resource consumption is used for billing. They've transitioned to newer models circa cloud technology, but it's mostly the same idea with more moving parts. The advantage of this model from a business operations standpoint is that you don't have to think about a single piece of hardware related to the mainframe. IBM will come out automagically and fix issues for you when the mainframe phones home about a problem. A majority of the system architecture is designed on purpose to enable seamless servicing of the machine without impacting operations. https://www.ibm.com/z/pricing https://www.ibm.com/z/pricing
- DaiPlusPlus 2y ago> IBM will come out automagically and fix issues for you when the mainframe phones home about a problem. A majority of the system architecture is designed on purpose to enable seamless servicing of the machine without impacting I'd rather have a fault-tolerant distributed software system running on commodity hardware, that way there's a plurality of hardware and support vendors to choose from. No lock-in.
- rbanffy 2y ago> I'd rather have a fault-tolerant distributed software system running on commodity hardware, that way there's a plurality of hardware and support vendors to choose from. No lock-in. But then you'd have to develop it yourself. IBM has been doing just that for 60 years (on the 360 and its descendants).
- bob1029 2y ago> distributed software system What if the business demands a certain level of serialized transaction throughput that is incompatible with ideas like paxos? You will never beat one fast machine at a serialized narrative, and it just so happens that most serious businesses require these semantics. How much does downtime cost you per hour? What are the consequences if your services become unavailable?
- UltraSane 2y agoI really don't understand why in 4 racks you can have only 4 CPU drawers and 12 I/O drawers. This seems like their IO is incredibly inefficient.
- rbanffy 2y agoHave you seen the size of those drawers? A single rack can only fit five of them, and you still would need to add processing, power/UPS, and cooling.
- UltraSane 2y agoThe CPU drawers are 5U while the I/O drawers are 8U. https://higherlogicdownload.s3.amazonaws.com/IMWUC/UploadedImages/wDe0H7MRHahpfSpY1qLE_z17_max_full_view.png https://higherlogicdownload.s3.amazonaws.com/IMWUC/UploadedI...
- froh 2y ago> zSeries IO seems insufficient because of the number of I/O drawers a surprising statement. mind to elaborate? asking because sustained, dependable and high I/O per floor space (combined with high throughput low latency compute) are the core selling points of zSeries
- os2warpman 2y agoEach I/O drawer is 16 x16 hot-swappable PCIe+ slots, an interconnect module, and a management unit. Each CPC drawer can up to 3 I/O drawers for up to 48 x16 hot-swappable PCIe+ slots and a system can have up to four CPC drawers for a total of 192 potential PCIe+ modules. If you set it up right any process running on any CPC drawer can directly access any PCIe+ device in the system no matter where it is. If you're super slick, any process running on any CPC drawer in a Z17 that is directly connected to another Z17, even one many miles away, can directly access any PCIe+ device in the remote system. I don't know if anyone is actually doing that. That would be silly. Doing that requires a lot of stuff, which takes up space.
- UltraSane 2y ago