12 ms·

IBM launches Linux-only mainframe system

178 points by jeo1234 11y ago

gauravphoenix 11y ago
"IBM said LinuxONE Emperor can scale up to 8,000 virtual machines or thousands of containers, which would be the most for any single Linux system." I wish article (or IBM) provided some specs. Thousands of containers is just too broad of a statement- what are these containers doing? Running just a bash script hello world program or minting bitcoins?
- stonewhite 11y ago
  Exactly. Even its official site lacks any clear explanation and solution brief pdf is 404. I don't see how this is any different from a regular server rack. http://www-03.ibm.com/systems/z/os/linux/linux-one.html http://www-03.ibm.com/systems/z/os/linux/linux-one.html
  - kkuduk 11y ago
    > I don't see how this is any different from a regular server rack well, the difference is that most rack cannot fit 10 TB of RAM and 140x5GHz cores: http://www-03.ibm.com/systems/z/hardware/z13_specs.html http://www-03.ibm.com/systems/z/hardware/z13_specs.html but surely those are super-expensive and I doubt all that effective when it comes to performance per watt. I'm actually a bit more excited about Power8 servers, but are there any non-IBM available yet?
    - ajdlinux 11y ago
      > I'm actually a bit more excited about Power8 servers, but are there any non-IBM available yet? Yep, OpenPOWER machines are starting to come out - Tyan has the Habanero server as well as the Palmetto reference/development platform. http://www.tyan.com/solutions/tyan_openpower_system.html http://www.tyan.com/solutions/tyan_openpower_system.html (disclosure: I'm an IBMer working on Power)
      - A010 11y ago
        > High-performance computing (HPC) > Data centers > Big data Can you give me some specific roles for this range of servers? What makes it's worth compared to x86 system?
        rwmj 11y ago
        The high-end POWER 7/8 hardware has incredible single core performance, beating the pants off Xeon. It uses huge amounts of power to do that, so it's not appropriate for all roles. Low end POWER is pretty niche. Freescale uses the architecture for telecoms applications. In general Linux upstream these days "just works" on ppc64 & ppc64le. There's RHEL for POWER already, and IBM have loaned hardware to the CentOS project so we'll get CentOS on POWER pretty soon. The licensing of (Open-)POWER is more open than x86 (but not as open as stuff like RISC-V), and there are several second sources for chips, in situations where that matters.
        Spooky23 11y ago
        You get a pretty decent multiple of performance on power vs. intel. Probably something like 4-7x for common use cases. Generally speaking, you buy the hardware because your workload needs the single core performance, or you're arbitraging vendor licensing costs for software. Also, IBM's bread and butter is "peaky" financial services and gov't business, so they have business models that makes it work from a $ pov. You can buy a box with 100 cores, pay for 20, and lease 30 more for a few days to meet your peak demands for tax/christmas/billing season.
    - rodgerd 11y ago
      > well, the difference is that most rack cannot fit 10 TB of RAM and 140x5GHz cores: http://www-03.ibm.com/systems/z/hardware/z13_specs.html http://www-03.ibm.com/systems/z/hardware/z13_specs.html I can fit 4 C7000 chassis in a single rack, which top out at an aggregate 32 TB of memory and over 2000 cores. 5 GHz cores running s390x microcode are not, FYI, twice as fast as Xeons. It would be great if people who weren't familiar with the state of the art in x86 kit didn't blindly assume IBM et al's advertising was reality.
      - jamestnz 11y ago
        I'm no expert, but my understanding is that the IBM "z" platform has a number of interesting features perhaps not found in more typical server platforms/configurations, beyond just scaling out to loads of cores and RAM. As with all IBM mainframes going right back to the 60s, the "z" systems are designed for continuous uptime, on the order of decades, and this is evident in many of the design decisions. For example, various subsystems and components have hot-spares available, so that even outright component failures will not cause downtime. Many components are then hot-swappable, even those that might not ordinarily be in other architectures, such as processors and main memory. No interruption to OS or application-level services is expected by hot-swapping such hardware. Across the useful life of the mainframe, most repairs and maintenance would be carried out without ever shutting it down. I understand the platform also has extensive internal integrity checking built in, a potentially important factor for various types of jobs. Its auditing service is capable of detecting unusual conditions in various subsystems or jobs ("I've just picked up a fault in the AE-35 unit"), automatically retrying instructions on the processor if they executed anomalously. If the fault continues, the suspect processor is routed-around with no interruption to OS or applications, the job is resumed from last checkpoint on another processor, and the system phones home to IBM to log a service call. This monitoring is not being performed by processes running in userland or by the kernel, but is in fact baked into the hardware/firmware platform. Furthermore, the systems can be configured with a variety of specialty offload processors or subsystems for tasks like encryption, key management, compression, and even logging -- which again might not be so commonly found on-board of some commodity servers. (And, of course, even if you could put together an analogous solution with commodity kit, it's IBM! For the sorts of companies looking at a mainframe in 2015, having the IBM name on the SLA has got to be a pretty big part of the equation, right?) Moreover, if you proposed to build and manage these sorts of capabilities from commodity x86 kit, I imagine IBM would claim that they'd have the lower TCO.
    - vidarh 11y ago
      > most rack cannot fit 10 TB of RAM and 140x5GHz cores Any rack can fit 10TB of RAM and the equivalent of 140x5GHz POWER cores. I can easily get ~1TB RAM and 64 2GHz+ x86 cores in 1U. A standard rack is at least 42U. In terms of density it's nothing special. What makes these different is that you get it in a box that someone else will be maintaining, and where all the dirty work of designing and building a high availability redundant system has been done for you. Most people never build anything remotely as redundant as these things tend to be.
      - vegabook 11y ago
        I'm guessing part of the appeal will also be super-wide bandwidth between the cores. Interconnect throughput and/or shared memory could make this significantly more interesting than a modular rack-based system for some workloads.
        rodgerd 11y ago
        > I'm guessing part of the appeal will also be super-wide bandwidth between the cores. If it's anything like previous-generations, that will be infiniband between books. The benefits on Z-class systems tend to be very big per-core, per-socket caches and, as the person you're replying to said, a lot of internal redundancy and automated failover (multiple backplanes with failover, spare memory and cores with failover, and so on and so forth).
    - jdboyd 11y ago
      There is the TYAN GN70-BP010, although they state it isn't for production use.
  - deleted 11y ago
    [deleted]
- blackbeard 11y ago
  That sounds like a whole lot of eggs in one very expensive basket. Plus we can get that density with standard kit I reckon.
  - creshal 11y ago
    That's kinda the point of a mainframe, is it not?
    - blackbeard 11y ago
      Spending money? Yep.
      - creshal 11y ago
        No, putting all your eggs into one (hopefully redundant) basket so you only need to yell at one person.
        rodgerd 11y ago
        Oh, that works great when you lose power to the rack. Or the datacentre. Or the SAN fails. Or the core routers. Or any of the many other SPOFs that can and do occur in a datacentre.
        easytiger 11y ago
        That's why you buy two and put them in different DCs
        creshal 11y ago
        And that's why IBM has their own bank branch to help customers figure out how to afford that?
        digi_owl 11y ago
- otis_inf 11y ago
  In an article which states "..IBM's z13 mainframe computer, which had been designed for high-volume mobile transactions.", all bets are off with respect to reality. I mean, what does 'designed for high-volume mobile transactions' even mean in the context of having Linux running on the hardware?
  - dublinclontarf 11y ago
    Probably a realtime(i.e. happens within a guaranteed amount of time) system but the statement is still too vague.
  - ams6110 11y ago
    Not sure if this is still correct for the z13 but in general: "System z servers offload such functions as I/O processing, cryptography, memory control, and various service functions (such as hardware configuration management and error logging) to dedicated processors." (from Wikipedia)
    - bridanp 11y ago
      It's called a zIIP engine (http://www-03.ibm.com/systems/z/hardware/features/ziip/ http://www-03.ibm.com/systems/z/hardware/features/ziip/) and it utilizes the IFL also (http://www-03.ibm.com/systems/z/os/linux/solutions/ifl.html http://www-03.ibm.com/systems/z/os/linux/solutions/ifl.html). We use the zIIP on our z13 basically to offload any work that takes a ton of I/O (DB2 data serves, dataset scans for performance monitoring, RMF and CICS reporting). It ends up saving a lot of money monthly because we are charged by maximum number of MSU's during any four-hour window of the month. The zIIP reduces the number of MSU's needed at any given time. In our case, 5-10. Back in 2005, I installed SuSe on an s/390 box. If you could throw a lot of memory at it or MIPs, then it would run pretty well, and could be used as a file or email server. But anything that crunched data was horribly slow. Can't say that it's any better now, but if it's utilizing the zIIP, I'm sure it's miles above where it was 10 years ago. Still, pretty high cost of ownership on the mainframe to get this. I can't imagine many people buying a million dollar mainframe just to run Linux. Seems like those who have a mainframe already would have been like us: let's partition out a Linux instance and try it out.
- deleted 11y ago
  [deleted]
rodgerd 11y ago
From my experience with zLinux environments I would be inclined to scrutinise any density claims very, very carefully. Interesting they're working with Canonical on this - the Z has historically been a SuSE stronghold, with a lot of the development of Linux-on-Z happening in Germany, and Red Hat trailing behind. I'd be wary it will be another increasingly typical adware effort from them. (I follow the main zLinux mailing lists for my day job, and I see a lot of folks from SuSE, some folks from Red Hat, a bunch of people from IBM, and basically no-one from the Ubuntu/Canonical world. Usual disclaimers apply.)
- kokey 11y ago
  I can think of two good reasons. One is if they want to offer these for virtual desktop environments (as an alternative to the VMWare on Cisco UCS setups many current IBM mainframe customers run) The other to offer an alternative platform to EC2 where Ubuntu seems to be very common. Both would indicate that they are probably aiming at the desktop/user/development side with this product.
- wglb 11y ago
  From my experience with zLinux environments Any specifics you can share?
  - rodgerd 11y ago
    Depends what you want to know.
    - wglb 11y ago
      Well, I find I would be inclined to scrutinise any density claims very, very carefully interesting. Why do they deserve very careful scrutiny?
      - rodgerd 11y ago
        So I can only speak fairly generally, but I've never even met anyone running the kinds of ratios IBM is talking about in their press releases. I have, however, seen some workloads run pretty well; Oracle is IME a very good performer on Z, needing a smaller SGA and fewer IFLs than the equivalent Intel setup. Depending on the deal IBM offer you, zLinux can be a very good way of running Oracle. Conversely, CPU intensive Java don't really seem to enjoy much, if any, advantage, which makes it a very, very expensive option for doing that. There are a bunch of factors to consider (for example, if you've got zLinux and zOS co-resident on the same system there are some interesting things you can do with your legacy and modern code bases), but in general I'd want to prove anything myself before diving onto a Z.
dang 11y ago
Also https://news.ycombinator.com/item?id=10071679 https://news.ycombinator.com/item?id=10071679.
cm-t 11y ago
Note: Canonical seems to be a partner to bring Ubuntu on it See http://techcrunch.com/2015/08/16/ibm-teams-with-canonical-on-linux-mainframe/ http://techcrunch.com/2015/08/16/ibm-teams-with-canonical-on...
- cm-t 11y ago
  Confirmed here: https://insights.ubuntu.com/2015/08/17/ibm-and-canonical-plan-ubuntu-support-on-ibm-z-systems-mainframe/ https://insights.ubuntu.com/2015/08/17/ibm-and-canonical-pla...
deleted 11y ago
[deleted]
- dijit 11y ago
  linux is only 23 years old or so. (1991 iirc) also, they have a history of supporting linux, why so bitter?
  - SagelyGuru 11y ago
    I meant they could have supported then some version of Unix. Linux wrote Linux only because the "big boys" did their best to ignore it.
    - noselasd 11y ago
      And they did. IBM have supported linux on mainframes since the earily days, they started sending patches in 1999. IBM also have their own version of UNIX, called AIX, for longer than that.
    - rodgerd 11y ago
      You... don't know much about Unix history or IBM products, do you?
    - rjsw 11y ago
      IBM did support Unix [1] and [2], several years before Linux was available. [1] https://en.wikipedia.org/wiki/Interactive_Systems_Corporation#PC.2FIX https://en.wikipedia.org/wiki/Interactive_Systems_Corporatio... [2] https://en.wikipedia.org/wiki/IBM_AIX https://en.wikipedia.org/wiki/IBM_AIX
    - skissane 11y ago
      Unix has a long history on the IBM mainframe, prior to Linux. It was running under VM/370 at Princeton in 1977 - https://www.bell-labs.com/usr/dmr/www/portpap.pdf https://www.bell-labs.com/usr/dmr/www/portpap.pdf page 3 - and in the early 1980s AT&T ran UNIX under TSS/370, which was used in development of the 5ESS switch - see https://www.bell-labs.com/usr/dmr/www/otherports/ibm.pdf https://www.bell-labs.com/usr/dmr/www/otherports/ibm.pdf. Amdahl UTS came out in 1981 (descended from the 1970s work at Princeton) and IBM/Interactive Systems VM/IX around 1984; then in 1988, the IBM/Locus AIX/370, followed by AIX/390. And then in 1993, MVS acquired a Unix compatibility subsystem (originally called OpenEdition, now called Unix Systems Services), and under its present name of z/OS is officially certified as a Unix, since Unix System Services passes the X/Open tests. I think the real problem for Unix on the mainframe has been the cost of the mainframe platform. Once you are developing portable applications, it is easy to shift them on to something else (AIX, Solaris, Linux, whatever), and you'll probably save money by doing so. When developing using a classic mainframe technology stack (COBOL, CICS, IMS, etc), replatforming is much more effort/risk, so those apps are more likely to survive on the mainframe in the long-term than Unix ones. (Disclaimer: I work for a competitor to IBM so I may well be biased.)
sgt 11y ago
When would this type of thing become cost effective? Let's say if you compare it with your typical X amounts of Dell servers also capable of running 8,000 virtual machines.
stefantalpalaru 11y ago
> (The story was refiled to correct the name of the server to "Emperor" from "Empire" in paragraph 4 and names of software to "MariaDB" and "PostgreSQL" from "Maria" and "Posture" in paragraph 5) These are the journalists specialized in IT news for a giant like Reuters?
- zymhan 11y ago
  As I read the bit about the correction, I figured Empire and Maria were understandable mistakes to make. But what in the hell is posture. Also, I know this is a newswire/business site, but this is super skimpy on the details.
  - chops 11y ago
    Autocorrect? My phone corrects postgres to "postures".
arca_vorago 11y ago
So for a business with a complex AS400 system still in place, would this be a replacement for that or an augmentation of that? Their page isn't very clear. http://www-03.ibm.com/systems/z/os/linux/linux-one.html http://www-03.ibm.com/systems/z/os/linux/linux-one.html
deleted 11y ago
[deleted]
bluedino 11y ago
At the last user conference for some iSeries-based software that we run, IBM had a booth where they displayed a 2U server with dual 16-core Power7 CPUs. They bragged that it only ran Linux and would save us a ton of money. They started out at about $20,000 US. How small is this market? You'd have to have apps that were written to take advantage of Power7. Equivalent x86 Linux servers are 1/4 the price.
- aus_ 11y ago
  Just to clarify: P-series (Power/POWER8) Linux is not the same as what is announced here. LinuxONE runs on the System z (mainframe / s390x) platform.
  - rodgerd 11y ago
    Yes, but they now share the same microarchitecture. s390x is mostly a difference in the microcode.
    - nickpsecurity 11y ago
      It has a custom processor: https://en.wikipedia.org/wiki/IBM_zEC12_%28microprocessor%29 https://en.wikipedia.org/wiki/IBM_zEC12_%28microprocessor%29
- thoughtpolice 11y ago
  I think the horsepower on those machines shouldn't be underestimated, because they are not entirely as equivalent as you think... I was thoroughly surprised when an unoptimized (but correct!) ChaCha20/8 implementation I wrote on a 3.0GHz POWER8 little-endian machine was about as fast as the latest 3.5gHz Xeons @ AES-256 with AESNI (about 1.3cpb vs 1.0cpb IIRC, but the latter has a dedicated hardware unit for it!) On that same Xeon, the ChaCha20 code only hit somewhere around 5cpb - that's software vs silicon! It also has 170 cores and was actually a QEMU instance (w/ hardware virtualization extensions) vs raw dedicated metal. If you're doing any kind of numerical or analytic workloads (even databases), I wouldn't throw them aside so quickly. You can even get CUDA for them these days, and certain physical addons like CAPI allow you to map and coherently share physical CPU address space with FPGAs or GPUs. If I could get those things in a reasonable workstation configuration, I'd probably go for it tbh. (I'd be more than willing to repeat this and post some more accurate numbers if anyone cares. I also need to get around to benchmarking AESNI vs that POWER8 machines _actual_ dedicated AES unit. The benchmark above was only flexing its vector/integer unit capabilities. ;)
jwildeboer 11y ago
I wonder if the KVM on zSeries thingy is actually open source and upstream. From what I can see they (IBM) say it's "based on open source" which is market droid speak for "actually it's not that open".
- monocasa 11y ago
  It appears to be pretty open source. http://lxr.free-electrons.com/source/arch/s390/kvm/ http://lxr.free-electrons.com/source/arch/s390/kvm/
tristor 11y ago
Frankly, my experience on mainframes is that UNIX is a stronghold there for a reason. I'd be far more interested in this ecosystem opening up by having strong mainframe vendor support for running Illumos and/or FreeBSD, and less concerned with running Linux. Among other things, Zones are more useful than lxc, and in this type of environment you often need strong kernel support for specialized high-speed interconnects and real-time operations, which Linux only has experimental support for but is integrated well in the UNIX world. I love Linux and what it's done for the world, but UNIX isn't dead for a reason, there are still many things it is superior at.
- the_why_of_y 11y ago
  This is an odd remark - apparently AIX/ESA, the AIX version for mainframe hardware was discontinued "in the late 1990s", roughly the same time when IBM started to invest in Linux. http://www.lookupmainframesoftware.com/soft_detail/dispsoft/1581 http://www.lookupmainframesoftware.com/soft_detail/dispsoft/... There was also Amdahl UTS, which was sold to a company UTS Global around 2000 that appears defunct now. https://en.wikipedia.org/wiki/Amdahl_UTS https://en.wikipedia.org/wiki/Amdahl_UTS So where is this stronghold of AT&T descendant UNIXes on mainframes then?
  - tolle 11y ago
    Nowhere, but some people call System I/(A/400)/I-Series a mainframe. It's midrange, but people call it what they call it. Anyway, IBM i and AIX run on the same hardware. They used to be called System I and System P. Now it's just Power Systems. Maybe thats what he ment. Or well, come to think of it. There is a UNIX side to z/OS itself.
aus_ 11y ago
By trade, I am a mainframe systems programmer at a large financial institution. My group is responsible for mainframe operating systems: z/OS (it may also be referred to as MVS), z/VM (the mainframe hypervisor), and z/Linux (Linux on a mainframe) and many of the software components that go along with these. I'm really excited about this announcement. I think IBM finally realized they need to be more open if they want to grow the Linux on z community. I just hope they are not too late to the game. There's a lot of mystery and misconception behind mainframes, so I am happy to answer any questions about mainframes that I can.
- mozumder 11y ago
  So, why use Linux instead of z/OS? Is it cheaper? Offer more tools?
  - noobermin 11y ago
    I'm curious too. Of course, anyone here can quote the usual benefits of open source (no licenses, familiarity with the technical crowd in general, the "many-eyes"), but I'm curious how valid these benefits are in the mainframe space.
  - clord 11y ago
    The reason from my perspective is that it is different than a traditional unix system, and what unix emulation it offers is limited (to put it in perspective, it feels like a bad vim emulation... your native skillset doesn't quite translate). Lots of issues with dealing with data formats (EBCDIC). Access to open source compilers and scripting languages was limited. Granted these were strengths of the platform as well, but it made our workload (maintaining a compiler for the target) harder than other unix-alikes for those of us not fully versed in JCL.
  - rdtsc 11y ago
    Maybe more software is supported on Linux. So it combined some of the hardware advantages of a mainframe + more recent/wide variety of packages?
  - aus_ 11y ago
nickpsecurity 11y ago
It's about time. This is long overdue. The combination of the mainframe architecture's strengths with Linux's API/ecosystem could be pretty awesome. Anyone wondering why buy a mainframe should focus on these areas: 1. Reliability. Some have gone 30 years without downtime. Probably strongest selling point. 2. Channel I/O [1]: dedicated I/O processors plus scheduling that lead to high utilization (80-90+%) and throughput vs commodity servers. Second strongest point in mainframe's favor. I wish my desktop & servers had this rather than a knockoff. 3. Hardware partitioning that's more robust and rated at stronger security than most virtualization. Certain cutting-edge projects in INFOSEC are doing similar things at CPU and I/O layers. Mainframe's version, although not as cool, is decent and field-proven. 4. Built-in, proven software virtualization. 5. Hardware acceleration for some things such as databases and crypto. 6. IBM's ecosystem of apps, third-party providers, and services. This might matter to existing IBM customers. So, those are a few advantages I hear from people who use mainframes. The z/OS-based mainframes have extra benefits in terms of software reliability, security through obscurity (obfuscation), and seemingly better use of both security (eg memory key) and functional (eg decimal) aspects of mainframe processors. The z/VM product has also been doing for decades what modern virtualization systems only recently do, even self-virtualizing since 70's. So, there's some things to ponder. Whether it makes since financially vs other setups is a whole, different discussion. However, mainframes do retain strong, technical advantages over commodity architectures. They were doing cloud in one box before it was a thing. Their reliability is still unmatched with only VMS clusters and NonStop architecture getting close. So, it's a sensible choice for a business to spend extra $$$ to get high-throughput with no downtime and strong isolation of logical partitions. Channel I/O [1] https://en.wikipedia.org/wiki/I/O_channel https://en.wikipedia.org/wiki/I/O_channel
ilaksh 11y ago
How much does the 8000 core thing cost?
- nickpsecurity 11y ago
  Yeah, I'd really like to see that number. I bet it's so big that it's highly negotiable and still highly profitable. ;)

178 points by jeo1234 11y ago

"IBM said LinuxONE Emperor can scale up to 8,000 virtual machines or thousands of containers, which would be the most for any single Linux system." I wish article (or IBM) provided some specs. Thousands of containers is just too broad of a statement- what are these containers doing? Running just a bash script hello world program or minting bitcoins?

stonewhite 11y ago
Exactly. Even its official site lacks any clear explanation and solution brief pdf is 404. I don't see how this is any different from a regular server rack. http://www-03.ibm.com/systems/z/os/linux/linux-one.html http://www-03.ibm.com/systems/z/os/linux/linux-one.html
- kkuduk 11y ago
  > I don't see how this is any different from a regular server rack well, the difference is that most rack cannot fit 10 TB of RAM and 140x5GHz cores: http://www-03.ibm.com/systems/z/hardware/z13_specs.html http://www-03.ibm.com/systems/z/hardware/z13_specs.html but surely those are super-expensive and I doubt all that effective when it comes to performance per watt. I'm actually a bit more excited about Power8 servers, but are there any non-IBM available yet?
  - ajdlinux 11y ago
    > I'm actually a bit more excited about Power8 servers, but are there any non-IBM available yet? Yep, OpenPOWER machines are starting to come out - Tyan has the Habanero server as well as the Palmetto reference/development platform. http://www.tyan.com/solutions/tyan_openpower_system.html http://www.tyan.com/solutions/tyan_openpower_system.html (disclosure: I'm an IBMer working on Power)
    - A010 11y ago
      > High-performance computing (HPC) > Data centers > Big data Can you give me some specific roles for this range of servers? What makes it's worth compared to x86 system?
      - rwmj 11y ago
        The high-end POWER 7/8 hardware has incredible single core performance, beating the pants off Xeon. It uses huge amounts of power to do that, so it's not appropriate for all roles. Low end POWER is pretty niche. Freescale uses the architecture for telecoms applications. In general Linux upstream these days "just works" on ppc64 & ppc64le. There's RHEL for POWER already, and IBM have loaned hardware to the CentOS project so we'll get CentOS on POWER pretty soon. The licensing of (Open-)POWER is more open than x86 (but not as open as stuff like RISC-V), and there are several second sources for chips, in situations where that matters.
      - Spooky23 11y ago
        You get a pretty decent multiple of performance on power vs. intel. Probably something like 4-7x for common use cases. Generally speaking, you buy the hardware because your workload needs the single core performance, or you're arbitraging vendor licensing costs for software. Also, IBM's bread and butter is "peaky" financial services and gov't business, so they have business models that makes it work from a $ pov. You can buy a box with 100 cores, pay for 20, and lease 30 more for a few days to meet your peak demands for tax/christmas/billing season.
  - rodgerd 11y ago
    > well, the difference is that most rack cannot fit 10 TB of RAM and 140x5GHz cores: http://www-03.ibm.com/systems/z/hardware/z13_specs.html http://www-03.ibm.com/systems/z/hardware/z13_specs.html I can fit 4 C7000 chassis in a single rack, which top out at an aggregate 32 TB of memory and over 2000 cores. 5 GHz cores running s390x microcode are not, FYI, twice as fast as Xeons. It would be great if people who weren't familiar with the state of the art in x86 kit didn't blindly assume IBM et al's advertising was reality.
    - jamestnz 11y ago
      I'm no expert, but my understanding is that the IBM "z" platform has a number of interesting features perhaps not found in more typical server platforms/configurations, beyond just scaling out to loads of cores and RAM. As with all IBM mainframes going right back to the 60s, the "z" systems are designed for continuous uptime, on the order of decades, and this is evident in many of the design decisions. For example, various subsystems and components have hot-spares available, so that even outright component failures will not cause downtime. Many components are then hot-swappable, even those that might not ordinarily be in other architectures, such as processors and main memory. No interruption to OS or application-level services is expected by hot-swapping such hardware. Across the useful life of the mainframe, most repairs and maintenance would be carried out without ever shutting it down. I understand the platform also has extensive internal integrity checking built in, a potentially important factor for various types of jobs. Its auditing service is capable of detecting unusual conditions in various subsystems or jobs ("I've just picked up a fault in the AE-35 unit"), automatically retrying instructions on the processor if they executed anomalously. If the fault continues, the suspect processor is routed-around with no interruption to OS or applications, the job is resumed from last checkpoint on another processor, and the system phones home to IBM to log a service call. This monitoring is not being performed by processes running in userland or by the kernel, but is in fact baked into the hardware/firmware platform. Furthermore, the systems can be configured with a variety of specialty offload processors or subsystems for tasks like encryption, key management, compression, and even logging -- which again might not be so commonly found on-board of some commodity servers. (And, of course, even if you could put together an analogous solution with commodity kit, it's IBM! For the sorts of companies looking at a mainframe in 2015, having the IBM name on the SLA has got to be a pretty big part of the equation, right?) Moreover, if you proposed to build and manage these sorts of capabilities from commodity x86 kit, I imagine IBM would claim that they'd have the lower TCO.
  - vidarh 11y ago
    > most rack cannot fit 10 TB of RAM and 140x5GHz cores Any rack can fit 10TB of RAM and the equivalent of 140x5GHz POWER cores. I can easily get ~1TB RAM and 64 2GHz+ x86 cores in 1U. A standard rack is at least 42U. In terms of density it's nothing special. What makes these different is that you get it in a box that someone else will be maintaining, and where all the dirty work of designing and building a high availability redundant system has been done for you. Most people never build anything remotely as redundant as these things tend to be.
    - vegabook 11y ago
      I'm guessing part of the appeal will also be super-wide bandwidth between the cores. Interconnect throughput and/or shared memory could make this significantly more interesting than a modular rack-based system for some workloads.
      - rodgerd 11y ago
        > I'm guessing part of the appeal will also be super-wide bandwidth between the cores. If it's anything like previous-generations, that will be infiniband between books. The benefits on Z-class systems tend to be very big per-core, per-socket caches and, as the person you're replying to said, a lot of internal redundancy and automated failover (multiple backplanes with failover, spare memory and cores with failover, and so on and so forth).
  - jdboyd 11y ago
    There is the TYAN GN70-BP010, although they state it isn't for production use.
- deleted 11y ago
  [deleted]
blackbeard 11y ago
That sounds like a whole lot of eggs in one very expensive basket. Plus we can get that density with standard kit I reckon.
- creshal 11y ago
  That's kinda the point of a mainframe, is it not?
  - blackbeard 11y ago
    Spending money? Yep.
    - creshal 11y ago
      No, putting all your eggs into one (hopefully redundant) basket so you only need to yell at one person.
      - rodgerd 11y ago
        Oh, that works great when you lose power to the rack. Or the datacentre. Or the SAN fails. Or the core routers. Or any of the many other SPOFs that can and do occur in a datacentre.
        easytiger 11y ago
        That's why you buy two and put them in different DCs
        creshal 11y ago
        And that's why IBM has their own bank branch to help customers figure out how to afford that?
        digi_owl 11y ago
otis_inf 11y ago
In an article which states "..IBM's z13 mainframe computer, which had been designed for high-volume mobile transactions.", all bets are off with respect to reality. I mean, what does 'designed for high-volume mobile transactions' even mean in the context of having Linux running on the hardware?
- dublinclontarf 11y ago
  Probably a realtime(i.e. happens within a guaranteed amount of time) system but the statement is still too vague.
- ams6110 11y ago
  Not sure if this is still correct for the z13 but in general: "System z servers offload such functions as I/O processing, cryptography, memory control, and various service functions (such as hardware configuration management and error logging) to dedicated processors." (from Wikipedia)
  - bridanp 11y ago
    It's called a zIIP engine (http://www-03.ibm.com/systems/z/hardware/features/ziip/ http://www-03.ibm.com/systems/z/hardware/features/ziip/) and it utilizes the IFL also (http://www-03.ibm.com/systems/z/os/linux/solutions/ifl.html http://www-03.ibm.com/systems/z/os/linux/solutions/ifl.html). We use the zIIP on our z13 basically to offload any work that takes a ton of I/O (DB2 data serves, dataset scans for performance monitoring, RMF and CICS reporting). It ends up saving a lot of money monthly because we are charged by maximum number of MSU's during any four-hour window of the month. The zIIP reduces the number of MSU's needed at any given time. In our case, 5-10. Back in 2005, I installed SuSe on an s/390 box. If you could throw a lot of memory at it or MIPs, then it would run pretty well, and could be used as a file or email server. But anything that crunched data was horribly slow. Can't say that it's any better now, but if it's utilizing the zIIP, I'm sure it's miles above where it was 10 years ago. Still, pretty high cost of ownership on the mainframe to get this. I can't imagine many people buying a million dollar mainframe just to run Linux. Seems like those who have a mainframe already would have been like us: let's partition out a Linux instance and try it out.
deleted 11y ago
[deleted]

rodgerd 11y ago

From my experience with zLinux environments I would be inclined to scrutinise any density claims very, very carefully. Interesting they're working with Canonical on this - the Z has historically been a SuSE stronghold, with a lot of the development of Linux-on-Z happening in Germany, and Red Hat trailing behind. I'd be wary it will be another increasingly typical adware effort from them. (I follow the main zLinux mailing lists for my day job, and I see a lot of folks from SuSE, some folks from Red Hat, a bunch of people from IBM, and basically no-one from the Ubuntu/Canonical world. Usual disclaimers apply.)

kokey 11y ago
I can think of two good reasons. One is if they want to offer these for virtual desktop environments (as an alternative to the VMWare on Cisco UCS setups many current IBM mainframe customers run) The other to offer an alternative platform to EC2 where Ubuntu seems to be very common. Both would indicate that they are probably aiming at the desktop/user/development side with this product.
wglb 11y ago
From my experience with zLinux environments Any specifics you can share?
- rodgerd 11y ago
  Depends what you want to know.
  - wglb 11y ago
    Well, I find I would be inclined to scrutinise any density claims very, very carefully interesting. Why do they deserve very careful scrutiny?
    - rodgerd 11y ago
      So I can only speak fairly generally, but I've never even met anyone running the kinds of ratios IBM is talking about in their press releases. I have, however, seen some workloads run pretty well; Oracle is IME a very good performer on Z, needing a smaller SGA and fewer IFLs than the equivalent Intel setup. Depending on the deal IBM offer you, zLinux can be a very good way of running Oracle. Conversely, CPU intensive Java don't really seem to enjoy much, if any, advantage, which makes it a very, very expensive option for doing that. There are a bunch of factors to consider (for example, if you've got zLinux and zOS co-resident on the same system there are some interesting things you can do with your legacy and modern code bases), but in general I'd want to prove anything myself before diving onto a Z.

dang 11y ago

Also https://news.ycombinator.com/item?id=10071679 https://news.ycombinator.com/item?id=10071679.

cm-t 11y ago

Note: Canonical seems to be a partner to bring Ubuntu on it See http://techcrunch.com/2015/08/16/ibm-teams-with-canonical-on-linux-mainframe/ http://techcrunch.com/2015/08/16/ibm-teams-with-canonical-on...

cm-t 11y ago
Confirmed here: https://insights.ubuntu.com/2015/08/17/ibm-and-canonical-plan-ubuntu-support-on-ibm-z-systems-mainframe/ https://insights.ubuntu.com/2015/08/17/ibm-and-canonical-pla...

deleted 11y ago

[deleted]

dijit 11y ago
linux is only 23 years old or so. (1991 iirc) also, they have a history of supporting linux, why so bitter?
- SagelyGuru 11y ago
  I meant they could have supported then some version of Unix. Linux wrote Linux only because the "big boys" did their best to ignore it.
  - noselasd 11y ago
    And they did. IBM have supported linux on mainframes since the earily days, they started sending patches in 1999. IBM also have their own version of UNIX, called AIX, for longer than that.
  - rodgerd 11y ago
    You... don't know much about Unix history or IBM products, do you?
  - rjsw 11y ago
    IBM did support Unix [1] and [2], several years before Linux was available. [1] https://en.wikipedia.org/wiki/Interactive_Systems_Corporation#PC.2FIX https://en.wikipedia.org/wiki/Interactive_Systems_Corporatio... [2] https://en.wikipedia.org/wiki/IBM_AIX https://en.wikipedia.org/wiki/IBM_AIX
  - skissane 11y ago
    Unix has a long history on the IBM mainframe, prior to Linux. It was running under VM/370 at Princeton in 1977 - https://www.bell-labs.com/usr/dmr/www/portpap.pdf https://www.bell-labs.com/usr/dmr/www/portpap.pdf page 3 - and in the early 1980s AT&T ran UNIX under TSS/370, which was used in development of the 5ESS switch - see https://www.bell-labs.com/usr/dmr/www/otherports/ibm.pdf https://www.bell-labs.com/usr/dmr/www/otherports/ibm.pdf. Amdahl UTS came out in 1981 (descended from the 1970s work at Princeton) and IBM/Interactive Systems VM/IX around 1984; then in 1988, the IBM/Locus AIX/370, followed by AIX/390. And then in 1993, MVS acquired a Unix compatibility subsystem (originally called OpenEdition, now called Unix Systems Services), and under its present name of z/OS is officially certified as a Unix, since Unix System Services passes the X/Open tests. I think the real problem for Unix on the mainframe has been the cost of the mainframe platform. Once you are developing portable applications, it is easy to shift them on to something else (AIX, Solaris, Linux, whatever), and you'll probably save money by doing so. When developing using a classic mainframe technology stack (COBOL, CICS, IMS, etc), replatforming is much more effort/risk, so those apps are more likely to survive on the mainframe in the long-term than Unix ones. (Disclaimer: I work for a competitor to IBM so I may well be biased.)

sgt 11y ago

When would this type of thing become cost effective? Let's say if you compare it with your typical X amounts of Dell servers also capable of running 8,000 virtual machines.

stefantalpalaru 11y ago

> (The story was refiled to correct the name of the server to "Emperor" from "Empire" in paragraph 4 and names of software to "MariaDB" and "PostgreSQL" from "Maria" and "Posture" in paragraph 5) These are the journalists specialized in IT news for a giant like Reuters?

zymhan 11y ago
As I read the bit about the correction, I figured Empire and Maria were understandable mistakes to make. But what in the hell is posture. Also, I know this is a newswire/business site, but this is super skimpy on the details.
- chops 11y ago
  Autocorrect? My phone corrects postgres to "postures".

arca_vorago 11y ago

So for a business with a complex AS400 system still in place, would this be a replacement for that or an augmentation of that? Their page isn't very clear. http://www-03.ibm.com/systems/z/os/linux/linux-one.html http://www-03.ibm.com/systems/z/os/linux/linux-one.html

deleted 11y ago

[deleted]

bluedino 11y ago

At the last user conference for some iSeries-based software that we run, IBM had a booth where they displayed a 2U server with dual 16-core Power7 CPUs. They bragged that it only ran Linux and would save us a ton of money. They started out at about $20,000 US. How small is this market? You'd have to have apps that were written to take advantage of Power7. Equivalent x86 Linux servers are 1/4 the price.

aus_ 11y ago
Just to clarify: P-series (Power/POWER8) Linux is not the same as what is announced here. LinuxONE runs on the System z (mainframe / s390x) platform.
- rodgerd 11y ago
  Yes, but they now share the same microarchitecture. s390x is mostly a difference in the microcode.
  - nickpsecurity 11y ago
    It has a custom processor: https://en.wikipedia.org/wiki/IBM_zEC12_%28microprocessor%29 https://en.wikipedia.org/wiki/IBM_zEC12_%28microprocessor%29
thoughtpolice 11y ago
I think the horsepower on those machines shouldn't be underestimated, because they are not entirely as equivalent as you think... I was thoroughly surprised when an unoptimized (but correct!) ChaCha20/8 implementation I wrote on a 3.0GHz POWER8 little-endian machine was about as fast as the latest 3.5gHz Xeons @ AES-256 with AESNI (about 1.3cpb vs 1.0cpb IIRC, but the latter has a dedicated hardware unit for it!) On that same Xeon, the ChaCha20 code only hit somewhere around 5cpb - that's software vs silicon! It also has 170 cores and was actually a QEMU instance (w/ hardware virtualization extensions) vs raw dedicated metal. If you're doing any kind of numerical or analytic workloads (even databases), I wouldn't throw them aside so quickly. You can even get CUDA for them these days, and certain physical addons like CAPI allow you to map and coherently share physical CPU address space with FPGAs or GPUs. If I could get those things in a reasonable workstation configuration, I'd probably go for it tbh. (I'd be more than willing to repeat this and post some more accurate numbers if anyone cares. I also need to get around to benchmarking AESNI vs that POWER8 machines _actual_ dedicated AES unit. The benchmark above was only flexing its vector/integer unit capabilities. ;)

jwildeboer 11y ago

I wonder if the KVM on zSeries thingy is actually open source and upstream. From what I can see they (IBM) say it's "based on open source" which is market droid speak for "actually it's not that open".

monocasa 11y ago
It appears to be pretty open source. http://lxr.free-electrons.com/source/arch/s390/kvm/ http://lxr.free-electrons.com/source/arch/s390/kvm/

tristor 11y ago

Frankly, my experience on mainframes is that UNIX is a stronghold there for a reason. I'd be far more interested in this ecosystem opening up by having strong mainframe vendor support for running Illumos and/or FreeBSD, and less concerned with running Linux. Among other things, Zones are more useful than lxc, and in this type of environment you often need strong kernel support for specialized high-speed interconnects and real-time operations, which Linux only has experimental support for but is integrated well in the UNIX world. I love Linux and what it's done for the world, but UNIX isn't dead for a reason, there are still many things it is superior at.

the_why_of_y 11y ago
This is an odd remark - apparently AIX/ESA, the AIX version for mainframe hardware was discontinued "in the late 1990s", roughly the same time when IBM started to invest in Linux. http://www.lookupmainframesoftware.com/soft_detail/dispsoft/1581 http://www.lookupmainframesoftware.com/soft_detail/dispsoft/... There was also Amdahl UTS, which was sold to a company UTS Global around 2000 that appears defunct now. https://en.wikipedia.org/wiki/Amdahl_UTS https://en.wikipedia.org/wiki/Amdahl_UTS So where is this stronghold of AT&T descendant UNIXes on mainframes then?
- tolle 11y ago
  Nowhere, but some people call System I/(A/400)/I-Series a mainframe. It's midrange, but people call it what they call it. Anyway, IBM i and AIX run on the same hardware. They used to be called System I and System P. Now it's just Power Systems. Maybe thats what he ment. Or well, come to think of it. There is a UNIX side to z/OS itself.

aus_ 11y ago

By trade, I am a mainframe systems programmer at a large financial institution. My group is responsible for mainframe operating systems: z/OS (it may also be referred to as MVS), z/VM (the mainframe hypervisor), and z/Linux (Linux on a mainframe) and many of the software components that go along with these. I'm really excited about this announcement. I think IBM finally realized they need to be more open if they want to grow the Linux on z community. I just hope they are not too late to the game. There's a lot of mystery and misconception behind mainframes, so I am happy to answer any questions about mainframes that I can.

mozumder 11y ago
So, why use Linux instead of z/OS? Is it cheaper? Offer more tools?
- noobermin 11y ago
  I'm curious too. Of course, anyone here can quote the usual benefits of open source (no licenses, familiarity with the technical crowd in general, the "many-eyes"), but I'm curious how valid these benefits are in the mainframe space.
- clord 11y ago
  The reason from my perspective is that it is different than a traditional unix system, and what unix emulation it offers is limited (to put it in perspective, it feels like a bad vim emulation... your native skillset doesn't quite translate). Lots of issues with dealing with data formats (EBCDIC). Access to open source compilers and scripting languages was limited. Granted these were strengths of the platform as well, but it made our workload (maintaining a compiler for the target) harder than other unix-alikes for those of us not fully versed in JCL.
- rdtsc 11y ago
  Maybe more software is supported on Linux. So it combined some of the hardware advantages of a mainframe + more recent/wide variety of packages?
- aus_ 11y ago

nickpsecurity 11y ago

It's about time. This is long overdue. The combination of the mainframe architecture's strengths with Linux's API/ecosystem could be pretty awesome. Anyone wondering why buy a mainframe should focus on these areas: 1. Reliability. Some have gone 30 years without downtime. Probably strongest selling point. 2. Channel I/O [1]: dedicated I/O processors plus scheduling that lead to high utilization (80-90+%) and throughput vs commodity servers. Second strongest point in mainframe's favor. I wish my desktop & servers had this rather than a knockoff. 3. Hardware partitioning that's more robust and rated at stronger security than most virtualization. Certain cutting-edge projects in INFOSEC are doing similar things at CPU and I/O layers. Mainframe's version, although not as cool, is decent and field-proven. 4. Built-in, proven software virtualization. 5. Hardware acceleration for some things such as databases and crypto. 6. IBM's ecosystem of apps, third-party providers, and services. This might matter to existing IBM customers. So, those are a few advantages I hear from people who use mainframes. The z/OS-based mainframes have extra benefits in terms of software reliability, security through obscurity (obfuscation), and seemingly better use of both security (eg memory key) and functional (eg decimal) aspects of mainframe processors. The z/VM product has also been doing for decades what modern virtualization systems only recently do, even self-virtualizing since 70's. So, there's some things to ponder. Whether it makes since financially vs other setups is a whole, different discussion. However, mainframes do retain strong, technical advantages over commodity architectures. They were doing cloud in one box before it was a thing. Their reliability is still unmatched with only VMS clusters and NonStop architecture getting close. So, it's a sensible choice for a business to spend extra $$$ to get high-throughput with no downtime and strong isolation of logical partitions. Channel I/O [1] https://en.wikipedia.org/wiki/I/O_channel https://en.wikipedia.org/wiki/I/O_channel

ilaksh 11y ago

How much does the 8000 core thing cost?

nickpsecurity 11y ago
Yeah, I'd really like to see that number. I bet it's so big that it's highly negotiable and still highly profitable. ;)