12 ms·

"IBM said LinuxONE Emperor can scale up to 8,000 virtual machines or thousands of containers, which would be the most for any single Linux system." I wish arti

"IBM said LinuxONE Emperor can scale up to 8,000 virtual machines or thousands of containers, which would be the most for any single Linux system." I wish article (or IBM) provided some specs. Thousands of containers is just too broad of a statement- what are these containers doing? Running just a bash script hello world program or minting bitcoins?

stonewhite 11y ago
Exactly. Even its official site lacks any clear explanation and solution brief pdf is 404. I don't see how this is any different from a regular server rack. http://www-03.ibm.com/systems/z/os/linux/linux-one.html http://www-03.ibm.com/systems/z/os/linux/linux-one.html
- kkuduk 11y ago
  > I don't see how this is any different from a regular server rack well, the difference is that most rack cannot fit 10 TB of RAM and 140x5GHz cores: http://www-03.ibm.com/systems/z/hardware/z13_specs.html http://www-03.ibm.com/systems/z/hardware/z13_specs.html but surely those are super-expensive and I doubt all that effective when it comes to performance per watt. I'm actually a bit more excited about Power8 servers, but are there any non-IBM available yet?
  - ajdlinux 11y ago
    > I'm actually a bit more excited about Power8 servers, but are there any non-IBM available yet? Yep, OpenPOWER machines are starting to come out - Tyan has the Habanero server as well as the Palmetto reference/development platform. http://www.tyan.com/solutions/tyan_openpower_system.html http://www.tyan.com/solutions/tyan_openpower_system.html (disclosure: I'm an IBMer working on Power)
    - A010 11y ago
      > High-performance computing (HPC) > Data centers > Big data Can you give me some specific roles for this range of servers? What makes it's worth compared to x86 system?
      - rwmj 11y ago
        The high-end POWER 7/8 hardware has incredible single core performance, beating the pants off Xeon. It uses huge amounts of power to do that, so it's not appropriate for all roles. Low end POWER is pretty niche. Freescale uses the architecture for telecoms applications. In general Linux upstream these days "just works" on ppc64 & ppc64le. There's RHEL for POWER already, and IBM have loaned hardware to the CentOS project so we'll get CentOS on POWER pretty soon. The licensing of (Open-)POWER is more open than x86 (but not as open as stuff like RISC-V), and there are several second sources for chips, in situations where that matters.
      - Spooky23 11y ago
        You get a pretty decent multiple of performance on power vs. intel. Probably something like 4-7x for common use cases. Generally speaking, you buy the hardware because your workload needs the single core performance, or you're arbitraging vendor licensing costs for software. Also, IBM's bread and butter is "peaky" financial services and gov't business, so they have business models that makes it work from a $ pov. You can buy a box with 100 cores, pay for 20, and lease 30 more for a few days to meet your peak demands for tax/christmas/billing season.
  - rodgerd 11y ago
    > well, the difference is that most rack cannot fit 10 TB of RAM and 140x5GHz cores: http://www-03.ibm.com/systems/z/hardware/z13_specs.html http://www-03.ibm.com/systems/z/hardware/z13_specs.html I can fit 4 C7000 chassis in a single rack, which top out at an aggregate 32 TB of memory and over 2000 cores. 5 GHz cores running s390x microcode are not, FYI, twice as fast as Xeons. It would be great if people who weren't familiar with the state of the art in x86 kit didn't blindly assume IBM et al's advertising was reality.
    - jamestnz 11y ago
      I'm no expert, but my understanding is that the IBM "z" platform has a number of interesting features perhaps not found in more typical server platforms/configurations, beyond just scaling out to loads of cores and RAM. As with all IBM mainframes going right back to the 60s, the "z" systems are designed for continuous uptime, on the order of decades, and this is evident in many of the design decisions. For example, various subsystems and components have hot-spares available, so that even outright component failures will not cause downtime. Many components are then hot-swappable, even those that might not ordinarily be in other architectures, such as processors and main memory. No interruption to OS or application-level services is expected by hot-swapping such hardware. Across the useful life of the mainframe, most repairs and maintenance would be carried out without ever shutting it down. I understand the platform also has extensive internal integrity checking built in, a potentially important factor for various types of jobs. Its auditing service is capable of detecting unusual conditions in various subsystems or jobs ("I've just picked up a fault in the AE-35 unit"), automatically retrying instructions on the processor if they executed anomalously. If the fault continues, the suspect processor is routed-around with no interruption to OS or applications, the job is resumed from last checkpoint on another processor, and the system phones home to IBM to log a service call. This monitoring is not being performed by processes running in userland or by the kernel, but is in fact baked into the hardware/firmware platform. Furthermore, the systems can be configured with a variety of specialty offload processors or subsystems for tasks like encryption, key management, compression, and even logging -- which again might not be so commonly found on-board of some commodity servers. (And, of course, even if you could put together an analogous solution with commodity kit, it's IBM! For the sorts of companies looking at a mainframe in 2015, having the IBM name on the SLA has got to be a pretty big part of the equation, right?) Moreover, if you proposed to build and manage these sorts of capabilities from commodity x86 kit, I imagine IBM would claim that they'd have the lower TCO.
  - vidarh 11y ago
    > most rack cannot fit 10 TB of RAM and 140x5GHz cores Any rack can fit 10TB of RAM and the equivalent of 140x5GHz POWER cores. I can easily get ~1TB RAM and 64 2GHz+ x86 cores in 1U. A standard rack is at least 42U. In terms of density it's nothing special. What makes these different is that you get it in a box that someone else will be maintaining, and where all the dirty work of designing and building a high availability redundant system has been done for you. Most people never build anything remotely as redundant as these things tend to be.
    - vegabook 11y ago
      I'm guessing part of the appeal will also be super-wide bandwidth between the cores. Interconnect throughput and/or shared memory could make this significantly more interesting than a modular rack-based system for some workloads.
      - rodgerd 11y ago
        > I'm guessing part of the appeal will also be super-wide bandwidth between the cores. If it's anything like previous-generations, that will be infiniband between books. The benefits on Z-class systems tend to be very big per-core, per-socket caches and, as the person you're replying to said, a lot of internal redundancy and automated failover (multiple backplanes with failover, spare memory and cores with failover, and so on and so forth).
  - jdboyd 11y ago
    There is the TYAN GN70-BP010, although they state it isn't for production use.
- deleted 11y ago
  [deleted]
blackbeard 11y ago
That sounds like a whole lot of eggs in one very expensive basket. Plus we can get that density with standard kit I reckon.
- creshal 11y ago
  That's kinda the point of a mainframe, is it not?
  - blackbeard 11y ago
    Spending money? Yep.
    - creshal 11y ago
      No, putting all your eggs into one (hopefully redundant) basket so you only need to yell at one person.
      - rodgerd 11y ago
        Oh, that works great when you lose power to the rack. Or the datacentre. Or the SAN fails. Or the core routers. Or any of the many other SPOFs that can and do occur in a datacentre.
        easytiger 11y ago
        That's why you buy two and put them in different DCs
        creshal 11y ago
        And that's why IBM has their own bank branch to help customers figure out how to afford that?
        digi_owl 11y ago
otis_inf 11y ago
In an article which states "..IBM's z13 mainframe computer, which had been designed for high-volume mobile transactions.", all bets are off with respect to reality. I mean, what does 'designed for high-volume mobile transactions' even mean in the context of having Linux running on the hardware?
- dublinclontarf 11y ago
  Probably a realtime(i.e. happens within a guaranteed amount of time) system but the statement is still too vague.
- ams6110 11y ago
  Not sure if this is still correct for the z13 but in general: "System z servers offload such functions as I/O processing, cryptography, memory control, and various service functions (such as hardware configuration management and error logging) to dedicated processors." (from Wikipedia)
  - bridanp 11y ago
    It's called a zIIP engine (http://www-03.ibm.com/systems/z/hardware/features/ziip/ http://www-03.ibm.com/systems/z/hardware/features/ziip/) and it utilizes the IFL also (http://www-03.ibm.com/systems/z/os/linux/solutions/ifl.html http://www-03.ibm.com/systems/z/os/linux/solutions/ifl.html). We use the zIIP on our z13 basically to offload any work that takes a ton of I/O (DB2 data serves, dataset scans for performance monitoring, RMF and CICS reporting). It ends up saving a lot of money monthly because we are charged by maximum number of MSU's during any four-hour window of the month. The zIIP reduces the number of MSU's needed at any given time. In our case, 5-10. Back in 2005, I installed SuSe on an s/390 box. If you could throw a lot of memory at it or MIPs, then it would run pretty well, and could be used as a file or email server. But anything that crunched data was horribly slow. Can't say that it's any better now, but if it's utilizing the zIIP, I'm sure it's miles above where it was 10 years ago. Still, pretty high cost of ownership on the mainframe to get this. I can't imagine many people buying a million dollar mainframe just to run Linux. Seems like those who have a mainframe already would have been like us: let's partition out a Linux instance and try it out.
deleted 11y ago
[deleted]

stonewhite 11y ago

Exactly. Even its official site lacks any clear explanation and solution brief pdf is 404. I don't see how this is any different from a regular server rack. http://www-03.ibm.com/systems/z/os/linux/linux-one.html http://www-03.ibm.com/systems/z/os/linux/linux-one.html

kkuduk 11y ago
> I don't see how this is any different from a regular server rack well, the difference is that most rack cannot fit 10 TB of RAM and 140x5GHz cores: http://www-03.ibm.com/systems/z/hardware/z13_specs.html http://www-03.ibm.com/systems/z/hardware/z13_specs.html but surely those are super-expensive and I doubt all that effective when it comes to performance per watt. I'm actually a bit more excited about Power8 servers, but are there any non-IBM available yet?
- ajdlinux 11y ago
  > I'm actually a bit more excited about Power8 servers, but are there any non-IBM available yet? Yep, OpenPOWER machines are starting to come out - Tyan has the Habanero server as well as the Palmetto reference/development platform. http://www.tyan.com/solutions/tyan_openpower_system.html http://www.tyan.com/solutions/tyan_openpower_system.html (disclosure: I'm an IBMer working on Power)
  - A010 11y ago
    > High-performance computing (HPC) > Data centers > Big data Can you give me some specific roles for this range of servers? What makes it's worth compared to x86 system?
    - rwmj 11y ago
      The high-end POWER 7/8 hardware has incredible single core performance, beating the pants off Xeon. It uses huge amounts of power to do that, so it's not appropriate for all roles. Low end POWER is pretty niche. Freescale uses the architecture for telecoms applications. In general Linux upstream these days "just works" on ppc64 & ppc64le. There's RHEL for POWER already, and IBM have loaned hardware to the CentOS project so we'll get CentOS on POWER pretty soon. The licensing of (Open-)POWER is more open than x86 (but not as open as stuff like RISC-V), and there are several second sources for chips, in situations where that matters.
    - Spooky23 11y ago
      You get a pretty decent multiple of performance on power vs. intel. Probably something like 4-7x for common use cases. Generally speaking, you buy the hardware because your workload needs the single core performance, or you're arbitraging vendor licensing costs for software. Also, IBM's bread and butter is "peaky" financial services and gov't business, so they have business models that makes it work from a $ pov. You can buy a box with 100 cores, pay for 20, and lease 30 more for a few days to meet your peak demands for tax/christmas/billing season.
- rodgerd 11y ago
  > well, the difference is that most rack cannot fit 10 TB of RAM and 140x5GHz cores: http://www-03.ibm.com/systems/z/hardware/z13_specs.html http://www-03.ibm.com/systems/z/hardware/z13_specs.html I can fit 4 C7000 chassis in a single rack, which top out at an aggregate 32 TB of memory and over 2000 cores. 5 GHz cores running s390x microcode are not, FYI, twice as fast as Xeons. It would be great if people who weren't familiar with the state of the art in x86 kit didn't blindly assume IBM et al's advertising was reality.
  - jamestnz 11y ago
    I'm no expert, but my understanding is that the IBM "z" platform has a number of interesting features perhaps not found in more typical server platforms/configurations, beyond just scaling out to loads of cores and RAM. As with all IBM mainframes going right back to the 60s, the "z" systems are designed for continuous uptime, on the order of decades, and this is evident in many of the design decisions. For example, various subsystems and components have hot-spares available, so that even outright component failures will not cause downtime. Many components are then hot-swappable, even those that might not ordinarily be in other architectures, such as processors and main memory. No interruption to OS or application-level services is expected by hot-swapping such hardware. Across the useful life of the mainframe, most repairs and maintenance would be carried out without ever shutting it down. I understand the platform also has extensive internal integrity checking built in, a potentially important factor for various types of jobs. Its auditing service is capable of detecting unusual conditions in various subsystems or jobs ("I've just picked up a fault in the AE-35 unit"), automatically retrying instructions on the processor if they executed anomalously. If the fault continues, the suspect processor is routed-around with no interruption to OS or applications, the job is resumed from last checkpoint on another processor, and the system phones home to IBM to log a service call. This monitoring is not being performed by processes running in userland or by the kernel, but is in fact baked into the hardware/firmware platform. Furthermore, the systems can be configured with a variety of specialty offload processors or subsystems for tasks like encryption, key management, compression, and even logging -- which again might not be so commonly found on-board of some commodity servers. (And, of course, even if you could put together an analogous solution with commodity kit, it's IBM! For the sorts of companies looking at a mainframe in 2015, having the IBM name on the SLA has got to be a pretty big part of the equation, right?) Moreover, if you proposed to build and manage these sorts of capabilities from commodity x86 kit, I imagine IBM would claim that they'd have the lower TCO.
- vidarh 11y ago
  > most rack cannot fit 10 TB of RAM and 140x5GHz cores Any rack can fit 10TB of RAM and the equivalent of 140x5GHz POWER cores. I can easily get ~1TB RAM and 64 2GHz+ x86 cores in 1U. A standard rack is at least 42U. In terms of density it's nothing special. What makes these different is that you get it in a box that someone else will be maintaining, and where all the dirty work of designing and building a high availability redundant system has been done for you. Most people never build anything remotely as redundant as these things tend to be.
  - vegabook 11y ago
    I'm guessing part of the appeal will also be super-wide bandwidth between the cores. Interconnect throughput and/or shared memory could make this significantly more interesting than a modular rack-based system for some workloads.
    - rodgerd 11y ago
      > I'm guessing part of the appeal will also be super-wide bandwidth between the cores. If it's anything like previous-generations, that will be infiniband between books. The benefits on Z-class systems tend to be very big per-core, per-socket caches and, as the person you're replying to said, a lot of internal redundancy and automated failover (multiple backplanes with failover, spare memory and cores with failover, and so on and so forth).
- jdboyd 11y ago
  There is the TYAN GN70-BP010, although they state it isn't for production use.
deleted 11y ago
[deleted]

blackbeard 11y ago

That sounds like a whole lot of eggs in one very expensive basket. Plus we can get that density with standard kit I reckon.

creshal 11y ago
That's kinda the point of a mainframe, is it not?
- blackbeard 11y ago
  Spending money? Yep.
  - creshal 11y ago
    No, putting all your eggs into one (hopefully redundant) basket so you only need to yell at one person.
    - rodgerd 11y ago
      Oh, that works great when you lose power to the rack. Or the datacentre. Or the SAN fails. Or the core routers. Or any of the many other SPOFs that can and do occur in a datacentre.
      - easytiger 11y ago
        That's why you buy two and put them in different DCs
        creshal 11y ago
        And that's why IBM has their own bank branch to help customers figure out how to afford that?
        digi_owl 11y ago

otis_inf 11y ago

In an article which states "..IBM's z13 mainframe computer, which had been designed for high-volume mobile transactions.", all bets are off with respect to reality. I mean, what does 'designed for high-volume mobile transactions' even mean in the context of having Linux running on the hardware?

dublinclontarf 11y ago
Probably a realtime(i.e. happens within a guaranteed amount of time) system but the statement is still too vague.
ams6110 11y ago
Not sure if this is still correct for the z13 but in general: "System z servers offload such functions as I/O processing, cryptography, memory control, and various service functions (such as hardware configuration management and error logging) to dedicated processors." (from Wikipedia)
- bridanp 11y ago
  It's called a zIIP engine (http://www-03.ibm.com/systems/z/hardware/features/ziip/ http://www-03.ibm.com/systems/z/hardware/features/ziip/) and it utilizes the IFL also (http://www-03.ibm.com/systems/z/os/linux/solutions/ifl.html http://www-03.ibm.com/systems/z/os/linux/solutions/ifl.html). We use the zIIP on our z13 basically to offload any work that takes a ton of I/O (DB2 data serves, dataset scans for performance monitoring, RMF and CICS reporting). It ends up saving a lot of money monthly because we are charged by maximum number of MSU's during any four-hour window of the month. The zIIP reduces the number of MSU's needed at any given time. In our case, 5-10. Back in 2005, I installed SuSe on an s/390 box. If you could throw a lot of memory at it or MIPs, then it would run pretty well, and could be used as a file or email server. But anything that crunched data was horribly slow. Can't say that it's any better now, but if it's utilizing the zIIP, I'm sure it's miles above where it was 10 years ago. Still, pretty high cost of ownership on the mainframe to get this. I can't imagine many people buying a million dollar mainframe just to run Linux. Seems like those who have a mainframe already would have been like us: let's partition out a Linux instance and try it out.

deleted 11y ago

[deleted]