6 ms·
Ignoring for a second that they claim these have the same performance without providing any actual benchmarks that I see... the server configurations alone make
by tw04 5y ago
Ignoring for a second that they claim these have the same performance without providing any actual benchmarks that I see... the server configurations alone make no sense. In 2021 nobody would order 12x HP proliants with less than 256GB of memory per server and then run 1,000 VMs on them. That averages out to what? Roughly 2GB of memory per server unless you're horribly oversubscribing them - which nobody who has ever designed a virtual environment would ever tell you to do - so your VMs are going to be memory starved pretty much 24/7. Even IBM tells you not to oversubscribe memory on their mainframes.
I also struggle to believe that oversubscribing the z CPUs 66:1 isn't going to result in performance issues. The documents on limits aren't clear but I foresee that mainframe spending a LOT of time on locks even with all the work IBM has done to help manage them.
This whole thing just really smells like a bad advertisement for IBM, which aligns with them announcing their latest gen z processor last week.
- oneplane 5y agoYeah, this whole LPAR thing is supposed to make oversubscribing less of an issue, but in practical terms it is comparable to Xen PV without HVM mode (Essentially Vt-i without Vt-d). It's not that it never works, it's just that you have to take severe shortcuts and snowflake kernels to get it to work in the first place (so now you're incompatible with the body of knowledge that is the internet), and then you essentially just do multiproces CPU core sharing, memory sharing with swapping etc... not great.
- evol262 5y agoVT-d is used for device passthrough. I suspect you're thinking of VT-x. But LPAR have been around forever, and aren't that different conceptually from jails, systemd-nspawn, LXC, and any other of a large number of OS-level virtualization solutions, except that you can stub in/out different kernels. Yes, the kernels need to be "snowflakes", but that's not necessarily an issue if this is functionality you really need.
- nix23 5y agoLAPR is not a OS-level virtualization, you can install Linux on a LPAR (however you do it normally with z/vm), it's more like a Hard, Hardware divider. https://en.wikipedia.org/wiki/Logical_partition https://en.wikipedia.org/wiki/Logical_partition
- oneplane 5y agoEssentially it's a microcode hypervisor. You can indeed install Linux on an LPAR, but it will be a useless linux unless you intend to use it for classic 'bog box' installations. (with big box referring to the cardboard boxes that come with installation floppies/CD/DVD media type of software 'distribution') While an LPAR could technically be HVM, the nature of the changes required to an OS make it PV-like.
- nix23 5y agoI just wanted to say that it's NOT like jails and it's not OS-Virtualization.
- oneplane 5y agoIf someone wanted to make a comparison, it's like additional 'rings' the x86 ring security model, but instead of nested rings it's parallel rings. You might as well have a single OS with containers.
- oneplane 5y agoYes, you are correct, I was thinking about the oldest of the VT's. The one that did accelerate but didn't even require an IOMMU.
- Misteur-Z 5y agoYou a need a shitload more RAM for sure and this also forgets about plugging a SAN, and any SAN from IBM is super expensive on hardware AND licenses.
- 300bps 5y agoThe article's price tag of $2.3 MM for 12 HP Servers with 24 cores per server and 166 GB of RAM each is ridiculous. We just paid just under $50k per server for on-prem physical servers that have 144 cores per server and 1.5 TB of RAM each. For 12 of them it would be less than $600k and you'd get a total of 1,728 cores and 18 TB of RAM to compare against the mainframe's 30 cores and 2 TB of RAM for over 4x the cost!
- throwaway98797 5y agoNobody gets fired for buying IBM.
- matmatmatmat 5y agoWell, maybe in the past.
- tyingq 5y agoUsing list pricing comparisons of course favors the mainframe in this comparison. Since commodity hardware is sold at a fraction of list price and mainframes are not.
- m4rtink 5y ago4x the cost is actually not bad for the massive redundancy, hotswap-everything, boatloads of IO, insane build quality and crtitical legacy software support a mainframe gives you!
- evol262 5y agoWhile I agree that this configuration needs more memory, mainframes are primarily used for very high throughput jobs. "Lots of memory to sit idle" isn't really a priority. That aside, samepage sharing has been a thing in virtualization for more than a decade. If you have 800 VMs running RHEL8 for POWER, a significant amount of the memory load is kept in identical pages, which lowers the burden of oversubscription. The recommended workload is unlikely to be 1000 unicorn/pet VMs, and more likely to be a large number of VMs running the same base OS, and some of the same workloads. Synchronous interrupt locking is also a solved problem.
- jfindley 5y agoIf by "samepage merging" you mean KSM[0], it's hugely expensive in CPU cycles (not to mention severe impacts on membw, l3 cache, etc) and outside of low-end VPS hosting has relatively few real-world applications. I'm not sure why it would be a useful thing to mention here. 0: https://www.kernel.org/doc/html/latest/admin-guide/mm/ksm.html https://www.kernel.org/doc/html/latest/admin-guide/mm/ksm.ht...
- evol262 5y agoYes, I mean KSM (and its equivalents in other hypervisors). The CPU cycles are a non-issue for the configuration on this mainframe, and it's fair to assume that, since IBM developed the silicon and IBM engineers wrote the hypervisor layer, that most of the detrimental effects of "we need this to work across multiple CPU vendors and multiple generations of the architecture" are covered. Even x86 CPUs get improvements to cache coherency in memory dedupe scenarios with generational improvement, and intelligent NUMA topology layout helps a lot. It's also worth noting that it's a mainframe, so essentially every configuration has a large number of other processors with a couple of instructions disabled so they don't "count" as CPUs for licensing, but they still perform work. I would not be surprised in the least if there were 1+ co-processors dedicated to offloading operations exactly like "LPAR || z/VM memory dedupe"
- jfindley 5y agoThat's a good point - I was speaking from experience with KSM on x86, but having a dedicated coprocessor on system Z for this application would make a lot of sense and fix many of the problems I've seen on x86.
- mathattack 5y agoWelcome to the world of IBM. This is consistent with their Watson marketing. They’ve stopped being an innovation business.
- nix23 5y agoLike Intel, Microsoft and Google?
- mathattack 5y agoWorse. The others still try.
- nix23 5y agoMicrosoft and Google try? But yeah Intel is trying really hard.
- spoonjim 5y agoThe purpose of this article isn’t to actually argue that mainframes are cheaper. It’s an asset for a CTO who’s staying on mainframes. When the CFO asks him “why aren’t we moving to Azure? I heard my buddy say he saved $40 million doing that” the CFO sends this article.
- ksec 5y agoIf anything this article shows why Financial companies aren't moving away from Mainframe. It is comparatively speaking such as small amount of money while the risk are incalculable. Upgrading to Z15 is a much easier option.
- Johnny555 5y agoIt's been a long time since I've run a VMWare cluster, but that's exactly what they told you to do - oversubscribe memory and let their memory dedupe, balloon driver + swapping handle the case where you run out of physical RAM.
- pickle-wizard 5y agoStarting with ESXi 6 VMware has disabled the transparent page sharing (Memory Dedup) by default. Back in 2015 or so there was a theoretical VM escape vector that used the shared memory pages. So VMware just changed the default to disabled. I usually recommend turning it back on in lab environments, but leave it off in production for security reasons.
- lucb1e 5y ago> Roughly 2GB of memory per server [...] so your VMs are going to be memory starved pretty much 24/7. Most applications don't need gigabytes of memory though? I don't know what one uses a mainframe for but web servers, file shares, remote login software, proxies, traffic scrubbers... lots of common applications use up to a few dozen megabytes of RAM; throughput is much more important there and memory to operate on is just for some state and buffers. Even a database with millions of rows and a dozen indexed columns, the index of that (I just checked a database of my own) is in the hundreds of megabytes, far from even one gigabyte. It seems a bit odd to assume each server always needs multiple gigabytes of RAM.