13 ms·

What does one use a mainframe for these days? I'm assuming projects that require massive parellelism, like research projects or engineering modeling and computa

• dijit 7y ago
  I know most airlines run on big iron mainframes. Mainframes are undoubtedly kings for things like high throughput highly atomicity databases. (Although I’m sure the precise problem set could be replicated with commodity hardware too).
  • tyingq 7y ago
    "Although I’m sure the precise problem set could be replicated with commodity hardware too" Only pretty recently. You would be surprised at the number of crazy smart people that failed to make a commercially viable TPF replacement. Including ITA, whom Google bought for ~$700M. Lots of top talent, lots of funding. They did build a reservation system, but nobody of note would use it. Amadeus only got rid of their TPF mainframes in the last year or so. As far as I know, Sabre hasn't finished. I don't know what progress VISA has made (another heavy TPF user). They are still hiring TPF programmers: https://usa.visa.com/careers/job-details.jobid.743999678230139.deptid.1146810.html https://usa.visa.com/careers/job-details.jobid.7439996782301...
    • selectodude 7y ago
      Even if these IBM mainframes are $10 million each, its such a rounding error in terms of cost that I can’t think of a good reason to migrate off of them. What does commodity hardware add other than complexity?
      • tyingq 7y ago
        Mostly a platform you can find programmers for. I believe that's really the main driver. If you have business logic in TPF, you need TPF literate 360 Assembler developers. Technically, you can program in C, but legacy integration and performance issues make that of limited use. People moving away usually tried to extract the business logic such that the TPF layer was mostly a distributed NoSQL store. That bought a lot of time, but it seems like the skill shortage is now hitting that layer.
      • coder543 7y ago
        I would have asked the opposite question: what does mainframe add other than complexity? It's a lot easier to find good engineers to write and maintain software for Linux on x86-64 than it is to find mainframe engineers, which makes things simpler, arguably.
        • TheOtherHobbes 7y ago
          A mainframe adds reliability and backward compatibility, and the knowledge that you don't have to rewrite your systems from the ground up. How long is Linux on x86-64 going to stay a thing? What are you going to do about staging Linux updates on business-critical facilities? Perhaps not so simple after all. Which suggests an interesting contrarian career path - train on COBOL + mainframe you'll never lack well-paid work. You'll also avoid the usual fad tracking.
          • coder543 7y ago
            > A mainframe adds reliability and backward compatibility Citation needed, especially on that implication that x86-64 lacks reliability. I’ve run reliable services on commodity hardware that literally processed over 5 billion transactions per day. > How long is Linux on x86-64 going to stay a thing? If something better comes along, why wouldn’t you want to move to it? But “better” in this context would imply popular support and plentiful access to developers, so you would have years of warning. It wouldn’t come as a surprise. As of now, it has been going for at least a decade, and shows no signs of stopping. > What are you going to do about staging Linux updates on business-critical facilities? This is not some huge, unsolved problem. It has been solved many times, in my opinion, so just learn how others do it and follow suit. I’m not here to teach sysadmin “tips and tricks.” I would be completely clueless on how to stage updates for a mainframe. Turn it off and back on? And I’m betting all of the good training material there is locked behind expensive paywalls. > Perhaps not so simple after all. Disagree, especially since you can hire people to solve these problems for a lot less money than you can get just the hardware for a mainframe, let alone hire the extremely rare (read: expensive) personnel needed to maintain and develop applications for mainframe. If IBM would focus on bringing the entry-level cost of mainframe down, they would probably be able to get more adoption, and more people would be able and motivated to learn their systems.
      • toast0 7y ago
        The migration itself may be valuable, if treated properly. Many airlines have had very public issues when their mainframe had problems running (often power issues, but sometimes other things); a migration to x86 includes accepting and handling more failures, which may include having a warm second site available. You can certainly have a warm second mainframe too, and I think most large banks are run that way, but it wouldn't be very fashionable to add that to your airline now, if it wasn't already there.
        • tyingq 7y ago
          TPF is inherently hot/hot, distributed across many mainframes. It's very different from z/OS.
    • mruts 7y ago
      According to Wikipedia it seems like a lot of people use or used ITA: “This system has been and is used by travel companies such as Bing Travel, CheapTickets, Kayak.com, and Orbitz, and by airlines such as Alitalia, American, ANA, Cape Air, Delta Air Lines, United Airlines, US Airways, and Virgin Atlantic.”
      • tyingq 7y ago
        That's the shopping engine, which was successful, but only a small subset of what a TPF reservation system does...albeit one of the more complex pieces. ITA did build the full thing, but only Cape Air used it. The "full thing" being shopping, schedules, inventory, booking, cancel, change, check-in, etc.
        • bogomipz 7y ago
          Oh interesting so ITA is really just a retail front-end for TPF then? I guess I was misinformed about it as well.
          • mruts 7y ago
            I was under the impression that ITA used exclusively Common Lisp to inplement their software. I could be wrong, though.
            • lispm 7y ago
              Other parts were written other languages. ITA also used languages like C++ (reported to be roughly 50% of the line count for QPX) and Java (for RES).
          • tyingq 7y ago
            The customers shop using ITA. The ITA shopping engine gets schedules, inventory and updates inventory from/to the backend (often TPF). The shopping engine itself isn't trivial. ITA has the best shopping engine available.
    • bogomipz 7y ago
      Is TPF running on Linux on commodity X86 now then? Is that also the eventual plan for Sabre?
      • monocasa 7y ago
        TPF itself, and the programs written in it are mainly in s/360 asm, so I doubt very much it's running on x86 now. It's so s/360 it doesn't have traditional stacks.
        • bogomipz 7y ago
          Is there a way to reconcile these two above responses? They seem to be very different answers. Are these both options that a customer can choose from?
          • monocasa 7y ago
            Yeah, it sounds like they rewrote it piece by piece to run on different architectures. Really similar to having a big monolith, and taking pieces out chunk by chunk rewritten to run as microservices until you're left with a new codebase with no sign of the original monolith or any of it's code.
          • deleted 7y ago
            [deleted]
      • tyingq 7y ago
        They slowly moved the business logic out of TPF ASM to x86 app servers, which left the TPF machines as mostly distributed NoSQL servers. Then they replaced that layer with Couchbase or similar. Took 15 years or so.
• jeremyjh 7y ago
  I'd guess they are only used to run legacy mainframe software. The accounting systems (loan accounting, deposits, etc) of large banks for example are all based on very old COBOL code bases and have lots of assumptions about their OS platform built into them. These code bases are not only very stable and well-tested but they are deeply integrated into hundreds of other systems in the bank. Porting them from Z/OS or replacing them is probably out of the question, but there more are modern tools for integrating them into a services-oriented architecture for example.
  • coleca 7y ago
    Surprisingly there are reasons why running financial type systems in COBOL makes a lot of sense. Evidently COBOL really excels at foxed point decimal operations more so than other “modern” languages like Python and Java. Porting then off the mainframe isn’t a trivial exercise it seems. https://medium.com/@bellmar/is-cobol-holding-you-hostage-with-math-5498c0eb428b https://medium.com/@bellmar/is-cobol-holding-you-hostage-wit...
    • jeremyjh 7y ago
      Java has had a fixed point type - BigDecimal - for a couple of decades now. Yes, there are people who don’t know what they are doing using floats for currency.
      • neverartful 7y ago
        This is true. The mainframe has hardware support for fixed point instructions (I don't know the details). IIRC, the POWER line (p and i) also have hardware support.
      • Retric 7y ago
        It’s far more than just having a single data type. COBOL was in effect designed for business use so fixed point arithmetic has a lot of support in the language, libraries, and tooling.
        • 7y ago
• jedbrown 7y ago
  Nope, computational science and engineering tends to use commodity server hardware (Xeon or similar) with or without GPUs. There may be many nodes or even many racks, but it's quite different from Z-series, which are a very poor pricepoint.
• tgtweak 7y ago
  Common in large transactional systems - stock exchanges & banks, airlines, shipping/logistics and academic/research - requiring tens of terabytes of data in-memory accessible from all cores without any networking latency or bottlenecks. There are a lot of systems that are simpler (cheaper and less risky) to scale up than re-engineer to be distributed.
  • zepearl 7y ago
    Yeah, and in my experience especially for anything that is extremely time-critical + has the potential to generate huge in/direct losses. For example at the end of each day a bank has to generate the final balances of the accounts that it has with other banks and then it has to check for irregularities and then finally set those balances just right (e.g. enough to cover payments made by its customers but without leaving there millions) => if it doesn't then it might not be able to use the money that is "parked" there by mistake, or it might have a huge position with a risky bank, or if the balance is too low other banks might decide not to perform the final credit into the target accounts involved in the customers' payments (which would then generate a lot of complaints + lower the bank's reputation with its long-term consequences), etc... . (the opposite happens as well - https://www.investopedia.com/ask/answers/051815/what-difference-between-nostro-and-vostro-account.asp https://www.investopedia.com/ask/answers/051815/what-differe... ) Nowadays even accounting (coupled with risk management) has become time-critical because of the many regulations - a 1-day delay filing the numbers with regulators and/or central banks can result in huge fines and loss of reputation (if the news about the problem becomes public it could happen that the news services try to highlight it increasing the focus on that negative publicity). In our case the detailed/low-level data (e.g. client X bought N shares of some company, client Y retrieved N$ from the ATM, etc) is all processed by the mainframe, which works fine 99.99999% of the time => then anything that comes afterwards which involves analysis/aggregation/reporting/etc of that data happens in other non-mainframe apps (they're all very different - from tiny to huge apps with distributed databases) and in general any such app has usually some kind of problem that results in a delay at least once every quarter due to the SW itself or even the HW => it's usually not a big problem (usually some hours are lost but there is a buffer) but if the central SW (running on the mainframe) would have the same problem then there would be a chain-reaction on all the other apps that need that data (and then their specific problems would be on top of that, and then there would be a hotspot of needed 100% CPU/RAM/network resources as they would all run at the same time as fast as possible which could cause further delays, and so on).
• Merrill 7y ago
  For processing large amounts of structured data using decimal arithmetic to do exact monetary calculations.
• jl6 7y ago
  I work for a bank. We use mainframes, but not for building new systems - just for keeping the legacy core running. “Just”. Eventually that core will move to other platforms as product cycles come and go, but there’s no active migrate-off-mainframe project. It will probably be 20 years before we’ve fully replaced them.
• svennek 7y ago
  High-IO jobs, where a single system-image (read "os") is important. Most of banking and alike problems. The main three benefits of the mainframes (as a non-mainframer): - crazy amounts of caches - crazy amounts of pcie-slots (and sufficient internal io and processing power to make it balanced) - production environment mentality for everyone involved, incl. extremely engineered (and redundant) hardware I think new customers are likely to run Linux on such a box in the future... The main cost-problem for mainframe customers is software cost (esp. z/OS, cobol etc..). I have heard stories (from actual mainframers) about the sql-performance and key-value performance (read like mongodb) on these boxes that are eye-watering..
  • lanstin 7y ago
    Yeah people don’t realize the IO bandwidth (off the central CPUs mostly) is the differentiator. And writing SQL for DB2 is easier than writing efficient spark jobs (with minimal shuffles across those cheap small machines).
    • rbanffy 7y ago
      People often forget how much computer science and engineering is in those machines that allow them to fully saturate two and a half full height racks of high performance IO while running COBOL programs that are blissfully unaware of all that goes behind their ABI and just see a ridiculously fast computer.
    • bogomipz 7y ago
      How is that level of IO bandwidth achieved exactly? Or how is it different from the x86 architecture?
      • altatyper 7y ago
        There is a ton of technical info out there, but in a nutshell, the hardware is extremely optimized for I/O bandwidth, cache's are able to be accessed by more cores at a time (I think the L3 cache is 32-way accessible for example). The goal is to have nearly everything in at least RAM (up to 40TB of it) or better yet in a cache closer to a CPU so there is no latency. And when it comes to reaching things outside of the mainframe itself, generally anything you can do to make accessing the data wider is better... Why use a single 16GB/sec HBA when you can interleave 8 or more FCAL paths to the same disk. Why use FCAL when you can use Infniband, etc etc.. They have a LOT of PCIe lanes available, and enough chips to keep most of them busy most of the time. TL;DR if it's expensive and offers more paths between things, it's probably an option for a mainframe.
• pjmlp 7y ago
  I had access to AS/400, nowadays known as IBM i, back in the early 90's. Had to perform regular backups on the system as part of my internship. Nowadays kids are all up with WebAssembly and WASI, well that is just how OS/400 works during the last 30+ years. Originally designed in a mix of PL/S and Assembly, everything else (RPG, Cobol, PL/I, C, C++) compiled into ILE (Intermediate Language Environment) and cross language calls are relatively easy to do. ILE applications are AOT compiled either at installation time, or anytime some critical hardware has changed or the application themselves have been updated. Nowadays there is also Metal C (real native not ILE), Java via IBM's own JVM (which in early versions converted JVM bytecodes into ILE ones), and the C and C++ compilers are also to target actual native code besides ILE. The database backed filesystem took some time getting used to, coming from Amiga/MS-DOS/Windows 3.x/Xenix experience, and the command line felt more cryptic those OSes, given the use of special characters as part of the name. The company I did my internship was using them for their accounting, everything else were MS-DOS/Windows computers connected via Novel Netware, zero UNIX flavour in sight. IBM z and Unisys ClearPath are two other mainframe models that also follow similar bytecode based deployment formats. So in a sense you can say that on every Android phone, watchOS or Windows PC lives a little mainframe.
  • monocasa 7y ago
    AS/400 isn't a mainframe, it's IBM's minicomputer arch.
    • pjmlp 7y ago
      Naming details,which I always get wrong, because nowadays PC servers have taken that role.
      • monocasa 7y ago
        It's just weird to bring up AS/400 in a conversation about mainframes.
        • rbanffy 7y ago
• wolf550e 7y ago
  A mainframe is not a supercomputer. Science is done on clusters of x86 chips and Nvidia gpus with custom interconnect and Linux. A system that runs a bank or an insurance company and wants good availability can be built in one of two ways: you either spend money on software that deals with the hardware being unreliable and save money on software (pioneered by Google) or you spend money on hardware that promises to be highly reliable and save on software. No new player believes mainframe (extremely expensive hardware) is cost effective, they all use commodity hardware. A bank that needs to run binaries from the 70s for which they don't have the source code can keep paying IBM and not investing in reverse engineering the binary and implementing it in Java. A bank that has a billion lines of code in Cobol can compile it to run on jvm on commodity hardware and run it in parallel with the mainframe for a year to validate and then switch over to the new system and stop overpaying for hardware, but that sounds risky, so they keep paying a million dollars for a system with the same performance as a 50 thousand dollar server.
  • pmart123 7y ago
    My guess is a mainframe would be much better at handling a high volume of transactions much better than a $50k server? I would also think it would better handle conditions where "eventually consistent" doesn't work? I'm sure most companies are running batch processes and data-intensive calculations that could be easily shifted to cloud workloads in practice, but it does seem like there are some use cases where a mainframe could be better?
    • marcosdumay 7y ago
      Yes if you have a huge volume of transactions and a huge amount of constraints that can not be partitioned. But the real world constraints can almost always be partitioned. For read-only batch computations you can always add some extra redundancy and partition the problem. So, I don't think it is likely that a mainframe would be useful here.
    • fhars 7y ago
      And it is especially quite a bit better in handling a high volume of transactions while physically replacing one of the CPUs and half of the RAM in the system than many $50k servers.
• marcosdumay 7y ago
  It must require massive parallelism and a huge amount of IPC, otherwise clusters will beat your mainframe with a much smaller budget. I don't know many of those problem (in fact, I can't think of any). Programmers are usually very eager in optimizing IPC out of any parallel algorithm. I have seen a handful of mainframes on use, but I have never seen one used for something that it is good for.
• crb002 7y ago
  In the CAP theorem, mainframe is the king of CA. You use them mostly for DB2 databases and Batch processing on that data to remain highly consistent and available.
  • bob1029 7y ago
    This is exactly the secret behind mainframe's ubiquitous persistence across mission-critical businesses. Consistency and availability are far more important than partition tolerance. You build your mainframe with >N+2 redundancy on any given component and then you build the datacenter around the mainframe. This is how you beat the CAP theorem. With brute-force engineering. Sure, you aren't truly defeating it in principle, but in practice it still seems to be the best option we've got. Batch is the other aspect that takes mainframes to unbeatable status. When you run a batch job, you basically say "ok all daily OLTP is halted, we are going into a completely new architectural mode". Obtaining an exclusive lock on a table or an entire database for a single series of processes to operate on can yield insane amounts of throughput when you are finalizing the data from the day's operations. A few years ago as a more junior developer, I will admit to being strongly "anti-mainframe" based on the principles I held at that time. Why can't we just put it all in the cloud, throw some MongoDB out there and pray it all works? After witnessing actual business cases for mainframe/batch unfold, I quickly started to change my tune. Mainframe is not for every business, but it does seem to be a tool you can reach for when someone says "everything just has to work always or someone dies", and "we have infinite money".
  • bogomipz 7y ago
    Is this built into the hardware then? Might you or someone else elaborate on how mainframe architecture provides for this level of CA?
• bogomipz 7y ago
  I had the same question recently and although high availability is certainly one reason an equally and perhaps more important reason is "cost per transaction." Think of the number of transactions entities such banks and airlines perform a day. This article talks about this in the section "what is a mainframe today": https://www.nanalyze.com/2017/10/ibm-mainframe-computers-today/ https://www.nanalyze.com/2017/10/ibm-mainframe-computers-tod...
• karmakaze 7y ago
  Vertical scaling. It's common with startups to move to a sharded cloud configuration. In cases like a transaction that can cross any imaginable shard boundary, scaling vertically makes more sense. Mainframes allow much greater vertical scaling as mentioned in other comments regarding I/O, caching, etc. Also being on-premise, fewer and more reliable machines require fewer people to maintain it.
• z3t4 7y ago
  There used to exist many type of high end servers like solaris and mainframes. But consumer grade hardware became so reliable that it became hard to justify the price of the high end serves. Some features was hot pluggable components like hard drives which has now also got into consumer grade desktop computers. Today when you have cloud database providers it makes even less sense to run your own high end server.
• coolkil 7y ago
  You use it in an environment where you need an extreme level of data integrity and availability. Think banks, airports, government, etc. Due to io offloading to co-processors and a whole range of supporting cpu types you can set up a system which can handle thousands of transaction per second. Most of the application that are used on mainframe's are databases or message que's Recently there is an increased interest in running Linux on mainframe's something which can be done since early 2000 you get the benefits of high available and secure hardware and the relative ease of Management of Linux. Another benefit is that you don't really need to train personnel in more exotic operating systems like z/OS.