4 ms·

Realistically, if, say, one of these big corps/banks with deeply-integrated COBOL systems spent a few years revamping their systems to something more modern and

Realistically, if, say, one of these big corps/banks with deeply-integrated COBOL systems spent a few years revamping their systems to something more modern and usable today, how much would it cost? Running mainframes isn't cheap. COBOL programmers aren't cheap. Not being able to improve these systems / innovate with them can cost the business a lot of time/money. And if we're talking COBOL, we're talking at least 25 year old corps. How long's the ROI on moving over? How expensive does the maintenance of antiquated systems get? It's easy to say that they're rock-solid and never need to be touched, but is that really true? For example, lot of banks today have a lot of absurd UX, often because of absurd internals and I know a lot of that usually is "the system is ancient and nobody really touches it anymore, that's just what it is now".

protomyth 10y ago
> Realistically, if, say, one of these big corps/banks with deeply-integrated COBOL systems spent a few years revamping their systems to something more modern and usable today, how much would it cost? What language would you suggest? Remember that decimal reliability and predictability is important. Plus, let's pick a language likely to have a very long service life because we need to have our investment rewarded. Frankly, the iSeries machine we have is pretty cheap to run. We bought it 10 years ago for about $15k and have had a maintenance contract that doesn't cost that much (about the same as our Red Hat service contract). We probably will buy another one since they just released a sub $10k one that will more than meet our needs. Never lost any data, easy enough backup that the biz office does it themselves[1], and IBM is pretty quick about any fixes. Much cheaper than any Linux, BSD, or Windows system we have ever had. Plus, I don't mess with it like the others. It just works. 1) we've tested it multiple times, after all, an unrestored backup is not a true backup
- scrollaway 10y ago
  I don't know enough about these systems to suggest a language, but I imagine within the "a few years of development" is budgeted some time to figure out what tool to use. Surely you're not telling me that mainframes running COBOL is the best tool for the job. What languages do younger banks use?
  - protomyth 10y ago
    "I don't know enough about these systems to suggest a language", so how can you say "Surely you're not telling me that mainframes running COBOL is the best tool for the job."?
    - scrollaway 10y ago
      Are you telling me I'm not allowed to ask questions?
      - protomyth 10y ago
        You weren't asking a question, you were making a statement about how foolish it was that banks are running COBOL. Question would be about the why and what value do people see in the situation that stays their hands at going to a new technology. Why did banks spend the money and time (which was not insignificant) to modify code for year 2000 instead of making the switch then at a natural change point? It would be the most logical point, yet they didn't do it. What does the mainframe and even the mini-mainframes bring that Docker does not? What made COBOL so hard to replace given later things like PL/1 which were directly supposed to replace it? Where do all the RPG programs fit into this? Maybe someone should be thinking about a COBOL replacement language instead of the plethora of C / C++ replacement languages. I imagine within the "a few years of development" is budgeted some time to figure out what tool to use It will be a lot more than a few. Things absolutely cannot go wrong or even change in production. One slip-up can result in real money being lost. This whole "go fast and break things" doesn't apply to bank accounts.
        scrollaway 10y ago
        > You weren't asking a question, you were making a statement about how foolish it was that banks are running COBOL. I've only asked questions and in return, you're being rude to me. I've made no statement about "foolishness"; in fact, I've made no statement of fact at all, outside of what I already know: A COBOL stack isn't cheap. I'd like you to get off your high horse now.
dragonwriter 10y ago
> Running mainframes isn't cheap. Price for performance for certain workloads, as I understand, it can be. > Not being able to improve these systems / innovate with them can cost the business a lot of time/money. The banks use other technology for peripheral systems that interact with the core functions all the time (look at the drivers behind AMQP, sometime.) The core functions, OTOH, are about impelmenting requirements that haven't changed in centuries; until there is a desire to implement a fundamental new way of doing things (like blockchain, which financial institutions are exploring and probably aren't using COBOL on mainframes for their experiments) theres nothing to innovate on.
shepardrtc 10y ago
These systems can be at the very core of the company's operations, so you're talking not just a revamp of a bunch of COBOL programs and mainframe servers, but a significant portion of the company's workflow would potentially need to be refactored or replaced. The mainframes themselves sit in a box, but many other systems rely on their data and their guaranteed processing. And all of those connections and data structures are custom. Even if you were to figure out some way to completely emulate the system (and that itself would probably take several years), you would then need to integrate your emulated system into other real-time systems. A bank just can't shut down for a day. And if they screw up a few minutes worth of transactions, then things get very bad. So your testing and integration will be slow and methodical and expensive. And even then, you're still emulating a crappy, old system. If you wanted to actually replace the entire system, how would you do it? These things aren't modular. They were custom-built with no intention of ever being parted-out. You would have to completely re-architect the company's infrastructure and workflow. Or you could not do any of that. You could just accept your situation. Mainframes are essentially a subscription service. For a yearly cost, IBM has offered you guaranteed performance and guaranteed hardware (they fix it immediately if there is a problem). So, all you have to do is pay your subscription and pay some people who like to have a steady, unexciting job and your business will continue to run like clockwork.
- scrollaway 10y ago
  I understand what you're saying, but does this logic hold over decades? I'm obviously not saying banks should just switch today cause it's easy and all, but if the ROI on switching is 10, even 20 years, it still makes sense to do if you're a bank.
  - semi-extrinsic 10y ago
    It's not 10 years, it's probably closer to 100. Reasons being a) the ridiculously large cost of rewriting and testing the new system on a real-time system that can't be taken down, and b) the ridiculously low cost of maintaining the current systems.
    - 10y ago
marktangotango 10y ago
> Realistically, if, say, one of these big corps/banks with deeply-integrated COBOL systems spent a few years revamping their systems to something more modern and usable today, how much would it cost? Success isn't guaranteed, maybe as low as 50%. When you say 'few years' it intimates you don't appreciate the scale of some these systems. In the end, retiring legacy systems isn't a technical problem, it's a product owner, product management, and business problem. The business owners have to make concerted, multi year effort to document their process as implemented, define the replacement, and manage it's (re)implementation. This never happens.
- scrollaway 10y ago
  When I say "few years" I mean 10, 15 years? 20? 30? In the context of a bank, which may be around hundreds of years, that's still not a lot. But maybe I'm still way off, hence my question: How long would it take? I see a lot of people here telling me I don't know what I'm talking about without actually giving an answer as to how long it would take and how much it would cost, which was my question in the first place and I'm being downvoted for asking it. Seriously, WTF HN?
  - marktangotango 10y ago
    Well, those questions depend on the system in question, and for any particular system, it's probably impossible to say what the cost of replacing would be, because no one understands everything the system does, and how it interacts with external systems. This is what others are alluding too, your question is unanswerable. What you can do is estimate the cost of implementation in today dollars. For example suppose Bank System X has been under development for 40 year with an averge of 10 developers per year. That's 400 person years. Suppose you find cheap labor at $100k per year, the sunk cost in personnel alone is $40M in todays dollars. So 40 people maybe could do it 10 years. The above analysis is not satisfying.
- Chaebixi 10y ago
  > The business owners have to make concerted, multi year effort to document their process as implemented, define the replacement, and manage it's (re)implementation. This never happens. I wouldn't say it never happens, but it certainly takes a couple tries and a lot of time and money. My employer is close to being off our mainframe(s), but it will probably take at least another 10 years to get rid of the last little dependencies. We've already spent about 15 years (with hundreds of developers) getting to the point where all the major online and batch processes have been reengineered and have very small mainframe dependencies. The mainframe development teams are very small and mostly focused on decommissioning work. But yeah, it's a massive undertaking. In our case, the business was motivated to pay for it because mainframes just couldn't scale for our main use cases.
skissane 10y ago
Most banks looking to move off mainframe/COBOL/CICS/etc don't look to rewrite everything themselves. They look to buy packaged software which they will then customise/enhance. I think most of those packaged software offerings are written in Java. (Disclaimer: My employer sells packaged banking software written in Java.) In fact, if you look at the existing COBOL banking applications, many of them aren't pure inhouse development, but packaged applications like CSC Hogan. Hogan runs on top of z/OS and CICS and is written in COBOL. Hogan shops will have a lot of their own COBOL they add to the Hogan system to implement customer-specific requirements. So for many banks moving off mainframe/COBOL, it is actually a migration from one packaged app to another, but of course they need to rewrite from scratch all their customisations. In my personal experience, most banks doing this don't try to do it as a big bang. They identify subsets of functionality where they feel the current mainframe system is most limited – e.g. mortgages, mobile banking, whatever – and start by replacing that function only with the new non-mainframe software, and have it integrate with the pre-existing mainframe app for other functions. Then, the idea is, progressively they will migrate more and more functionality to the new system, and eventually the mainframe/COBOL systems can be retired. A piecemeal approach often involves less risk, costs less (in the short term) and is easier to justify to the business since they can see tangible results much easier.
nikdaheratik 10y ago
It cost one large bank in Australia $1 Billion over 5 years to convert their system. That's 2-3% of their annual profit over 5 years to fix the problem, so we can assume the costs of running the current system are probably some small multiple of that. And they would still need to pay to run the new system, and most likely both systems at the same time towards the end of the transition.

protomyth 10y ago

> Realistically, if, say, one of these big corps/banks with deeply-integrated COBOL systems spent a few years revamping their systems to something more modern and usable today, how much would it cost? What language would you suggest? Remember that decimal reliability and predictability is important. Plus, let's pick a language likely to have a very long service life because we need to have our investment rewarded. Frankly, the iSeries machine we have is pretty cheap to run. We bought it 10 years ago for about $15k and have had a maintenance contract that doesn't cost that much (about the same as our Red Hat service contract). We probably will buy another one since they just released a sub $10k one that will more than meet our needs. Never lost any data, easy enough backup that the biz office does it themselves[1], and IBM is pretty quick about any fixes. Much cheaper than any Linux, BSD, or Windows system we have ever had. Plus, I don't mess with it like the others. It just works. 1) we've tested it multiple times, after all, an unrestored backup is not a true backup

scrollaway 10y ago
I don't know enough about these systems to suggest a language, but I imagine within the "a few years of development" is budgeted some time to figure out what tool to use. Surely you're not telling me that mainframes running COBOL is the best tool for the job. What languages do younger banks use?
- protomyth 10y ago
  "I don't know enough about these systems to suggest a language", so how can you say "Surely you're not telling me that mainframes running COBOL is the best tool for the job."?
  - scrollaway 10y ago
    Are you telling me I'm not allowed to ask questions?
    - protomyth 10y ago
      You weren't asking a question, you were making a statement about how foolish it was that banks are running COBOL. Question would be about the why and what value do people see in the situation that stays their hands at going to a new technology. Why did banks spend the money and time (which was not insignificant) to modify code for year 2000 instead of making the switch then at a natural change point? It would be the most logical point, yet they didn't do it. What does the mainframe and even the mini-mainframes bring that Docker does not? What made COBOL so hard to replace given later things like PL/1 which were directly supposed to replace it? Where do all the RPG programs fit into this? Maybe someone should be thinking about a COBOL replacement language instead of the plethora of C / C++ replacement languages. I imagine within the "a few years of development" is budgeted some time to figure out what tool to use It will be a lot more than a few. Things absolutely cannot go wrong or even change in production. One slip-up can result in real money being lost. This whole "go fast and break things" doesn't apply to bank accounts.
      - scrollaway 10y ago
        > You weren't asking a question, you were making a statement about how foolish it was that banks are running COBOL. I've only asked questions and in return, you're being rude to me. I've made no statement about "foolishness"; in fact, I've made no statement of fact at all, outside of what I already know: A COBOL stack isn't cheap. I'd like you to get off your high horse now.

dragonwriter 10y ago

> Running mainframes isn't cheap. Price for performance for certain workloads, as I understand, it can be. > Not being able to improve these systems / innovate with them can cost the business a lot of time/money. The banks use other technology for peripheral systems that interact with the core functions all the time (look at the drivers behind AMQP, sometime.) The core functions, OTOH, are about impelmenting requirements that haven't changed in centuries; until there is a desire to implement a fundamental new way of doing things (like blockchain, which financial institutions are exploring and probably aren't using COBOL on mainframes for their experiments) theres nothing to innovate on.

shepardrtc 10y ago

These systems can be at the very core of the company's operations, so you're talking not just a revamp of a bunch of COBOL programs and mainframe servers, but a significant portion of the company's workflow would potentially need to be refactored or replaced. The mainframes themselves sit in a box, but many other systems rely on their data and their guaranteed processing. And all of those connections and data structures are custom. Even if you were to figure out some way to completely emulate the system (and that itself would probably take several years), you would then need to integrate your emulated system into other real-time systems. A bank just can't shut down for a day. And if they screw up a few minutes worth of transactions, then things get very bad. So your testing and integration will be slow and methodical and expensive. And even then, you're still emulating a crappy, old system. If you wanted to actually replace the entire system, how would you do it? These things aren't modular. They were custom-built with no intention of ever being parted-out. You would have to completely re-architect the company's infrastructure and workflow. Or you could not do any of that. You could just accept your situation. Mainframes are essentially a subscription service. For a yearly cost, IBM has offered you guaranteed performance and guaranteed hardware (they fix it immediately if there is a problem). So, all you have to do is pay your subscription and pay some people who like to have a steady, unexciting job and your business will continue to run like clockwork.

scrollaway 10y ago
I understand what you're saying, but does this logic hold over decades? I'm obviously not saying banks should just switch today cause it's easy and all, but if the ROI on switching is 10, even 20 years, it still makes sense to do if you're a bank.
- semi-extrinsic 10y ago
  It's not 10 years, it's probably closer to 100. Reasons being a) the ridiculously large cost of rewriting and testing the new system on a real-time system that can't be taken down, and b) the ridiculously low cost of maintaining the current systems.
  - 10y ago

marktangotango 10y ago

> Realistically, if, say, one of these big corps/banks with deeply-integrated COBOL systems spent a few years revamping their systems to something more modern and usable today, how much would it cost? Success isn't guaranteed, maybe as low as 50%. When you say 'few years' it intimates you don't appreciate the scale of some these systems. In the end, retiring legacy systems isn't a technical problem, it's a product owner, product management, and business problem. The business owners have to make concerted, multi year effort to document their process as implemented, define the replacement, and manage it's (re)implementation. This never happens.

scrollaway 10y ago
When I say "few years" I mean 10, 15 years? 20? 30? In the context of a bank, which may be around hundreds of years, that's still not a lot. But maybe I'm still way off, hence my question: How long would it take? I see a lot of people here telling me I don't know what I'm talking about without actually giving an answer as to how long it would take and how much it would cost, which was my question in the first place and I'm being downvoted for asking it. Seriously, WTF HN?
- marktangotango 10y ago
  Well, those questions depend on the system in question, and for any particular system, it's probably impossible to say what the cost of replacing would be, because no one understands everything the system does, and how it interacts with external systems. This is what others are alluding too, your question is unanswerable. What you can do is estimate the cost of implementation in today dollars. For example suppose Bank System X has been under development for 40 year with an averge of 10 developers per year. That's 400 person years. Suppose you find cheap labor at $100k per year, the sunk cost in personnel alone is $40M in todays dollars. So 40 people maybe could do it 10 years. The above analysis is not satisfying.
Chaebixi 10y ago
> The business owners have to make concerted, multi year effort to document their process as implemented, define the replacement, and manage it's (re)implementation. This never happens. I wouldn't say it never happens, but it certainly takes a couple tries and a lot of time and money. My employer is close to being off our mainframe(s), but it will probably take at least another 10 years to get rid of the last little dependencies. We've already spent about 15 years (with hundreds of developers) getting to the point where all the major online and batch processes have been reengineered and have very small mainframe dependencies. The mainframe development teams are very small and mostly focused on decommissioning work. But yeah, it's a massive undertaking. In our case, the business was motivated to pay for it because mainframes just couldn't scale for our main use cases.

skissane 10y ago

Most banks looking to move off mainframe/COBOL/CICS/etc don't look to rewrite everything themselves. They look to buy packaged software which they will then customise/enhance. I think most of those packaged software offerings are written in Java. (Disclaimer: My employer sells packaged banking software written in Java.) In fact, if you look at the existing COBOL banking applications, many of them aren't pure inhouse development, but packaged applications like CSC Hogan. Hogan runs on top of z/OS and CICS and is written in COBOL. Hogan shops will have a lot of their own COBOL they add to the Hogan system to implement customer-specific requirements. So for many banks moving off mainframe/COBOL, it is actually a migration from one packaged app to another, but of course they need to rewrite from scratch all their customisations. In my personal experience, most banks doing this don't try to do it as a big bang. They identify subsets of functionality where they feel the current mainframe system is most limited – e.g. mortgages, mobile banking, whatever – and start by replacing that function only with the new non-mainframe software, and have it integrate with the pre-existing mainframe app for other functions. Then, the idea is, progressively they will migrate more and more functionality to the new system, and eventually the mainframe/COBOL systems can be retired. A piecemeal approach often involves less risk, costs less (in the short term) and is easier to justify to the business since they can see tangible results much easier.

nikdaheratik 10y ago

It cost one large bank in Australia $1 Billion over 5 years to convert their system. That's 2-3% of their annual profit over 5 years to fix the problem, so we can assume the costs of running the current system are probably some small multiple of that. And they would still need to pay to run the new system, and most likely both systems at the same time towards the end of the transition.