6 ms·

Any company that knows better has done a modernization to get to get off it asap. The legacy nature of the products means that the few outlets that still suppor

by adra 4y ago

Any company that knows better has done a modernization to get to get off it asap. The legacy nature of the products means that the few outlets that still support them or their mainframes are charging exorbitant rates. How much? I had a customer (non-gov) that paid a million$ a month to keep their product leased and up on their support agreement, that $12m a year without even talking about the costs of finding Cobol devs. Whos still using them? Any bureaucracy terrible enough to have not transitioned years (decades) ago.

analog31 4y ago
I wonder if second system effect looms large in the minds of those who are maintaining those COBOL programs. At least with a legacy program, you probably know that it can keep running for another year for a predictable cost. On the other hand, spinning up a scratch build of its replacement is potentially a black hole. I've witnessed second system effect.
skissane 4y ago
Many places where COBOL survives, it survives because of at least one (sometimes more than one) mega-failure in which management spends tens of millions on replacing it with a new system written in Java/.NET/whatever, only for that replacement to turn into such a complete disaster that it ends up completely abandoned and the whole development cost written-off
- Kamq 4y ago
  This tends to be the case when you need a bug-for-bug re-implementation of the old program, but it isn't stated up front.
  - skissane 4y ago
    Sometimes implementations fail for essentially the opposite reason – the new system is too generic and insufficiently close to the original program. One place I worked (a university), they tried to replace their mainframe COBOL system (Fujitsu FACOM GS8400 mainframe [0] [1], along with Software AG's ADABAS database and some use of their Natural 4GL as well) with PeopleSoft Campus. However, PeopleSoft Campus was designed for the US market, and many of its features were rather irrelevant in Australia; conversely, key features needed to cope with Australia's rather different regulatory environment were lacking, forcing the project to reimplement large parts of the original COBOL system as add-on code. PeopleSoft sales (this was in the mid-to-late 1990s, so before Oracle bought them) had painted (whether by intention or ignorance) a fundamentally misleading picture of how suitable the product was for the Australian market. Eventually the whole project was written off as an expensive multi-million dollar failure, and the mainframe COBOL system got a reprieve on its execution. I was still in high school when it happened, but when I worked there several years later, the "PeopleSoft disaster" was a frequently repeated part of local institutional lore. Then, they tried again with a competing system from a local Australian ERP firm (TechnologyOne), running on a Microsoft application stack and Oracle RDBMS; and this time they succeeded in replacing the mainframe COBOL system, in part because (unlike PeopleSoft) it was actually designed for the Australian market. [0] https://www.fujitsu.com/au/about/corporate/history/products/computer/mainframe/gs8600.html https://www.fujitsu.com/au/about/corporate/history/products/... [1] http://museum.ipsj.or.jp/en/computer/main/0088.html http://museum.ipsj.or.jp/en/computer/main/0088.html
    - ngcc_hk 4y ago
      Adabas and natural … those were the days my friends. Still have nightmare when it failed and have to fly in expert who actually does not know. …
    - Kamq 4y ago
      > the new system is too generic and insufficiently close to the original program I may have been unclear, but this was the scenario I was describing. The need for a bug-for-bug re-implementation is a special case where the acceptable deviation from the original program is quite low.
      - skissane 4y ago
        They didn’t need a bug for bug reimplementation. Their problem was they tried to replace it with COTS software which wasn’t properly localised for their market. That’s more commonly a problem for smaller markets such as Australia, and also a bigger problem in highly regulated sectors such as education and health, where you are forced to deal with all sorts of government regulations which can be rather specific to the sector and jurisdiction you are operating in. The other problem, which is a more common problem with COTS software, is it is easy to be given one impression during the sales process, but then only once you’ve bought it and have started spending money to implement it that you realise how wrong that impression was. I think the people buying it on the customer side were senior university executives (who almost always come from an academic background) and they didn’t have a good enough appreciation for the nuts and bolts of student administration systems-and senior executives being sold on something where they don’t understand the details enough to see why it won’t work is yet another common cause for these kinds of projects to fail.
hajile 4y ago
I know of an insurance company that has tried 3-4 times at a cost probably in the hundreds of millions to switch off COBOL. The simple fact is that they've spent tons of time slowly coding in hundreds of thousands of little, undocumented edge cases all over the place. The rewrite gets 90% there then falls flat with all the little things and you can't switch these kinds of businesses until it is 100% there.
- charcircuit 4y ago
  >The rewrite gets 90% there then falls flat with all the little things and you can't switch these kinds of businesses until it is 100% there. Is it not possible to find a safe subset of cases that can be handled by a new codebase. You use shadow testing to help prove the rewrite handles the cases the same as the old implementation and then grow that subset until it encompasses the whole thing. As the old implementation is used less and less you can run it on a smaller computer and have to maintain it less.
  - hajile 4y ago
    It takes years or decades to do the swap. You’ll have both teams working in parallel all that time which costs the company a TON of money. This process also outlasts aging managers. Chances are that leadership will shift at least once and decide it’s taking too long So they kill the replacement, switch languages and/or teams, and try yet again.
- derefr 4y ago
  It sounds like, rather than a ground-up rewrite, COBOL should be treated as an object-code language, and "hand-decompiled" (ala efforts like https://github.com/n64decomp/sm64 https://github.com/n64decomp/sm64) into an HLL that can, at every point, be losslessly transpiled back into the original COBOL. I know the tooling for doing that doesn't currently exist... but paying someone to develop it would be cheaper than any one of these ground-up rewrite projects!
  - Nextgrid 4y ago

by adra 4y ago

analog31 4y ago

I wonder if second system effect looms large in the minds of those who are maintaining those COBOL programs. At least with a legacy program, you probably know that it can keep running for another year for a predictable cost. On the other hand, spinning up a scratch build of its replacement is potentially a black hole. I've witnessed second system effect.

skissane 4y ago

Many places where COBOL survives, it survives because of at least one (sometimes more than one) mega-failure in which management spends tens of millions on replacing it with a new system written in Java/.NET/whatever, only for that replacement to turn into such a complete disaster that it ends up completely abandoned and the whole development cost written-off

Kamq 4y ago
This tends to be the case when you need a bug-for-bug re-implementation of the old program, but it isn't stated up front.
- skissane 4y ago
  Sometimes implementations fail for essentially the opposite reason – the new system is too generic and insufficiently close to the original program. One place I worked (a university), they tried to replace their mainframe COBOL system (Fujitsu FACOM GS8400 mainframe [0] [1], along with Software AG's ADABAS database and some use of their Natural 4GL as well) with PeopleSoft Campus. However, PeopleSoft Campus was designed for the US market, and many of its features were rather irrelevant in Australia; conversely, key features needed to cope with Australia's rather different regulatory environment were lacking, forcing the project to reimplement large parts of the original COBOL system as add-on code. PeopleSoft sales (this was in the mid-to-late 1990s, so before Oracle bought them) had painted (whether by intention or ignorance) a fundamentally misleading picture of how suitable the product was for the Australian market. Eventually the whole project was written off as an expensive multi-million dollar failure, and the mainframe COBOL system got a reprieve on its execution. I was still in high school when it happened, but when I worked there several years later, the "PeopleSoft disaster" was a frequently repeated part of local institutional lore. Then, they tried again with a competing system from a local Australian ERP firm (TechnologyOne), running on a Microsoft application stack and Oracle RDBMS; and this time they succeeded in replacing the mainframe COBOL system, in part because (unlike PeopleSoft) it was actually designed for the Australian market. [0] https://www.fujitsu.com/au/about/corporate/history/products/computer/mainframe/gs8600.html https://www.fujitsu.com/au/about/corporate/history/products/... [1] http://museum.ipsj.or.jp/en/computer/main/0088.html http://museum.ipsj.or.jp/en/computer/main/0088.html
  - ngcc_hk 4y ago
    Adabas and natural … those were the days my friends. Still have nightmare when it failed and have to fly in expert who actually does not know. …
  - Kamq 4y ago
    > the new system is too generic and insufficiently close to the original program I may have been unclear, but this was the scenario I was describing. The need for a bug-for-bug re-implementation is a special case where the acceptable deviation from the original program is quite low.
    - skissane 4y ago
      They didn’t need a bug for bug reimplementation. Their problem was they tried to replace it with COTS software which wasn’t properly localised for their market. That’s more commonly a problem for smaller markets such as Australia, and also a bigger problem in highly regulated sectors such as education and health, where you are forced to deal with all sorts of government regulations which can be rather specific to the sector and jurisdiction you are operating in. The other problem, which is a more common problem with COTS software, is it is easy to be given one impression during the sales process, but then only once you’ve bought it and have started spending money to implement it that you realise how wrong that impression was. I think the people buying it on the customer side were senior university executives (who almost always come from an academic background) and they didn’t have a good enough appreciation for the nuts and bolts of student administration systems-and senior executives being sold on something where they don’t understand the details enough to see why it won’t work is yet another common cause for these kinds of projects to fail.

hajile 4y ago

I know of an insurance company that has tried 3-4 times at a cost probably in the hundreds of millions to switch off COBOL. The simple fact is that they've spent tons of time slowly coding in hundreds of thousands of little, undocumented edge cases all over the place. The rewrite gets 90% there then falls flat with all the little things and you can't switch these kinds of businesses until it is 100% there.

charcircuit 4y ago
>The rewrite gets 90% there then falls flat with all the little things and you can't switch these kinds of businesses until it is 100% there. Is it not possible to find a safe subset of cases that can be handled by a new codebase. You use shadow testing to help prove the rewrite handles the cases the same as the old implementation and then grow that subset until it encompasses the whole thing. As the old implementation is used less and less you can run it on a smaller computer and have to maintain it less.
- hajile 4y ago
  It takes years or decades to do the swap. You’ll have both teams working in parallel all that time which costs the company a TON of money. This process also outlasts aging managers. Chances are that leadership will shift at least once and decide it’s taking too long So they kill the replacement, switch languages and/or teams, and try yet again.
derefr 4y ago
It sounds like, rather than a ground-up rewrite, COBOL should be treated as an object-code language, and "hand-decompiled" (ala efforts like https://github.com/n64decomp/sm64 https://github.com/n64decomp/sm64) into an HLL that can, at every point, be losslessly transpiled back into the original COBOL. I know the tooling for doing that doesn't currently exist... but paying someone to develop it would be cheaper than any one of these ground-up rewrite projects!
- Nextgrid 4y ago