6 ms·

These corporations may not see it, because they do not understand it, but having such legacy systems a risk to their operations. The longer they wait to migrat

by mey 16y ago

These corporations may not see it, because they do not understand it, but having such legacy systems a risk to their operations. The longer they wait to migrate, the more expensive it will be, and the harder it will be to attract talent to the project. There may come a point where they can no longer find anyone to support their hardware or software, and it'll force them to stagnate against their competition who may be able to provide better or different offerings because they are not tied to a system they can't modify. My understanding of business is that you should never stagnate, and always keep advancing. People that I've talked to who have worked with such systems indicated that they built giant shims around the systems in other languages to allow them to be adaptable. In theory this gives them a clear interface definition (it may be huge, thousands of method signatures, and possible direct schema access, but it can still be a clear interface) to boot strap a core system replacement in another language and hardware system.

jacquesm 16y ago
The laws of supply and demand will make sure that there will always be people available to maintain those systems. The price will go up until some 'modern' programmer is tempted enough to open a 3 decades old book on COBOL and JCL. Your understanding of business is flawed, businesses are advancing by marketing, not by shacking up with every new hot tech that comes along in favour of their old, trusted and reliable tech. The larger, older and more profitable a business is the more likely that they are going to be ultra-conservative about using new technology at the core of their business. Eventually we'll get rid of this stuff but I wouldn't bet on it happening in the next two decades, unless someone finds a way to emulate a mainframe on a cluster with similar reliability. And then we still need to get rid of the software.
- logicalmind 16y ago
  You don't think that there is room for a new high-tech bank to step in and eat the lunch of the big banks? If you started a bank from scratch, built with modern technology, you could provide all kinds of valuable services and adapt to change much more quickly. Pretty much every service provided, particularly on the web, is some application that screenscrapes CICS screens or reads data from VSAM. These applications a tricky to build and take much more time than they should.
  - jacquesm 16y ago
    Starting a bank from scratch will have you worry about other things than the technology of your IT department, mostly to do with solvency and staying clear of SO. Sure, at some level the technology will come in to play but nobody will 'start a high tech bank'. For instance, ING direct (one of the efforts of the banking industry to leverage the use of technology) was jump started from the legacy ING systems and as far as I know still ties in to them for all of their back-end stuff. When I was the 'systems administrator' for the corporate division of a bank (in 1986 or so) we had daily rounds of getting the signatures of the various people responsible that the totals for the night batch runs were correct. That's the sort of responsibility you're talking about, I can't imagine that an MD5 or some digital signature would be sufficient today, it's all about procedures and accountability, emphatically not about technology. I got one of the first PCs the bank bought on account of our project manager being very forward thinking but nobody ever thought of it as more than a curiosity and something that might be a handy replacement for the terminals. That attitude has changed a bit but not by much and that's 25 years.
    - logicalmind 16y ago
      Well, nobody would have built another search engine either, but google did it. Part of the problem is the legacy mindset of the higher ups at financial institutions combined with their extremely conservative nature. I currently work in a fairly large financial institution, giving them a web presence for various services and the backend system is no longer a mainframe but an AS/400. Though it still runs COBOL-based apps, so it's really no different than the standard mainframe.
      - jacquesm 16y ago
        I'm not sure I follow you on the google link here. Google built another search engine, which ran on a cluster of PCs and did so more efficiently than the big competitor at the time, AltaVista, which ran (afaik) on a cluster of Dec Alphas. Because google needed lot of horsepower and had a tight budget they came up with some pretty clever packaging ideas for their cluster. In that sense both these companies took the best route to a solution. Now a search engine is materially different from a bank in that if something goes wrong in a bank the potential damage (not just to the bank, but also to the customers of the bank) is enormous. A search engine that spits out 'wrong' results is still a search engine. A bank that doesn't know where your money went is no longer a bank (at least, not for very long). Financial institutions rely heavily on their image of reliability, much more so than any web company.
rbranson 16y ago
He's talking about FedEx. They understand that it's a risk to their business continuity. The guy who hijacked a FedEx plane planned to fly it into their datacenter near the airport, which houses the primary mainframe that runs FedEx's most critical business software. He wasn't an IT employee either, so it's fairly common knowledge. AFAIK, they are now running in a hybrid mainframe / client+server model. Most of the package transactions are processed simultaneously. Apparently all of the tracking on FedEx.com uses the client+server system to back it. Obviously they are taking it slowly and carefully (it's like a decade-long project) to avoid any screwups.
- Hoff 16y ago
  When you're working at this scale, geographic redundancy of DCs is commonplace. I know of several cases where entire DCs were destroyed, and production continued unabated. There are commercial platforms that are explicitly designed for this sort of DC redundancy over 800+ km spans. Where you have 400 km between various volumes in your RAIDset.
  - mkr-hn 16y ago
    Do you have links? That's a scenario I've always been curious about, and would like to read about situations where it actually happened.
    - Hoff 16y ago
      http://h71000.www7.hp.com/openvms/brochures/commerzbank/commerzbank.pdf http://h71000.www7.hp.com/openvms/brochures/commerzbank/comm... A moderately technical article on some of the issues that arise, and (for your question) see the end of the following: http://h71000.www7.hp.com/openvms/journal/v1/disastertol.pdf http://h71000.www7.hp.com/openvms/journal/v1/disastertol.pdf
      - wallflower 16y ago
        Fascinating. "At 2 a.m. on Dec. 29, 1999, an active stock market trading day, the audio alert on a UPS system alarmed a security guard on his first day on the job. He pressed the emergency power-off switch, taking down the entire datacenter. Because a disaster-tolerant OpenVMS cluster was in place, the cluster continued to run at the opposite site, with no disruption. The brokerage ran through that stock-trading day on the one site alone, and performed a shadow full-copy operation to restore redundancy in the evening, after trading hours. They procured a replacement for the failed security guard by the next day." The last sentence.
Hoff 16y ago
Don't presume that you're operating with sufficient domain knowledge to make those judgements, and don't presume that the folks that are making those judgements are ignorant of the options and alternatives. When you're operating at the juggernaught-scale of business or with industrial-scale production, the rules and the requirements and the competitive landscapes are different. And the risks can come not from the existing environments - which generally work - but from making the changes to the environment, and from the risks of outages and downtime. Every vendor that can make even a tangental case for a platform or application or process migration or update or new hardware is also certainly pressing that, too; that's an automatic education for the folks involved. If you can't process customer orders for a week or a month or can't ship orders or your outage slags a whole production line, you could well end up out of business. I know of places that have had similar meltdowns during updates and during migrations, and it's Not Pretty.
fmora 16y ago
You have to remember that they use these systems to run their business and they've been tested for thousands or millions of hours so the system is extremely reliable. It is naive to suggest to switch to a newer platform or language. Those things are always changing. Also, once you are a seasoned software developer in multiple languages how long do you really think it will take you to learn cobol? Not much I can assure you. You'll probably be familiar with the basics in a couple of days, and maybe even enough to be dangerous. Switching to a platform just for the sake of switching is extremely dangerous and potentially quite costly. Especially when from the customers point of view it really makes no difference whether behind the scenes the business is using cobol, fortran, java, ruby or whatever new fad of the day. If the system is going to behave exactly the same whether you use java or cobol then what is the point of switching?
- jacquesm 16y ago
  A million hours is more than a century, I think that is somewhat unlikely.
  - fmora 16y ago
    No is not. Thousands if not millions of people use their systems every year. Every person that uses it counts as time testing the system. Notice how quickly the hours add up if you look at it this way.
    - jacquesm 16y ago
      Ok, I was thinking in terms of the run-time of the software, not in terms of the time people spent using the system. But that's a valid way to look at it.
    - whatusername 16y ago
      You could say the same about Windows XP.

by mey 16y ago

jacquesm 16y ago

The laws of supply and demand will make sure that there will always be people available to maintain those systems. The price will go up until some 'modern' programmer is tempted enough to open a 3 decades old book on COBOL and JCL. Your understanding of business is flawed, businesses are advancing by marketing, not by shacking up with every new hot tech that comes along in favour of their old, trusted and reliable tech. The larger, older and more profitable a business is the more likely that they are going to be ultra-conservative about using new technology at the core of their business. Eventually we'll get rid of this stuff but I wouldn't bet on it happening in the next two decades, unless someone finds a way to emulate a mainframe on a cluster with similar reliability. And then we still need to get rid of the software.

logicalmind 16y ago
You don't think that there is room for a new high-tech bank to step in and eat the lunch of the big banks? If you started a bank from scratch, built with modern technology, you could provide all kinds of valuable services and adapt to change much more quickly. Pretty much every service provided, particularly on the web, is some application that screenscrapes CICS screens or reads data from VSAM. These applications a tricky to build and take much more time than they should.
- jacquesm 16y ago
  Starting a bank from scratch will have you worry about other things than the technology of your IT department, mostly to do with solvency and staying clear of SO. Sure, at some level the technology will come in to play but nobody will 'start a high tech bank'. For instance, ING direct (one of the efforts of the banking industry to leverage the use of technology) was jump started from the legacy ING systems and as far as I know still ties in to them for all of their back-end stuff. When I was the 'systems administrator' for the corporate division of a bank (in 1986 or so) we had daily rounds of getting the signatures of the various people responsible that the totals for the night batch runs were correct. That's the sort of responsibility you're talking about, I can't imagine that an MD5 or some digital signature would be sufficient today, it's all about procedures and accountability, emphatically not about technology. I got one of the first PCs the bank bought on account of our project manager being very forward thinking but nobody ever thought of it as more than a curiosity and something that might be a handy replacement for the terminals. That attitude has changed a bit but not by much and that's 25 years.
  - logicalmind 16y ago
    Well, nobody would have built another search engine either, but google did it. Part of the problem is the legacy mindset of the higher ups at financial institutions combined with their extremely conservative nature. I currently work in a fairly large financial institution, giving them a web presence for various services and the backend system is no longer a mainframe but an AS/400. Though it still runs COBOL-based apps, so it's really no different than the standard mainframe.
    - jacquesm 16y ago
      I'm not sure I follow you on the google link here. Google built another search engine, which ran on a cluster of PCs and did so more efficiently than the big competitor at the time, AltaVista, which ran (afaik) on a cluster of Dec Alphas. Because google needed lot of horsepower and had a tight budget they came up with some pretty clever packaging ideas for their cluster. In that sense both these companies took the best route to a solution. Now a search engine is materially different from a bank in that if something goes wrong in a bank the potential damage (not just to the bank, but also to the customers of the bank) is enormous. A search engine that spits out 'wrong' results is still a search engine. A bank that doesn't know where your money went is no longer a bank (at least, not for very long). Financial institutions rely heavily on their image of reliability, much more so than any web company.

rbranson 16y ago

He's talking about FedEx. They understand that it's a risk to their business continuity. The guy who hijacked a FedEx plane planned to fly it into their datacenter near the airport, which houses the primary mainframe that runs FedEx's most critical business software. He wasn't an IT employee either, so it's fairly common knowledge. AFAIK, they are now running in a hybrid mainframe / client+server model. Most of the package transactions are processed simultaneously. Apparently all of the tracking on FedEx.com uses the client+server system to back it. Obviously they are taking it slowly and carefully (it's like a decade-long project) to avoid any screwups.

Hoff 16y ago
When you're working at this scale, geographic redundancy of DCs is commonplace. I know of several cases where entire DCs were destroyed, and production continued unabated. There are commercial platforms that are explicitly designed for this sort of DC redundancy over 800+ km spans. Where you have 400 km between various volumes in your RAIDset.
- mkr-hn 16y ago
  Do you have links? That's a scenario I've always been curious about, and would like to read about situations where it actually happened.
  - Hoff 16y ago
    http://h71000.www7.hp.com/openvms/brochures/commerzbank/commerzbank.pdf http://h71000.www7.hp.com/openvms/brochures/commerzbank/comm... A moderately technical article on some of the issues that arise, and (for your question) see the end of the following: http://h71000.www7.hp.com/openvms/journal/v1/disastertol.pdf http://h71000.www7.hp.com/openvms/journal/v1/disastertol.pdf
    - wallflower 16y ago
      Fascinating. "At 2 a.m. on Dec. 29, 1999, an active stock market trading day, the audio alert on a UPS system alarmed a security guard on his first day on the job. He pressed the emergency power-off switch, taking down the entire datacenter. Because a disaster-tolerant OpenVMS cluster was in place, the cluster continued to run at the opposite site, with no disruption. The brokerage ran through that stock-trading day on the one site alone, and performed a shadow full-copy operation to restore redundancy in the evening, after trading hours. They procured a replacement for the failed security guard by the next day." The last sentence.

Hoff 16y ago

Don't presume that you're operating with sufficient domain knowledge to make those judgements, and don't presume that the folks that are making those judgements are ignorant of the options and alternatives. When you're operating at the juggernaught-scale of business or with industrial-scale production, the rules and the requirements and the competitive landscapes are different. And the risks can come not from the existing environments - which generally work - but from making the changes to the environment, and from the risks of outages and downtime. Every vendor that can make even a tangental case for a platform or application or process migration or update or new hardware is also certainly pressing that, too; that's an automatic education for the folks involved. If you can't process customer orders for a week or a month or can't ship orders or your outage slags a whole production line, you could well end up out of business. I know of places that have had similar meltdowns during updates and during migrations, and it's Not Pretty.

fmora 16y ago

You have to remember that they use these systems to run their business and they've been tested for thousands or millions of hours so the system is extremely reliable. It is naive to suggest to switch to a newer platform or language. Those things are always changing. Also, once you are a seasoned software developer in multiple languages how long do you really think it will take you to learn cobol? Not much I can assure you. You'll probably be familiar with the basics in a couple of days, and maybe even enough to be dangerous. Switching to a platform just for the sake of switching is extremely dangerous and potentially quite costly. Especially when from the customers point of view it really makes no difference whether behind the scenes the business is using cobol, fortran, java, ruby or whatever new fad of the day. If the system is going to behave exactly the same whether you use java or cobol then what is the point of switching?

jacquesm 16y ago
A million hours is more than a century, I think that is somewhat unlikely.
- fmora 16y ago
  No is not. Thousands if not millions of people use their systems every year. Every person that uses it counts as time testing the system. Notice how quickly the hours add up if you look at it this way.
  - jacquesm 16y ago
    Ok, I was thinking in terms of the run-time of the software, not in terms of the time people spent using the system. But that's a valid way to look at it.
  - whatusername 16y ago
    You could say the same about Windows XP.