11 ms·

It's a trap. The problem with those old codebases that governments, hospitals, big businesses are struggling with is not really the language, it's the engineer

• brnt 7y ago
  Just finished a contract with a hospital that built a lot of stuff in house in the 80ies based on a C (still pre C89 in many places) and Delphi/Pascal. The problem is indeed volume (over 10MLoC), two or three wizards supporting it all, but firmly coding like it's '83. No training of newcomers whatsoever, and thus really no way to contribute. If you manage to get some support, it will be once a year and in the form of code they wrote for you without much feedback possible. Management prefers not to think about long term, because management obviously does not think long term.
  • throwaway_pdp09 7y ago
    Is it possibly not a management problem at the core? I mean, I know exactly what you mean about suits like to ignore the future, but I've worked with a few 'wizards' who, once they've got the job under their belt, use it to keep other people out. They'll not comment, help, document or whatever, and once they're doing that, they are uncontrollable. They can't be sacked because they're the only ones keeping the system running, and they won't help others train up. That seems a very difficult situation for the suits to deal with, even if they want to.
    • bryanrasmussen 7y ago
      well if you have 3 guys that are the wizards that can't be sacked, hire 3 contractors for a year long project to document things sufficiently that these guys become more sackable. Contractors of course have to be highly experienced as well, and well-remunerated, so maybe management doesn't want to go down that expensive avenue, which means it's a management problem. Not to mention the years where they did not comment, help, document or whatever, and became uncontrollable sounds like a management problem over those years.
      • throwaway_pdp09 7y ago
        Your first suggestion might work, but having been there I know obnoxious programmers can be very obstructive, and put up some major barriers to newbies. That said, I think yours is a good solution if it can be afforded. Per your 2nd point about long-term management problems, no doubt of it at all, but sometimes the new (and sometimes good) management simply inherits what previous mismanagement left behind. Also perhaps you underestimate the power that programmers in well-bedded-in positions have. They can outright ignore management orders - experience speaking. But a good post nonetheless, thanks.
        • alxtt 7y ago
          The other side of the coin is that new hires can be very obstructive. 80% of new hires here are relentless politicians who snitch to management that they are "underappreciated" while making one idiotic suggestion after the next to show their relevance. In reality they cannot do anything, so they scheme to get rid of people who can. During the battle no useful work is being done.
          • rbanffy 7y ago
            If you hire people like this you need to involve HR in better selecting who the org hires.
            • Red-Ted 7y ago
              I've made a point of not hiring intersectional people in general been burnt before by employees who care more about 'woke' politics then actually getting the job done.
    • zhte415 7y ago
      I worked with a guy like this. He bragged he was the only one that knew the entire legacy codebase and he only shared parts with colleagues on his pretty sizable team to ensure he had a job and fat salary for life. The the multinational decides a change in direction and fired the entire office, relocating it for regional diversification (this was not offshoring, spreading the work out across multiple teams). This was pretty niche stuff, and he became unemployable until moving across the country. And I know of a company that similar was outsourced to. Fully outsourced with seemingly no internal expertise. Bleeding their client for increasing support prices year-by-year imagining they could do this for ever. I was in the team in that client that for two years, going from-scratch re-implementing the functionality, no cheap proposition, but within 3 years of in-sourcing the savings were already there. Not all wizards know magic.
      • brnt 7y ago
        Indeed, the word wizard was perhaps ill-chosen. These guys know this particular codebase, in all it's hairy glory, but not much else.
        • zhte415 7y ago
          I don't think ill-chosen. In their eyes, they are wizards.
      • m0xte 7y ago
        That smells familiar. I worked with one of those wizards and he designed everything to keep him the job. That was until he rode a motorbike into the side of a car at 100mph and someone else (me) inherited it. I had to start again because it was that impenetrable. Consequentially it turned out it didn't do a lot and I'd rewritten the bulk of it in ASP/SQL Server at the time in a couple of weeks reducing the cost of the entire platform to an hour or two a week rather than an entire "tier B" (whatever that was but I was told it was a lot and was tier F myself) salary. When I quit it took a few hours to hand over to the next guy.
    • lotsofpulp 7y ago
      Management that allows one or a few persons to become that indispensable is a management problem.
    • vageli 7y ago
      > Is it possibly not a management problem at the core? I mean, I know exactly what you mean about suits like to ignore the future, but I've worked with a few 'wizards' who, once they've got the job under their belt, use it to keep other people out. > They'll not comment, help, document or whatever, and once they're doing that, they are uncontrollable. They can't be sacked because they're the only ones keeping the system running, and they won't help others train up. That seems a very difficult situation for the suits to deal with, even if they want to. This sounds _exactly_ like a failing of management to me. If developers are expected or allowed to "just code" without documenting anything or training anyone, management is absolutely to blame for allowing it to go on.
• atoav 7y ago
  > keeping the old UI and UX (which often is incredibly ergonomic) I have the feeling that with the success of the iPhone many people forgot that a thing like a UI can have a target audience as well. If you make a tool that is beeing used twice a week for a minute at a time it has to look fundamentally different from a tool that is used 50 times every day. With the former beeing intuitive is more valuable, while with the latter reducing friction is more valuable. This is a choice which has to be made – and sadly I often don't see it beeing made. People just make a UI that is akin to the ones Google or Apple make and call it a day.
  • deleted 7y ago
    [deleted]
  • finnthehuman 7y ago
    >People just make a UI that is akin to the ones Google or Apple make and call it a day. It's worse than that. Lots of people involved in the creation of software don't just follow the trends, they have internalized the idea a UI exposing any complexity is inherently bad. That if something can't be easily expressed in the interaction language currently fashionable in mobile, then it must be a misfeature. A distant but perhaps illustratively analogous example can be seen in non-nerdy teens and young adults. Take one that does class writing assignments in a google doc on their phone (they're not hard to find, you can even find some that try to do CAD on mobile devices). Try suggesting that if they learned to properly touch type on a real keyboard they'd find the whole process easier and faster. Then tell them apple's bluetooth keyboard can pair to iPhones. Compare the reactions. tl;dr: In the TV show Metalocalypse the characters derisively called acoustic guitars "grandpa's guitars." That's the UX world in a nutshell.
• augustk 7y ago
  I completely agree. I work with a large APL codebase and the main problem is not the language but the culture. Overly long expressions, one-letter variable names, gotos etc. make the code obfuscated.
  • rbanffy 7y ago
    And APL requires supreme discipline to prevent that from happening.
  • useful 7y ago
    I was the team lead to rewrite a large APL codebase with a very small team as part of a much larger group that refused to change and I'll agree with this. Once we started documenting and building tests for the code that replaced APL it became clear that the complicated bits were easy to replace and the hard part was just man hours converting all the conditional logic.
  • Koshkin 7y ago
    > one-letter variable names Just like in math!
    • msla 7y ago
      Except in math, the notation is surrounded by natural-language prose which carries the brunt of the semantic load.
• rgmt 7y ago
  But who would rewrite it for the low salaries? Most people I know are not interested in low-level correctness at all, the open source C++ projects on GitHub with a high churn rate always have tons of corner case bugs. If you don't get the right people, the rewrite would be an overengineered object oriented nightmare with an endless stream of bugs.
• spindle 7y ago
  Absolutely (and I say that as an erstwhile mainframe COBOL programmer). The language itself is dead easy.
  • Koshkin 7y ago
    The language isn't dead (and doesn't have to be). If anything, it must be taught as part of a programming language design course. In a way COBOL is similar to SQL which is far from dying. (IMHO it would have been useful if they had merged into one language; the available embedding of SQL into COBOL was kinda clunky.)
    • chihuahua 7y ago
      The person you're responding to didn't claim that COBOL is dead. He/she wrote that COBOL is "dead easy", meaning "very easy".
• Shivetya 7y ago
  Other than constraints most of the issues you raise are just as likely to happen in modern languages as well. That comes down to how the team was managed. If anything I have found a lot of older code over documented. My difficulty when I had assist in migrating some old COBOL from CICS on a zSeries was understanding their file structure techniques. However that was easily remedied by understanding the task as hand so that the data was better represented on the new system. Is it a trap? Well if you want a secure position in managing a code base and maybe eventually working with others to move it to a new platform I do not see how. I have been around enough new languages to know we are always going to run into code bases we just want nothing to do with but here we are. The problem to me is you may land in a development shop that is not well maintained. The code has worked for so long that management outside the department just went assumed that everyone knew everything.
• zxcvbn4038 7y ago
  Not to mention GOTO. If your one of the people who hyperventilates when you see a goto because you learned that it was considered harmful in programmer school then Cobol might not be for you. ;) You might be surprised about the comments though - depending on the age of the codebase. Mainframes were rented back in the day, you paid by resources consumed, terminal time was precious, and mainframes were often turned off outside business hours. Because of this a lot of the development actually happened between terminal sessions in flowcharts, pseudo code, documentation, and peer review before the programs were ever modified and run. If you ever run across really old comp-sci books you’ll typically see them divided into three sections - first section was usually a guide to the author’s terminology and symbology, second part was usually a guide to flow charting and documentation (IBM had standardized forms for developers to use), and then the remainder of the book was the content with lots of explanations of how to work the datasets hugely larger then the memory available to you. But as time passed and computer time became cheaper many of those formal development practices started to get lax.
  • walshemj 7y ago
    I doubt that mainframes where turned on and off every day even later with super mini's we left them powered up
    • nosborn 7y ago
      Every mainframe I worked on until the mid-80s was turned on by the first person to arrive in the morning and turned off by the last to leave in the evening.
    • p_l 7y ago
      Many sites had operations plan that involved weekly or biweekly "IPLs" done on weekends.
      • walshemj 7y ago
        Doesn't mean the power was off
  • 7y ago
• edw519 7y ago
  lack of comments, bad variable naming, bad structure (little or no procedures or readability), and just sheer volume of it This has nothing to do with technology and everything to do with people. For all the progress we've made with technology in the past 50 years, we've made little or no progress with any of the items in this list. The second biggest problem building software has always been programmers too small for the task at hand. The biggest problem has been managers who are even smaller. So don't blame COBOL or any other technology. Fix the people and you can build anything excellent from almost any tech.
  • zozbot234 7y ago
    > This has nothing to do with technology Oh, it has everything to do with technology. Wouldn't want to waste that perfectly serviceable punch-card by punching a * in column 7.
• cmiles74 7y ago
  I think it would make a lot more sense to pay COBOL developers at the same rate as other software developers. The era where we had analysts writing psuedo-code so clear that it could simply be "coded" into COBOL is over, if it ever really existed in the first place. In addition, paying COBOL developers more is so obviously less expensive than trying to completely re-write all of that legacy code. It is likely true that many people would not be excited to learn COBOL. That said, I do think there would be a good amount of developers, perhaps new to the field, that would be willing to write the code in return for a fair wage and the work experience. But this attitude that COBOL code is somehow worth less than JavaScript code needs to be worked out of the system. It simply is not true and it is clearly doing harm.
• vsareto 7y ago
  Reverse engineers can manage to stare at assembly and figure things out. Every programmer can learn the skills of staring at really fucked up representations of logic and eventually figure things out. It's definitely slower of course. Now is definitely not the time for a rewrite. As much of a trap this seems to be, it's actually the best move given the circumstances. I may actually take IBM up on this since my father was a COBOL programmer, but I wouldn't plan to make a career of it.
• bdavis__ 7y ago
  I propose that most of the code written in the old days, in COBOL in particular, was not done by computer scientists. There were many lanes for learning COBOL, and they included community colleges and industry training courses. The amount of "computer science / software engineering" concepts that were widely known (or even possible) in those days was very limited. 're-engineering' the code is a valid operation, but the amount of code is extreme. and without a deep testing regime, a dangerous journey. so, we keep on keeping on. the same way.
  • FlyMoreRockets 7y ago
    This is true. Back in the day, COBOL was marketed to businiess people as an easy to learn code to just get stuff done. Serious programming was done with FORTRAN.
    • bdavis__ 7y ago
      still true today. (not really, but there is a lot of f77 out there...)
      • eyegor 7y ago
        I don't know anyone who uses f77 anymore, but f90 and up is still popular in scientific research.
    • Koshkin 7y ago
      Actually, FORTRAN was marketed to scientists, not for "serious programming" which was still done in assembler.
      • WorldMaker 7y ago
        Why sometimes still today someone's Jupyter notebook of data science Python is calling into FORTRAN libraries under the hood.
  • perl4ever 7y ago
• forinti 7y ago
  Now imagine 30 years from now when you are going to have to track down the documentation for version X of the web framework for version Y of some language and version Z of frontend framework.
  • jerf 7y ago
    So... don't do that. There are plenty of options that have a reasonable probability of being stable. (Also, consider committing the docs right in to the repo. In the 1970s such an idea would have been absurd. Today, a lot of my projects technically already have this, thanks to vendoring and docs embedded into the programs themselves.)
  • LeonM 7y ago
    > Now imagine 30 years from For the javascript ecosystem this is already true for projects that are > 2 year old. Not kidding, try to build a 2 year old React web application, you'll see what I mean.
    • earthboundkid 7y ago
      TBF, part of the problem is that dependency locking only caught on recently. (Yarn came out in 2016.) That makes it more possible.
  • Zenst 7y ago
    best documentation is getting the business logic mapped out and the code itself, along with data mapped-out with what does what to it when and how. As any code documentation will be out of date in way way or another, even best sites it will be case of getting that documentation and then mapping a few decades worth of change management, bug tracking and other avenues that modified that code. I will say though, every migration project I worked up, the documentation was carefully worded in the contract as being the customers liability and with that, code gets migrated logic for logic, bug for bug and testing so anal that it will show that what goes in is the same that comes out in the migrated code. That and the business documentation will still be good, even much of the code documentation if high enough level, but the code itself will be the best documentation. Until we get a standard in which the documentation produces the code and all changes done to the documentation over quickly hacking the code, the disparity between any documentation and the code will always be adrift. So you see many bespoke solutions to go thru the code and produce documentation from that to varying levels of success, however that success for one sites quirks in code may not work as well with others. Hence even the best documentation will wisely get treated with a pinch of salt and in many instances, be like comparing a book to the movie it spawned in many ways, some close to the original, many not even close. That is documentation and code in a nutshell. Always best to map the data first and that can be done easier and more automated, more so databases and generating a schema and then map what code talks with what and gradual get to see what is happening.
• commandlinefan 7y ago
  > modern engineering practices Heck, I’d be willing to settle for seeing modern software languages used with modern engineering practices.
• robomartin 7y ago
  > The language is not the problem - lack of comments, bad variable naming, bad structure (little or no procedures or readability), and just sheer volume of it, is. I wonder what basis, evidence or data you might be using to make this assertion. You are assuming quite a bit there as well as generalizing all problems across all affected systems to have your list of issues as the root cause. Could it be that nobody bothered to maintain and modernize these systems because spending more money on software that "works" isn't going to earn anyone in government points? Government and politics have metrics and fitness functions that do not align very well with the real world (anything outside of government or large stagnant companies). And yet, at the same time, have you looked at open source libraries lately? The phrase that comes to mind is: rotten smelly stinking mess. I just had to deal with one of those a few weeks ago. No comments, horrible code structure, massive class hierarchies, just awful stuff. The complexity and thickness of the interface they created was astounding. We re-wrote the entire thing in about thirty lines of code. Yeah. A massive multi-source-file library got boiled down to just a handful of clean code, no classes, just clean, simple and easy to understand code with comments anyone could understand. I know it might be difficult to modern programmers to understand the kinds of constraints software developers had to work with in the '80's or before. A simple example of this would be single character variable names. When you only have a few thousand bytes of memory and you are working with an interpreted language, variable names consume memory you desperately need, not to mention CPU cycles. So, yes, people resorted to use single character names to conserve memory and improve execution time. Context is important. I took one semester of COBOL back in the dark ages, FORTRAN also. Thankfully I never had to use them professionally. I started professional life using APL, C and FORTH. I realized, years later, how lucky I was to have been shoved into that path by a physics professor who insisted I veer away from COBOL/FORTRAN and take his APL class.
• Spooky23 7y ago
  It’s not a trap, just IBM pr. The legacy stuff is usually fine, it’s the layers of middleware scaffolds around the mainframe. Mainframe jobs are 90% batch, so even under stress, it can handle it. Your circa 2002 scaffolds are the problems.
• Yhippa 7y ago
  > It would be very interesting to see the old systems rewritten in a modern language, with modern engineering practices, but keeping the old UI and UX (which often is incredibly ergonomic) - so as to limit scope and not mess it all up by trying to introduce mouse navigation and windowing mess. I was the tech lead on one of these projects. Personally I was sad and frustrated we had to keep the old UX/UI. I would have much rather have made something more ergonomic for our users. Alas retraining would have been too expensive to do that even though it would have been probably more intuitive. I do agree with you that there is some benefit to being able to do everything on a keyboard without having to deal with the baggage of what we consider modern.
  • hindsightbias 7y ago
    At some point in the web, ergonomic went from something that meant all form and minimal viable function. Efficiency just isn’t a real noun in the UX/UI vocabulary when it somes to complex entry. 30 years ago, those of us on green screens at Big Org knew more shortcuts than any emacs user. Now whenever I have to use a “modern” CRM it’s the most anti-productive aspect of my job.
    • vbezhenar 7y ago
      Web UI optimized for user who can close page any time if he does not like anything. You have 100 competitors and user can flee to anyone of them. Enterprise UI optimized for speed of trained personnel. They can't close page if they don't like it, they are paid for work. It's completely different situations and when someone mixes them, it leads to bad UI.
• cat199 7y ago
  anyone familiar with current node/npm/front end web dev would have very little trouble dismantling the false notions of implicit chronological progress belying this argument.
• ngcc_hk 7y ago
  It always mess up by introduce not the mouse or any technical things. But your keyword introduce ... What do you think to get the proposal off the ground. Just reimplement granddad system is not it I am afraid. Saw that so many times ...