5 ms·
AI migrated legacy COBOL programs to Java, bugs included
- deleted 2mo ago[deleted]
- madduci 2mo agoI remember a similar story shared this year at JAX2026 from the Sparkasse Group, they said they were using AI to migrate from COBOL, but they still were in the middle of the migration. Maybe they faced the same issues / problems? It seemed pretty zealous to me, that everything was working smoothly, but this article highlights the limitations
- wolfi1 2mo agosome years ago I talked to a software engineer at a bank, he said it would be too risky for them to move away from the mainframe what with regulations and all, I'm not sure if AI could make the banks more risk friendly, so if Sparkasse does that already, I, for one, am eager to learn the result (and am happy to have no account there)
- BigJono 2mo agoYeah nah maybe fix the bugs before swapping the average COBOL dev for the average Java dev.
- nicman23 2mo agosometimes if the bug exists for enough time it is intended behavior
- surfingdino 2mo agoIndeed, there may even be a whole lot of code that depends on it.
- wwind123 2mo agoYeah. Bug-for-bug migration is a real thing in large code-bases in the industry. You want to replicate all behavior of the code regardless whether the behavior is a feature or a bug. See Hyrum's Law: https://www.hyrumslaw.com/ https://www.hyrumslaw.com/
- nkjoep 2mo agoYep, and when it’s about money you don’t want any unexpected.
- tapland 2mo agoAbsolutely, and you'll have all kinds of fixes for the symptoms throughout the project. COBOL isn't hard, the tooling around it on old systems are a pain though.
- Maxion 2mo agoAnything running on COBOL to day is a large enterprise system. You'll have reports running in other systems in subsidiary companies that rely on bugs in the upstream cobol code.
- skissane 2mo agoOne of the hinges on our front screen door was coming off, the screws were coming loose from the wooden door jam. So I screwed it back on. My wife and son then told me I "broke" the front door. I was confused, I thought I had fixed it Turns out, with the door coming off the hinges, it wasn't closing properly, which meant it was ajar most of the time, so our dog could push it open with her body. Now I'd fixed it, it closed properly all the time, so our dog started whining for someone to open the front screen door for her So I loosened the screws, and hence "fixed" it by breaking it again
- pasc1878 2mo agoSo a form of https://xkcd.com/1172/ https://xkcd.com/1172/
- yomismoaqui 2mo agoAh, the classic load bearing bug.
- vrighter 2mo agohyrum's law applies here
- pacaro 2mo agoThe largest test case was 4kloc. There are hundreds of billions of lines of cobol in production. The IRS alone has approx 160 cobol programs, averaging 230kloc each.
- mpfh 2mo agoWhen they migrate to Java how are they migrating things like the messaging (send, receive) functionality?
- toplinesoftsys 2mo agoThe biggest problem is not that bugs are migrated with COBOL, but that lots of new bugs are going to be introduced. AI is not deterministic, it will be making tons of mistakes. The only realistic low-error approach is incremental step-by-step migration using Cursor or similar tools. However, it requires much more time as each step must be prompted, tested and committed manually. Any hope that one-shot migraton of a large code base will not introduce enormous number of bugs is very naive. LLM is very bad on handling long context - it is their nature unfortunately. There is no answer to this problem yet.
- pinkgolem 2mo agoDoes it? I found llms to be great for straight conversions At least if tescoverage is good, but well... That's something llms can also be used for
- selcuka 2mo ago> AI is not deterministic, it will be making tons of mistakes. From the paper: > The COBOL source is passed through an internal deterministic Migrator to produce a generated Java target. Also, humans are not deterministic either. Give the same COBOL -> Java translation to multiple developers and each will come up with a different solution. Heck, even the same developer will produce a different output for the same task, depending on the day of the week.
- maoberlehner 2mo ago> Also, humans are not deterministic either. Indeed, and that's why so few dare to migrate them, and so many who do fail or blow the budget many times over. I think that was the implicit point of the comment: don't expect that with AI, suddenly we can convert all those COBOL apps with a single prompt.
- dwroberts 2mo agoBut it says that the agentic authoring step patches the migrator when things get stuck. So while the execution of this stuff is deterministic, its actual content is not.
- fock 2mo ago300 to 4000 lines of "production like" (whatever that is) cobol code which is easily ported to a non-mainframe env. Our's sometimes uses assembler in its innards, so good luck with real legacy code spanning a dozen files and 50k loc... I recently threw in (want to check those intelligence metrics!) some real production code into a non-agentic system (just to get a feel how things perform without a custom harness) and results where ... interesting. The particular program uses some preprocessor no LLM we have access to (newest was GPT 5.5) has any clue about - so they confabulate what it could do (Gemini 2.5 didn't even notice there was a preprocessor...). This is expected of course but it somehow seems the problem of this technology that unless you feed it masses of data or mechanically break up the tasks in rote subunits, it just doesn't do anything sensible still...
- TimByte 2mo agoYeah that’s why they delegated code gen to deterministic tooling and saved the model for input fuzzing - let it fight the data instead of legacy syntax
- hexasquid 2mo agoTo IBM As specified, please find 99997 correct parts and the 3 defects (do not use)
- bigbuppo 2mo agoCOBOL will never die. Whatever this is will only result in more COBOL being written.
- anthk 2mo agoCOBOL is not about the language, it's about the whole environment in the mainframe. LLM fanboys won't understand this. You need something like a mainframe with a 99.99% uptime no matter what happens in hardware, with live CPU swapping and such.
- raverbashing 2mo agoYes and that mainframe runs several other technologies besides COBOL, and more importantly, that don't depend on it (COBOL fanboys won't understand this ;) )
- pjmlp 2mo agoGiven the option, I would go with DB 2 on z/OS instead of COBOL, for example.
- jjmarr 2mo agoErlang/Elixir/OTP has a resilient environment with hot swapping. Is that truly problematic?
- michaelmrose 2mo agoCan you explain why you need one machine never falling over ever instead of a cluster of machines never falling over at once collectively?
- Maxion 2mo agoYou don't. Either would work. Getting rid of a system running cobol on a mainframe is perhaps the hardest work possible in software engineering.
- Surac 2mo agoHow does ai implement all the not Cobol parts a Cobol program rely on? Job Contol, CICS, sort processors? Cobol and mainframe technologies are non existant in java on any modern machine
- p_l 2mo agoBatch job control is handled by alternative systems (already often triggering the work on mainframes anyway) like Control-M CICS is the big issue but AFAIK there were attempts. Everything else, including sort and VSAM, has various options provided usually by COBOL compiler vendors.
- TimByte 2mo agoFor the article they just mocked it out for unit tests. But in reality you really can't - that’s a massive pain in the ass. Rewriting pure math is easy, but mocking CICS transaction isolation in java means spinning up these monstrous adapter frameworks that just tank performance
- renezander030 2mo ago[dead]
- pjmlp 2mo agoIf the systems are to stay on mainframe/micro land, all IBM, Fujitsu, Unisys platforms support Java. This is one set of ecosystems, where .NET going cross platform still misses out to Java, and neither of those companies cares about supporting it on their platforms. There are some half way supported Python, Go and Rust versions as well, although I think they need the "WSL" of the respective mainframe/micro OS, aka UNIX services.
- aldente0630 2mo agoAs I understand it, the translation isn't done by an LLM but by a deterministic AST-based migrator. Also, carrying over the bugs is the stated goal.
- luciana1u 2mo ago[flagged]
- pjmlp 2mo agoStuff like this are already a commercial product, it isn't only Zig to Rust rewrites going around. https://www.pega.com/insights/resources/break-free-legacy-mainframe-systems https://www.pega.com/insights/resources/break-free-legacy-ma... https://www.ibm.com/products/watsonx-code-assistant-z https://www.ibm.com/products/watsonx-code-assistant-z https://global.fujitsu/en-global/pr/news/2026/03/30-01 https://global.fujitsu/en-global/pr/news/2026/03/30-01 https://www.rocketsoftware.com/en-us/insights/ai-powered-cobol-development https://www.rocketsoftware.com/en-us/insights/ai-powered-cob...
- kukkeliskuu 2mo agoWhile not everything can be easily converted (IMS, CICS, reports, batch processing etc.), there are many situations where automated tooling can be helpful in migrations. Related to this, I created a tool for situations where you want to compare COBOL code with Java code. It includes a preprocessing step where IMS etc. calls are converted to mocks that return JSON (from file), and also use JSON for input/output, and GnuCOBOL to run the program. More a proof-of-concept than production, but here is a link if somebody finds it helpful. https://github.com/mikko-ahonen/coboltwin https://github.com/mikko-ahonen/coboltwin
- crewlesslab 2mo ago[flagged]
- mtct88 2mo agoWhy should I convert COBOL to Java? LLM can write COBOL just as fine.
- deleted 2mo ago[deleted]
- Leonard_of_Q 2mo agoBecause humans don't want to spend time to learn how to use "dead" languages and humans still play a role in programming. Having LLMs churn out more COBOL instead of Java means more of the code base becomes 'terra incognita' for the involved humans. Learning COBOL isn't that hard - which was one of its stated purposes - but it is still seen as decidedly 'uncool' and 'legacy'. Maybe the increased role of LLMs in coding will change this and make coding in 'legacy' languages 'cool' again just like e.g. the availability of accurate time sources like mobile devices made mechanical watches trendy again or the direct access to music through streaming services made vinyl and now CDs (i.e. 'physical media') regain market share. Maybe. Still not something you want to risk your business on.
- pjmlp 2mo agoThe irony is that talking to AI could not be any closer than COBOL's original goal.
- kome 2mo agoyes i was wondering the same... what's wrong with COBOL? especially in an era of LLM... also, i find COBOL to be way more readable and intuitive than Java.
- TimByte 2mo ago[flagged]
- matsemann 2mo agoBefore, only the senior cobol programmers at the company understood and knew the codebase. Now, no one does. I can see the allure of moving away from a legacy cobol system. But an AI rewrite doesn't actually solve any of the issues with having a legacy cobol codebase. You just have a new system no one knows or understands.
- someone654 2mo ago> Now, no one does. I got a chuckle over this, but in my experience working on legacy systems, this is the case already. Using AI to translate/explain the code brings extra understanding.
- Zenst 2mo agoWhilst true, it's not without its gotchas, and I saw a video the other day that elegantly highlights this: https://www.youtube.com/watch?v=t6MAZW-joCo https://www.youtube.com/watch?v=t6MAZW-joCo. A false sense of confidence is bad; one that is offloaded onto somebody else is another.
- m4rtink 2mo agoCan it really explain the code if the information is not there anymore ? It was in a binder that got shredded 40 years ago by accident. More like it will hallucinate some explanation & cause even more confusion.
- free_bip 2mo agoIf what you want is the "Why", you're right it probably won't get it right. It'll just give you baseless speculation. But if what you want is the "What" or the "How", I think it can be useful.
- sharts 2mo agoThe code is the information. If you need binders of info then it wasn’t codified
- pixel_popping 2mo ago
- h_mirin 2mo ago[dead]
- dzonga 2mo agothere was some London company migrating Cobol to Rust - with 'a.i' assistance. I don't remember their name, hopefully they stayed in business. since probably convincing financial firms that using Rust is better than Java is a mammoth task.
- Taikonerd 2mo agoThis sounds like what the company Mechanical Orchard offers: https://www.mechanical-orchard.com/ https://www.mechanical-orchard.com/ Their pitch is like, "we take your COBOL code, and all the real-world data you can give us. Then we model your COBOL program as a graph, where each node has inputs and outputs. Then we use AI to port each node, making sure that it has the same (input => output) mapping for all the test data you gave us."
- ASalazarMX 2mo ago> for all the test data you gave us This is the secret sauce. You'll be responsible for any defects if you didn't prepare your test data to act like a massive unit test. Wonder how hard is to prepare the ideal test data vs writing the unit tests themselves in COBOL and verifying their translation. I guess if you throw enough data at it, the effort is minimal while the coverage becomes good enough to fix any bugs manually.
- zoom6628 2mo agoAll this ranting about COBOL. Makes me fairly sure AI hasn't tackled RPG3 yet :-D
- prirun 2mo agoI just asked Google AI how LLMs translate COBOL code containing GOTOs into Java, a language that doesn't have GOTO. It gave several examples, one of them being this: COBOL code: PROCESS-DATA. ADD 1 TO COUNTER. DISPLAY COUNTER. IF COUNTER < 10 GO TO PROCESS-DATA. Java translation: while (counter < 10) { counter++; System.out.println(counter); } If "counter" is 10 on entry, the COBOL code prints 11 while the Java code prints nothing. So not only keeping old bugs, but apparently introducing new ones too! I wrote COBOL code for a few years at a job when I was a teenager. What makes legacy COBOL code difficult IMO is it can sometimes be very hard to maintain a mental execution state when examining the code, for several reasons: 1. all variables are global, aka, WORKING-STORAGE. You list all the variables used in the program and they are accessible to the entire program. 2. programs are divided into paragraphs. Control normally flows sequentially top to bottom through paragraphs, one executing after another. Except that the PERFORM statement can drastically alter this normal flow control, and you can't tell by looking at a paragraph how it will be executed. To do that, you have to look at all PERFORM statements that mention this paragraph or any paragraph physically before it, because in COBOL you can say PERFORM PARA1 THROUGH PARA27. If PARA13 is physically between PARA1 and PARA27, it's potentially going to get executed. 3. In true legacy COBOL, before structured COBOL was a thing (circa 1985), the main control flow statement in addition to PERFORM was GOTO. Lots of flag setting, and lots of GOTOs. So in the previous example, you can't tell if PARA13 is going to get executed because any prior statement might be a GOTO PARA14, skipping execution of PARA13. But even worse, you are still under the influence of the PERFORM THRU, so after PARA27 is executed, control returns to the statement following the PERFORM THRU, wherever that was. But if you GOTO PARA27, without being under a PERFORM THRU, then PARA27 is executed followed by the next sequential paragraph. Trying to figure this out statically by looking at the program can be very difficult, especially considering PERFORMs that are nested at runtime but may not be anywhere near each other in a code listing.
- moojacob 2mo agoI have been translating Motorola 6809 assembly into C. It is not that bad. I don’t find AI saves me much time writing the code because understanding it is still the bottleneck, but man it is good at explaining stuff.