6 ms·

IBM Watson's Next Challenge: Modernize Legacy Code

35 points by headalgorithm 6y ago

skywhopper 6y ago
"There was no more clear a demonstration of AI’s capability than when IBM had its Watson supercomputer defeat all the human champions in the game show Jeopardy." The only thing that event proved was that a computer can press a button faster than a human can. The system was getting a digital signal when the signaling devices were unlocked, so unless Watson was unsure of the correct response, it could _always_ beat the humans to the punch. It also had the advantage of parsing the text version of the answers, rather than the audio of Alex speaking in real time, meaning it had a head start to begin with. Unlike the similar Chess and Go challenges, the Jeopardy event was incredibly unfair to the humans involved.
- bsaul 6y ago
  I haven't watched the whole game, but watson jeopardy was to me the most amazing things i've witnessed in AI, and has only been superseded by GPT-3. And this wasn't at all due to its ability to answer fast, but rather to answer correctly to questions that required not only raw knowledge, but beeing able to understand pun, double-meaning, references, etc... This seems to me much more impressive than playing go or chess..
- thrower123 6y ago
  That's pretty much the only success Watson has to point at. The cognitive services stuff that they sell is utterly commonplace crap.
JacobiX 6y ago
The idea of automagic conversion of legacy code is great and the market is huge. But then Watson failed to deliver in the past and I doubt that this time it will live up to its claim.
- Tokelin 6y ago
  Do you have any links on the past failures? I of course heard some of the attempts, but not the failures.
  - mikequinlan 6y ago
    "How IBM Watson Overpromised and Underdelivered on AI Health Care" https://spectrum.ieee.org/biomedical/diagnostics/how-ibm-watson-overpromised-and-underdelivered-on-ai-health-care https://spectrum.ieee.org/biomedical/diagnostics/how-ibm-wat...
- hinkley 6y ago
  I was expecting to have a few more years yet before AI beat a Go grandmaster, but that ship already sailed. For Watson to be able to convert code, it's going to have to be able to determine the intention of each block of code, and know how to translate those intention into the new code. You will, of course, end up with "You can write Fortran in any language" output, but if they do manage to succeed, then this will be the first time in my career that I've ever been remotely worried about the prospect of computers taking over the job of programming. In all likelihood however, it'll turn the process of creating software into a conversation, not unlike the negotiations between a builder, the customer, and the architect. Honestly though, if we'd have ever cracked the modularity problem, we'd just be loading this old code as opaque blobs, possibly through a cross-compiler. But we never succeeded, and over a time span of decades, lots of coding conventions start to resemble premature optimization, making it painful for modern eyes to look at this old code. In fact the latter may be the stumbling block for automated translation. If the old code wasn't so 'quirky' (read: byzantine as hell) we would have ported it already.
  - nullsense 6y ago
    I don't know about IBM but Microsoft could probably do it now they own GitHub.
- bigbubba 6y ago
  'Watson' is just a brand that IBM uses for a variety of unrelated bullshit.
Flankk 6y ago
It looks like you're writing a program. Would you like help? * Get help with writing the program. * Enable autocorrect and code prediction. * Translate the program to COBOL. * Write the program without help. [ ] Pin Watson to taskbar.
arcticbull 6y ago
I suspect this kind of task is actually pretty well suited to ML. It's taking a broadly accepted set of patterns at one time and mapping it onto a new set of broadly accepted patterns, plus or minus, with a margin of error. A proper test suite (haha) and some manual review would likely show decent results IMO.
hinkley 6y ago
Has anyone done an analysis of "the google translate game" with an IR like wasm or the JVM to decompile code from one language into another language? I'm curious to what degree their solution will end up being isomorphic (in the graph theory sense) to that sort of exercise.
ekiwi 6y ago
It seems like the real challenge would not be the translation of individual code snippets to a newer programming language, but more about adapting the code base to a different library eco-system. For example you would have to understand the exact behavior of the Cobol and Java standard libraries if you wanted to correctly translate code between the two languages. You might also have implicit dependencies in the legacy code on the schema of some external database, all of which would need to be taken into account when migrating. Also, tests might be missing, so how do you even know that your migration was successful?
2OEH8eoCRo0 6y ago
That's pretty rich coming from IBM. I spend my day on AIX 6 using shit like Clearcase and DOORs. I wouldn't wish this on my worst enemy.
protomyth 6y ago
Is there a repository of a lot of example COBOL code? IBM might actually be the only company capable of doing automatic translation because they probably have access to enough code to actually test their methodology on. Also, lost in the conversation is the other language that keeps showing up: RPG. It was and is used in at least the AS/400 - iSeries arena. Lastly, I would imagine translated code would actually fit into something like orchestrations in BizTalk or related tool. I get the feeling if you did an actual analysis, some form of workflow language would be developed and be a better fit.
- kevin_thibedeau 6y ago
  They already have a tool to convert COBOL to Java. It doesn't take ML to do this job.
  - protomyth 6y ago
    Apparently the market does not consider it a fully working option if they went for this initiative. I'm doubtful it requires ML, but I truly wonder if anyone other than IBM has access to enough COBOL code to make a go of it.
deleted 6y ago
[deleted]
pronoiac 6y ago
I've seen a successful effort to turn COBOL into Java, from 2018: Phase 1 (18 months): COBOL-to-Java code automated refactoring to x86/RHEL platform. This was the most complex and risky phase. Phase 2 (12 months): Code advanced refactoring to remove COBOL design overtones Phase 3 (3 months): Infrastructure moved to AWS GovCloud (US) Article: https://aws.amazon.com/blogs/apn/automated-refactoring-of-a-u-s-department-of-defense-mainframe-to-aws/ https://aws.amazon.com/blogs/apn/automated-refactoring-of-a-... Discussion here: https://news.ycombinator.com/item?id=19839563 https://news.ycombinator.com/item?id=19839563
fmakunbound 6y ago
This confirms my suspicions that IBM Watson is on the bullshit end of the AI spectrum.
aaron695 6y ago
Why does HN continually talk about technology that's decades old yet we have no proof that it works. Has Watson ever done anything? Is it used anywhere? I don't want a PR puff piece. Don't want theories. Don't want, I used it for a week. I don;t want to know what you claim it can do. I want an article or youtube from an independent person who's used it in the field for a year and confirms it works at all. How about we start there. And FYI Watson cheated at Jeopardy. It couldn't even do that properly.
- eska 6y ago
  This article answers all of that with an emphatic "no" https://spectrum.ieee.org/biomedical/diagnostics/how-ibm-watson-overpromised-and-underdelivered-on-ai-health-care https://spectrum.ieee.org/biomedical/diagnostics/how-ibm-wat...
xvilka 6y ago
If someone wants to help with more COBOL code analysis adding[1] it's support in GitHub Semantic would be awesome. The first step would be to finish the Tree-Sitter parser[2] for it. Then it would require to add support in Haskell bindings [3]. [1] https://github.com/github/semantic/issues/452 https://github.com/github/semantic/issues/452 [2] https://github.com/Neppord/tree-sitter-cobol https://github.com/Neppord/tree-sitter-cobol [3] https://github.com/tree-sitter/haskell-tree-sitter https://github.com/tree-sitter/haskell-tree-sitter

35 points by headalgorithm 6y ago

"There was no more clear a demonstration of AI’s capability than when IBM had its Watson supercomputer defeat all the human champions in the game show Jeopardy." The only thing that event proved was that a computer can press a button faster than a human can. The system was getting a digital signal when the signaling devices were unlocked, so unless Watson was unsure of the correct response, it could _always_ beat the humans to the punch. It also had the advantage of parsing the text version of the answers, rather than the audio of Alex speaking in real time, meaning it had a head start to begin with. Unlike the similar Chess and Go challenges, the Jeopardy event was incredibly unfair to the humans involved.

bsaul 6y ago
I haven't watched the whole game, but watson jeopardy was to me the most amazing things i've witnessed in AI, and has only been superseded by GPT-3. And this wasn't at all due to its ability to answer fast, but rather to answer correctly to questions that required not only raw knowledge, but beeing able to understand pun, double-meaning, references, etc... This seems to me much more impressive than playing go or chess..
thrower123 6y ago
That's pretty much the only success Watson has to point at. The cognitive services stuff that they sell is utterly commonplace crap.

JacobiX 6y ago

The idea of automagic conversion of legacy code is great and the market is huge. But then Watson failed to deliver in the past and I doubt that this time it will live up to its claim.

Tokelin 6y ago
Do you have any links on the past failures? I of course heard some of the attempts, but not the failures.
- mikequinlan 6y ago
  "How IBM Watson Overpromised and Underdelivered on AI Health Care" https://spectrum.ieee.org/biomedical/diagnostics/how-ibm-watson-overpromised-and-underdelivered-on-ai-health-care https://spectrum.ieee.org/biomedical/diagnostics/how-ibm-wat...
hinkley 6y ago
I was expecting to have a few more years yet before AI beat a Go grandmaster, but that ship already sailed. For Watson to be able to convert code, it's going to have to be able to determine the intention of each block of code, and know how to translate those intention into the new code. You will, of course, end up with "You can write Fortran in any language" output, but if they do manage to succeed, then this will be the first time in my career that I've ever been remotely worried about the prospect of computers taking over the job of programming. In all likelihood however, it'll turn the process of creating software into a conversation, not unlike the negotiations between a builder, the customer, and the architect. Honestly though, if we'd have ever cracked the modularity problem, we'd just be loading this old code as opaque blobs, possibly through a cross-compiler. But we never succeeded, and over a time span of decades, lots of coding conventions start to resemble premature optimization, making it painful for modern eyes to look at this old code. In fact the latter may be the stumbling block for automated translation. If the old code wasn't so 'quirky' (read: byzantine as hell) we would have ported it already.
- nullsense 6y ago
  I don't know about IBM but Microsoft could probably do it now they own GitHub.
bigbubba 6y ago
'Watson' is just a brand that IBM uses for a variety of unrelated bullshit.

Flankk 6y ago

It looks like you're writing a program. Would you like help? * Get help with writing the program. * Enable autocorrect and code prediction. * Translate the program to COBOL. * Write the program without help. [ ] Pin Watson to taskbar.

arcticbull 6y ago

I suspect this kind of task is actually pretty well suited to ML. It's taking a broadly accepted set of patterns at one time and mapping it onto a new set of broadly accepted patterns, plus or minus, with a margin of error. A proper test suite (haha) and some manual review would likely show decent results IMO.

hinkley 6y ago

Has anyone done an analysis of "the google translate game" with an IR like wasm or the JVM to decompile code from one language into another language? I'm curious to what degree their solution will end up being isomorphic (in the graph theory sense) to that sort of exercise.

ekiwi 6y ago

It seems like the real challenge would not be the translation of individual code snippets to a newer programming language, but more about adapting the code base to a different library eco-system. For example you would have to understand the exact behavior of the Cobol and Java standard libraries if you wanted to correctly translate code between the two languages. You might also have implicit dependencies in the legacy code on the schema of some external database, all of which would need to be taken into account when migrating. Also, tests might be missing, so how do you even know that your migration was successful?

2OEH8eoCRo0 6y ago

That's pretty rich coming from IBM. I spend my day on AIX 6 using shit like Clearcase and DOORs. I wouldn't wish this on my worst enemy.

protomyth 6y ago

Is there a repository of a lot of example COBOL code? IBM might actually be the only company capable of doing automatic translation because they probably have access to enough code to actually test their methodology on. Also, lost in the conversation is the other language that keeps showing up: RPG. It was and is used in at least the AS/400 - iSeries arena. Lastly, I would imagine translated code would actually fit into something like orchestrations in BizTalk or related tool. I get the feeling if you did an actual analysis, some form of workflow language would be developed and be a better fit.

kevin_thibedeau 6y ago
They already have a tool to convert COBOL to Java. It doesn't take ML to do this job.
- protomyth 6y ago
  Apparently the market does not consider it a fully working option if they went for this initiative. I'm doubtful it requires ML, but I truly wonder if anyone other than IBM has access to enough COBOL code to make a go of it.

deleted 6y ago

[deleted]

pronoiac 6y ago

I've seen a successful effort to turn COBOL into Java, from 2018: Phase 1 (18 months): COBOL-to-Java code automated refactoring to x86/RHEL platform. This was the most complex and risky phase. Phase 2 (12 months): Code advanced refactoring to remove COBOL design overtones Phase 3 (3 months): Infrastructure moved to AWS GovCloud (US) Article: https://aws.amazon.com/blogs/apn/automated-refactoring-of-a-u-s-department-of-defense-mainframe-to-aws/ https://aws.amazon.com/blogs/apn/automated-refactoring-of-a-... Discussion here: https://news.ycombinator.com/item?id=19839563 https://news.ycombinator.com/item?id=19839563

fmakunbound 6y ago

This confirms my suspicions that IBM Watson is on the bullshit end of the AI spectrum.

aaron695 6y ago

Why does HN continually talk about technology that's decades old yet we have no proof that it works. Has Watson ever done anything? Is it used anywhere? I don't want a PR puff piece. Don't want theories. Don't want, I used it for a week. I don;t want to know what you claim it can do. I want an article or youtube from an independent person who's used it in the field for a year and confirms it works at all. How about we start there. And FYI Watson cheated at Jeopardy. It couldn't even do that properly.

eska 6y ago
This article answers all of that with an emphatic "no" https://spectrum.ieee.org/biomedical/diagnostics/how-ibm-watson-overpromised-and-underdelivered-on-ai-health-care https://spectrum.ieee.org/biomedical/diagnostics/how-ibm-wat...

xvilka 6y ago

If someone wants to help with more COBOL code analysis adding[1] it's support in GitHub Semantic would be awesome. The first step would be to finish the Tree-Sitter parser[2] for it. Then it would require to add support in Haskell bindings [3]. [1] https://github.com/github/semantic/issues/452 https://github.com/github/semantic/issues/452 [2] https://github.com/Neppord/tree-sitter-cobol https://github.com/Neppord/tree-sitter-cobol [3] https://github.com/tree-sitter/haskell-tree-sitter https://github.com/tree-sitter/haskell-tree-sitter