3 ms·
As far as I can tell, this article gives no definition of what they mean by "successfully translate" in their testing. I would think it should be equivalent to
by bmitc 6y ago
As far as I can tell, this article gives no definition of what they mean by "successfully translate" in their testing. I would think it should be equivalent to the typical use of porting, which usually means you move the codebase from one language to another, reproduce all existing functionality, introduce no new bugs, and maintain relative performance. I am not seeing anything in this article other than their translator translates functions to functions. It also doesn't mention what failure looks like. Does it mean bugs are introduced? Does it mean the translator doesn't return? How are the percentages arrived at?
Also, I don't fully understand the why, which is also not really addressed. What problem is automatically translating a mess of a code base (I am assuming it's a mess, otherwise, why would you want to port it) to another programming language, leaving the mess in place, solving? Isn't the idea of porting to another language usually to help improve the codebase and/or to help integrate it into some other system? Is the idea of automatic translators that it does the "heavy" lifting that is then gone in and cleaned up by experts of the new language? How do you manage the failures, since that would seem you'd still need an expert in the original language?
- jakear 6y agoCould be helpful to integrate existing libraries with languages/runtimes lacking performant ffi.
- nestorD 6y ago> To better measure the performance of TransCoder and other code translation techniques, we’ve created a new metric called computational accuracy, which evaluates whether the hypothesis function generates the same outputs as the reference when given the same inputs. We are also releasing our test set and the scripts and unit tests we used to compute this metric. It reads as if they used a test suite to confirm wether the translation represents the same function.