4 ms·

To what degree must one consume the "prediction is more fundamental than understanding" koolaid before work like this looks reasonable?

junipertea 6y ago
In Table 2, we observe that TransCoder significantly outperforms both baselines in terms of computational accuracy, with 74.8% and 68.7% in the C++ → Java and Java → Python directions, compared to 61% and 38.3% It beats existing commercial products that try to do the same, so to me it seems it has some value right now.
- qayxc 6y ago
  > It beats existing commercial products that try to do the same, so to me it seems it has some value right now. While I agree with your conclusion, a minor correction is in order here: it beats a (as in a single selected) commercial product and a single free OSS transpiler. The improvement for the commercial product is 61% vs. 75% accuracy (i.e. 23% better) which - while impressive given the unsupervised learning aspect - isn't a game changer (yet!).
whymauri 6y ago
This is a rude comment. The paper has no grander claim of AGI or 'fundamental' understanding. It's a novel algorithm that outperforms existing solutions. Not everything needs to be part of work towards 'understanding.'
- linux-hog 6y ago
  It outperforms existing solutions on a publicly unavailable corpus with no clear way to reproduce the results? Hmm... The authors literally ran a series of experiments and published them without giving a clear way to reproduce the results. How is that useful to anyone except to self advertise? My comment about prediction vs understanding was simply meant to underscore this, albeit it might rub some people the wrong way. If you publish a paper purely about prediction (ie a set of experiments) be prepared to release all pertinent information to reproduce said experiments. If you choose to publish a paper that aims to improve mankind's understanding of the problem at hand, you are intrinsically required to provide all proof in your exposition. Otherwise we might as well just believe everything anyone ever says with no proof.

junipertea 6y ago

In Table 2, we observe that TransCoder significantly outperforms both baselines in terms of computational accuracy, with 74.8% and 68.7% in the C++ → Java and Java → Python directions, compared to 61% and 38.3% It beats existing commercial products that try to do the same, so to me it seems it has some value right now.

qayxc 6y ago
> It beats existing commercial products that try to do the same, so to me it seems it has some value right now. While I agree with your conclusion, a minor correction is in order here: it beats a (as in a single selected) commercial product and a single free OSS transpiler. The improvement for the commercial product is 61% vs. 75% accuracy (i.e. 23% better) which - while impressive given the unsupervised learning aspect - isn't a game changer (yet!).

whymauri 6y ago

This is a rude comment. The paper has no grander claim of AGI or 'fundamental' understanding. It's a novel algorithm that outperforms existing solutions. Not everything needs to be part of work towards 'understanding.'

linux-hog 6y ago
It outperforms existing solutions on a publicly unavailable corpus with no clear way to reproduce the results? Hmm... The authors literally ran a series of experiments and published them without giving a clear way to reproduce the results. How is that useful to anyone except to self advertise? My comment about prediction vs understanding was simply meant to underscore this, albeit it might rub some people the wrong way. If you publish a paper purely about prediction (ie a set of experiments) be prepared to release all pertinent information to reproduce said experiments. If you choose to publish a paper that aims to improve mankind's understanding of the problem at hand, you are intrinsically required to provide all proof in your exposition. Otherwise we might as well just believe everything anyone ever says with no proof.