2 ms·
> They need to be trained to provide trust. The AlphaCode model generates code, but does not generate documentation or tests that would build trust in the code.
by neilv 4y ago
> They need to be trained to provide trust. The AlphaCode model generates code, but does not generate documentation or tests that would build trust in the code.
I don't understand how this would build trust.
If they generate test cases, you have to validate the test cases.
If they generate documentation, you have to validate the documentation.
For a one-shot drop of code from an unknown party, test cases and docs have been signals that the writer know that's a thing, and they at least put effort into typing it. So maybe we assume more likely that they also used good practices with the code.
But that's signalling to build trust, and adding those to build trust without addressing the reasons we shouldn't have trust in the code (as this article points out) seems like it would be building misplaced trust.
(Though there is some benefit to doc for validation, due to the idea behind the old saying "if your code and documentation disagree, then both are probably wrong".)
- low_tech_love 4y agoI think it’s indeed a bit weird to “require” the AI to generate test cases but not impossible, actually it might be even desirable. The thing is that you don’t need to necessarily validate the test cases themselves, as long as the method to generate them is well known and trustworthy. A good suite of test cases (manually or automatically generated) will never guarantee anything, but it is certainly very useful, and I believe an AI could generate much more and better test cases than we do, as long as it is trained correctly. For example, you can have reinforcement learning with rewards for finding test cases that fail, and it could even be an adversary to the other AI that is generating the code itself…?