3 ms·
I think you could use linting tools that check for things like this and filter the output. Or use outputs that fail the lint as negative training examples.
by fshbbdssbbgdd 5y ago
I think you could use linting tools that check for things like this and filter the output. Or use outputs that fail the lint as negative training examples.
- bee_rider 5y agoI don't know anything about Copilot's design, but surely they passed all the code they fed it in the training stage through some pretty strict linters, right? I mean that's just common sense...
- PeterisP 5y agoNo, definitely not. The Copilot essentially tries to use "all code in the wild as it is". In the Copilot design, code blocks are essentially treated as independent units of text, not attempting to put different files in context of a larger project - IIRC they were taking size-limited chunks of files as the independent units - much less attempt to build or lint the projects, because if your basic unit is 100 lines from the middle of a file, it obviously won't even compile (because of external references), and there's no straightforward way to handle deploying a working build/test environment (w. dependencies) automatically for a million projects. Furthermore, what's your expectation of the proportion of code on Github that would pass "through some pretty strict linters" ? My guess is that for serious, widely used and actively maintained projects it's perhaps 5% (almost no projects pass linters without something getting flagged, but there's a small minority of projects who actively fight all the "identified lint"), and for random code it's roughly 0%, with only specific teaching/toy projects passing.