4 ms·
I can't get Copilot to generate Python that adds numbers together correctly sometimes. Getting an LLM to generate correct, working code for a language that hard
by bastawhiz 3y ago
I can't get Copilot to generate Python that adds numbers together correctly sometimes. Getting an LLM to generate correct, working code for a language that hardly anybody writes anymore is almost assuredly going to lead to failure.
- encoderer 3y agoyeah I agree but when you look at the slope not the y-intercept it’s getting obviously better. one advantage the government would have is training/fine-tuning on a hundred million lines of domain specific cobol.
- bastawhiz 3y agoThe slope doesn't really matter, because the target is "better than a human, and able to identify and fix its own errors". The slope will decrease as you approach this threshold. It's also wildly bad to plan to train and fine tune on code that you know has bugs. Already we have Copilot generating code with trivial vulnerabilities because that's what it's trained on.
- encoderer 3y agoRoughly ~all code has bugs. We were all trained on trivial examples and buggy code - just like llms. Honestly I think editing code will be easier than creating wholly new applications to precise spec.