3 ms·
If a new programming language is created in the future, how long would it take the LLM to become proficient in it for everyday use?
by deadbabe 2y ago
If a new programming language is created in the future, how long would it take the LLM to become proficient in it for everyday use?
- hansvm 2y agoIME, not long. ChatGPT (v3) did poorly with Zig when it first came out, despite some Zig in its training code, and despite equivalent Python tasks being handled trivially. I worked around that for a bit by including a moderately lengthy prompt prefix with example Zig code and descriptions of interesting language features, which worked well for bulk boilerplate at the time. A year later, the model was updated (presumably with additional GH Zig source as training data) and works fine without any prompt prefix strategy, at least for the simple sorts of tasks I'd use an LLM for.
- thorum 2y agoThe goal is for the LLMs we have to be so good at long context tasks and reasoning about code that you can simply paste the entire documentation for your new language and they’ll figure it out. Once you have that working, you can generate synthetic examples to train the next version of the LLM, or for open source coding models, a small LoRA file that anyone can download to add support to their model of choice.
- gadflyinyoureye 2y agoA question might be, will there be new languages? If we start using AI so much that humans lose there job, where will the examples come from on which we’ll train the AI?
- cpeterso 2y agoWhat might a programming language that was designed to be more easily or correctly generated by LLMs look like? Probably something Lispy. Without an organic corpus of existing code, it would need to be trained entirely on synthetic examples.
- blharr 2y agoLLMs are probably the worst at programming in Lisp in my experience though