3 ms·
From the article: > Although most LLMs are trained on corpora that include several programming languages, we often observe differential performance across lang
by ekidd 2y ago
From the article:
> Although most LLMs are trained on corpora that include several programming languages, we often observe differential performance across languages, especially languages like Rust that are not well represented in popular training datasets.
Interesting! My experience with Rust and Copilot is that Rust is actually a relatively strong language for Copilot, in many ways. I suspect that this might be a mix of different things:
- Is the training set code for Rust relatively high-quality?
- Do strongly-typed languages have a better chance of catching incorrect completions quickly?
- cageface 2y agoMy very limited experience with GPT and Rust is that the code it generates rarely satisfies the borrow checker out of the box. I’ve had much better luck with garbage collected languages.
- ekidd 2y agoInteresting. I find that Copilot is quite good with the borrow checker, at least if it can "see" the signature of the function that it's calling. Sometimes it inserts an extra "&" where it really shouldn't. That's a no-op, clippy catches it, and I take it out. But I tend to write my code in a style that easily passes the borrow checker 99% of the time. I keep mutation very local, and rely on a more "functional" architecture. So Copilot is naturally going to write new code in the same borrow-checker-friendly style, which may keep it out of trouble. When using ChatGPT, I've actually had more luck asking it to write Python, and then to translate the Python into Rust. ChatGPT seems to be better at algorithms in Python, and it's surprisingly good (in my personal experience) at inserting the fine details of Rust types and references when translating.
- cageface 2y agoDoes your more functional style involve making a lot of copies?