3 ms·

From my intuition (and experience) with LLMs, they make more of what is in the context. If you use a language that is often used for business CRUD software, the

From my intuition (and experience) with LLMs, they make more of what is in the context. If you use a language that is often used for business CRUD software, the assistant will be great it giving you business CRUD software, of typical business CRUD software quality. Using copilot for work, my experience is that it will happily offer ugly but context-plausible solutions (that often don't even work), but if you stop and ask "maybe there's a better way to do this", it often will suggest good simplifications. Especially if you give it a hint ("Maybe it's worth trying...") I haven't really used copilot with, say, Rust or Haskell, but if the average quality of code is better in those languages, maybe it will take less coaxing to actually suggest good code? Either way, LLMs are great at generalizing between human languages, so I expect they transfer a lot between programming languages too. I think your language has to be pretty weird for copilot to not get a grip on it with enough context.

bluGill 3y ago
I had someone submit a C++ code review where AI was used to write the code. It looked like C++98: new and delete all over, even though modern c++ is 12 years old by now. There are a lot of other things that scream I learned to program C++ in 1997 and see no reason to update my skills to anything newer. Modern C++ is really a different language and yet there is a lot of not modern code out there that AIs don't know is bad.
- vintermann 3y ago
  It's not that they don't know it's bad, it's that fundamentally, they're not trying to help you. They're trying to match a probability distribution. And if there's lot of 1997-quality C++ code out there (there is!), that's highly likely what you're going to get. If you provide high quality context, it's more likely to produce high quality code too - but it can also spiral into creating bad code if it accidentally starts off on the wrong foot. The more bad code you have, the more bad code it suggests. That's why I think it's very important to clean up the code it suggests, even if it's just cosmetically.
- bsder 3y ago
  Unsurprising as StackOverflow has exactly the same problem. There are lots of answers that are flat out wrong 5+ years after they were answered. There is no way to go back and "unmark" that answer. > Modern C++ is really a different language and yet there is a lot of not modern code out there that AIs don't know is bad. This is a human problem and why I won't use C++ anymore. Instead of "modern C++" we needed a new language to replace C++ and leave the old nastiness behind. We'll see if any of the new replacements like Carbon get any traction.
  - Arnavion 3y ago
    You can downvote wrong SO answers, and there is a way to sort that gives new votes more priority.
- AlchemistCamp 3y ago
  I see this issue with JavaScript too, sometimes.

bluGill 3y ago

I had someone submit a C++ code review where AI was used to write the code. It looked like C++98: new and delete all over, even though modern c++ is 12 years old by now. There are a lot of other things that scream I learned to program C++ in 1997 and see no reason to update my skills to anything newer. Modern C++ is really a different language and yet there is a lot of not modern code out there that AIs don't know is bad.

vintermann 3y ago
It's not that they don't know it's bad, it's that fundamentally, they're not trying to help you. They're trying to match a probability distribution. And if there's lot of 1997-quality C++ code out there (there is!), that's highly likely what you're going to get. If you provide high quality context, it's more likely to produce high quality code too - but it can also spiral into creating bad code if it accidentally starts off on the wrong foot. The more bad code you have, the more bad code it suggests. That's why I think it's very important to clean up the code it suggests, even if it's just cosmetically.
bsder 3y ago
Unsurprising as StackOverflow has exactly the same problem. There are lots of answers that are flat out wrong 5+ years after they were answered. There is no way to go back and "unmark" that answer. > Modern C++ is really a different language and yet there is a lot of not modern code out there that AIs don't know is bad. This is a human problem and why I won't use C++ anymore. Instead of "modern C++" we needed a new language to replace C++ and leave the old nastiness behind. We'll see if any of the new replacements like Carbon get any traction.
- Arnavion 3y ago
  You can downvote wrong SO answers, and there is a way to sort that gives new votes more priority.
AlchemistCamp 3y ago
I see this issue with JavaScript too, sometimes.