4 ms·

Until there is a radically new version of {popular programming language} with breaking changes and no new and correct answers to train on.

• 1123581321 1y ago
  These models can figure out syntax and language features they haven’t seen before. Try it with a few code snippets of your own made-up language. It’s a little freaky.
  • zahlman 1y ago
    They can implicitly assume that your made-up language is designed to be easy to use by native language speakers, and thus apply their existing understanding of "code" to it, sure.
    • 1123581321 1y ago
      Yes, that’s part of it, but it can also correctly reason the language is designed to be hard to understand. If you try this exercise, it’ll list its reasoning for what it thinks each unfamiliar piece of syntax might be, and one of its approaches is to bring in analogues to other languages, including other esoterica attempts. If you give it something more inventive, it’ll reason its way to other academic fields to come up with solutions. It’s a good check to make sure you haven’t accidentally made something too simple or similar to another language, too; that’ll be spotted immediately.
  • int_19h 1y ago
    They can indeed, but 1) this takes up an inordinate amount of context, and 2) the more you force model to think about that, the less effective it is at actually writing code.