3 ms·

Next step: minimize the number of iterations by using another language model to write commit messages.

• eru 4y ago
  I wonder whether you should generate the commit messages first, and then base your code changes on that. (Might also be an interesting idea for human programmers. In analogy to Test-Driven-Development, this would be called Commit-Message-Driven-Development.)
  • blackbear_ 4y ago
    I would actually train a model to write the commit message by looking at the latest performance, sort of trying to identify the best next step.
  • gwern 4y ago
    Definitely. It'd be very Decision Transformer and prompt programmingesque: you declare what the high-level intended result should be, with requirements, test cases, examples, and then let the model condition on that and figure out the implementation. Why would you do it backwards? You're going to have to explain what you want anyway, after all. (And of course, you would ideally do it both directions, similar to image/text: neither direction is privileged, you want to be able to both generate images given a description, and generate a description of an image, and both tasks help the other.)
    • eru 4y ago
      Oh, I meant have the computer generate first the commit message, and then have another computer generate the commit. Though you are probably better off creating both at the same time. What you say makes a lot of sense, if you want a human to create the commit message as a sort-of prompt engineering to ask for specific changes.