4 ms·

>using private, internal LLMs that are probably fine-tuned with Meta’s codebase. what does this mean ? i would have thought they would simply use codellama. is

• mdaniel 3y ago
  I am not privy to Meta's situation, and to be honest don't have any hand-to-hand experience with finetuned code LLMs, but my mental model is that any corpus of rules will always produce better outcomes when taking local norms into consideration. It's a silly one, but code formatting styles is a perfect example: a hypothetical Google one that has been finetuned on the Google codebase will more easily produce code that already follows their documented code style merely because it has seen more "already correct" examples. Variable nomenclature, method ordering, what things are versus are not documented, any nullability annotations (where appropriate to the language), etc are more that spring to mind More germane to this discussion, I would guess a locally tuned model will also recognize the kinds of things they care about testing, up to and including spotting any bug fix tests that were hard won and can carry forward in any such generated tests for future code