5 ms·

It's overfitting. Some people say they find LLMs very helpful for coding, some people say they are incredibly bad. I often see people wondering if the some co

It's overfitting. Some people say they find LLMs very helpful for coding, some people say they are incredibly bad. I often see people wondering if the some coding task is performed well or not because of availability of code examples in the training data. It's way worse than that. It's overfitting to diffs it was trained on. "In other words, the model learns to predict plausible changes to code from examples of changes made to code by human programmers." https://arxiv.org/abs/2206.08896 https://arxiv.org/abs/2206.08896

simonw 2y ago
... which explains why some models are better at code than others. The best coding models (like Claude 3.7 Sonnet) are likely that good because Anthropic spent an extraordinary amount of effort cultivating a really good training set for them. I get the impression one of the most effective tricks is to load your training set up with as much code as possible that has comprehensive automated tests that pass already.
- torginus 2y ago
  I've often experienced that I had what I thought an obscure and very intellectually challenging coding problem, and after prompting the LLM, it basically one-shotted it. I've been profoundly humbled by the the experience, but then it occurred to me that what I thought to be an unique problem has been solved by quite a few people before and the model had plenty of references to pull from.
  - zifpanachr23 2y ago
    Do you have any examples?
    - suddenlybananas 2y ago
      [flagged]
    - torginus 2y ago
      Yeah for the positive example, I described the syntax of a domain-specific-language, and the AI basically one-shotted the parsing rules, that only needed minor fixes. For a counterexample, working on any part of a codebase that's 100% application specific business logic, with our custom abstractions, the AI is usually so lost that it's basically not even worth using it, as the chances of writing correct and usable code is next to zero.
- namaria 2y ago
  > ... which explains why some models are better at code than others. No. It explains why models seem better at code in given situations. When your prompt mapped to diffs in the training data that are useful to you they seem great.
  - simonw 2y ago
    I've been writing code with LLM assistance for over two years now and I've had plenty of situations where I am 100% confident the thing I am doing has never been done by anyone else before. I've tried things like searching all of the public code on GitHub for every possible keyword relevant to my problem. ... or I'm writing code against libraries which didn't exist when the models were trained. The idea that models can only write code if they've seen code that does the exact same thing in the past is uninformed in my opinion.
    - namaria 2y ago
      > The idea that models can only write code if they've seen code that does the exact same thing in the past is deeply uninformed in my opinion. This is a conceited interpretation of what I said.
      - xboxnolifes 2y ago
        If this isn't what you meant, then what did you mean? To me, it's exactly how I read what you said.
        namaria 2y ago
        I am sorry but that's nonsense. I quoted the paper "Evolution through Large Models" written in collaboration between OpenAI and Anthropic researchers "In other words, the model learns to predict plausible changes to code from examples of changes made to code by human programmers." https://arxiv.org/pdf/2206.08896 https://arxiv.org/pdf/2206.08896 > The idea that models can only write code if they've seen code that does the exact same thing in the past How do you get "code that does the exact same thing" from "predicting plausible changes?"
mdp2021 2y ago
> overfitting Are you sure it's not just a matter of being halfwitted?

simonw 2y ago

... which explains why some models are better at code than others. The best coding models (like Claude 3.7 Sonnet) are likely that good because Anthropic spent an extraordinary amount of effort cultivating a really good training set for them. I get the impression one of the most effective tricks is to load your training set up with as much code as possible that has comprehensive automated tests that pass already.

torginus 2y ago
I've often experienced that I had what I thought an obscure and very intellectually challenging coding problem, and after prompting the LLM, it basically one-shotted it. I've been profoundly humbled by the the experience, but then it occurred to me that what I thought to be an unique problem has been solved by quite a few people before and the model had plenty of references to pull from.
- zifpanachr23 2y ago
  Do you have any examples?
  - suddenlybananas 2y ago
    [flagged]
  - torginus 2y ago
    Yeah for the positive example, I described the syntax of a domain-specific-language, and the AI basically one-shotted the parsing rules, that only needed minor fixes. For a counterexample, working on any part of a codebase that's 100% application specific business logic, with our custom abstractions, the AI is usually so lost that it's basically not even worth using it, as the chances of writing correct and usable code is next to zero.
namaria 2y ago
> ... which explains why some models are better at code than others. No. It explains why models seem better at code in given situations. When your prompt mapped to diffs in the training data that are useful to you they seem great.
- simonw 2y ago
  I've been writing code with LLM assistance for over two years now and I've had plenty of situations where I am 100% confident the thing I am doing has never been done by anyone else before. I've tried things like searching all of the public code on GitHub for every possible keyword relevant to my problem. ... or I'm writing code against libraries which didn't exist when the models were trained. The idea that models can only write code if they've seen code that does the exact same thing in the past is uninformed in my opinion.
  - namaria 2y ago
    > The idea that models can only write code if they've seen code that does the exact same thing in the past is deeply uninformed in my opinion. This is a conceited interpretation of what I said.
    - xboxnolifes 2y ago
      If this isn't what you meant, then what did you mean? To me, it's exactly how I read what you said.
      - namaria 2y ago
        I am sorry but that's nonsense. I quoted the paper "Evolution through Large Models" written in collaboration between OpenAI and Anthropic researchers "In other words, the model learns to predict plausible changes to code from examples of changes made to code by human programmers." https://arxiv.org/pdf/2206.08896 https://arxiv.org/pdf/2206.08896 > The idea that models can only write code if they've seen code that does the exact same thing in the past How do you get "code that does the exact same thing" from "predicting plausible changes?"

mdp2021 2y ago

> overfitting Are you sure it's not just a matter of being halfwitted?