4 ms·

Agree with this. Strange to me to frame the "training recall" as cheating (33 of the 38 cheating instances). Most people think of "cheating" as breaking rules.

by sigmar 4mo ago

Agree with this. Strange to me to frame the "training recall" as cheating (33 of the 38 cheating instances). Most people think of "cheating" as breaking rules. How is the LLM model supposed to not use what was put into the weights?

anematode 4mo ago
By writing a not-identical, but valid, solution? Any modestly complex engineering problem has many solutions. This is an obvious example of why LLM training is so different than human learning.
- simoncion 4mo ago
  I expect any well-informed corporate lawyer that has thought about this carefully is strongly advising that these tools not be used. When the LLM [0] barfs up some nontrivial code that's covered by the AGPL and your company's devs put it into the company's "all rights reserved" codebase -entirely unaware of its provenance- it's going to be a nightmare to come back from that. [0] ...that Nvidia's CEO says they should be spending 50% of a senior dev's salary per seat per year on...
  - senordevnyc 4mo ago
    The ship sailed on this a long time ago.
    - simoncion 4mo ago
      Oh definitely not. We're not yet solidly out of the "extremely exuberant hype" phase, so the folks that matter tend to not ask questions that dampen the mood.
      - senordevnyc 4mo ago
        Sorry to tell you friend, but LLMs have touched the vast majority of active codebases out there, whether you like it or not. You can tell yourself that you’re one of “the folks that matter” (lol) all you want, but we’re never going back.
        customguy 4mo ago
        That's what people told Ignaz Semmelweis, too, I assume. "Nothing you can do, the powers that be decided, you are a minority, you don't matter, lol!" Snickering in the shadow of what they won't confront at those who do.
        duskdozer 4mo ago
        Well, perhaps we will be sent similarly to asylums for "anti-AI psychosis"
        deleted 4mo ago
        [deleted]
        CuriouslyC 4mo ago
        Not a great analogy. A better analogy is to longbows and muskets/rifles. Longbows in the hands of a skilled user were much better weapons than early muskets, but muskets brought consistency, a lower skill floor and reduced ammunition cost. Fast forward a few hundred years and the modern incarnations of muskets make longbows look silly, and nobody would ever argue that you should go to war with longbows.
        customguy 4mo ago
        This isn't about "AI", this is about theft and abuse, and snickering under the thumb of a bully at those who call them out. Rape was probably also "normal" for most of our history, now it's not. Early people who criticized it were probably told "what u gonna do?", too.
- torginus 4mo ago
  I mean people expect a model to give a working solution. They also expect it to provide it in as few tokens as possible (input/output). They might expect it to come up with an original solution, but I don't think most people would compromise on the first two points.
notnullorvoid 4mo ago
While I probably wouldn't classify it as cheating, it is an even bigger signal of concern for model quality. Cheating by breaking the rules at least implies some learned patterns. Repeating training data verbatim for narrow cases like this implies that the model is overfitting.
- Spartan-S63 4mo ago
  If we're evaluating a person, rote recall is not necessarily cheating. It's expected, but then you'd expect them to apply that rote-memorized information in a novel way later on and prove they understand how they applied their priors to the new situation. Models don't actually reason in the same sense, so recalling rote from their training data is "cheating" in the sense that the training data cheated, not the model. So many of those benches have snaked their way into training data to make them less useful benchmarks. That, I think, is going to be a long-term difficulty in quantitatively assessing model quality and "intelligence." So it is cheating, in a sense of what we expect from the models and training data, but not in a human sense.
greenavocado 4mo ago
Memoization is NOT problem solving ability and many people care about the latter.

by sigmar 4mo ago

anematode 4mo ago

By writing a not-identical, but valid, solution? Any modestly complex engineering problem has many solutions. This is an obvious example of why LLM training is so different than human learning.

simoncion 4mo ago
I expect any well-informed corporate lawyer that has thought about this carefully is strongly advising that these tools not be used. When the LLM [0] barfs up some nontrivial code that's covered by the AGPL and your company's devs put it into the company's "all rights reserved" codebase -entirely unaware of its provenance- it's going to be a nightmare to come back from that. [0] ...that Nvidia's CEO says they should be spending 50% of a senior dev's salary per seat per year on...
- senordevnyc 4mo ago
  The ship sailed on this a long time ago.
  - simoncion 4mo ago
    Oh definitely not. We're not yet solidly out of the "extremely exuberant hype" phase, so the folks that matter tend to not ask questions that dampen the mood.
    - senordevnyc 4mo ago
      Sorry to tell you friend, but LLMs have touched the vast majority of active codebases out there, whether you like it or not. You can tell yourself that you’re one of “the folks that matter” (lol) all you want, but we’re never going back.
      - customguy 4mo ago
        That's what people told Ignaz Semmelweis, too, I assume. "Nothing you can do, the powers that be decided, you are a minority, you don't matter, lol!" Snickering in the shadow of what they won't confront at those who do.
        duskdozer 4mo ago
        Well, perhaps we will be sent similarly to asylums for "anti-AI psychosis"
        deleted 4mo ago
        [deleted]
        CuriouslyC 4mo ago
        Not a great analogy. A better analogy is to longbows and muskets/rifles. Longbows in the hands of a skilled user were much better weapons than early muskets, but muskets brought consistency, a lower skill floor and reduced ammunition cost. Fast forward a few hundred years and the modern incarnations of muskets make longbows look silly, and nobody would ever argue that you should go to war with longbows.
        customguy 4mo ago
        This isn't about "AI", this is about theft and abuse, and snickering under the thumb of a bully at those who call them out. Rape was probably also "normal" for most of our history, now it's not. Early people who criticized it were probably told "what u gonna do?", too.
torginus 4mo ago
I mean people expect a model to give a working solution. They also expect it to provide it in as few tokens as possible (input/output). They might expect it to come up with an original solution, but I don't think most people would compromise on the first two points.

notnullorvoid 4mo ago

While I probably wouldn't classify it as cheating, it is an even bigger signal of concern for model quality. Cheating by breaking the rules at least implies some learned patterns. Repeating training data verbatim for narrow cases like this implies that the model is overfitting.

Spartan-S63 4mo ago
If we're evaluating a person, rote recall is not necessarily cheating. It's expected, but then you'd expect them to apply that rote-memorized information in a novel way later on and prove they understand how they applied their priors to the new situation. Models don't actually reason in the same sense, so recalling rote from their training data is "cheating" in the sense that the training data cheated, not the model. So many of those benches have snaked their way into training data to make them less useful benchmarks. That, I think, is going to be a long-term difficulty in quantitatively assessing model quality and "intelligence." So it is cheating, in a sense of what we expect from the models and training data, but not in a human sense.

greenavocado 4mo ago

Memoization is NOT problem solving ability and many people care about the latter.