4 ms·

> Of course, it's random and by chance - tokens are literally sampled from a predicted probability distribution. Picking randomly out of a non-random distribut

> Of course, it's random and by chance - tokens are literally sampled from a predicted probability distribution. Picking randomly out of a non-random distribution doesn't give you a random result. And you don't have to use randomness to pick tokens. > If you mean chance=uniform probability you have to articulate that. Don't be a pain. This isn't about uniform distribution versus other generic distribution. This is about the very elaborate calculations that exist on a per-token basis specifically to make the next token plausible and exclude the vast majority of tokens. > My point is that matching sequence length and distributional similarity are both quantifiable. Where do you draw the line? Any reasonable line has examples that cross it from many models. Very long segments that can be reproduced. Because many models were trained in a way that overfits certain pieces of code and basically causes them to be memorized.

kelseyfrog 11mo ago
> Very long segments that can be reproduced Right, and very short segments can also be reproduced. Let's say that "//" is an arbitrarily short segment that matches some source code. This is trivially true. I could write "//" on a coin and half the time it's going to land "//". Let's agree that's a lower bound. I don't even disagree that there is an upper bound. Surely reproducing a repo in its entirety is a match. So there must exist a line between the two that divides too short and too long. Again, by what basis do you draw a line between a 1 token reproduction and a 1,000 token reproduction? 5, 10, 20, 50? How is it justified? Purely "reasonableness"?
- Dylan16807 11mo ago
  Why do you want me to pick a number so bad? There are very very long examples that are clearly memorization. Like, if a model was trained on all the code in the world except that specific example, the chance of it producing that snippet is less than a billionth of a billionth of a percent. But that snippet got fed in so many times it gets treated like a standard idiom and memorized in full. Is that a clear enough threshold for you? I don't know where the exact line is, but I know it's somewhere inside this big ballpark, and there are examples that go past the entire ballpark. I don't care where specifically the bound is.
  - kelseyfrog 11mo ago
    [flagged]
    - Dylan16807 11mo ago
      That is not good faith, my dude.
      - kelseyfrog 11mo ago
        It sounds like you do care where it is then, no?
        Dylan16807 11mo ago
        I care that it's within the ballpark I spent considerable detail explaining. I don't care where inside the ballpark it is. You gave an exaggerated upper limit, so extreme there's no ambiguity, of "entire repo". I gave my own exaggerated upper limit, so extreme there's no ambiguity. And mine has examples of it actually happening. Incidents so extreme they're clear violations. Maybe an analogy will help: The point at which a collection of sand grains becomes a heap is ambiguous. But when we have documented incidents involving a kilogram or more of sand in a conical shape, we can skip refining the threshold and simply declare that yes heaps are real. Incidents of major LLMs copying code, in a way that is full-on memorization and not just recreating things via chance and general code knowledge, are real. You're the only person I've seen ever imply that true copying incidents are a statistical illusion, akin to a random die. Normally the debate is over how often and impactful they are, who is going to be held responsible, and what to do about them.
        deleted 11mo ago
        [deleted]
        kelseyfrog 11mo ago
        To recap, the original statement was, "Llm's do not verbatim disgorge chunks of the code they were trained on." We obviously both disagree with it. While you keep trying to drag this toward an upper bound, I'm trying to illustrate that a coin with "//" reproduces a chunk of code. Again. I don't see much of a disagreement on that point either. What I continue to fail to elicit from you is the salient difference between the two. I'm trying to find a scissor that distills your vibes into a consistent rule and each time it's the rebutted like I'm trying to make an argument. If your system doesn't have consistency, just say so.

kelseyfrog 11mo ago

> Very long segments that can be reproduced Right, and very short segments can also be reproduced. Let's say that "//" is an arbitrarily short segment that matches some source code. This is trivially true. I could write "//" on a coin and half the time it's going to land "//". Let's agree that's a lower bound. I don't even disagree that there is an upper bound. Surely reproducing a repo in its entirety is a match. So there must exist a line between the two that divides too short and too long. Again, by what basis do you draw a line between a 1 token reproduction and a 1,000 token reproduction? 5, 10, 20, 50? How is it justified? Purely "reasonableness"?

Dylan16807 11mo ago
Why do you want me to pick a number so bad? There are very very long examples that are clearly memorization. Like, if a model was trained on all the code in the world except that specific example, the chance of it producing that snippet is less than a billionth of a billionth of a percent. But that snippet got fed in so many times it gets treated like a standard idiom and memorized in full. Is that a clear enough threshold for you? I don't know where the exact line is, but I know it's somewhere inside this big ballpark, and there are examples that go past the entire ballpark. I don't care where specifically the bound is.
- kelseyfrog 11mo ago
  [flagged]
  - Dylan16807 11mo ago
    That is not good faith, my dude.
    - kelseyfrog 11mo ago
      It sounds like you do care where it is then, no?
      - Dylan16807 11mo ago
        I care that it's within the ballpark I spent considerable detail explaining. I don't care where inside the ballpark it is. You gave an exaggerated upper limit, so extreme there's no ambiguity, of "entire repo". I gave my own exaggerated upper limit, so extreme there's no ambiguity. And mine has examples of it actually happening. Incidents so extreme they're clear violations. Maybe an analogy will help: The point at which a collection of sand grains becomes a heap is ambiguous. But when we have documented incidents involving a kilogram or more of sand in a conical shape, we can skip refining the threshold and simply declare that yes heaps are real. Incidents of major LLMs copying code, in a way that is full-on memorization and not just recreating things via chance and general code knowledge, are real. You're the only person I've seen ever imply that true copying incidents are a statistical illusion, akin to a random die. Normally the debate is over how often and impactful they are, who is going to be held responsible, and what to do about them.
        deleted 11mo ago
        [deleted]
        kelseyfrog 11mo ago
        To recap, the original statement was, "Llm's do not verbatim disgorge chunks of the code they were trained on." We obviously both disagree with it. While you keep trying to drag this toward an upper bound, I'm trying to illustrate that a coin with "//" reproduces a chunk of code. Again. I don't see much of a disagreement on that point either. What I continue to fail to elicit from you is the salient difference between the two. I'm trying to find a scissor that distills your vibes into a consistent rule and each time it's the rebutted like I'm trying to make an argument. If your system doesn't have consistency, just say so.