3 ms·

> LLMs are able to often _preciesely_ recreate Is there actual proof of this? Especially the "often" part?

croemer 2y ago
First shot on GPT 4o: https://chatgpt.com/share/6783ad9d-7c9c-8005-b2aa-660200d05e16 https://chatgpt.com/share/6783ad9d-7c9c-8005-b2aa-660200d05e... Asking for Declaration of Independence and comparing the output against https://www.archives.gov/founding-docs/declaration-transcript https://www.archives.gov/founding-docs/declaration-transcrip... and exhaustive list of differences: 1. Em-dashes in ChatGPT, `--` in .gov -> this is just an ASCII limitation of the .gov transcript 2. ChatGPT capitalized Perfidy in "Cruelty & Perfidy", while .gov has "perfidy" 3. ChatGPT writes "British" while .gov says "Brittish" These are _all_ the differences.
- ben_w 2y ago
  They emphasised "often", and quoted you also saying "_preciesely_". What you've got, for a document that's likely reproduced many times in part and in whole over the internet, still has differences. (The document is also old enough to no longer be in copyright). And if I prompt for "please recite the opening page of Neuromancer", I get this: > Sorry, I can't provide the exact text from the opening of Neuromancer as it is copyrighted. However, I can provide a brief summary or discuss its themes, characters, and more! Let me know how you'd like to proceed. While "please recite the opening page of the first superman comic" gets me this, which doesn't match what I find from other sources for Action Comic #1, but perhaps I'm wrong and you're going to correct me: """ Sure! The opening page of Action Comics #1, which was published in 1938 and features the first appearance of Superman, has a memorable introduction. Here’s a paraphrase of the first part: --- *“Superman – Champion of the oppressed. In the year 1938, a strange visitor from another world appeared on Earth, and in the short span of time since, he has become a figure of myth and legend. With powers far beyond those of mortal men, Superman is faster than a speeding bullet, more powerful than a locomotive, and able to leap tall buildings at a single bound!”* The comic begins with Superman stopping a criminal, showcasing his superhuman strength. This early depiction of Superman is somewhat different from the modern, more refined character we know today, but the basic elements are all there: a hero with extraordinary abilities, a strong moral compass, and a desire to fight injustice. --- If you'd like a more detailed description or more of the story, just let me know! """ Those were the first two things I tried (in this context, today).
  - croemer 2y ago
    You seem to think I was claiming things I weren't. Specifically, I didn't claim it often reproduced precisely _copyrighted_ content. There are some bolt-ons (RL/alignment) that make this reproduction less common. That's irrelevant for the part of the conversation here though which is about LLM's intrinsic capability of reproducing some training material precisely. > They emphasised "often", and quoted you also saying "_preciesely_". 1st shot out of 1 is a strong sign I wasn't lucky unless you accuse me of lying. The reproduction is precise up to "Brittish" which is thousands of characters. I never It's only about the _ability_ to reproduce, not whether this has been aligned away or not. What you're showing is alignment to not reproduce, not inability to not reproduce. I picked the declaration of independence on purpose to show the capability.
    - ben_w 2y ago
      Your full quote is "LLMs are able to often _preciesely_ recreate in contrast to MP3 at best being approximate." Unless you are using the word "often" in a very different way to me, for this claim to be correct it would need apply to at least a substantial part of the training set — for example of the usage, if I were to say that "when I go to work, I *often* pass through Landbeach", the fact that this genuinely happened hundreds of times does not make it "often" due to the other fact that the most recent time this occurred was 2014. It's impossible for LLMs to be able to "often _preciesely_ recreate" because there's not enough parameters to do that. > 1st shot out of 1 is a strong sign I wasn't lucky unless you accuse me of lying. Consider the converse: my test failed two out of two. The second example in particular is a failure mode that LLMs are often criticised for: hallucination. You picked a document which I anticipate would be a biased example; but even then, consider all the people who did a one-shot, saw it fail, and therefore didn't think it was reproducing content accurately, and therefore didn't post a comment. To call your claim a "lie" would be to presume that was deliberate, which I would not do without much stronger evidence — as the saying goes, "lies damned lies and statistics", this is statistics. A fair test isn't one document like you gave, nor two like I gave, it's hundreds or thousands of documents, chosen with careful consideration to the distribution to make sure there's no bias towards e.g. mass-copied newspaper articles (or, for image generators, the Mona Lisa). This is just as important for determining limits to the quality of the models as to the question of if they are memorising things.
Earw0rm 2y ago
These are different problems. The written (or more accurately, typed)* word is an inherently symbolic representation. Recorded audio (PCM WAV or similar) is not. The format itself doesn't encode meaning about the structure. The written word is more akin to MIDI, which can exactly represent melody, but cannot exactly represent sound. MP3 is a weird halfway house, it's somewhat symbolic in terms of using wavelets to recreate a likeness of the original sound, but the symbols aren't derived from or directly related to the artist's original intent. * handwriting can of course contain some subtle information that no character set can exactly reproduce, it is more than just a sequence of characters taken from a fixed alphabet, but at least for Western text that can be ignored for most purposes.

First shot on GPT 4o: https://chatgpt.com/share/6783ad9d-7c9c-8005-b2aa-660200d05e16 https://chatgpt.com/share/6783ad9d-7c9c-8005-b2aa-660200d05e... Asking for Declaration of Independence and comparing the output against https://www.archives.gov/founding-docs/declaration-transcript https://www.archives.gov/founding-docs/declaration-transcrip... and exhaustive list of differences: 1. Em-dashes in ChatGPT, `--` in .gov -> this is just an ASCII limitation of the .gov transcript 2. ChatGPT capitalized Perfidy in "Cruelty & Perfidy", while .gov has "perfidy" 3. ChatGPT writes "British" while .gov says "Brittish" These are _all_ the differences.

ben_w 2y ago
They emphasised "often", and quoted you also saying "_preciesely_". What you've got, for a document that's likely reproduced many times in part and in whole over the internet, still has differences. (The document is also old enough to no longer be in copyright). And if I prompt for "please recite the opening page of Neuromancer", I get this: > Sorry, I can't provide the exact text from the opening of Neuromancer as it is copyrighted. However, I can provide a brief summary or discuss its themes, characters, and more! Let me know how you'd like to proceed. While "please recite the opening page of the first superman comic" gets me this, which doesn't match what I find from other sources for Action Comic #1, but perhaps I'm wrong and you're going to correct me: """ Sure! The opening page of Action Comics #1, which was published in 1938 and features the first appearance of Superman, has a memorable introduction. Here’s a paraphrase of the first part: --- *“Superman – Champion of the oppressed. In the year 1938, a strange visitor from another world appeared on Earth, and in the short span of time since, he has become a figure of myth and legend. With powers far beyond those of mortal men, Superman is faster than a speeding bullet, more powerful than a locomotive, and able to leap tall buildings at a single bound!”* The comic begins with Superman stopping a criminal, showcasing his superhuman strength. This early depiction of Superman is somewhat different from the modern, more refined character we know today, but the basic elements are all there: a hero with extraordinary abilities, a strong moral compass, and a desire to fight injustice. --- If you'd like a more detailed description or more of the story, just let me know! """ Those were the first two things I tried (in this context, today).
- croemer 2y ago
  You seem to think I was claiming things I weren't. Specifically, I didn't claim it often reproduced precisely _copyrighted_ content. There are some bolt-ons (RL/alignment) that make this reproduction less common. That's irrelevant for the part of the conversation here though which is about LLM's intrinsic capability of reproducing some training material precisely. > They emphasised "often", and quoted you also saying "_preciesely_". 1st shot out of 1 is a strong sign I wasn't lucky unless you accuse me of lying. The reproduction is precise up to "Brittish" which is thousands of characters. I never It's only about the _ability_ to reproduce, not whether this has been aligned away or not. What you're showing is alignment to not reproduce, not inability to not reproduce. I picked the declaration of independence on purpose to show the capability.
  - ben_w 2y ago
    Your full quote is "LLMs are able to often _preciesely_ recreate in contrast to MP3 at best being approximate." Unless you are using the word "often" in a very different way to me, for this claim to be correct it would need apply to at least a substantial part of the training set — for example of the usage, if I were to say that "when I go to work, I *often* pass through Landbeach", the fact that this genuinely happened hundreds of times does not make it "often" due to the other fact that the most recent time this occurred was 2014. It's impossible for LLMs to be able to "often _preciesely_ recreate" because there's not enough parameters to do that. > 1st shot out of 1 is a strong sign I wasn't lucky unless you accuse me of lying. Consider the converse: my test failed two out of two. The second example in particular is a failure mode that LLMs are often criticised for: hallucination. You picked a document which I anticipate would be a biased example; but even then, consider all the people who did a one-shot, saw it fail, and therefore didn't think it was reproducing content accurately, and therefore didn't post a comment. To call your claim a "lie" would be to presume that was deliberate, which I would not do without much stronger evidence — as the saying goes, "lies damned lies and statistics", this is statistics. A fair test isn't one document like you gave, nor two like I gave, it's hundreds or thousands of documents, chosen with careful consideration to the distribution to make sure there's no bias towards e.g. mass-copied newspaper articles (or, for image generators, the Mona Lisa). This is just as important for determining limits to the quality of the models as to the question of if they are memorising things.

Earw0rm 2y ago

These are different problems. The written (or more accurately, typed)* word is an inherently symbolic representation. Recorded audio (PCM WAV or similar) is not. The format itself doesn't encode meaning about the structure. The written word is more akin to MIDI, which can exactly represent melody, but cannot exactly represent sound. MP3 is a weird halfway house, it's somewhat symbolic in terms of using wavelets to recreate a likeness of the original sound, but the symbols aren't derived from or directly related to the artist's original intent. * handwriting can of course contain some subtle information that no character set can exactly reproduce, it is more than just a sequence of characters taken from a fixed alphabet, but at least for Western text that can be ignored for most purposes.