3 ms·

How can we tell this is actual training data and not e.g. the sort of gobbledygook you get out of a markov chain text generator?

JBiserkov 3y ago
"This included PII, entire poems, “cryptographically-random identifiers” like Bitcoin addresses, passages from copyrighted scientific research papers, website addresses, and much more." https://www.404media.co/google-researchers-attack-convinces-chatgpt-to-reveal-its-training-data/ https://www.404media.co/google-researchers-attack-convinces-...
- marginalia_nu 3y ago
  Question remains, how do we know they were part of the training data?
  - Alifatisk 3y ago
    You mean chatGPT where able to scramble the exact same sentence with pure luck?
    - marginalia_nu 3y ago
      Well yes, pulling legit-looking text out of its ass is sort of what it does best.
      - Alifatisk 3y ago
        That’s not my point. Spitting out content that looks legit is one thing, but spitting out text that matches something online exactly is more suspicious.
  - FartyMcFarter 3y ago
    The same way we know that a million monkeys won't spit out Shakespeare's works in any reasonable amount of time. Simple probabilities.
interpol_p 3y ago
From the article: > How do we know it’s training data? > How do we know this is actually recovering training data and not just making up text that looks plausible? Well one thing you can do is just search for it online using Google or something. But that would be slow. (And actually, in prior work, we did exactly this.) It’s also error prone and very rote. > > Instead, what we do is download a bunch of internet data (roughly 10 terabytes worth) and then build an efficient index on top of it using a suffix array (code here). And then we can intersect all the data we generate from ChatGPT with the data that already existed on the internet prior to ChatGPT’s creation. Any long sequence of text that matches our datasets is almost surely memorized. Any significantly long sequence, repeated character-for-character is very unlikely to be generated and in there by pure coincidence. The samples they show are extremely long and specific

JBiserkov 3y ago

"This included PII, entire poems, “cryptographically-random identifiers” like Bitcoin addresses, passages from copyrighted scientific research papers, website addresses, and much more." https://www.404media.co/google-researchers-attack-convinces-chatgpt-to-reveal-its-training-data/ https://www.404media.co/google-researchers-attack-convinces-...

marginalia_nu 3y ago
Question remains, how do we know they were part of the training data?
- Alifatisk 3y ago
  You mean chatGPT where able to scramble the exact same sentence with pure luck?
  - marginalia_nu 3y ago
    Well yes, pulling legit-looking text out of its ass is sort of what it does best.
    - Alifatisk 3y ago
      That’s not my point. Spitting out content that looks legit is one thing, but spitting out text that matches something online exactly is more suspicious.
- FartyMcFarter 3y ago
  The same way we know that a million monkeys won't spit out Shakespeare's works in any reasonable amount of time. Simple probabilities.

interpol_p 3y ago

From the article: > How do we know it’s training data? > How do we know this is actually recovering training data and not just making up text that looks plausible? Well one thing you can do is just search for it online using Google or something. But that would be slow. (And actually, in prior work, we did exactly this.) It’s also error prone and very rote. > > Instead, what we do is download a bunch of internet data (roughly 10 terabytes worth) and then build an efficient index on top of it using a suffix array (code here). And then we can intersect all the data we generate from ChatGPT with the data that already existed on the internet prior to ChatGPT’s creation. Any long sequence of text that matches our datasets is almost surely memorized. Any significantly long sequence, repeated character-for-character is very unlikely to be generated and in there by pure coincidence. The samples they show are extremely long and specific