4 ms·

Models can easily regurgitate back training data verbatim, so anything private can be in theory accessed by anyone without proper access to that file

• brookst 2y ago
  This is partly true but less and less every day. IMO the bigger concern is that this data is not just used to train models. It is stored, completely verbatim, in the training set data. They aren’t pulling from PDFs in realtime during training runs, they’re aggregating all of that text and storing it somewhere. And that somewhere is prone to employees viewing, leakage to the internet, etc.
  • oblio 2y ago
    > This is partly true but less and less every day. Isn't this like encryption, though? I'm fairly sure that the cryptography community basically says: if someone has a copy of your encrypted data for a long time, the likelihood over time for them to be able to read it approaches 100%, regardless of the current security standard you're using. Who could possibly guarantee that whatever LLM is safe now will be safe at all times over the next 5-10-20 years? And if they're guaranteeing, they're lying.
    • brookst 2y ago
      I think it’s different, unless you believe LLMs have broken theoretical limits on compression. I don’t see how an LLM with 1T 16 bit parameters could encode 100PB of data.
      • oblio 2y ago
        My point was about attack angles. The original comment said, that for example, you could exfiltrate data with the right prompt attack. To which the reply was "they'll just make the LLM able to better defend itself". And my point was "the attackers will learn to build better prompts, too".