4 ms·
Such an oracle would be a devastating thing to lose to industrial espionage.
by ericbarrett 3y ago
Such an oracle would be a devastating thing to lose to industrial espionage.
- jmfldn 3y agoIndeed it would. But then the source training data would be bad to lose too. That already exists. I understand it would be a higher level of damage to lose the model of course, but the benefits outweigh the risks, and these could be managed via a very security-hardened product.
- ericbarrett 3y agoI think being able to steal a language model that can instruct you on things like the notes of every meeting, employee relationships, back-channel drama, security discussions, etc. and how these things relate to corporate and technical assets, all contained in a single artifact/tarball, really ups the ante for what kind of "security hardening" you'd need for this. Even exfiltration of API access for a few well-prompted queries could be really bad.
- jmfldn 3y agoYep for sure. Loads to consider here! My initial comment was just a brainstorm /bluesky thinking kind of idea. Clearly the implications are incredibly deep for security, data retention and a host of other concerns.
- plugin-baby 3y agoPresumably threat actors can already feed stolen document dumps to LLMs to quickly find interesting data.
- ericbarrett 3y agoOP mentioned meeting notes (like otter.ai) and other ephemera. I think that really changes the risk profile.