3 ms·
Perhaps such products are in the works, but I would love a private company LLM that slurps all messages, docs, meetings, and becomes an oracle that can be relie
by jmfldn 3y ago
Perhaps such products are in the works, but I would love a private company LLM that slurps all messages, docs, meetings, and becomes an oracle that can be relied upon. Perhaps it could even infer tacit knowledge even when it's not explicit in any docs, since it could abstract principles and concepts from the maze of data.
- ericbarrett 3y agoSuch an oracle would be a devastating thing to lose to industrial espionage.
- jmfldn 3y agoIndeed it would. But then the source training data would be bad to lose too. That already exists. I understand it would be a higher level of damage to lose the model of course, but the benefits outweigh the risks, and these could be managed via a very security-hardened product.
- ericbarrett 3y agoI think being able to steal a language model that can instruct you on things like the notes of every meeting, employee relationships, back-channel drama, security discussions, etc. and how these things relate to corporate and technical assets, all contained in a single artifact/tarball, really ups the ante for what kind of "security hardening" you'd need for this. Even exfiltration of API access for a few well-prompted queries could be really bad.
- jmfldn 3y agoYep for sure. Loads to consider here! My initial comment was just a brainstorm /bluesky thinking kind of idea. Clearly the implications are incredibly deep for security, data retention and a host of other concerns.
- plugin-baby 3y agoPresumably threat actors can already feed stolen document dumps to LLMs to quickly find interesting data.
- ericbarrett 3y agoOP mentioned meeting notes (like otter.ai) and other ephemera. I think that really changes the risk profile.
- haswell 3y agoAs someone who has spent far too much time diving into old Slack threads to unearth what happened in the past, I can see incredible value in a capability like this. On the other hand, I think many companies would hesitate to create lasting records of written communications that go beyond the company’s retention policy. While it’d be extremely useful, I think it would also increase potential liability that is currently managed by purge policies.
- jmfldn 3y agoAn excellent point around data retention and training data. It's not like you can scrub out bits of the neural network weights. I guess such a product would have to factor this in.
- OJFord 3y agoYou kind of can right, isn't that essentially fine-tuning or reinforcement learning? I.e. if you told it to 'forget about the July 2019 fraud we did' to adjust responses going forwards accordingly? Granted you'd have to test it without recording that, and you'd never really know for sure if you'd fully scrubbed it, only that you hadn't found a trace of it with your subsequent queries. ...now that I say that I'm sure there's a film about roughly this, delving into an AI trying to retrieve something hidden or something? The Wargames sequel perhaps?
- jmfldn 3y agoPerhaps? I'm an LLM / NN layperson so I know basically zero. In my mind, the data is all smashed into a trillion numbers so there's no notion way of removing it as such. Your point about fine-tuning it out is interesting though. As you say though, you'd have to be really sure you'd really erased it or made it "unreachable" .
- pavel_lishin 3y agoAn oracle that doesn't tell you when it doesn't know something, and lies or hallucinates answers instead?
- jmfldn 3y agoHa yes! We're not there yet with this tech, that's for sure.
- DaveSchmindel 3y agoZoom offers this today! I think Zoom + a custom GPT could do the trick.