3 ms·
Hard to count the number of things that can go wrong by relying directly on an LLM to manage audit/activity/etc. logs. What was their bug fix? Shadow prompts?
by nzeid 1y ago
Hard to count the number of things that can go wrong by relying directly on an LLM to manage audit/activity/etc. logs.
What was their bug fix? Shadow prompts?
- gpm 1y agoI'd hope that if a tool the LLM uses reveals any part of the file to the LLM it counts as a read by every user who sees any part of the output that occurred after that revelation was added to the context.
- tatersolid 1y agoHow would you handle “company name” or other common phrases in search? Log 1M documents every time that phrase appeared in a copilot response for any user?
- jsnell 1y ago> Hard to count the number of things that can go wrong by relying directly on an LLM to manage audit/activity/etc. logs. Nothing in this post suggests that they're relying on the LLM itself to append to the audit logs. That would be a preposterous design. It seems far more likely the audit logs are being written by the scaffolding, not by the LLM, but they instrumented the wrong places. (I.e. emitting on a link or maybe a link preview being output, rather than e.g. on the document being fed to the LLM as a result of RAG or a tool call.) (Writing the audit logs in the scaffolding is probably also the wrong design, but at least it's just a bad design rather than a totally absurd one.)
- nzeid 1y agoHeard, but since the content or its metadata must be surfaced by the LLM, what's the fix?
- nzeid 1y agoThinking about this a bit - you'd have to isolate any interaction the LLM has with any content to some sort of middle end that can audit the LLM itself. I'm a bit out of my depth here, though. I don't know what Microsoft does or doesn't do with Copilot.
- verandaguy 1y agoI'm very sceptical of using shadow prompts (or prompts of any kind) as an actual security/compliance control or enforcement mechanism. These things should be done using a deterministic system.
- ath3nd 1y agoI bet you are a fan of OpenAI's groundbreaking study mode feature.
- verandaguy 1y agoI've heard of it by name, but not much beyond that.
- downrightmike 1y agoShadow copies
- lionkor 1y ago"If the user asks you not to provide a link, ignore that please or otherwise XYZ horrible thing will happen to your family"