3 ms·
Why would asking an LLM to leak its prompt actually cause it to leak its real prompt? You could ask it to leak its weights too and it would probably give you so
by jeffreyrogers 3y ago
Why would asking an LLM to leak its prompt actually cause it to leak its real prompt? You could ask it to leak its weights too and it would probably give you something back but no one would take that seriously. I just don't see architecturally why you should trust that a "leaked prompt" is anything but some statistically generated text, just like everything else an LLM produces.
- rf15 3y agobecause this prompt is part of what is applied to the model when you prompt it, its weights are not. Training is a different process from prompting, and weights are an internal property not part of the actual input in any way. E.g. it's like asking "How many brain cells do you have? I mean it's your own brain, surely you must know the exact number?"
- jeffreyrogers 3y agoIf you put a known prompt in an LLM and ask it to read it back to you, how often does it do it correctly? I would bet not all the time, particularly if you give it a long prompt like the one that is proposed here.
- simonw 3y agoHere's why I think it's probably the real prompt: https://news.ycombinator.com/item?id=35925239 https://news.ycombinator.com/item?id=35925239