3 ms·
You're simply incorrect here: the point is that the quarantined LLM has no ability to execute code and all inputs and outputs are treated as untrusted strings.
by sdenton4 3y ago
You're simply incorrect here: the point is that the quarantined LLM has no ability to execute code and all inputs and outputs are treated as untrusted strings. Thanks to the history of the Internet, handling untrusted strings is a thing we understand how to do.
The privileged LLM doesn't see the untrusted text, and is prompted by the user - which is fine until the user does something dumb with the untrusted text. (Thus, the social engineering section.)
Nothing about this is security by obscurity... It may be flawed ( feel free to provide an example that would cause a failure), but it's not just hiding a problem under a layer of rot13...