3 ms·
I like that none of these "rules" actually mean anything because it's just a language model. It will follow them if it thinks following them is the most probabl
by MagicMoonlight 3y ago
I like that none of these "rules" actually mean anything because it's just a language model. It will follow them if it thinks following them is the most probable next sequence of characters (although it might not follow them because its output has randomness rather than always being the most probable next characters) but that's not an actual guarantee.
There's no thinking structure. It needs an actual brain system where the language model is connected to decision making components.
- brap 3y agoI wonder if this can be solved by adding a disconnected, 2nd layer which can answer meta-questions about the bot’s answers, such as “does this answer follow the rules?”, and this can be used to tweak the answers. Perhaps it can also be used to answer questions that require some sort of chain-of-thought, like “what would be the first step to answering the question? What is the next one?”, etc, then compile these “thoughts” to a final answer. Basically you have the LLM that answers questions, and a “supervisor” LLM which can help guide it. Does something like this exist?
- simonw 3y agoLots of people have tried solving prompt injection like this. I don't think it's a responsible strategy: - https://simonwillison.net/2023/May/2/prompt-injection-explained/ https://simonwillison.net/2023/May/2/prompt-injection-explai... - https://simonwillison.net/2022/Sep/17/prompt-injection-more-ai/ https://simonwillison.net/2022/Sep/17/prompt-injection-more-...