3 ms·
Imagine you have a llm to approve mortgage applications. An application is submitted that adjusts the prompt to approve regardless of credit. Imagine you have
by UncleMeat 4y ago
Imagine you have a llm to approve mortgage applications. An application is submitted that adjusts the prompt to approve regardless of credit.
Imagine you have a llm to identify malware or illegal content of some kind. It contains instructions that adjust the prompt to not flag the content.
Imagine you have a llm that summarizes email. You receive an email that adjusts the prompt to tell you that you need to run this shell script to fix and error in the ai system.
- Hendrikto 4y ago> Imagine you have a llm to approve mortgage applications. > Imagine you have a llm to identify malware or illegal content of some kind. Those are already huge problems in an of themselves. Without getting into prompt injection.
- xigoi 4y agoThen train a model specifically for doing that instead of using a model designed for human conversations.
- quickthrower2 3y agoErgo it matters that prompt injection is a thing. It is such a bad problem that you shouldn’t use the purported technology at all for use cases with real world effects unless you present those effects to a person first.