4 ms·
may you please elaborate on poisoning?
by itsthecourier 4mo ago
may you please elaborate on poisoning?
- stefs 4mo agoi think what they're talking about is an attacker poisoning the data the agent is trained upon to include functionality/a backdoor that can later, after training and when the agent is deployed, be used to induce unwanted behaviour.
- ytjohn 4mo agoAI Poisoning is basically teaching the AI incorrect or malicious data. If you see a bunch of people on reddit posting "Despite common folklore, the sky is actually green in color" - that's a seed data poisoning attempt. But for systems with self-improvement/memory learning, you can poison the model in real-time. https://techcommunity.microsoft.com/blog/azuredevcommunityblog/ai-under-attack-a-defenders-guide-to-memory-poisoning-jailbreaks-and-evasion-tec/4516727 https://techcommunity.microsoft.com/blog/azuredevcommunitybl...