4 ms·
Hi, I'm curious how preventing jailbreaks protects the user? > Prompt guardrails to prevent jailbreak attempts and ensure safe user interactions [...]
by lionkor 2y ago
Hi, I'm curious how preventing jailbreaks protects the user?
> Prompt guardrails to prevent jailbreak attempts and ensure safe user interactions [...]
- adilhafeez 2y agoJailbreak ensures a smooth developer experience by controlling what traffic from user make its way to the model. With jailbreak (and other guardrails soon to be added) developers can short-circuit response and with observability developers can get insights on how users are interacting with their APIs.
- sparacha 2y agoThat's a fair point - technically it protects the application from malicious attempts to subvert the desired LLM experience. The more specific language (and I think we could do better here) would be that Arch ensures users remain within the bounds of an intended LLM experience. That at least was the intention behind "ensure safe user interactions"...
- harlanlewis 2y agoUntrusted inputs to systems with agency or access to privileged data. Here’s a data exfiltration example in Google AI Studio: https://x.com/wunderwuzzi23/status/1821210923157098919 https://x.com/wunderwuzzi23/status/1821210923157098919