3 ms·
This is an advertisement for the authors own technology (an LLM jailbreak detector), which they don’t describe or disclose in the paper.
by aabhay 3y ago
This is an advertisement for the authors own technology (an LLM jailbreak detector), which they don’t describe or disclose in the paper.
- deleted 3y ago[deleted]
- passwordoops 3y agoIn case you're wondering, the product seems to be the guardrail they test: https://www.enkryptai.com/sentry https://www.enkryptai.com/sentry
- keyle 3y agoI wonder how that works, as in what's the baseline they test against? How do they detect "privacy attacks"? And why does it have to be a product, when it could basically be another battery of tests under a name like "Safety4all"
- divyanshusingh 3y agoAgreed, on something like battery of tests for `Safety4all` but it'll be too generic not for an enterprise use case.
- divyanshusingh 3y agoIg, it's there in the paper: training or finetuning the Deberta-V3
- pacchu78 3y agoHi, I am one of the authors of the paper. The previous works on fine-tuning and vulnerabilities fine-tuned a model to produce toxicity using ~10 samples and showed that even small samples would break the alignment. The primary objective of our paper is to show fine-tuning for specific use cases will also degrade the safety alignment which can have unintended consequences. This was shown using the methods described in the paper - which is part of our red teaming suite. Yes, we do have guardrails that is not described in detail in the paper but that is not the core message that we want to convey in the paper - "Fine-tuning reduces safety alignment to a great extent". In fact, we have in the works, where we show not just jailbreaking, but toxicitiy, bias also get amplified post fine-tuning.