Hi, I am one of the authors of the paper. The previous works on fine-tuning and vulnerabilities fine-tuned a model to produce toxicity using ~10 samples and sho

Hi, I am one of the authors of the paper. The previous works on fine-tuning and vulnerabilities fine-tuned a model to produce toxicity using ~10 samples and showed that even small samples would break the alignment. The primary objective of our paper is to show fine-tuning for specific use cases will also degrade the safety alignment which can have unintended consequences. This was shown using the methods described in the paper - which is part of our red teaming suite. Yes, we do have guardrails that is not described in detail in the paper but that is not the core message that we want to convey in the paper - "Fine-tuning reduces safety alignment to a great extent". In fact, we have in the works, where we show not just jailbreaking, but toxicitiy, bias also get amplified post fine-tuning.