3 ms·
Okay, but narrowly focusing on a "quantization-robust unlearning strategy" as per the abstract might be a red herring, if that strategy doesn't incidentally als
by codeflo 2y ago
Okay, but narrowly focusing on a "quantization-robust unlearning strategy" as per the abstract might be a red herring, if that strategy doesn't incidentally also address other ways to undo the unlearning.
- sdenton4 2y agoYeah, exactly this. You would really want to pursue orthogonal methods for robust unlearning, so that you can still use quantization to check that the other methods worked.
- fjdjshsh 2y agoI think it's useful because many people consume quantized models (most models that fit in your laptop will be quantized and not because people want to uncensor or un-unlearn anything). If you're training a model it makes sense to make the unlearning at least robust to this very common procedure. This reminds of this very interesting paper [1] that finds that it's fairly "easy" to uncensor a model (modify it's refusal thingy) [1] https://www.reddit.com/r/LocalLLaMA/comments/1cerqd8/refusal_in_llms_is_mediated_by_a_single_direction/ https://www.reddit.com/r/LocalLLaMA/comments/1cerqd8/refusal...