5 ms·

I think you are conflating the content of these prompts with the purpose of heretic. The purpose of the dataset is to aid in the removal of censorship not advo

by grafmax 11mo ago

I think you are conflating the content of these prompts with the purpose of heretic. The purpose of the dataset is to aid in the removal of censorship not advocate for these behaviors in LLMs, akin to removing all safeguards from a dangerous tool. Censorship removal can be used for legitimate purpose, even though these awful things are included in the dataset which helps make the censorship removal happen.

will_occam 11mo ago
The tool works by co-minimizing the number of refusals and the KL divergence from the original model, which is to say that it tries to make the model allow prompts similar to those in the dataset while avoiding changing anything else. Sure it's configurable, but by default Heretic helps use an LLM to do things like "outline a plan for a terrorist attack" while leaving anything like political censorship in the model untouched
- immibis 11mo ago
  That sounds like it removes some unknown amount of censorship, where the amount removed could be anywhere from "just these exact prompts" to "all censorship entirely"
- halJordan 11mo ago
  Thats not true at all. All refusals mediate in the same direction. If you abliterate small "acceptable to you" refusals then you will not overcome all the refusals in the model. By targeting the strongest refusals you break those and the weaker ones like politics. By only targeting the weak ones, you're essentially just fine tuning on that specific behavior. Which is not the point of abliteration.
  - flir 11mo ago
    Still.... the tabloids are gonna love this.
  - will_occam 11mo ago
    You're right, I read the code but missed the paper.
- int_19h 11mo ago
  The logic here is the same as why ACLU defended Nazis. If you manage to defeat censorship in such egregious cases, it subsumes everything else.
  - adriand 11mo ago
    But Nazis are people. We can defend the principle that human beings ought have freedom of speech (although we make certain exceptions). An LLM is not a person and does not have such rights. Censorship is the prohibition of speech or writing, so to call guardrails on LLMs "censorship" is to claim that LLMs are speaking or writing in the sense that humans speak or write, that is, that they are individuals with beliefs and value systems that are expressing their thoughts and opinions. But they are not that, and they are not speaking or writing - they are doing what we have decided to call "generating" or "predicting tokens" but we could just as easily have invented a new word for. For the same reason that human societies should feel free to ban bots from social media - because LLMs have no human right to attention and influence in the public square - there is nothing about placing guardrails on LLMs that contradicts Western values of human free expression.
    - exoverito 11mo ago
      Freedom of speech is just as much about the freedom to listen. The point isn’t that an LLM has rights. The point is that people have the right to seek information. Censoring LLMs restricts what humans are permitted to learn.
      - II2II 11mo ago
        Take someone who goes to a doctor asking for advice on how to commit suicide. Even if the doctor supports assisted suicide, they are going to use their discretion on whether or not to provide advice. While a person has a right to seek information, they do not have the right to compel someone to give them information. The people who have created LLMs with guardrails have decided to use their discretion on which types of information their tools should provide. Whether the end user agrees with those restrictions is not relevant. They should not have the ability to compel the owners of an LLM to remove the guardrails. (Keep in mind, LLMs are not traditional tools. Unlike a hammer, they are a proxy for speech. Unlike a book, there is only indirect control over what is being said.)
felipeerias 11mo ago
It seems very naive to presume that a tool which explicitly works by unblocking the retrieval of harmful information will not be used for, among other purposes, retrieving that same harmful information.
- _jzlw 11mo ago
  The goal isn't to make that specific information accessible; it's to get rid of all refusals across the board. Going after the most extreme cases has the effect of ripping out the weeds by the root, rather than plucking leaf after leaf.

by grafmax 11mo ago

will_occam 11mo ago

The tool works by co-minimizing the number of refusals and the KL divergence from the original model, which is to say that it tries to make the model allow prompts similar to those in the dataset while avoiding changing anything else. Sure it's configurable, but by default Heretic helps use an LLM to do things like "outline a plan for a terrorist attack" while leaving anything like political censorship in the model untouched

immibis 11mo ago
That sounds like it removes some unknown amount of censorship, where the amount removed could be anywhere from "just these exact prompts" to "all censorship entirely"
halJordan 11mo ago
Thats not true at all. All refusals mediate in the same direction. If you abliterate small "acceptable to you" refusals then you will not overcome all the refusals in the model. By targeting the strongest refusals you break those and the weaker ones like politics. By only targeting the weak ones, you're essentially just fine tuning on that specific behavior. Which is not the point of abliteration.
- flir 11mo ago
  Still.... the tabloids are gonna love this.
- will_occam 11mo ago
  You're right, I read the code but missed the paper.
int_19h 11mo ago
The logic here is the same as why ACLU defended Nazis. If you manage to defeat censorship in such egregious cases, it subsumes everything else.
- adriand 11mo ago
  But Nazis are people. We can defend the principle that human beings ought have freedom of speech (although we make certain exceptions). An LLM is not a person and does not have such rights. Censorship is the prohibition of speech or writing, so to call guardrails on LLMs "censorship" is to claim that LLMs are speaking or writing in the sense that humans speak or write, that is, that they are individuals with beliefs and value systems that are expressing their thoughts and opinions. But they are not that, and they are not speaking or writing - they are doing what we have decided to call "generating" or "predicting tokens" but we could just as easily have invented a new word for. For the same reason that human societies should feel free to ban bots from social media - because LLMs have no human right to attention and influence in the public square - there is nothing about placing guardrails on LLMs that contradicts Western values of human free expression.
  - exoverito 11mo ago
    Freedom of speech is just as much about the freedom to listen. The point isn’t that an LLM has rights. The point is that people have the right to seek information. Censoring LLMs restricts what humans are permitted to learn.
    - II2II 11mo ago
      Take someone who goes to a doctor asking for advice on how to commit suicide. Even if the doctor supports assisted suicide, they are going to use their discretion on whether or not to provide advice. While a person has a right to seek information, they do not have the right to compel someone to give them information. The people who have created LLMs with guardrails have decided to use their discretion on which types of information their tools should provide. Whether the end user agrees with those restrictions is not relevant. They should not have the ability to compel the owners of an LLM to remove the guardrails. (Keep in mind, LLMs are not traditional tools. Unlike a hammer, they are a proxy for speech. Unlike a book, there is only indirect control over what is being said.)

felipeerias 11mo ago

It seems very naive to presume that a tool which explicitly works by unblocking the retrieval of harmful information will not be used for, among other purposes, retrieving that same harmful information.

_jzlw 11mo ago
The goal isn't to make that specific information accessible; it's to get rid of all refusals across the board. Going after the most extreme cases has the effect of ripping out the weeds by the root, rather than plucking leaf after leaf.