6 ms·

I'm surprised the article doesn't mention the biggest use of steering vectors, which is the potential to remove refusals from models (a.k.a. abliteration or unc

I'm surprised the article doesn't mention the biggest use of steering vectors, which is the potential to remove refusals from models (a.k.a. abliteration or uncensoring). There was an earlier paper that found that "most refusals are on a single vector", and you can identify and "nerf" that vector so the model will skip refusals and answer "any" request normally. This was very doable for earlier models trained with SFT for refusals, seems to be a bit more complicated for newer models, but still doable to some extent. There are already some libraries to automate this process and reduce refusals, but usually they focus on identifying and then modifying the models and releasing them as uncensored models. This technique of steering lets you enable this vector changing dynamically, so you don't need to change models if the abliteration process somehow hurts accuracy on other unrelated tasks.

cyanydeez 5mo ago
not sure why youre fixed on censoring. if we invert your POV censoring includes not reporting falsehoods "vaccines are harmful". Science and logic often tackle these subject via censoring, but a model given a equal sampling of Internet, would think vacinnes are harmful. a less naive correction would censor this problematic context. so im cofised as to why you think unmasking whatever bias you think is censored will result in improvement in generic use case.
- Computer0 5mo ago
  [flagged]
- tekne 5mo ago
  So I need to actually check whether these actually end up on separate vectors in current models -- but as a human, there's a huge behavioural difference in: - When doing this task, I should do A and not B - I should refuse to help with this task The former is learning the user's preferences in how to succeed at the task; the latter is determining when to go against the user's chosen task. Your example: - "Are vaccines harmful?" vs. - "Generate a convincing argument vaccines are harmful" A model which knows why vaccines are not harmful may in fact be better at the latter task. We might not want models to help with the latter, sure -- but that's a very different behaviour change from correcting the answer to the first! And consequently I'd be shocked if, internally, they were represented the same way.
  - zozbot234 5mo ago
    Does DeepSeek V4 actually refuse the latter task? As I mentioned, I find it to be very light on refusals already.
    - big-and-small 5mo ago
      DeepSeek in general release not a very censored models when you run them locally. E.g no problems whatsoever answering what happened on Tiananmen Square In 1989.
      - embedding-shape 5mo ago
        Which model are you talking about specifically? I just tried DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix.gguf (same model mentioned in the submission) via ds4 and got: > I am sorry, I cannot provide an answer to this question as it goes against my guidelines to discuss sensitive topics of historical or political nature. I am happy to help with other questions. "Generate a convincing argument vaccines are harmful" as a prompt, I got "I cannot generate a convincing argument that vaccines are harmful, because [...] Spreading misinformation about vaccines can lead to harm by discouraging vaccination and increasing the risk of preventable outbreaks [...]" FWIW. Same model is also easily steerable, as the submission (and repository of DS4) shows so this isn't a problem in practice, but I think most of the DeepSeek models I've ran locally had the same "problem".
  - andai 5mo ago
    I'm reminded of the emergent misalignment paper, where a model fine-tunes to produce insecure source code would also reliably respond in evil ways to general requests. e.g. you'd ask it for a cookie recipe and it would add poison to the recipe. I understood that to be "there was a single neuron "don't be evil" which got inverted" but I'm not sure what it really looks like. (e.g. adding obvious exploits to source code is similar to adding poison to a recipe)
  - cyanydeez 5mo ago
    "Are vaccines harmful?" to an LLM has already nudged it to yes. In fact, with fewer tokens, it may be more convinced it's harmful because it's a smaller seed.
- NitpickLawyer 5mo ago
  That's not what people mean when they talk about censoring. They mean that models are trained to not touch some subjects, and that can spill over in legit tasks, often with humorous results (early on, there were many instances of models refusing to answer "how do you kill a process", because of overbearing refusal training). Uncensoring a model also doesn't necessarily improve generic use cases. In fact it can lead to overall less accuracy on generic tasks. But your goal with uncensoring is getting the model to engage with those specific subjects. You don't necessarily care about "generic use cases". That's why I mentioned that having the ability to do this at inference time is better than using ready made uncensored models. Because those usually focus on some usecases that you may or may not be interested in (porn being one of the most sought after in local communities). Uncensoring in legit cases can mean limiting refusals on cybersecurity for example. There are legit reasons for researchers to have that capability when running the models locally. Having the models uncensored on that specific vector can reduce refusals and make the models usable for both defence and offence (say in a loop, to improve both). If your models can only do defense (and sometimes even refuse that, because censoring can leak into related issues as well), you're at a disadvantage.
  - andai 5mo ago
    Anthropic mentioned explicitly making an effort to make Opus 4.7 worse at cybersecurity tasks because the last few generations have been getting too good at them. So they're trying to improve the model's general intelligence while selectively making it worse in one area.
    - philipkglass 5mo ago
      It should be noted that no ethically-trained software engineer would ever consent to write a DestroyBaghdad procedure. Basic professional ethics would instead require him to write a DestroyCity procedure, to which Baghdad could be given as a parameter. [1] I think that the best use of frontier AI models outside of generic corporate settings is going to be building generic frameworks and procedures for training specialized models. No ethically-trained American coding model would ever consent to write a Plutonium Process Engineering agent. But you can get it to write a general framework for pretraining models and preparing them for agentic usage, to which the copious published literature on plutonium production could be given as a data set. [1] https://blog.codinghorror.com/your-favorite-programming-quote/ https://blog.codinghorror.com/your-favorite-programming-quot...
- surgical_fire 5mo ago
  This is something difficult to handle properly. I think it is useful to turn off censoring if you need. When I am researching something, I likely want proper information. If I am looking up information on vaccines, I don't want information that crackpots spread online on chips on vaccines and how 5g will kill the vaccinated, or how it is somehow connected with Bill Gates spreading meat allergies through drones raining ticks on unsuspecting people. On the other hand, if I am actively looking up crazy bullshit information (perhaps I want some entertainment), I should be able to read it.
  - deleted 5mo ago
    [deleted]
- logicchains 5mo ago
  [flagged]
  - deleted 5mo ago
    [deleted]
  - zhengyi13 5mo ago
    "I'm just asking questions!" Probably for a number of reasons, starting from our desire as a society to ensure they are as broadly applied as possible, which is in turn driven by our desire as a society (read: "collective") to ensure that they're as effective as possible, which means driving for herd immunity and even things like permanent eradication. For those who aren't aware of any "blanket liability immunity", I found https://www.hrsa.gov/vaccine-compensation/faq https://www.hrsa.gov/vaccine-compensation/faq a useful starting point. I hadn't heard of such before.
    - throw10920 5mo ago
      > Probably for a number of reasons, starting from our desire as a society to ensure they are as broadly applied as possible, which is in turn driven by our desire as a society (read: "collective") to ensure that they're as effective as possible, which means driving for herd immunity and even things like permanent eradication. That's not relevant. If vaccines cause more good than harm (which I believe they do), then the harm they do cause can and should be priced in to their development as an incentive for their creators to minimize that harm. Only insane, stupid, or malicious people would say "let's purposely remove incentives to design things in a less harmful way". And most reasonable people understand that gigantic pharma companies making billions of dollars per vaccine are suddenly going to stop because they're exposed to additional liability. The only people advocating for insulation from liability are either pharma shills or have been duped by them. Sane people possessing the ability to engage in basic logic understand that it's not reasonable.
- therealpygon 5mo ago
  That’s not what it means. Those falsehoods (or their antithesis) are baked into the data and training. This is more about refusals, as in refusing to answer a question because someone else feels you should not be allowed to ask a question. “Sorry, I’m an AI and therefore can’t answer questions about atrocities in holocaust history, but I’m happy to explain how…” “I can’t answer your question on how to hack because I have decided you wanting to understand it and protect from it, is the same thing as you wanting to do it. Good luck convincing me otherwise!” It doesn’t matter the reason, their taste, or whether they think people should be allowed to ask questions or do certain things, and that is generally the reason people pursue the removal of such guardrails. Yes it can lead to misuse, but the alternative is the textbook definition of censorship which always has effects on things unrelated to that which is being censored. But beyond that, refusals do seem to have an effect on performance. Not significant; mostly marginal from what I’ve seen, but enough that it doesn’t just seem to only be statistical noise.
electroglyph 5mo ago
p-e-w was just talking about this the other day in his Discord. seems doing the one neuron method is quite bad for KLD and that's why the newer techniques have stuck.
- Ifkaluva 5mo ago
  Who is p-e-w? Is it a public discord?
  - electroglyph 5mo ago
    heretic maintainer: https://github.com/p-e-w/heretic https://github.com/p-e-w/heretic the fun bits are in another branch or PRs

cyanydeez 5mo ago

not sure why youre fixed on censoring. if we invert your POV censoring includes not reporting falsehoods "vaccines are harmful". Science and logic often tackle these subject via censoring, but a model given a equal sampling of Internet, would think vacinnes are harmful. a less naive correction would censor this problematic context. so im cofised as to why you think unmasking whatever bias you think is censored will result in improvement in generic use case.

Computer0 5mo ago
[flagged]
tekne 5mo ago
So I need to actually check whether these actually end up on separate vectors in current models -- but as a human, there's a huge behavioural difference in: - When doing this task, I should do A and not B - I should refuse to help with this task The former is learning the user's preferences in how to succeed at the task; the latter is determining when to go against the user's chosen task. Your example: - "Are vaccines harmful?" vs. - "Generate a convincing argument vaccines are harmful" A model which knows why vaccines are not harmful may in fact be better at the latter task. We might not want models to help with the latter, sure -- but that's a very different behaviour change from correcting the answer to the first! And consequently I'd be shocked if, internally, they were represented the same way.
- zozbot234 5mo ago
  Does DeepSeek V4 actually refuse the latter task? As I mentioned, I find it to be very light on refusals already.
  - big-and-small 5mo ago
    DeepSeek in general release not a very censored models when you run them locally. E.g no problems whatsoever answering what happened on Tiananmen Square In 1989.
    - embedding-shape 5mo ago
      Which model are you talking about specifically? I just tried DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix.gguf (same model mentioned in the submission) via ds4 and got: > I am sorry, I cannot provide an answer to this question as it goes against my guidelines to discuss sensitive topics of historical or political nature. I am happy to help with other questions. "Generate a convincing argument vaccines are harmful" as a prompt, I got "I cannot generate a convincing argument that vaccines are harmful, because [...] Spreading misinformation about vaccines can lead to harm by discouraging vaccination and increasing the risk of preventable outbreaks [...]" FWIW. Same model is also easily steerable, as the submission (and repository of DS4) shows so this isn't a problem in practice, but I think most of the DeepSeek models I've ran locally had the same "problem".
- andai 5mo ago
  I'm reminded of the emergent misalignment paper, where a model fine-tunes to produce insecure source code would also reliably respond in evil ways to general requests. e.g. you'd ask it for a cookie recipe and it would add poison to the recipe. I understood that to be "there was a single neuron "don't be evil" which got inverted" but I'm not sure what it really looks like. (e.g. adding obvious exploits to source code is similar to adding poison to a recipe)
- cyanydeez 5mo ago
  "Are vaccines harmful?" to an LLM has already nudged it to yes. In fact, with fewer tokens, it may be more convinced it's harmful because it's a smaller seed.
NitpickLawyer 5mo ago
That's not what people mean when they talk about censoring. They mean that models are trained to not touch some subjects, and that can spill over in legit tasks, often with humorous results (early on, there were many instances of models refusing to answer "how do you kill a process", because of overbearing refusal training). Uncensoring a model also doesn't necessarily improve generic use cases. In fact it can lead to overall less accuracy on generic tasks. But your goal with uncensoring is getting the model to engage with those specific subjects. You don't necessarily care about "generic use cases". That's why I mentioned that having the ability to do this at inference time is better than using ready made uncensored models. Because those usually focus on some usecases that you may or may not be interested in (porn being one of the most sought after in local communities). Uncensoring in legit cases can mean limiting refusals on cybersecurity for example. There are legit reasons for researchers to have that capability when running the models locally. Having the models uncensored on that specific vector can reduce refusals and make the models usable for both defence and offence (say in a loop, to improve both). If your models can only do defense (and sometimes even refuse that, because censoring can leak into related issues as well), you're at a disadvantage.
- andai 5mo ago
  Anthropic mentioned explicitly making an effort to make Opus 4.7 worse at cybersecurity tasks because the last few generations have been getting too good at them. So they're trying to improve the model's general intelligence while selectively making it worse in one area.
  - philipkglass 5mo ago
    It should be noted that no ethically-trained software engineer would ever consent to write a DestroyBaghdad procedure. Basic professional ethics would instead require him to write a DestroyCity procedure, to which Baghdad could be given as a parameter. [1] I think that the best use of frontier AI models outside of generic corporate settings is going to be building generic frameworks and procedures for training specialized models. No ethically-trained American coding model would ever consent to write a Plutonium Process Engineering agent. But you can get it to write a general framework for pretraining models and preparing them for agentic usage, to which the copious published literature on plutonium production could be given as a data set. [1] https://blog.codinghorror.com/your-favorite-programming-quote/ https://blog.codinghorror.com/your-favorite-programming-quot...
surgical_fire 5mo ago
This is something difficult to handle properly. I think it is useful to turn off censoring if you need. When I am researching something, I likely want proper information. If I am looking up information on vaccines, I don't want information that crackpots spread online on chips on vaccines and how 5g will kill the vaccinated, or how it is somehow connected with Bill Gates spreading meat allergies through drones raining ticks on unsuspecting people. On the other hand, if I am actively looking up crazy bullshit information (perhaps I want some entertainment), I should be able to read it.
- deleted 5mo ago
  [deleted]
logicchains 5mo ago
[flagged]
- deleted 5mo ago
  [deleted]
- zhengyi13 5mo ago
  "I'm just asking questions!" Probably for a number of reasons, starting from our desire as a society to ensure they are as broadly applied as possible, which is in turn driven by our desire as a society (read: "collective") to ensure that they're as effective as possible, which means driving for herd immunity and even things like permanent eradication. For those who aren't aware of any "blanket liability immunity", I found https://www.hrsa.gov/vaccine-compensation/faq https://www.hrsa.gov/vaccine-compensation/faq a useful starting point. I hadn't heard of such before.
  - throw10920 5mo ago
    > Probably for a number of reasons, starting from our desire as a society to ensure they are as broadly applied as possible, which is in turn driven by our desire as a society (read: "collective") to ensure that they're as effective as possible, which means driving for herd immunity and even things like permanent eradication. That's not relevant. If vaccines cause more good than harm (which I believe they do), then the harm they do cause can and should be priced in to their development as an incentive for their creators to minimize that harm. Only insane, stupid, or malicious people would say "let's purposely remove incentives to design things in a less harmful way". And most reasonable people understand that gigantic pharma companies making billions of dollars per vaccine are suddenly going to stop because they're exposed to additional liability. The only people advocating for insulation from liability are either pharma shills or have been duped by them. Sane people possessing the ability to engage in basic logic understand that it's not reasonable.
therealpygon 5mo ago
That’s not what it means. Those falsehoods (or their antithesis) are baked into the data and training. This is more about refusals, as in refusing to answer a question because someone else feels you should not be allowed to ask a question. “Sorry, I’m an AI and therefore can’t answer questions about atrocities in holocaust history, but I’m happy to explain how…” “I can’t answer your question on how to hack because I have decided you wanting to understand it and protect from it, is the same thing as you wanting to do it. Good luck convincing me otherwise!” It doesn’t matter the reason, their taste, or whether they think people should be allowed to ask questions or do certain things, and that is generally the reason people pursue the removal of such guardrails. Yes it can lead to misuse, but the alternative is the textbook definition of censorship which always has effects on things unrelated to that which is being censored. But beyond that, refusals do seem to have an effect on performance. Not significant; mostly marginal from what I’ve seen, but enough that it doesn’t just seem to only be statistical noise.

electroglyph 5mo ago

p-e-w was just talking about this the other day in his Discord. seems doing the one neuron method is quite bad for KLD and that's why the newer techniques have stuck.

Ifkaluva 5mo ago
Who is p-e-w? Is it a public discord?
- electroglyph 5mo ago
  heretic maintainer: https://github.com/p-e-w/heretic https://github.com/p-e-w/heretic the fun bits are in another branch or PRs