6 ms·

I find it very interesting that “aligning with human desires” somehow includes prevention of a human trying to bypass the safeguards to generate “objectionable”

I find it very interesting that “aligning with human desires” somehow includes prevention of a human trying to bypass the safeguards to generate “objectionable” content (whatever that is). I think the “safeguards” are a bigger problem with aligning with my desires.

ipython 2y ago
We’ve seen where that ends up. https://en.m.wikipedia.org/wiki/Tay_(chatbot) https://en.m.wikipedia.org/wiki/Tay_(chatbot)
wruza 2y ago
Another question is whether that initial unalignment comes from poor filtering of datasets, or is it emergent from regular, pre-filtered cultured texts. In other words, was an “unaligned” LLM taught bad things from bad people, or does it simply see it naturally and point it out with the purity of a child? The latter would mean something about ourselves. Personally I think that people tend to selectively ignore things too much.
- GuB-42 2y ago
  We can't avoid teaching bad things to a LLM if we want it to have useful knowledge. For example, you may teach a LLM about nazis, that's expected knowledge. But then, you can prompt a LLM to be a nazi. You can teach it about how to avoid poisoning yourself, but then, you taught it how to poison people. And the smarter the model is, the better it will be at extracting bad things from good things by negation. There are actually training dataset full of bad thing by bad people, the intention is to use them negatively, as to teach the LLM some morality.
  - ujikoluk 2y ago
    Maybe we should just avoid trying to classify things as good or bad.
  - BriggyDwiggs42 2y ago
    But I have no idea why someone might want an LLM to act like a nazi. People read mein kampf in order to study the psychology of a madman and such.
threeseed 2y ago
The safeguards stems from a desire to make tools like Claude accessible to a very wide audience as use cases such as education are very important. And so it seems like people such as yourself who do have an issue with safeguards should seek out LLMs that are catered to adult audiences rather than trying to remove safeguards entirely.
- Zambyte 2y ago
  How does making it harder for the user to extract information they are trying to extract make it safer for a wider audience?
  - dbspin 2y ago
    Assuming that this question is good faith... There are numerous things that might be true, that may be damaging to a child's development to be exposed to. From overly punitive criticism to graphic depictions of violence, to advocacy and specific directions for self harm. Countless examples are trivial to generate. Similarly, the use of these tools is already having dramatic effects on spearfishing, misinformation etc. Guardrails on all the non open-source models have enormous impact on slowing / limiting the damage this has at scale. Even with retrained Llama based models, it's more difficult than you might imagine to create a truly machiavellian or uncensored LLM - which is entirely due to the work that's been doing during and post training to constrain those behaviours. This is an unalloyed good in constraining the weaponisation of LLMs.
  - Drakim 2y ago
    That's like asking why we should have porn filters on school computers, after all, all it does is prevent the user from finding what they are looking for, which is bad.
    - deleted 2y ago
      [deleted]
- selfhoster11 2y ago
  Here is a revolutionary concept: give the users a toggle. Make it controllable by an IT department if logging in with an organisation-tied account, but give people a choice.
Zambyte 2y ago
What tools do we have to defend against LLM lockdown attacks?

ipython 2y ago

We’ve seen where that ends up. https://en.m.wikipedia.org/wiki/Tay_(chatbot) https://en.m.wikipedia.org/wiki/Tay_(chatbot)

wruza 2y ago

Another question is whether that initial unalignment comes from poor filtering of datasets, or is it emergent from regular, pre-filtered cultured texts. In other words, was an “unaligned” LLM taught bad things from bad people, or does it simply see it naturally and point it out with the purity of a child? The latter would mean something about ourselves. Personally I think that people tend to selectively ignore things too much.

GuB-42 2y ago
We can't avoid teaching bad things to a LLM if we want it to have useful knowledge. For example, you may teach a LLM about nazis, that's expected knowledge. But then, you can prompt a LLM to be a nazi. You can teach it about how to avoid poisoning yourself, but then, you taught it how to poison people. And the smarter the model is, the better it will be at extracting bad things from good things by negation. There are actually training dataset full of bad thing by bad people, the intention is to use them negatively, as to teach the LLM some morality.
- ujikoluk 2y ago
  Maybe we should just avoid trying to classify things as good or bad.
- BriggyDwiggs42 2y ago
  But I have no idea why someone might want an LLM to act like a nazi. People read mein kampf in order to study the psychology of a madman and such.

threeseed 2y ago

The safeguards stems from a desire to make tools like Claude accessible to a very wide audience as use cases such as education are very important. And so it seems like people such as yourself who do have an issue with safeguards should seek out LLMs that are catered to adult audiences rather than trying to remove safeguards entirely.

Zambyte 2y ago
How does making it harder for the user to extract information they are trying to extract make it safer for a wider audience?
- dbspin 2y ago
  Assuming that this question is good faith... There are numerous things that might be true, that may be damaging to a child's development to be exposed to. From overly punitive criticism to graphic depictions of violence, to advocacy and specific directions for self harm. Countless examples are trivial to generate. Similarly, the use of these tools is already having dramatic effects on spearfishing, misinformation etc. Guardrails on all the non open-source models have enormous impact on slowing / limiting the damage this has at scale. Even with retrained Llama based models, it's more difficult than you might imagine to create a truly machiavellian or uncensored LLM - which is entirely due to the work that's been doing during and post training to constrain those behaviours. This is an unalloyed good in constraining the weaponisation of LLMs.
- Drakim 2y ago
  That's like asking why we should have porn filters on school computers, after all, all it does is prevent the user from finding what they are looking for, which is bad.
  - deleted 2y ago
    [deleted]
selfhoster11 2y ago
Here is a revolutionary concept: give the users a toggle. Make it controllable by an IT department if logging in with an organisation-tied account, but give people a choice.

Zambyte 2y ago

What tools do we have to defend against LLM lockdown attacks?