4 ms·
Another question is whether that initial unalignment comes from poor filtering of datasets, or is it emergent from regular, pre-filtered cultured texts. In oth
by wruza 2y ago
Another question is whether that initial unalignment comes from poor filtering of datasets, or is it emergent from regular, pre-filtered cultured texts.
In other words, was an “unaligned” LLM taught bad things from bad people, or does it simply see it naturally and point it out with the purity of a child? The latter would mean something about ourselves. Personally I think that people tend to selectively ignore things too much.
- GuB-42 2y agoWe can't avoid teaching bad things to a LLM if we want it to have useful knowledge. For example, you may teach a LLM about nazis, that's expected knowledge. But then, you can prompt a LLM to be a nazi. You can teach it about how to avoid poisoning yourself, but then, you taught it how to poison people. And the smarter the model is, the better it will be at extracting bad things from good things by negation. There are actually training dataset full of bad thing by bad people, the intention is to use them negatively, as to teach the LLM some morality.
- ujikoluk 2y agoMaybe we should just avoid trying to classify things as good or bad.
- BriggyDwiggs42 2y agoBut I have no idea why someone might want an LLM to act like a nazi. People read mein kampf in order to study the psychology of a madman and such.