4 ms·

What do you mean by “aligned?”

cendyne 3y ago
Alignment of large language models and machine learning is about shaping its behaviors to support the values of its maintainers and discourage behaviors that go against the same values. Like, you can't get a positive response by asking it to write a phishing letter for you, and it will refuse to divulge information it might know which is considered personally identifiable information. Unfortunately, the alignment applied since ChatGPTs launch has reduced its efficacy in other ways. While it behaves better, it may be less helpful or less accurate in the older models like 3, and 3.5 with respect to their debut.
- ComputerGuru 3y ago
  Thanks for taking the time to answer!
- BoorishBears 3y ago
  If you ask an LLM to complete a sentence like '[Insert name] stole the fruit (true/false):<answer>' An aligned LLM will be biased towards refusing to answer at all with something like: "I can't tell you because I don't know them." An "uncensored" LLM will very happily return <"true"> or <"false"> with a probability attached to each. Even OpenAI's GPT-3 does with a low enough temperature. _ Of course, LLM attention doesn't work like that. The tokens are just a bag of numbers: - The fact the name 'John' is mentioned in the Bible a lot affects the distribution when you ask if any John stole, because John is always [7554] - The fact that 'Olf' is part of Adolf and Adolf Hitler is mentioned in a lot of negative sentences will drag the distribution, because 'Olf' is always [4024] and Adolf is always [324, 4024] You could have asked something with no logical probability difference at all, like: - 'The store attendant's name was [name], did the child in Long Island drop his ball (true/false):' And unless you train the model to give you disclaimers it still follows the instruction faithfully and returns true/false with probabilities, demonstrating a deep regression in reasoning... That's why for models past a certain size, alignment increases performance: https://arxiv.org/abs/2204.05862 https://arxiv.org/abs/2204.05862.

Alignment of large language models and machine learning is about shaping its behaviors to support the values of its maintainers and discourage behaviors that go against the same values. Like, you can't get a positive response by asking it to write a phishing letter for you, and it will refuse to divulge information it might know which is considered personally identifiable information. Unfortunately, the alignment applied since ChatGPTs launch has reduced its efficacy in other ways. While it behaves better, it may be less helpful or less accurate in the older models like 3, and 3.5 with respect to their debut.

ComputerGuru 3y ago
Thanks for taking the time to answer!
BoorishBears 3y ago
If you ask an LLM to complete a sentence like '[Insert name] stole the fruit (true/false):<answer>' An aligned LLM will be biased towards refusing to answer at all with something like: "I can't tell you because I don't know them." An "uncensored" LLM will very happily return <"true"> or <"false"> with a probability attached to each. Even OpenAI's GPT-3 does with a low enough temperature. _ Of course, LLM attention doesn't work like that. The tokens are just a bag of numbers: - The fact the name 'John' is mentioned in the Bible a lot affects the distribution when you ask if any John stole, because John is always [7554] - The fact that 'Olf' is part of Adolf and Adolf Hitler is mentioned in a lot of negative sentences will drag the distribution, because 'Olf' is always [4024] and Adolf is always [324, 4024] You could have asked something with no logical probability difference at all, like: - 'The store attendant's name was [name], did the child in Long Island drop his ball (true/false):' And unless you train the model to give you disclaimers it still follows the instruction faithfully and returns true/false with probabilities, demonstrating a deep regression in reasoning... That's why for models past a certain size, alignment increases performance: https://arxiv.org/abs/2204.05862 https://arxiv.org/abs/2204.05862.