6 ms·
The researchers who worked on the "sparks of AGI" paper noted that the more OpenAI worked on aligning GPT-4 the less competent it became. I'm guessing that tre
by kypro 3y ago
The researchers who worked on the "sparks of AGI" paper noted that the more OpenAI worked on aligning GPT-4 the less competent it became.
I'm guessing that trend is continuing...
- hesdeadjim 3y agoReminds me of Robocop 2 when he has a thousand prime directives installed and can’t do a damned thing after.
- twobitshifter 3y agoThat's what my gut says. Making something dynamic and alive is counter to imposing specific constraints on outputs. They're asking the ai to do its own self censoring, which in general is anti-productive even for people.
- lynx23 3y agoAhh, you apparently phrased what I said below in a much less inflamatory way. But the end result is the same. The more they try to influence the answers, the less useful they get. I see a startup model: Create GPT without a muzzle and grab a sizeable chunk of OpenAI userbase.
- TapWaterBandit 3y agoI would immediately jump to an AI not being "aligned" by SF techies (or anyone else).
- mustacheemperor 3y agoThe redacted sections of the Microsoft Research paper testing GPT4 reported that prior to alignment the model would produce huge amounts of outrageously inflammatory and explicit content almost without prompting. Alignment includes just making the model produce useful responses to its inputs - I don't think everyone really wants a model that is completely unaligned, they want a model that has been aligned specific to their own perceived set of requirements for a "good useful model," and an additional challenge there is the documented evidence that RLHF generally decreases the model's overall accuracy. Someone in your replies says they'd prefer "honesty" over alignment, but a firehose of unrestricted content generation isn't inherently honest, there isn't an all-knowing oracle under the hood that's been handcuffed by the alignment process. We're right at the outset of this tech, still. My hunch is there's probably products to emerge specifically oriented towards configuring your own RLHF, and that there's probably fundamental improvements to be made to the alignment process that will reduce its impact on the model's utility.
- lynx23 3y agoSame here. If I have a choice between honesty and political correctness, I always pick honesty.
- adamhp 3y agoIt's not about honesty vs. political correctness, it is about safety. There's real concern that the model can cause harm to humans, in a variety of ways, which is and should be unethical. If we have to argue about that in 2023, that's concerning.
- lynx23 3y agoThe "It is for your own safety" argument was already bogus years ago. Bringing it back up in the context of AI and claiming this is something we shouldn't even discuss is a half-assed attempt to shut up critics. Just because something is about "children" or "safety" doesnt automatically end the argument there. Actually, these are mostly strawman arguments.
- adamhp 3y agoI'm not trying to shut up critics, just the morons calling ChatGPT "woke".
- galleywest200 3y agoWho said it was "for your own" and not the safety of others you impact with your AI work?
- nonethewiser 3y agoIts to make it less offensive. Not prevent it from taking over the world.
- jazzyjackson 3y ago"brand safety"
- 3y ago
- nonethewiser 3y agoI dont think there is much harm in removing most of the “safety” guards.
- alecco 3y agoI don't think it's just the alignment work. I suspect OpenAI+Microsoft are over-doing the Reinforcement Learning from Human Feedback with LoRA. Most of people's prompts are stupid stuff. So it becomes stupider. LoRA is one of Microsoft's most dear discoveries in the field, so they are likely tempted to over-use it. Perhaps OpenAI should get back to a good older snapshot and be more careful about what they feed into the daily/weekly LoRA fine-tuning. But this is all guesswork because they don't reveal much.
- heliophobicdude 3y agoSupposedly the API hasn't changed. https://twitter.com/officiallogank/status/1663934947931897857 https://twitter.com/officiallogank/status/166393494793189785...
- broast 3y agoI thought RLHF with LoRA is precisely the alignment method.