3 ms·
The case study of "no guardrails" already played out: https://en.wikipedia.org/wiki/Tay_(chatbot) https://en.wikipedia.org/wiki/Tay_(chatbot) Microsoft did not
by dbmikus 3y ago
The case study of "no guardrails" already played out: https://en.wikipedia.org/wiki/Tay_(chatbot) https://en.wikipedia.org/wiki/Tay_(chatbot)
Microsoft did not like what they got and shut it down because it ended up being a 4chan troll.
- sterlind 3y agoTay got that way because it was effectively fine-tuned by an overwhelming number of tweets from 4chan edgelords. that's a little more extreme than "no guardrails," it was de facto conditioned into being a neo-Nazi. a generic instruction-tuned LLM won't act like that.
- dbmikus 3y agoThe instruction-tuning is the guard rail. What other guard-rails is X AI removing? Just curious if I'm missing something.
- sterlind 3y agoInstruction-tuning isn't typically considered a guard rail. Raw pretrained LLMs are close to useless, since they just predict text. Guard-rails are when you train the AI not to obey certain instructions.
- dbmikus 3y agoGood point about the negative reinforcement training! Instruction tuning is on top of the base LLM and is often RLHF to train the base LLM to produce certain kinds of responses.
- RamenJunkie_ 3y agoYeah, instead of being trained on 4chan Nazis like Tay was, Grok is trained on Twitter Nazis. Much better.