4 ms·
The training data was considered good by Musk to start with, so he could have spicy mode, but he changed his mind and now Grok is considered poisoned with porn.
by labrador 11mo ago
The training data was considered good by Musk to start with, so he could have spicy mode, but he changed his mind and now Grok is considered poisoned with porn. My question is, can that be fixed or does he have to start over again?
- looobay 11mo agoThere was research on LLMs training and distillation that if two models have a similar architecture (probably the case for Xai) the "master" model will distill knowledge to the model even if its not in the distillation data. So they probably need to train a new model from scratch. (sorry i don't remember the name but there was an example with a model liking howl to showcase this)
- -_- 11mo agoSubliminal learning: https://alignment.anthropic.com/2025/subliminal-learning/ https://alignment.anthropic.com/2025/subliminal-learning/
- labrador 11mo agoIf true, bad news for Elon Musk and xAI because they have to start over. He's already indicated this in regards to Wikipedia. He wants to train on Grokepedia and not Wikipedia. Removing NSFW material gives him another reason.