5 ms·
Why would these be independent?
by black_knight 4mo ago
Why would these be independent?
- janalsncm 4mo agoMore specifically, political lobotomy shouldn’t affect coding ability.
- Hamuko 4mo agoIt's all a bunch of weights isn't it? Why wouldn't fiddling with some parts of the weights have cascading effects?
- Discordian93 4mo agoYet empirically it does
- girvo 4mo agoYou’d be quite surprised, I think. Fine tuning a model on one axis can have drastic impacts on another that as a human we would expect to be completely unrelated.
- janalsncm 4mo agoI have never seen anyone argue that this cannot be overcome with more high quality RLVR data. The practical reality is that the Chinese and American models might have very different politics. But the most relevant factor in model performance is the quality and volume of training data, not ideology of the base model. Unless you are suggesting something very particular about the way Grok was neutered.