4 ms·
There is bias in the training data as well as the fine-tuning. LLMs are stochastic, which means that every time you call it, there's a chance that it will accid
by pgkr 2y ago
There is bias in the training data as well as the fine-tuning. LLMs are stochastic, which means that every time you call it, there's a chance that it will accidentally not censor itself. However, this is only true for certain topics when it comes to DeepSeek-R1. For other topics, it always censors itself.
We're in the middle of conducting research on this using the fully self-hosted open source version of R1 and will release the findings in the next day or so. That should clear up a lot of speculation.
- eru 2y ago> LLMs are stochastic, which means that every time you call it, there's a chance that it will accidentally not censor itself. A die is stochastic, but that doesn't mean there's a chance it'll roll a 7.
- pgkr 2y agoWe were curious about this, too. Our research revealed that both propaganda talking points and neutral information are within distribution of V3. The full writeup is here: https://news.ycombinator.com/item?id=42918935 https://news.ycombinator.com/item?id=42918935