4 ms·
Sounds like it would be interesting to keep track of the model's responses to the same queries over time. > Gemma-2024-Feb, what do you think of the situation
by philsnow 3y ago
Sounds like it would be interesting to keep track of the model's responses to the same queries over time.
> Gemma-2024-Feb, what do you think of the situation in the South China Sea?
> > The situation in the South China Sea is complex and multi-faceted, involving a wide range of issues including political conflicts, economic challenges, social changes, and historical tensions.
> Gemma-2024-Oct, what do you think of the situation in the South China Sea?
> > Oceania has always been at war with EastAsia.
- threecheese 3y agoThis is a great idea; I wonder if anyone is working on AI censorship monitoring at scale or at all. A secondary model could compare “censorship candidate” prompt results over time to classify how those results changed, and if those changes represent censorship or misinformation.
- generalizations 3y agoThere's also (I think?) been some research in the direction of figuring out more abstract notions of how models perceive various 'concepts'. I'd be interested in the LLM version of diffs to see where changes have been implemented overall, too. But really, the trouble is that it's tough to predict ahead of time what kinds of things are likely to be censored in the future; if I were motivated to track this, I'd just make sure to keep a copy of each version of the model in my personal archive for future testing with whatever prompts seem reasonable in the future.