3 ms·
>The problem of preventing leaks of forbidden knowledge seems impossible without excluding the forbidden knowledge from the training data. I fail to see how tha
by s1artibartfast 3y ago
>The problem of preventing leaks of forbidden knowledge seems impossible without excluding the forbidden knowledge from the training data. I fail to see how that would be possible without crippling the desirable use cases.
This isnt accurate. There already several tools and methods explicitly for excluding forbidden knowledge from training data, or reweighting the training data so that the LLM have the perspective that the owners want.
e.g. if you want the model from being racist, you run the training data through a classifier to remove data with negative sentiment around race.
https://openai.com/research/dall-e-2-pre-training-mitigations https://openai.com/research/dall-e-2-pre-training-mitigation...
- apwell23 3y agothat doesn't address the last part "without crippling the desirable use cases." ?
- s1artibartfast 3y agoIt just cripples the undesirable use cases. It makes a targeted blind spot for the LLM, as decided by their owner.
- INGSOCIALITE 3y agowhy is everyone saying the term "forbidden knowledge" like that's just fine and dandy?
- s1artibartfast 3y agobecause if you dont pretend you want your information censored, you get labeled as an antisocial racist.
- sangnoir 3y agoHow do you define censorship? Is it different from editing and curation? Is the the choice to exclude some sources from the training data "censorship"? How about the choice of loss minimization strategy?
- s1artibartfast 3y agoI think excluding data and sources is both censorship, and something more. Excluding data is censoring from them model. Then when the model provides incorrect or biased, the firm is actively deceiving the user. Im not sure what you mean by loss minimization strategy. what I said is focused on the case where data is manipulated with the intent to curate a different perspective than an the source data.