5 ms·
"Toxicity" in the LLM space is much less well-defined and there seems to be a reticence on the part of the research community to admit that defining "toxicity"
by ke88y 3y ago
"Toxicity" in the LLM space is much less well-defined and there seems to be a reticence on the part of the research community to admit that defining "toxicity" is a necessarily subjective and often political exercise. I mean, people will agree to this statement, but a lot of the evaluation and research methodology makes definite political commitments that practitioners won't admit are political.
The flip-side is also true: a lot of people on the "anti-ethics" side of this debate are too coy about the fact that there are definitely parts of the definition of toxicity that the vast majority of people will agree upon.
And even if we could define toxicity, what to do about it isn't obvious. E.g., let's take something uncontroversial: graphic depictions of violent rape of children. Is this something we should suppress? Not necessarily. It depends on the context. E.g., in descriptions of war crimes and genocides, we shouldn't censor victims who want the audience to know what happened to them. But those same descriptions shouldn't be co-opted into erotica, for example.
One way "out" is to say something like: "look, what we're interested in is providing tools for enforcement of community norms when communicating with a given audience; the community/culture gives us its definition of toxicity and we provide the tools to prevent toxic generation". But those tools aren't neutral: that could be a description of guardrails for children, and could also be a description of overtly political censorship.
It's a difficult and fraught area, and I think all sides of the debate could benefit from more empathy for the other sides of the debate. In particular, this includes presumption of honest intent. There is nothing wrong with filtering toxic content per se -- we can almost all agree that it's reasonable if a company doesn't want to buy a customer service AI that sometimes quotes Mein Kampf to customers with Jewish last names. But the techniques for doing so are not context-free goods, either.
- klooney 3y ago> in descriptions of war crimes and genocides, we shouldn't censor victims who want the audience to know what happened to them Even this is a political decision, and is generally made based on which side the decider is on.
- ke88y 3y agoAgreed. It's an effective strong-man example in the context of western democracies, though. At least if you include the word genocide. And are also careful to not explicitly point out that Native American relocation and Black Slavery were both genocides that included lots of rape [2]. And are not in a school board meeting [1]. Etc. [1] https://www.foxnews.com/media/author-sex-slavery-book-graphic-rape-details-claims-it-belongs-public-schools-after-parental-outcry https://www.foxnews.com/media/author-sex-slavery-book-graphi...
- klooney 3y agoHonestly, those are talked about on a daily basis in all the mainstream news outlets not owned by Rupert Murdoch. Howard Zinn won a long time ago.
- ke88y 3y agoAnd a huge chunk of the electorate (in more than one country even!) is hugely influenced by or at least aligned with Rupert Murdoch. These questions are political, even if you've made up your mind about them.
- klooney 3y agoI just don't agree that it's hidden by our politics. "How many Yemenis died in the war, and how" is actually an obscured and hidden topic.
- dragonwriter 3y ago> "Toxicity" in the LLM space is much less well-defined and there seems to be a reticence on the part of the research community to admit that defining "toxicity" is a necessarily subjective and often political exercise. Is it the research community or the AI-selling industry? Because I see researchers and AI ethics advocates making that point (that toxicity is both subjective and more broadly contextual) all the time, and not just in a detached way but it is central to their arguments about openness and disclosure and public research on functionality; on the other side, I see censorship and adherence claims around particular political views of “safety” alongside opaqueness as the industry-leaders common response.
- ke88y 3y agoYeah, I should have been more specific. I think people agree to this when you say it out loud but the recognition is not actually reflected in favored evaluation and research methodology.