The perceived goal is “detect toxicity”, but let’s unwind this goal a bit. Is it the lofty “make people be nice to each other”? Well, the paradox is that bein

The perceived goal is “detect toxicity”, but let’s unwind this goal a bit. Is it the lofty “make people be nice to each other”? Well, the paradox is that being nice is possible with the strongest choice of words, while being very harsh can sound most fluffy bunnies on the surface. In fact, in human relationships there are degrees of mutual familiarity where being exceedingly polite and not “insulting” your counterparty would be perceived as negative—where insults are not taken at face value, but rather as signifiers of friendliness (there’s a line, of course). Shall we unwind the perceived goal differently? Of course, the platform’s actual customers are the advertisers (we are talking about a hypothetical platform, but where is it really different?), and by being free to the user it participates in a very limited oligopoly of big social so no, it doesn’t really care about what anyone really meant or intended, and it definitely isn’t going to hire real humans who’d make an effort at grasping the context of the conversation. The real objective is for the platform to not have problems with law enforcement when one user complains about another user for being naughty, discriminatory, threatening, etc.—and, of course, we shouldn’t expect anything more from toxicity detectors geared towards that goal. As long as it’s not egregious enough that users leave en masse to a competitor (which can hardly exist, no honest business could reasonably compete with “free”) the platform wouldn’t care since users only matter to advertising revenue as cattle in numbers.