> Have several employees do tagging and if enough give the same tag, apply it. When the scope of what you're tagging is expansive enough, inter-rater agreement

> Have several employees do tagging and if enough give the same tag, apply it. When the scope of what you're tagging is expansive enough, inter-rater agreement isn't as useful as you'd think, primarily because you get correlated biases across your rater population. These days the pendulum seems to have swung towards the left being fond of censorship and advocating for institutional control of "truth", so to use a left-coded example: good luck finding finding a population of raters in the US that doesn't fail the Implicit Association Test[1] wrt anti-black racism. The "obvious" answer is to make sure your raters on each ad are balanced wrt biases, but doing so upon every axis that Google ads may refer to is literally impossible. You probably can't even articulate all the relevant axes. It seems to be hard for people to imagine any ambiguity or flaws in the official truth that contemporary institutions push, so an easy thought exercise is to consider what this sort of an approach would have looked like during the Cold War, or even post-9/11. If you think those are anomalies and that we're in a post-lies, universal-truth era, I don't know what to tell you other than to suggest picking up a history book and realizing that there's never been such a thing. [1] I'm aware of the flaws of this test and the associated studies, but what matters here is the perception of its usefulness.