Obviously this depends a lot on how you define the metric. It seems to me (superficially) that "like-for-like" metrics such as "use of passive voice" should be

Obviously this depends a lot on how you define the metric. It seems to me (superficially) that "like-for-like" metrics such as "use of passive voice" should be fairly independent of reality; for a similar example, see the American tendency to refer to "officer-involved shootings" (but, in a clear case of bias, to never say "dog-involved bitings"). On the other hand, as you note, "use of violent terms" could be a reflection of real-world differences in violent actions, so it might be a more fraught metric. I'm really intrigued by the idea that we can bring this kind of quantitative analysis to bear to reveal hidden bias, though; it sounds like it can be a powerful approach. But you're certainly right about potential pitfalls.