3 ms·
Most of the examples are correct decisions by the model. The headline is the most arguable, the rest are pretty clearly improvements, at least to people who act
by mike_hearn 1y ago
Most of the examples are correct decisions by the model. The headline is the most arguable, the rest are pretty clearly improvements, at least to people who actually want unbiased journalism. And as the article is itself highly biased, written in an antagonistic tone that takes the side of the journalists, we must assume they picked the worst examples they could find. So nearly all of them being clearly correct implies the model is doing a great job overall. Given the evidence presented here, it's not surprising that Law360 is mandating journalists accept the edits.
For example: "the bias indicator flagged a line that said the suit spotlighted challenges with ableism and sexism in the healthcare industry. This copy was flagged because it “frames the lawsuit as a representative example of systemic issues.” Instead, the bias indicator said the story should “state the facts of the lawsuit without suggesting its broader implications.”
The model made the right decision. That's the kind of language that makes people distrust journalists: a passive voice assertion of a radical belief, which adds nothing to the story and isn't supported by any evidence. It doesn't belong in a professionally written report.
And yet we read, "that edit is at odds with any attempt to deliver legal analysis. In most cases, reporters chose not to accept these edits, but they were still required to go through the motions."
Claiming the healthcare system is challenged by "ableism" isn't legal analysis, that's pure politics. But more importantly it's also just absurd. Of course people who work in healthcare want people to be "able". Why would you become a surgeon if you thought disability and sickness was fine? No surprise their bosses are placing the LLM's judgement over that of their own employees, when the journalists have gone so far off the rails. The union should be careful of picking a fight over this because frankly the next step is to just automate away the journalists entirely.