7 ms·

Law360 mandates reporters use AI "bias" detection on all stories

63 points by 12_throw_away 1y ago

jerf 1y ago
Today, somewhere in the world, some people made an event occur. Some people approved and some people disapproved. It impacted some people more than others, which some people think may be a bad thing, although others think that it may have been a good thing. The event stemmed from many past events, which many people have various strong opinions about. Politicians made various strong statements about the event, as well as a few celebrities. The future impact of the event is under debate, though experts agree that the situation is complicated and people should probably think about it very carefully before coming to conclusions. It is unclear what the market will do in response to this event.
- Herring 1y ago
  > Party A: wants to abolish slavery > Party B: wants way more slaves > Media: record low percentage of Americans satisfied with agriculture today! Impartiality is very important.
  - bilbo0s 1y ago
    In unrelated news: General Sherman Begins Demolition in Georgia For Urgently Needed Infrastructure Projects I mean, you could ask ChatGPT or something to make some pretty braindead impartial newspaper headlines about, like, the Holocaust or slavery. I guess you're right, in some cases, impartiality will come off as a bit ridiculous to the common sense segment of society.
- nxobject 1y ago
  Welp, I’ve read every newspaper article - at least I have an excuse not to read the news anymore for my sanity.
- encom 1y ago
  Breaking News: Some Bullshit Happening Somewhere: <https://www.youtube.com/watch?v=9U4Ha9HQvMo https://www.youtube.com/watch?v=9U4Ha9HQvMo>
- deleted 1y ago
  [deleted]
add-sub-mul-div 1y ago
"Bias" and "narrative" have become meaningless words that people use when encountering ideas they don't like. Or when they believe that neither side could ever be right about an issue, that it's virtuous for a stance to be in the middle of the current Overton window.
- nh23423fefe 1y ago
  I don't like this idea. Bias is lying to advance your position. Narrative is telling a story to persuade instead of making an argument to persuade.
  - JoshTriplett 1y ago
    While I'd favor defining words rather than claiming they're meaningless, those aren't accurate definitions. "Narrative" in this context is more like "attempting to steer the current discourse by making connections among things and positioning them as part of a coherent story", which can be positive (if the result is accurate and reasonable and helps people better understand what's going on) or negative (if it's spin or manufacturing consent). It's "narrative" to say "you should be afraid of X, it's the cause of all your problems". It's also "narrative" to say "here are the five different things we're currently doing to improve Y, and how they tie together into a coherent picture". Also, bias would be easier to deal with if it were always "lying", or if it advanced a coherent position. It's much broader than that.
- Gormo 1y ago
  I don't think it's entirely unreasonable to presume that more valid positions are likely to be found closer to the midpoint of the current Overton window. I suspect that as a matter of Bayesian probability, more extreme positions are more likely to be wrong.
  - MangoToupe 1y ago
    On which topic? Saying the average person can't decide between two viewpoints on any given topic seems ridiculous.
owisd 1y ago
The problem with trying too hard to neutrally report both sides of a story is that's not unbiased either, it's just biased in favour of the side that can lie the most convincingly.
- wredcoll 1y ago
  I am so, so tired, of "both sides" reporting. How about just report the facts instead?
  - schmidtleonard 1y ago
    It's a rare issue that you can't swing both ways by selecting the subset of facts to focus on. Attention doesn't just matter, it's all that matters. This is why adversarial forums are so important even if they so often degenerate into shouting past one another: each side has the ability and motivation to bring its strongest facts, so they are at least present in the discussion, even if debate tactics add tons of noise and confirmation bias dampens the effects.
    - wredcoll 1y ago
      Yes, in principle, but I object to "reporters" merely quoting people instead of attempting to actually verify if what they said has any basis in reality. That being said, I'm not sure how much this actually matters, trump seems to pretty comprehensively prove that people would rather hear lies, even if they know they are lies.
  - an0malous 1y ago
    Because there’s basically no such thing and choosing which facts you report is a huge source of bias anyway
  - kube-system 1y ago
    Nearly all political issues are rooted in philosophical differences of opinion. "The facts" are merely the icing on the cake that various factions use to lure people to their ideology. Factual reporting is better than direct opinion reporting, but both are inherently interpreted by people in the context of the Overton window.
JoshTriplett 1y ago
Timeless quote: > If Jimmy Allred says it’s raining, and W. Lee O’Daniel says it isn’t raining, Texas newspapermen quote them both, and don’t look out the window to see which is lying, and to tell the readers what the truth is at the moment. https://quoteinvestigator.com/2023/11/14/rain-look/ https://quoteinvestigator.com/2023/11/14/rain-look/ If an AI tool, or for that matter a meddling editor, says a headline is "framing the action as unprecedented in a way that might subtly critique the administration", the correct response is "yes, that was the idea".
- sandwichsphinx 1y ago
  reminds me of the essay "Politics and the English Language" George Orwell wrote in 1946, it's a good read
- zaphar 1y ago
  I don't think most reporting bias takes the form of incorrect facts. It takes the form of picking which facts to share driven by the facts that matter the most to the particular reporter sharing them. This results in slanted coverage even if technically it's "factual".
  - JoshTriplett 1y ago
    Take a look at the headline flagged in the article, which the tool flagged as "framing the action as unprecedented in a way that might subtly critique the administration". The headline is factually accurate, and the facts are not cherry-picked. Yes, some reporting is biased. But some reporting is simply accurately reporting damaging information, and "biased" is a way of attacking that without addressing the substance of the problem.
    - zaphar 1y ago
      I agree that article's headline is pretty factual nor are the facts cherry-picked. But "some reporting is biased" is heavily understating the problem. And I hypothesize that the reason there is a leap to "This is biased" today on reporting is because the news media organizations have participated in the cherry-picked facts case to enough of a degree that trust has eroded. Which is perhaps why organizations are searching for ways to not appear biased and begin to restore that trust. I think the reason the LLM approach is likely to fail is because it is unable to detect the "missing facts" case and can only really advise about sentiment and phrasing. Which is not I think the actual problem that needs fixing.
BrenBarn 1y ago
Who checks the bias detector for bias?
- ethan_smith 1y ago
  This creates an infinite regress problem - bias detection systems are themselves trained on data reflecting human judgments about what constitutes "bias."
- kelseyfrog 1y ago
  That's the nice thing. You don't. The conceit of centrism/Liberalism is that extreme voices cancel out. Just ingest a large enough corpus and the central limit theorum takes care of the rest.
jasonthorsness 1y ago
LLMs and ML algorithms are beginning to influence the entire lifecycle of articles: researching, writing, editing, publication, discovery (TikTok), and consumption (ChatGPT summarize this). With the few big players, it could be the same model involved at every step. It's scary how a small change to a system prompt could subtly influence things across the board and guide popular opinion.
wredcoll 1y ago
Remember kids, if you don't like the truth, just accuse it of being biased and force it to change!
- bilbo0s 1y ago
  Truth is biased. That's what makes it truth. If you wanted unbiased information, you should have asked for facts.. and then collected as many of them as you possibly could.
  - MangoToupe 1y ago
    More people should really read Baudrillard. Objectivity and language don't combine very well. Show me a person who wants the facts and I'll show you someone who is easily manipulated.
deleted 1y ago
[deleted]
saaaaaam 1y ago
I said to a colleague the other day that Law360’s reporting seems to be more tortuous and weird than usual. I wonder if this is why. There was one piece on a case I’ve been following quite closely where I was genuinely unable to make sense of what had happened from the Law360 report without sitting down and spelling my way through it.
throwaway984393 1y ago
[dead]
pkilgore 1y ago
What I do not understand is this top-down mandate that people use specific tools. Mandate productivity, mandate quality; the tools will be adopted if they are sharp. But elevating the way work is performed above the work itself? Weird.
- lovich 1y ago
  I don't know how you read this article and came away that they were merely trying to force adoption of AI tools. This is LexisNexis requiring all of their workers run their output through a censorship machine to comply with the whims of the current administration. It has zero benefits to the workers so of course they have to mandate it
- mike_hearn 1y ago
  It's because the executives don't trust the journalists to do the work properly. The article says that explicitly: customers have complained about the quality of the work. The business owners view unhappy customers as a threat to their business, and so have introduced automation to make product quality higher and more even. The workers are unhappy because they don't want to be automated away and resent the implication that their artisanally hand-crafted articles aren't good enough. This is a very old story that's been going on for hundreds of years. There's nothing special about it.
jauntywundrkind 1y ago
> Several sentences in the story were flagged as biased, including this one: "It’s the first time in 60 years that a president has mobilized a state’s National Guard without receiving a request to do so from the state’s governor." According to the bias indicator, this sentence is "framing the action as unprecedented in a way that might subtly critique the administration." It was best to give more context to “balance the tone." "Might subtly critique"?! This isn't protecting versus bias, this isn't even charitably regardable as cowardice. It's an attack on reason & reporting. To shield the worst cry bullies & offenders. It's ghastly how basic simple reporting so offends, that the public interest is sabotaged by these foes. We face such a need to improve the media, and this is the opposite path, of craven blunt un-intelligently intermediated corporatism.
iFire 1y ago
Does this mean automated censorship by AI "bias" detection?
MangoToupe 1y ago
To me, a newsroom's bias IS its value. If you let me know your values, I can actually interpret what you're saying. It's the pretense of impartiality that makes me distrust someone.
croes 1y ago
Who watches the watchers?
exiguus 1y ago
I have so many question. Let's say a media outlet, like a newspaper, commits to common journalism best practices, such as neutrality. In my opinion, this does not mean avoiding certain words; it means presenting different perspectives on a topic based on evidence. Besides that, if someone writes an Op-Ed or commentary, the author expresses their personal opinion. How does this work with this AI thingy? Currently it looks like plain censorship, isn't it? And why should I buy a newspaper that does not represent my political or personal opinion? I thought that was how it worked.
- eddythompson80 1y ago
  well, in this particular example I think the answer is straightforward: it's just an ad so it doesn't matter. This is a marketing/sales piece for LexisNexis by their marketing agency as they try to sell AI services to their customers (law firms and other legal professions). It gives them a news article to share with their customers and tell them something like "see, we're taking the AI plunge. You should too. We offer a bunch of legal AI services, lets talk". Now their CEO and sales people can keep saying "I'm sure you've seen the news about Law360. We think AI is finally ready for legal tasks. You know, as an assistant." In reality, they are probably to run the article, look at what the AI says and shrug and move on.

63 points by 12_throw_away 1y ago

jerf 1y ago

Today, somewhere in the world, some people made an event occur. Some people approved and some people disapproved. It impacted some people more than others, which some people think may be a bad thing, although others think that it may have been a good thing. The event stemmed from many past events, which many people have various strong opinions about. Politicians made various strong statements about the event, as well as a few celebrities. The future impact of the event is under debate, though experts agree that the situation is complicated and people should probably think about it very carefully before coming to conclusions. It is unclear what the market will do in response to this event.

Herring 1y ago
> Party A: wants to abolish slavery > Party B: wants way more slaves > Media: record low percentage of Americans satisfied with agriculture today! Impartiality is very important.
- bilbo0s 1y ago
  In unrelated news: General Sherman Begins Demolition in Georgia For Urgently Needed Infrastructure Projects I mean, you could ask ChatGPT or something to make some pretty braindead impartial newspaper headlines about, like, the Holocaust or slavery. I guess you're right, in some cases, impartiality will come off as a bit ridiculous to the common sense segment of society.
nxobject 1y ago
Welp, I’ve read every newspaper article - at least I have an excuse not to read the news anymore for my sanity.
encom 1y ago
Breaking News: Some Bullshit Happening Somewhere: <https://www.youtube.com/watch?v=9U4Ha9HQvMo https://www.youtube.com/watch?v=9U4Ha9HQvMo>
deleted 1y ago
[deleted]

add-sub-mul-div 1y ago

"Bias" and "narrative" have become meaningless words that people use when encountering ideas they don't like. Or when they believe that neither side could ever be right about an issue, that it's virtuous for a stance to be in the middle of the current Overton window.

nh23423fefe 1y ago
I don't like this idea. Bias is lying to advance your position. Narrative is telling a story to persuade instead of making an argument to persuade.
- JoshTriplett 1y ago
  While I'd favor defining words rather than claiming they're meaningless, those aren't accurate definitions. "Narrative" in this context is more like "attempting to steer the current discourse by making connections among things and positioning them as part of a coherent story", which can be positive (if the result is accurate and reasonable and helps people better understand what's going on) or negative (if it's spin or manufacturing consent). It's "narrative" to say "you should be afraid of X, it's the cause of all your problems". It's also "narrative" to say "here are the five different things we're currently doing to improve Y, and how they tie together into a coherent picture". Also, bias would be easier to deal with if it were always "lying", or if it advanced a coherent position. It's much broader than that.
Gormo 1y ago
I don't think it's entirely unreasonable to presume that more valid positions are likely to be found closer to the midpoint of the current Overton window. I suspect that as a matter of Bayesian probability, more extreme positions are more likely to be wrong.
- MangoToupe 1y ago
  On which topic? Saying the average person can't decide between two viewpoints on any given topic seems ridiculous.

owisd 1y ago

The problem with trying too hard to neutrally report both sides of a story is that's not unbiased either, it's just biased in favour of the side that can lie the most convincingly.

wredcoll 1y ago
I am so, so tired, of "both sides" reporting. How about just report the facts instead?
- schmidtleonard 1y ago
  It's a rare issue that you can't swing both ways by selecting the subset of facts to focus on. Attention doesn't just matter, it's all that matters. This is why adversarial forums are so important even if they so often degenerate into shouting past one another: each side has the ability and motivation to bring its strongest facts, so they are at least present in the discussion, even if debate tactics add tons of noise and confirmation bias dampens the effects.
  - wredcoll 1y ago
    Yes, in principle, but I object to "reporters" merely quoting people instead of attempting to actually verify if what they said has any basis in reality. That being said, I'm not sure how much this actually matters, trump seems to pretty comprehensively prove that people would rather hear lies, even if they know they are lies.
- an0malous 1y ago
  Because there’s basically no such thing and choosing which facts you report is a huge source of bias anyway
- kube-system 1y ago
  Nearly all political issues are rooted in philosophical differences of opinion. "The facts" are merely the icing on the cake that various factions use to lure people to their ideology. Factual reporting is better than direct opinion reporting, but both are inherently interpreted by people in the context of the Overton window.

JoshTriplett 1y ago

Timeless quote: > If Jimmy Allred says it’s raining, and W. Lee O’Daniel says it isn’t raining, Texas newspapermen quote them both, and don’t look out the window to see which is lying, and to tell the readers what the truth is at the moment. https://quoteinvestigator.com/2023/11/14/rain-look/ https://quoteinvestigator.com/2023/11/14/rain-look/ If an AI tool, or for that matter a meddling editor, says a headline is "framing the action as unprecedented in a way that might subtly critique the administration", the correct response is "yes, that was the idea".

sandwichsphinx 1y ago
reminds me of the essay "Politics and the English Language" George Orwell wrote in 1946, it's a good read
zaphar 1y ago
I don't think most reporting bias takes the form of incorrect facts. It takes the form of picking which facts to share driven by the facts that matter the most to the particular reporter sharing them. This results in slanted coverage even if technically it's "factual".
- JoshTriplett 1y ago
  Take a look at the headline flagged in the article, which the tool flagged as "framing the action as unprecedented in a way that might subtly critique the administration". The headline is factually accurate, and the facts are not cherry-picked. Yes, some reporting is biased. But some reporting is simply accurately reporting damaging information, and "biased" is a way of attacking that without addressing the substance of the problem.
  - zaphar 1y ago
    I agree that article's headline is pretty factual nor are the facts cherry-picked. But "some reporting is biased" is heavily understating the problem. And I hypothesize that the reason there is a leap to "This is biased" today on reporting is because the news media organizations have participated in the cherry-picked facts case to enough of a degree that trust has eroded. Which is perhaps why organizations are searching for ways to not appear biased and begin to restore that trust. I think the reason the LLM approach is likely to fail is because it is unable to detect the "missing facts" case and can only really advise about sentiment and phrasing. Which is not I think the actual problem that needs fixing.

BrenBarn 1y ago

Who checks the bias detector for bias?

ethan_smith 1y ago
This creates an infinite regress problem - bias detection systems are themselves trained on data reflecting human judgments about what constitutes "bias."
kelseyfrog 1y ago
That's the nice thing. You don't. The conceit of centrism/Liberalism is that extreme voices cancel out. Just ingest a large enough corpus and the central limit theorum takes care of the rest.

jasonthorsness 1y ago

LLMs and ML algorithms are beginning to influence the entire lifecycle of articles: researching, writing, editing, publication, discovery (TikTok), and consumption (ChatGPT summarize this). With the few big players, it could be the same model involved at every step. It's scary how a small change to a system prompt could subtly influence things across the board and guide popular opinion.

wredcoll 1y ago

Remember kids, if you don't like the truth, just accuse it of being biased and force it to change!

bilbo0s 1y ago
Truth is biased. That's what makes it truth. If you wanted unbiased information, you should have asked for facts.. and then collected as many of them as you possibly could.
- MangoToupe 1y ago
  More people should really read Baudrillard. Objectivity and language don't combine very well. Show me a person who wants the facts and I'll show you someone who is easily manipulated.

deleted 1y ago

[deleted]

saaaaaam 1y ago

I said to a colleague the other day that Law360’s reporting seems to be more tortuous and weird than usual. I wonder if this is why. There was one piece on a case I’ve been following quite closely where I was genuinely unable to make sense of what had happened from the Law360 report without sitting down and spelling my way through it.

throwaway984393 1y ago

[dead]

pkilgore 1y ago

What I do not understand is this top-down mandate that people use specific tools. Mandate productivity, mandate quality; the tools will be adopted if they are sharp. But elevating the way work is performed above the work itself? Weird.

lovich 1y ago
I don't know how you read this article and came away that they were merely trying to force adoption of AI tools. This is LexisNexis requiring all of their workers run their output through a censorship machine to comply with the whims of the current administration. It has zero benefits to the workers so of course they have to mandate it
mike_hearn 1y ago
It's because the executives don't trust the journalists to do the work properly. The article says that explicitly: customers have complained about the quality of the work. The business owners view unhappy customers as a threat to their business, and so have introduced automation to make product quality higher and more even. The workers are unhappy because they don't want to be automated away and resent the implication that their artisanally hand-crafted articles aren't good enough. This is a very old story that's been going on for hundreds of years. There's nothing special about it.

jauntywundrkind 1y ago

> Several sentences in the story were flagged as biased, including this one: "It’s the first time in 60 years that a president has mobilized a state’s National Guard without receiving a request to do so from the state’s governor." According to the bias indicator, this sentence is "framing the action as unprecedented in a way that might subtly critique the administration." It was best to give more context to “balance the tone." "Might subtly critique"?! This isn't protecting versus bias, this isn't even charitably regardable as cowardice. It's an attack on reason & reporting. To shield the worst cry bullies & offenders. It's ghastly how basic simple reporting so offends, that the public interest is sabotaged by these foes. We face such a need to improve the media, and this is the opposite path, of craven blunt un-intelligently intermediated corporatism.

iFire 1y ago

Does this mean automated censorship by AI "bias" detection?

MangoToupe 1y ago

To me, a newsroom's bias IS its value. If you let me know your values, I can actually interpret what you're saying. It's the pretense of impartiality that makes me distrust someone.

croes 1y ago

Who watches the watchers?

exiguus 1y ago

I have so many question. Let's say a media outlet, like a newspaper, commits to common journalism best practices, such as neutrality. In my opinion, this does not mean avoiding certain words; it means presenting different perspectives on a topic based on evidence. Besides that, if someone writes an Op-Ed or commentary, the author expresses their personal opinion. How does this work with this AI thingy? Currently it looks like plain censorship, isn't it? And why should I buy a newspaper that does not represent my political or personal opinion? I thought that was how it worked.

eddythompson80 1y ago
well, in this particular example I think the answer is straightforward: it's just an ad so it doesn't matter. This is a marketing/sales piece for LexisNexis by their marketing agency as they try to sell AI services to their customers (law firms and other legal professions). It gives them a news article to share with their customers and tell them something like "see, we're taking the AI plunge. You should too. We offer a bunch of legal AI services, lets talk". Now their CEO and sales people can keep saying "I'm sure you've seen the news about Law360. We think AI is finally ready for legal tasks. You know, as an assistant." In reality, they are probably to run the article, look at what the AI says and shrug and move on.