12 ms·

Truth goggles sniff out suspicious sentences in news

78 points by mjbellantoni 15y ago

kls 15y ago
There are few ideas that I see that have the potential to change the world, but I would say this one may. A lot of people are too busy or to intellectually lazy to research facts. As such they default to non-verbal ques in candidates. The strongest being confidence. A candidate can be confident while completely fabricating information and more people will believe them than a candidate with no confidence and truth. Having a tool like this to refute the information immediately will be valuable. The trick will be the credibility of the "truth" provided. We all have biases and keeping them out of our creations can be more tricky than it would appear. No doubt if the product gains wide traction, many will accuse it of bias the author needs to make sure that they have double and triple checked the facts so that it can withstand the onslaught of accusations that will surly come from both sides.
rplnt 15y ago
Don't know why, but the page is completely broken when rendered in Opera. Perhaps the custom font?
- jodrellblank 15y ago
  Perhaps the page isn't broken, the browser is.
  - burgerbrain 15y ago
    1) The user doesn't care. Therefore it is the site's problem. 2) Works fine for me with Opera (linux). so shrug.
    - danilocampos 15y ago
      Nah. When the browser has a single digit share, it's on the browser to make things render correctly. The user went to some trouble to get that arcane tool.
      - burgerbrain 15y ago
        "The user went to some trouble to get that arcane tool." Troll much?
        danilocampos 15y ago
        Only when the mood strikes. It didn't above. "Who uses Opera?" seems to me a perfectly valid sentiment in favor of not spending one's limited energies on making sure something renders correctly for it. Hell, in a cousin, the abundantly reasonable admin of the linked site reports it's less than 1% of their traffic. Like I said, arcane. Worry about what real users use, I say.
        burgerbrain 15y ago
        The trolly-ness was specifically the sentence I quoted. Anyone who is already aware of Opera (more people than you might think, usage != awareness) will not have any trouble acquiring it. I have numerous non-technical acquaintances that use it on their often aging hardware. It's not like we're talking about elinks here or something.
        rplnt 15y ago
        Except it has share in tens of percents in some markets. The fact that it has 0.5% market share in US doesn't mean it has no market share at all. You are right that it doesn't make sense for US site to focus on Opera but your other statements are quite invalid.
- crabwalk 15y ago
  Looks fine to me in Opera Mac too, fwiw. (I run niemanlab.org, and I confess I don't do any testing in Opera, so I honestly appreciate any bug reports.) FYI, our browser breakdown: 31% Chrome, 28% Firefox, 15% desktop Safari, 11% IE, 10% iOS Safari, 3% Android browser, 0.74% Opera, 0.26% Opera Mini, 0.23% RockMelt (!).
  - rplnt 15y ago
    Might be some userscript I have installed, thanks for the feedback. And as a bonus, browser stats from country I'm from: http://gs.statcounter.com/#browser-SK-monthly-201010-201110-bar http://gs.statcounter.com/#browser-SK-monthly-201010-201110-... (You can check more countries from this region, you will see similiar results for Opera (Ukraine stands out though)). The reason for that is that Opera was the best browser without any doubt (think pre-firefox2 age). Only flaw was that it was not free - it was ad-sponsored. But you could get it for free of course, and users in Easterish Europe didn't really care.
mjbellantoni 15y ago
I've been thinking a lot lately about how technology can provide transparency to the political process. Among these ideas have been "reading aides" like this: tools that will provide verification and context for what you're reading or watching. Another such tool I've been thinking about would show you the relationship of the writer and quoted sources (people or institutions) to the material being covered. For example, consider the case where spokesperson for a non-profit organization is quoted defending a particular industry. This tool might tell you that this person was once an executive at a for-profit corporation in that industry or have some other non-obvious relationship. If anyone knows of anyone working on something like this, I'd love to learn about it.
- espeed 15y ago
  I have been working on Whybase (http://whybase.com/ http://whybase.com/), which allows you to connect ideas and analyze statements made by people. You can use it to discover insights into the reasons behind why things are true and how things are interconnected.
  - noodly 15y ago
    Can you give some example ?
rubidium 15y ago
Too bad it employs the PolitiFact database, as that itself is suspect to having its own bias. (http://blog.lib.umn.edu/cspg/smartpolitics/2011/02/selection_bias_politifact_rate.php http://blog.lib.umn.edu/cspg/smartpolitics/2011/02/selection...) Using a biased fact-checker to check facts doesn't really work well. Constructing a non-biased fact-checker is the real challenge here. I'm not sure it can be done, due to the great amount of spin that can be put on most anything political. EDIT: BTW, I'm not meaning to take sides on this, just pointing out the very real difficulties in establishing a "fact-checker". It's worth knowing that others have contested PolitiFact's methodologies. If the trustworthiness of the statement is based on a person's judgement, then the fact-checker has bias.
- Cushman 15y ago
  Uh, it looks like that accusation of bias comes down to PolitiFact grading Republicans as wrong more often that Democrats... I don't want to get all political, but there's another potential explanation for that, yes? (There's also the accusation that they target Republicans more than Democrats, but that's easily explained by their methodology, from that post: "If we look at something and we think that an elected official or talk show host is wrong, then we will fact-check it." If we posit that Republicans are wrong more often than Democrats, we would expect them to show up more often.)
  - jerf 15y ago
    Yes, but the parsimonious explanation is that neither Democratic politicians nor Republican politicians nor any other kind of politicians are particularly careful with their facts, and which you think are more prone to "lying" has more to do with the listeners than the speakers. The idea that any politician is routinely dealing in truth is laughable... and I don't mean that so much in terms of cynicism per se, but because they really have very little incentive to do the necessary huge amounts of work it would take that would bring them no gain, or even less than no gain inasmuch as it would limit their verbal weaponry. Just as one example of the sort of thing that makes it difficult for me to take Politifact ratings seriously in 30 seconds of browsing, look at: http://www.politifact.com/truth-o-meter/statements/2011/sep/22/mitt-romney/mitt-romney-repeats-claim-obama-went-around-world-/ http://www.politifact.com/truth-o-meter/statements/2011/sep/... Mitt Romney gets a "pants on fire" rating for claiming that Obama has apologized to the world for the US on various occasions. To my eyes, it's a desperate attempt to avoid the accusation by deliberately and stubbornly being as naive as possible, insisting that it wasn't anything remotely resembling an apology ("pants on fire" remember, not just "half truth", which I would actually accept just fine on this matter) because it doesn't contain the word "sorry" and never quite fully says "apology". I call it "naive" because in politics everybody knows that there's more to what's going on that just what is being said; would Politifact expect me to be naive enough that when diplomats say they had the traditional "frank exchange of views" that means the negotiations are going well, because nobody said outright they were going poorly? Of course Obama never said "I apologize for the US", but there's rather more wiggle room for interpretation than giving someone who says there were an apology a "pants on fire" rating. And to be perfectly clear, I see equally dubious straining-at-gnats attacks from conservatives on liberals as well routinely. The only way these "truth goggles" could actually work is to draw from a wide variety of sources and hope the bias cancels.
jobu 15y ago
"... software that flags suspicious claims in news articles and helps readers determine their truthiness." +1 for use of the word "truthiness". I can't believe the impact people like Stephen Colbert have on our culture and lexicon.
noodly 15y ago
If you're looking for truth in media, you're looking in wrong place.
- michael_dorfman 15y ago
  What? The article is about direct quotes from politicians. There are a lot of things you can blame the media for, but this isn't one of them.
  - noodly 15y ago
    The article is about "suspicious sentences in news", and media are news producers. This is one thing I can blame them for, due to: media needn't be objective - that's not their mission - their mission is to make money, unless they're non-profit (how many of them is ?). They use deception, to make you believe that news they produced is objective, and you shouldn't look for sources (which often are well hidden or absent), and assuming that news they produce is compatible with your worldview, you won't look for sources - you're their customer. I'm sure you're able to divide media by political views they support - if they're objective why it's possible ? The use of deception, means they're not good source of truth. The main reason I found HN and stayed here, is its news quality (almost no political bs, and really close to the source).
    - slifty 15y ago
      So how do you propose we function as a democracy if any media source that covers political issues is so full of shit that there is no point in even considering it? Your perspective is extreme, but it is a very important concern. If only someone could try to create additional information layers to help you navigate a medium that is potentially riddled with dubious claims. Maybe that person could call the project "Truth Goggles" or something.
      - adbge 15y ago
        The media's political analysis absolutely is worthless. If that's all that keeps democracy functioning, we might as well flip coins.
buckwild 15y ago
I don't understand how this can be any more effective than just cited sources at the end of a paper...
- DrewG 15y ago
  A lack of cited sources in an article doesn't necessarily imply that its false.
  - jquery 15y ago
    And cited sources does not automatically make something true.
- slifty 15y ago
  This is like saying "I don't understand how watching a movie is any more effective than just looking at a script." Not everybody has access to the script. Not everyone has time or capacity to take that script and turn it into a movie in their head. Scripts tell you a lot more than you need to know. Scripts aren't designed for audiences. The list probably goes on but hopefully you get the point.
michaelkeenan 15y ago
I've dreamed about the possibilities for automated fact-checking to improve online debate, and considered attempting to write it myself (using data sources like Wolfram Alpha and Google Public Data, and parsing natural language with a big dirty pile of regular expressions, which can't be the best approach but it's an approach I could attempt in a weekend). For example, a program could examine a text for sentences matching "[X country] is [comparison] than [Y country]" and check that against online data sources. So if you're typing a comment into reddit or wherever, it would first check your comment for claims it can parse, and alert you if you are mistaken about something so you can change it. And if you are correct about something, but didn't supply a citation, it could add it automatically. This could have several awesome effects: * It would make online debates more accurate, spreading truth and knowledge. * It might set a standard of factual accuracy; people might come to expect assertions to be backed up with citations, and regard claims as suspicious if no supporting reference is provided. * Just knowing their facts will be checked (possibly by a future version of the software being re-run on old forum comments and highlighting inaccuracies for all to see) might make people more careful in their assertions. If voice recognition technology improves, then this might eventually fact-check TV in real-time.
acgourley 15y ago
This is excellent. There has been a certain class of "debate" sites that gather opposing expert opinions on some controversy but they always had a distribution problem. The people that need this kind of information most and the least likely to go check these sites. This is a great distribution mechanism of the output of those sites. If there is a trending, controversial statement in the news it would be great to be one click away from seeing smart people discussing both sides. Now I want other kinds of goggles. Like goggles that spot popular Anachronisms. Or trending memes.
dylangs1030 15y ago
This is really awesome, but I think it would be even better if a program could be written such to flag sentences without a third party judge. Using the stylistic tendencies of lying, for example, in rhetoric. This might not be possible or plausible, but if it could be written, I think it would be at least equal to this technology.
- cynwoody 15y ago
  There are at least two reasons that's impossible. First, liars are smart. Second, not everyone spouting an untruth is a liar. If a person doesn't know he's uttering a falsehood, then the fact is unlikely to influence his style — or his speech spectrum or cadence or direction of gaze, etc.
  - dylangs1030 15y ago
    Well, I see a solution around the first reason. Just flag anything that is worded in a suspicious way. The example, "Obama has the worst ratings in modern times" sounds suspicious. You shouldn't need a fact checker to know you need to check that fact. If you could implement the same sorting method common sense has to a fact checker, you might be able to make it work. However, I admit, you might be faced with the problem that an overwhelming number of statements would be flagged. I'm not sure how you would get rid of false positives.
philwelch 15y ago
"Goggles" "sniff"? Do people even pay attention to their own metaphors anymore?

78 points by mjbellantoni 15y ago

kls 15y ago

There are few ideas that I see that have the potential to change the world, but I would say this one may. A lot of people are too busy or to intellectually lazy to research facts. As such they default to non-verbal ques in candidates. The strongest being confidence. A candidate can be confident while completely fabricating information and more people will believe them than a candidate with no confidence and truth. Having a tool like this to refute the information immediately will be valuable. The trick will be the credibility of the "truth" provided. We all have biases and keeping them out of our creations can be more tricky than it would appear. No doubt if the product gains wide traction, many will accuse it of bias the author needs to make sure that they have double and triple checked the facts so that it can withstand the onslaught of accusations that will surly come from both sides.

rplnt 15y ago

Don't know why, but the page is completely broken when rendered in Opera. Perhaps the custom font?

jodrellblank 15y ago
Perhaps the page isn't broken, the browser is.
- burgerbrain 15y ago
  1) The user doesn't care. Therefore it is the site's problem. 2) Works fine for me with Opera (linux). so shrug.
  - danilocampos 15y ago
    Nah. When the browser has a single digit share, it's on the browser to make things render correctly. The user went to some trouble to get that arcane tool.
    - burgerbrain 15y ago
      "The user went to some trouble to get that arcane tool." Troll much?
      - danilocampos 15y ago
        Only when the mood strikes. It didn't above. "Who uses Opera?" seems to me a perfectly valid sentiment in favor of not spending one's limited energies on making sure something renders correctly for it. Hell, in a cousin, the abundantly reasonable admin of the linked site reports it's less than 1% of their traffic. Like I said, arcane. Worry about what real users use, I say.
        burgerbrain 15y ago
        The trolly-ness was specifically the sentence I quoted. Anyone who is already aware of Opera (more people than you might think, usage != awareness) will not have any trouble acquiring it. I have numerous non-technical acquaintances that use it on their often aging hardware. It's not like we're talking about elinks here or something.
        rplnt 15y ago
        Except it has share in tens of percents in some markets. The fact that it has 0.5% market share in US doesn't mean it has no market share at all. You are right that it doesn't make sense for US site to focus on Opera but your other statements are quite invalid.
crabwalk 15y ago
Looks fine to me in Opera Mac too, fwiw. (I run niemanlab.org, and I confess I don't do any testing in Opera, so I honestly appreciate any bug reports.) FYI, our browser breakdown: 31% Chrome, 28% Firefox, 15% desktop Safari, 11% IE, 10% iOS Safari, 3% Android browser, 0.74% Opera, 0.26% Opera Mini, 0.23% RockMelt (!).
- rplnt 15y ago
  Might be some userscript I have installed, thanks for the feedback. And as a bonus, browser stats from country I'm from: http://gs.statcounter.com/#browser-SK-monthly-201010-201110-bar http://gs.statcounter.com/#browser-SK-monthly-201010-201110-... (You can check more countries from this region, you will see similiar results for Opera (Ukraine stands out though)). The reason for that is that Opera was the best browser without any doubt (think pre-firefox2 age). Only flaw was that it was not free - it was ad-sponsored. But you could get it for free of course, and users in Easterish Europe didn't really care.

mjbellantoni 15y ago

I've been thinking a lot lately about how technology can provide transparency to the political process. Among these ideas have been "reading aides" like this: tools that will provide verification and context for what you're reading or watching. Another such tool I've been thinking about would show you the relationship of the writer and quoted sources (people or institutions) to the material being covered. For example, consider the case where spokesperson for a non-profit organization is quoted defending a particular industry. This tool might tell you that this person was once an executive at a for-profit corporation in that industry or have some other non-obvious relationship. If anyone knows of anyone working on something like this, I'd love to learn about it.

espeed 15y ago
I have been working on Whybase (http://whybase.com/ http://whybase.com/), which allows you to connect ideas and analyze statements made by people. You can use it to discover insights into the reasons behind why things are true and how things are interconnected.
- noodly 15y ago
  Can you give some example ?

rubidium 15y ago

Too bad it employs the PolitiFact database, as that itself is suspect to having its own bias. (http://blog.lib.umn.edu/cspg/smartpolitics/2011/02/selection_bias_politifact_rate.php http://blog.lib.umn.edu/cspg/smartpolitics/2011/02/selection...) Using a biased fact-checker to check facts doesn't really work well. Constructing a non-biased fact-checker is the real challenge here. I'm not sure it can be done, due to the great amount of spin that can be put on most anything political. EDIT: BTW, I'm not meaning to take sides on this, just pointing out the very real difficulties in establishing a "fact-checker". It's worth knowing that others have contested PolitiFact's methodologies. If the trustworthiness of the statement is based on a person's judgement, then the fact-checker has bias.

Cushman 15y ago
Uh, it looks like that accusation of bias comes down to PolitiFact grading Republicans as wrong more often that Democrats... I don't want to get all political, but there's another potential explanation for that, yes? (There's also the accusation that they target Republicans more than Democrats, but that's easily explained by their methodology, from that post: "If we look at something and we think that an elected official or talk show host is wrong, then we will fact-check it." If we posit that Republicans are wrong more often than Democrats, we would expect them to show up more often.)
- jerf 15y ago
  Yes, but the parsimonious explanation is that neither Democratic politicians nor Republican politicians nor any other kind of politicians are particularly careful with their facts, and which you think are more prone to "lying" has more to do with the listeners than the speakers. The idea that any politician is routinely dealing in truth is laughable... and I don't mean that so much in terms of cynicism per se, but because they really have very little incentive to do the necessary huge amounts of work it would take that would bring them no gain, or even less than no gain inasmuch as it would limit their verbal weaponry. Just as one example of the sort of thing that makes it difficult for me to take Politifact ratings seriously in 30 seconds of browsing, look at: http://www.politifact.com/truth-o-meter/statements/2011/sep/22/mitt-romney/mitt-romney-repeats-claim-obama-went-around-world-/ http://www.politifact.com/truth-o-meter/statements/2011/sep/... Mitt Romney gets a "pants on fire" rating for claiming that Obama has apologized to the world for the US on various occasions. To my eyes, it's a desperate attempt to avoid the accusation by deliberately and stubbornly being as naive as possible, insisting that it wasn't anything remotely resembling an apology ("pants on fire" remember, not just "half truth", which I would actually accept just fine on this matter) because it doesn't contain the word "sorry" and never quite fully says "apology". I call it "naive" because in politics everybody knows that there's more to what's going on that just what is being said; would Politifact expect me to be naive enough that when diplomats say they had the traditional "frank exchange of views" that means the negotiations are going well, because nobody said outright they were going poorly? Of course Obama never said "I apologize for the US", but there's rather more wiggle room for interpretation than giving someone who says there were an apology a "pants on fire" rating. And to be perfectly clear, I see equally dubious straining-at-gnats attacks from conservatives on liberals as well routinely. The only way these "truth goggles" could actually work is to draw from a wide variety of sources and hope the bias cancels.

jobu 15y ago

"... software that flags suspicious claims in news articles and helps readers determine their truthiness." +1 for use of the word "truthiness". I can't believe the impact people like Stephen Colbert have on our culture and lexicon.

noodly 15y ago

If you're looking for truth in media, you're looking in wrong place.

michael_dorfman 15y ago
What? The article is about direct quotes from politicians. There are a lot of things you can blame the media for, but this isn't one of them.
- noodly 15y ago
  The article is about "suspicious sentences in news", and media are news producers. This is one thing I can blame them for, due to: media needn't be objective - that's not their mission - their mission is to make money, unless they're non-profit (how many of them is ?). They use deception, to make you believe that news they produced is objective, and you shouldn't look for sources (which often are well hidden or absent), and assuming that news they produce is compatible with your worldview, you won't look for sources - you're their customer. I'm sure you're able to divide media by political views they support - if they're objective why it's possible ? The use of deception, means they're not good source of truth. The main reason I found HN and stayed here, is its news quality (almost no political bs, and really close to the source).
  - slifty 15y ago
    So how do you propose we function as a democracy if any media source that covers political issues is so full of shit that there is no point in even considering it? Your perspective is extreme, but it is a very important concern. If only someone could try to create additional information layers to help you navigate a medium that is potentially riddled with dubious claims. Maybe that person could call the project "Truth Goggles" or something.
    - adbge 15y ago
      The media's political analysis absolutely is worthless. If that's all that keeps democracy functioning, we might as well flip coins.

buckwild 15y ago

I don't understand how this can be any more effective than just cited sources at the end of a paper...

DrewG 15y ago
A lack of cited sources in an article doesn't necessarily imply that its false.
- jquery 15y ago
  And cited sources does not automatically make something true.
slifty 15y ago
This is like saying "I don't understand how watching a movie is any more effective than just looking at a script." Not everybody has access to the script. Not everyone has time or capacity to take that script and turn it into a movie in their head. Scripts tell you a lot more than you need to know. Scripts aren't designed for audiences. The list probably goes on but hopefully you get the point.

michaelkeenan 15y ago

I've dreamed about the possibilities for automated fact-checking to improve online debate, and considered attempting to write it myself (using data sources like Wolfram Alpha and Google Public Data, and parsing natural language with a big dirty pile of regular expressions, which can't be the best approach but it's an approach I could attempt in a weekend). For example, a program could examine a text for sentences matching "[X country] is [comparison] than [Y country]" and check that against online data sources. So if you're typing a comment into reddit or wherever, it would first check your comment for claims it can parse, and alert you if you are mistaken about something so you can change it. And if you are correct about something, but didn't supply a citation, it could add it automatically. This could have several awesome effects: * It would make online debates more accurate, spreading truth and knowledge. * It might set a standard of factual accuracy; people might come to expect assertions to be backed up with citations, and regard claims as suspicious if no supporting reference is provided. * Just knowing their facts will be checked (possibly by a future version of the software being re-run on old forum comments and highlighting inaccuracies for all to see) might make people more careful in their assertions. If voice recognition technology improves, then this might eventually fact-check TV in real-time.

acgourley 15y ago

This is excellent. There has been a certain class of "debate" sites that gather opposing expert opinions on some controversy but they always had a distribution problem. The people that need this kind of information most and the least likely to go check these sites. This is a great distribution mechanism of the output of those sites. If there is a trending, controversial statement in the news it would be great to be one click away from seeing smart people discussing both sides. Now I want other kinds of goggles. Like goggles that spot popular Anachronisms. Or trending memes.

dylangs1030 15y ago

This is really awesome, but I think it would be even better if a program could be written such to flag sentences without a third party judge. Using the stylistic tendencies of lying, for example, in rhetoric. This might not be possible or plausible, but if it could be written, I think it would be at least equal to this technology.

cynwoody 15y ago
There are at least two reasons that's impossible. First, liars are smart. Second, not everyone spouting an untruth is a liar. If a person doesn't know he's uttering a falsehood, then the fact is unlikely to influence his style — or his speech spectrum or cadence or direction of gaze, etc.
- dylangs1030 15y ago
  Well, I see a solution around the first reason. Just flag anything that is worded in a suspicious way. The example, "Obama has the worst ratings in modern times" sounds suspicious. You shouldn't need a fact checker to know you need to check that fact. If you could implement the same sorting method common sense has to a fact checker, you might be able to make it work. However, I admit, you might be faced with the problem that an overwhelming number of statements would be flagged. I'm not sure how you would get rid of false positives.

philwelch 15y ago

"Goggles" "sniff"? Do people even pay attention to their own metaphors anymore?