22 ms·

Twitter to label deepfakes and other deceptive media

155 points by jonbaer 7y ago

irjustin 7y ago
I'll keep an eye on this one. I'm hopeful because we need something like this, but man it's going to be hard with LOTS of problems on both sides of the line. Guys who get labeled as fake who didn't deserve it and guys who should be labeled not getting it. All the false positive/negatives will have a terrible time.
- libertine 7y ago
  The problem of these scalable approaches is well documented (not scientifically as far as I know), but in several forms of media. You said well the consequences: - False positives - the wrongfully labelled ones ; - """hackers""" - because there's not enough amount of '"' to make it sound different but bare with me, it's just those who learn/are lucky to manipulate/circumvent the system; Even if this are small percentage cases, it will be in the order of hundreds of thousands, if not millions of users. How do you give support to this people? Is it going to be another Youtube fuck-fest where you end up talking to bots? Because this shouldn't be allowed - at all. If you want to make automated control measures, then people should have the right to talk directly to a human who can actually solve their problems. Yet that seems to be impossible, because of the scale.
zpeti 7y ago
This seems like an insanely hard problem with many false positives/negatives. But of course the media hysteria against tech and fake news might eat it up and praise twitter. Nice PR move for non techies.
- klundqist 7y ago
  Yeah I'm most concerned about the false sense of security this gives. It won't find deep fakes from unpublished methods.
- michaelt 7y ago
  > Nice PR move for non techies. Depends if the praise they get for "at least they're doing something/not pretending the problem doesn't exist" outweighs the criticism they get for false positives, false negatives, inconsistency/bias, arbitrary judgements, response times, or labelling so many things fake that it becomes meaningless. Way back in 1994, Time magazine published a photo of OJ Simpson that was photoshopped to make his skin darker [1] - but having some colour-correction of magazine cover photos is inevitable. The line between regular and deceptive colour correction is blurry, and some people will be upset no matter where you try to draw it. [1] https://people.southwestern.edu/~bednarb/su_netWorks/projects/enyioha/O.J.Simpson.html https://people.southwestern.edu/~bednarb/su_netWorks/project...
  - makomk 7y ago
    If they get the right false positives, false negatives, and bias, it might enhance their media reputation overall. For example, a while ago the White House posted a video of an altercation with a journalist which a lot of journalists believed had been manipulated to make it look worse than it was. Now, this obviously wasn't true on looking at the video, and indeed a couple of outlets hired video forensics experts who concluded this - but AP managed to find a non-expert who was merely a video editor and confirmed their belief by mistaking signs of frame rate conversion for the video being sped up, and the press ran with it. Meanwhile, there was an a very convincing deceptively-edited video circulating on Twitter which clamed to show parts had been sped up by overlaying the White House version on the original - it was very vivid, with the overlay seeming to jump ahead in fast-moving parts. This was an illusion created by running the videos 7 frames or so out of sync the whole way through and abusing the fact the slight differences weren't noticable in slow-moving parts during normal playback. No-one noticed. Youtube fighter of misinformation Captain Disillusion even helped spread it and gave his approval. My general reckoning is that Twitter would get much more praise for labelling the non-tampered video as misleadingly edited and endorsing the accuracy of the video that was misleadingly edited to make the other one appear that way, than the other way around.
- Ragnarork 7y ago
  Deep fakes are a thing. Fake news are a thing. Not sure what's achieved by pointing out some fantasized "hysteria" of the media about it, especially when that bundles up everyone in the same basket, although some of these are actually relying on these methods to push their views. Attempting to label these so people aren't manipulated by it is something that deserves praise. It's not a free pass of course, it does seem insanely hard, and it better materializes into something concrete with a real impact that does solve the issues at hand.
  - kmlx 7y ago
    > Attempting to label these so people aren't manipulated by it is something that deserves praise. it's pretty incredible how we went from "give people the benefit of the doubt" to "people are manipulated".
    - Ragnarork 7y ago
      I admit I could have phrased it better. What I implied is that the end result has been that people got manipulated by such techniques (which I don't blame them for, as these methods have been engineered to be extremely persuasive and credible-looking). I did not imply that everyone was de-facto manipulated by those. Only that it can already happen.
  - finnthehuman 7y ago
    >label these so people aren't manipulated by it is something that deserves praise. Except for all the purposefully manipulative things they allow on their platform and don't label or attempt to debunk. You could say they're targeting the biggest problem first, but twitter isn't going to touch politician's tweets or the mainstream news en mass. If you're worried about the average twitter users' critical thinking skills, don't you think those important-sounding sources of information should be THE key targets of a anti-misleading or anti-manipulative campaigns?
L_226 7y ago
Very interested to know how this would work technically, or if it is just another task for the army of human content moderators. Is the end result of this just going to be that deepfake generators face natural selection pressure to improve beyond the exclusion threshold of this feature?
Fnoord 7y ago
Would not surprise me when, eventually, browsers are going to mark content as fake. If there's DBs of ad networks, there might also be DBs of deepfakes.
- diegoperini 7y ago
  How come this hasn't happened for news articles without video? Or has it? I am aware of the bias issue but that never stopped volunteers before.
- mkl 7y ago
  New ad networks are not continuously automatically generated. I think DBs will be an impossible way to do it, as they will always lag far behind. Automatic detection is probably the only possibility, and it is likely to become very fallible as the generated content gets better.
- syshum 7y ago
  The problem with is always how do you tell if something is actually fake. Most of the time it ends up with Truth = Deep Pockets, i.e CNN, NY Times, Fox News, MSNBC, LA Times, etc are all "True" even if they do spread false info. and Random YouTuber's or Blogs are labeled as "Fake" if if the info is true.
- Nasrudith 7y ago
  If anything looking for "unreal" patterns to the pixels would probably work better than a database and even that has limits. I know before the moral panic of deep fakes started there were subreddits with them on movie scenes and they had obvious distortions in flesh tones.
- EasyTiger_ 7y ago
  I could definitely see Chrome start doing this.
- bagacrap 7y ago
  If browsers were in the business of general censorship wouldn't they already apply a DB based blocking system to child porn? When deepfakes actually become easy to produce, and pervasive, then humans will stop trusting videos in the same way most folks do not blindly trust still images (an image of Trump's head perched on top of a Mr Olympia body would not fool anyone). At that point I doubt browsers will need to take any explicit censory action.
ailideex 7y ago
I have yet to see a deep fake that was passed of as real. Is there any major incidents of this happening that I'm not aware of?
- slipheen 7y ago
  https://www.cbsnews.com/news/doctored-nancy-pelosi-video-highlights-threat-of-deepfake-tech-2019-05-25/ https://www.cbsnews.com/news/doctored-nancy-pelosi-video-hig...
  - busymom0 7y ago
    That’s not a deep fake. If that’s deep fake, then simply slowing down a video gets qualified as a deep fake.
    - Mangalor 7y ago
      They're both forms of misleading manipulation. DL is just more capable.
    - slipheen 7y ago
      The story says that Twitter will be labeling any "deceptively edited forms of media", not just the product of machine learning algorithms. > Twitter under its new policy will similarly apply a “false” > warning label to any photos or videos that have been > “significantly and deceptively altered or fabricated,” > although it will not differentiate between the technologies > used to manipulate a piece of media.
- 3fe9a03ccd14ca5 7y ago
  The cynical side of me says all of this clamoring to stop “deep fakes” from politicians is just a bunch of people getting ready to mount a defense in case a particular video of them visiting Epstein’s island becomes public.
kmlx 7y ago
1. how is this not a slippery slope? 2. have we completely given up on critical thinking and education? 3. free speech vs not so free speech. i guess we're choosing "not so free"?
- krapp 7y ago
  1. How is allowing the unchecked proliferation of false and misleading media not also a slippery slope, and why is that never a concern? 2. False dichotomy - Twitter attempting to label deepfakes and deceptive media has no negative effect on critical thinking or education, just as Twitter not doing has no positive effect on either. Twitter is not setting education policy for any society or government, here. 3. There is no government or society in the world which does not place some limits on speech in some form, even the US has laws against terrorist threats, slander, libel, lying under oath, misleading labels on food and drugs, misleading contracts, etc ... demonstrating that one common and widely accepted basis for limiting speech is deception and misleading information. Also, merely classifying data is not preventing its spread, and is not, therefore, a limit on free speech even in theory. If that were the case, then every tag on Twitter would also somehow be "not free speech." Also, Twitter is a privately owned platform, not a government or a public square.
  - kmlx 7y ago
    > 1. How is allowing the proliferation of false and misleading media not also a slippery slope, and why is that never a concern? we have had this for the past 400 years (since the birth of the modern press). why do we need it now? how did you, for example, simply decide you can't rely on other people's good judgement? what triggered this change?
    - krapp 7y ago
      >we have had this for the past 400 years (since the birth of the modern press). why do we need it now? We've also had experts and arbiters of truth for 400 years. Why do we need to abandon the premise that truth has value in civil society? >how did you, for example, simply decide you can't rely on other people's good judgement? what triggered this change? I personally have never made any such decision. Trust is unavoidable either way. Twitter wants people to trust their good judgement in determining what is and isn't a deepfake, fabricated of false information. If you choose not to trust Twitter, you inevitably choose to trust the source, or the community, or your own bias (created and fed from third party sources.) No one is able to independently and objectively verify every claim they encounter. The only (as yet undetermined) relevant factor here is whether algorithms are any better at determining truthiness than the web zeitgeist. If so, then I see no harm done. If not, the feature will probably be abandoned. I don't particularly buy into the implied third option that Twitter and other platforms will be using features like this to intentionally mislead the public in order to further some radical political agenda of misinformation and thought control.
ducttape12 7y ago
You don't need advanced AI to create hyper realistic deep fake news. You just need to take a picture of X candidate, stick a made up quote beneath them, and put it on Facebook. There's enough people on social media who will share that without verifying it.
- 45ure 7y ago
  The fake-ness is even more effective, if the said candidate is the originator and controls the narrative.
  - pferde 7y ago
    If a person makes up a quote and distributes it with their own picture, is it really a made-up quote, is it "fake"? :)
    - bilekas 7y ago
      "So proud to be the Man of the year 2020" - Me Its fake, just because I said it, doesn't make the content true.
      - dTal 7y ago
        This seems to be a language drift worth nipping in the bud. It's not "fake". It's a lie. When something purports to be something it's not, "fake" is an adjective describing that thing. "Lie" is the adjective that describes the claim. If you tweet that you are Man of the Year 2020, you are fake, a phony. The tweet, on the other hand, is simply a lie.
        bilekas 7y ago
        But the content is the article in this case. The content is fake.. No ? Maybe I have messed up the semantics. I'm not trying to be difficult, but is it not valid in this context to correlate "lie" with "fake" ?
sschueller 7y ago
If we are going to mark news and tweets as "out of context" or "altered video" I would like a link to be included to the original full source so I can see it for myself.
- deleted 7y ago
  [deleted]
sascha_sl 7y ago
Based on twitter engineering reliability and how well Twitter deals with reports, I'd give every flagged image about a 50% change to be real. The other 50% will be obvious jokes.
ReptileMan 7y ago
Truth is too important to let the public decide what is true. I think that twitter doesn't have enough resources to bear this burden alone. So it is better if the state give a helping hand. A dedicated ministry of truth should be there when Twitter cannot hold the line against the ever more devious deceptions. Also since a lot of the speech online is hate speech we should probably also make a state organization that helps people to love each other. A ministry of love.
- geofft 7y ago
  This seems reasonable, but I'm a bit worried you're being sarcastic. Are you worried about this for some reason? Maybe scared of something you read in a fiction book once? Is this a good idea in reality?
  - pequalsnp 7y ago
    He is paraphrasing parts of 1984 by George Orwell. Your comment must be sarcastic as well. Please tell me that you don't actually believe that the government should hold a monopoly on truth...
    - geofft 7y ago
      Yeah, I'm aware there's a fictional book being referenced. Back in the real world - what does "monopoly on truth" mean?
- SkyBelow 7y ago
  I like the general idea of where you are going, but you go way too fast towards the end and jump the tracks. I think this post would work much better if you dropped the ministry of love bit and were slower introducing the ministry of truth. I would also suggest avoiding the phrase "ministry of truth" and stick to something a bit more generic, such as "a ministry dedicated to uncovering falsehoods". Still very on point to the extent of having a feeling of lacking subtlety, but without the over the top forced feeling of using the exact phrase.
MisterTea 7y ago
Twitter ignites cat and mouse war with deepfakes and other deceptive media FTFY.
pjc50 7y ago
There are obvious cases of deceptive media, where things are simply made up, but the most invidious ones are just "differential spin" and very hard to pin down. The recent Harry/Megan fiasco has provided a great set of examples on how the same action can be reported as a positive or negative depending on the paper's preferred spin: https://www.buzzfeednews.com/article/ellievhall/meghan-markle-kate-middleton-double-standards-royal https://www.buzzfeednews.com/article/ellievhall/meghan-markl...
- wyldfire 7y ago
  Goodness! The avocado one was two stories fewer than eighteen months apart. It almost looks like they were intending to get caught.
  - pjc50 7y ago
    More that they don't care if they get caught. You could run diametrically opposing articles a week apart and not expect the readership to care enough to stop buying the paper. Several papers run opposite stories locally: DMG Media (owners of the Mail) notoriously did this with the HPV vaccine. Irish Mail were in favour, UK Daily Mail were against, on the same day. The Sun vs Scottish Sun is another repeat offender.
    - dTal 7y ago
      Sounds a bit like A/B testing with public opinion.
      - dsfyu404ed 7y ago
        Or catering to two different groups. Like selling arms to both sides.
      - 45ure 7y ago
        I would say they are Red-teaming (any relation to red-tops is unintentional). They already know their audience, so they strive to reduce the unknowns or false-positives.
geofft 7y ago
Are they gonna label fake screenshots to? Because unlike deepfakes, that's an actual problem today.
dsfyu404ed 7y ago
By allowing us to combine audio, video and text on a whim modern computing technology has opened the door to a whole new kind of communication and like any new form of communication anyone who strongly believes in their cause uses it as a tool to further their cause. Right now we're going through a rough patch because the effort required to forge images/video is dropping and the old heuristic of "if it's a video it's probably legit" is going out the window. Simultaneously the internet is re-arranging who gets to gate keep mass communication and to what extent. Both these things are going to cause strife. I think it's useful to remember what the printing press did to Christianity in Europe in the 1500s and what mass literacy did to monarchies in the late 18th through 19th century. Some blood was spilled but I think we can agree the printing press and mass literacy have been massive net positives to civilization. I won't be losing any sleep over whether TwatBookGram does or doesn't censor anything. It will all work itself out in the end.
tsukurimashou 7y ago
So, where do we exactly draw the line with these things? Considering everything you see on media has been altered in some ways. To me it looks like the same problem as "fake news", how do you define "fake news" considering every piece of news has at least some bias from the person that wrote it. I think it would be a better idea to educate people about these things, just like when "Photoshop" and other similar software started to pick up, a lot of people started creating "fake images", nowadays I feel like most people are aware that you cannot trust a picture, because they can be so easily edited.
- bilekas 7y ago
  > I think it would be a better idea to educate people about these things This argument is nonsense an innefective. Try and explain to your elders about what a deep fake is, and how to identify one. The movie : "The Irishman" used a lot of the technology successfully to take years off the cast. Applying labels to known false media is not the same as an opinion based 'fact' or alternative facts. So don't come with this absolute strawman point of : "Ooo where is the line." - The line is at fake. Always has been. Whats been blurred is the idea that Facts are not Facts. They are. An opinion is not a fact. A deepFake is Fake. Hence the name. And its about time people grew up and used their intelligence to realise whats a FACT and whats not a FACT. Whats real and whats fake. It's a shame its up to companies like Twitter to do this for everyone, but clearly too many people believe everything they see & hear without question.
  - blackearl 7y ago
    > and other deceptive media Ok where's the line on deceptive media? A deepfake obviously has concrete criteria to identify it. What about a little spin? Cropping out a piece of a picture or a few seconds of a video? Muting a crucial bit of audio? What about that silly scenario where a reporter goes out in a kayak to show flooded streets and a cheeky passerby walks behind them in knee-high galoshes? Those all seem deceptive. I don't trust Twitter to have my best interests at heart when deciding. Then again, I don't use Twitter so I don't really care.
    - 7y ago
jug 7y ago
I honestly think oversimplified clickbait titles which inspire people to not even read the article but rather generate rings in their echo chamber as they share it is way more dangerous than the occasional deepfake. When are we going to start shaming clickbait? Hey, another thing in this topic: It would be very cool if a social network could somehow indicate if a shared/retweeted link had not been read for a while in advance. Imagine if a network would also have this system implemented for a good while, like a year, to then retroactively show this information on all links. I wonder what it would look like...
izzydata 7y ago
It will become a good training tool for the deepfake software. It clearly isn't deepfake enough if twitter can label it fake or not.
nathias 7y ago
like whatever can cause harm is so vague that its just a legitimation of censorship for profit/ideology
option 7y ago
Yesterday, I searched for a children song from an old soviet movie (my childhood) to play to my kids. Nothing political whatsoever. Youtube promptly showed me the warning for that channel saying that “VGTRK is funded by the Russian government” (which is true). Out of curiosity, I browsed throw tons of (political) clips posted by BBC and did not see a similar warning (regarding UK goverment) or anything close to it.
- jedieaston 7y ago
  I’m getting the warning for the BBC on this video: https://youtu.be/dRggIb5Je_w https://youtu.be/dRggIb5Je_w And I just checked, the CBC gives a similar warning as well.
  - option 7y ago
    yes, I see it now too. Though it doesn’t say “funded by British government”, it says “BBC is a British public broadcaster”
kyrieeschaton 7y ago
Of course they didn't restrict themselves to "deepfakes"; they made sure to encompass anything "deceptive", "manipulated" media, "tropes", and "unwanted attention" - which, given every video clip has a start and end point, seems tailor made to allow them to flag eg clips of Ralph Northam endorsing infanticide or Hillary Clinton collapsing in public as leading to an incorrect political conclusion rather than being actually false.
- Nuzzerino 7y ago
  Will they go as far as to censor statements by politicians such as this? https://dailycaller.com/2016/10/09/media-bias-nbc-news-fact-check-declares-false-that-hillary-acid-washed-her-server/ https://dailycaller.com/2016/10/09/media-bias-nbc-news-fact-...
  - iscrewyou 7y ago
    Hopefully. Same as certain individuals yelling innocence while obstructing and tampering jury as well as evidence.
    - Nuzzerino 7y ago
      That was meant to be rhetorical, but I forgot that Poe's law was a thing.
      - iscrewyou 7y ago
        Looking at the rest of the thread, it was hard to get a rhetorical vibe from your comment. But I get it now.
- Mangalor 7y ago
  Personally, given 2016 and now Iowa, I would rather err on the side of allowing users to find "on the fence" content outside of Twitter, rather than letting the propaganda chips fall where they may. These are ultra-powerful tools, and our society clearly hasn't mastered them (yet).
golemotron 7y ago
They are setting themselves up to become the humor police.
nabnob 7y ago
Twitter and other social media companies already heavily censor users that fall outside of the overton window (left or right). And now they have another justificiation to keep doing this, with all the buzz around deepfakes. It amazes me that people who consider themselves leftists are ok with massive corporations deciding what information they're able to see.
- ravenstine 7y ago
  > It amazes me that people who consider themselves leftists are ok with massive corporations deciding what information they're able to see. I used to be amazed, but not anymore. I've come to accept that people will always practice tribalism and will defy any principles to achieve their end.
v7x 7y ago
Twitter labeled it, so it must be fake! Twitter didn't label it, so it must be real!
lwb 7y ago
Considering who Twitter gives blue check marks to these days, I'm not convinced that the "false" label will be applied with much more integrity.
friendlybus 7y ago
This problem is in the human mind, there will always be multiple ways to interpret information. It's fundamental. The move to force two different interpretations into one is a cultural and political shift, not a technological move. The fires of cognitive dissonance and a fixation on hate will be ended. Valid interpretations will be set by a handful of tech companies regardless of the value lost. This is the Thanos snap that has been coming, two halves of the populace into one. Hate is dead, long live the heartless.
Bartweiss 7y ago
Twitter has an actual post up announcing the policy, with a fair bit more detail on tiers of actions and reasons for actions. (The big one that alarms me is "reduced visibility", which seems far more manipulative than labeling or deletion.) Link: https://blog.twitter.com/en_us/topics/company/2020/new-approach-to-synthetic-and-manipulated-media.html https://blog.twitter.com/en_us/topics/company/2020/new-appro...
deleted 7y ago
[deleted]
SeanFerree 7y ago
Love it! Good job Twitter
deminature 7y ago
Facebook received enormous pressure after the 2016 election to get their platform under control and hired over 30,000 manual moderators to try to prevent their platform from being leveraged by malicious actors to spread misinformation. [1] It's reasonable that Twitter should begin to be less permissive on their platform too, to get ahead of controversies that might occur on their platform in the upcoming election. Twitter political threads are still lawless wastelands full of incendiary comments from suspiciously bot-like accounts, so clearly something has to be done. [1] https://www.telegraph.co.uk/technology/2019/12/17/google-facebooks-moderators-superheroes-need-protection/ https://www.telegraph.co.uk/technology/2019/12/17/google-fac...
sixstringtheory 7y ago
Glad to see they’ve solved the problem of harassment and fake accounts and are moving on to much harder challenges.
- pradn 7y ago
  Multiple teams can work on separate abuse patterns, you know.
zadkey 7y ago
What happens when content from mainstream media gets flagged as a deepfake? What kind of verification is used to ensure that there are not false positives, or misuse for nefarious purposes? You know twitter is a company, it has a CEO, a board of directors, and is beholden to the wishes of the shareholders. What happens if shareholders want certain types of content flagged in a way itself is deceptive or is conducive to false conclusions about the status or quality of content? What happens when governments apply pressure? What kinds of safeguards in place to prevent this kind of thing? How do you secure such a system from becoming a tool of manipulation?

155 points by jonbaer 7y ago

I'll keep an eye on this one. I'm hopeful because we need something like this, but man it's going to be hard with LOTS of problems on both sides of the line. Guys who get labeled as fake who didn't deserve it and guys who should be labeled not getting it. All the false positive/negatives will have a terrible time.

libertine 7y ago
The problem of these scalable approaches is well documented (not scientifically as far as I know), but in several forms of media. You said well the consequences: - False positives - the wrongfully labelled ones ; - """hackers""" - because there's not enough amount of '"' to make it sound different but bare with me, it's just those who learn/are lucky to manipulate/circumvent the system; Even if this are small percentage cases, it will be in the order of hundreds of thousands, if not millions of users. How do you give support to this people? Is it going to be another Youtube fuck-fest where you end up talking to bots? Because this shouldn't be allowed - at all. If you want to make automated control measures, then people should have the right to talk directly to a human who can actually solve their problems. Yet that seems to be impossible, because of the scale.

zpeti 7y ago

This seems like an insanely hard problem with many false positives/negatives. But of course the media hysteria against tech and fake news might eat it up and praise twitter. Nice PR move for non techies.

klundqist 7y ago
Yeah I'm most concerned about the false sense of security this gives. It won't find deep fakes from unpublished methods.
michaelt 7y ago
> Nice PR move for non techies. Depends if the praise they get for "at least they're doing something/not pretending the problem doesn't exist" outweighs the criticism they get for false positives, false negatives, inconsistency/bias, arbitrary judgements, response times, or labelling so many things fake that it becomes meaningless. Way back in 1994, Time magazine published a photo of OJ Simpson that was photoshopped to make his skin darker [1] - but having some colour-correction of magazine cover photos is inevitable. The line between regular and deceptive colour correction is blurry, and some people will be upset no matter where you try to draw it. [1] https://people.southwestern.edu/~bednarb/su_netWorks/projects/enyioha/O.J.Simpson.html https://people.southwestern.edu/~bednarb/su_netWorks/project...
- makomk 7y ago
  If they get the right false positives, false negatives, and bias, it might enhance their media reputation overall. For example, a while ago the White House posted a video of an altercation with a journalist which a lot of journalists believed had been manipulated to make it look worse than it was. Now, this obviously wasn't true on looking at the video, and indeed a couple of outlets hired video forensics experts who concluded this - but AP managed to find a non-expert who was merely a video editor and confirmed their belief by mistaking signs of frame rate conversion for the video being sped up, and the press ran with it. Meanwhile, there was an a very convincing deceptively-edited video circulating on Twitter which clamed to show parts had been sped up by overlaying the White House version on the original - it was very vivid, with the overlay seeming to jump ahead in fast-moving parts. This was an illusion created by running the videos 7 frames or so out of sync the whole way through and abusing the fact the slight differences weren't noticable in slow-moving parts during normal playback. No-one noticed. Youtube fighter of misinformation Captain Disillusion even helped spread it and gave his approval. My general reckoning is that Twitter would get much more praise for labelling the non-tampered video as misleadingly edited and endorsing the accuracy of the video that was misleadingly edited to make the other one appear that way, than the other way around.
Ragnarork 7y ago
Deep fakes are a thing. Fake news are a thing. Not sure what's achieved by pointing out some fantasized "hysteria" of the media about it, especially when that bundles up everyone in the same basket, although some of these are actually relying on these methods to push their views. Attempting to label these so people aren't manipulated by it is something that deserves praise. It's not a free pass of course, it does seem insanely hard, and it better materializes into something concrete with a real impact that does solve the issues at hand.
- kmlx 7y ago
  > Attempting to label these so people aren't manipulated by it is something that deserves praise. it's pretty incredible how we went from "give people the benefit of the doubt" to "people are manipulated".
  - Ragnarork 7y ago
    I admit I could have phrased it better. What I implied is that the end result has been that people got manipulated by such techniques (which I don't blame them for, as these methods have been engineered to be extremely persuasive and credible-looking). I did not imply that everyone was de-facto manipulated by those. Only that it can already happen.
- finnthehuman 7y ago
  >label these so people aren't manipulated by it is something that deserves praise. Except for all the purposefully manipulative things they allow on their platform and don't label or attempt to debunk. You could say they're targeting the biggest problem first, but twitter isn't going to touch politician's tweets or the mainstream news en mass. If you're worried about the average twitter users' critical thinking skills, don't you think those important-sounding sources of information should be THE key targets of a anti-misleading or anti-manipulative campaigns?

L_226 7y ago

Very interested to know how this would work technically, or if it is just another task for the army of human content moderators. Is the end result of this just going to be that deepfake generators face natural selection pressure to improve beyond the exclusion threshold of this feature?

Fnoord 7y ago

Would not surprise me when, eventually, browsers are going to mark content as fake. If there's DBs of ad networks, there might also be DBs of deepfakes.

diegoperini 7y ago
How come this hasn't happened for news articles without video? Or has it? I am aware of the bias issue but that never stopped volunteers before.
mkl 7y ago
New ad networks are not continuously automatically generated. I think DBs will be an impossible way to do it, as they will always lag far behind. Automatic detection is probably the only possibility, and it is likely to become very fallible as the generated content gets better.
syshum 7y ago
The problem with is always how do you tell if something is actually fake. Most of the time it ends up with Truth = Deep Pockets, i.e CNN, NY Times, Fox News, MSNBC, LA Times, etc are all "True" even if they do spread false info. and Random YouTuber's or Blogs are labeled as "Fake" if if the info is true.
Nasrudith 7y ago
If anything looking for "unreal" patterns to the pixels would probably work better than a database and even that has limits. I know before the moral panic of deep fakes started there were subreddits with them on movie scenes and they had obvious distortions in flesh tones.
EasyTiger_ 7y ago
I could definitely see Chrome start doing this.
bagacrap 7y ago
If browsers were in the business of general censorship wouldn't they already apply a DB based blocking system to child porn? When deepfakes actually become easy to produce, and pervasive, then humans will stop trusting videos in the same way most folks do not blindly trust still images (an image of Trump's head perched on top of a Mr Olympia body would not fool anyone). At that point I doubt browsers will need to take any explicit censory action.

ailideex 7y ago

I have yet to see a deep fake that was passed of as real. Is there any major incidents of this happening that I'm not aware of?

slipheen 7y ago
https://www.cbsnews.com/news/doctored-nancy-pelosi-video-highlights-threat-of-deepfake-tech-2019-05-25/ https://www.cbsnews.com/news/doctored-nancy-pelosi-video-hig...
- busymom0 7y ago
  That’s not a deep fake. If that’s deep fake, then simply slowing down a video gets qualified as a deep fake.
  - Mangalor 7y ago
    They're both forms of misleading manipulation. DL is just more capable.
  - slipheen 7y ago
    The story says that Twitter will be labeling any "deceptively edited forms of media", not just the product of machine learning algorithms. > Twitter under its new policy will similarly apply a “false” > warning label to any photos or videos that have been > “significantly and deceptively altered or fabricated,” > although it will not differentiate between the technologies > used to manipulate a piece of media.
3fe9a03ccd14ca5 7y ago
The cynical side of me says all of this clamoring to stop “deep fakes” from politicians is just a bunch of people getting ready to mount a defense in case a particular video of them visiting Epstein’s island becomes public.

kmlx 7y ago

1. how is this not a slippery slope? 2. have we completely given up on critical thinking and education? 3. free speech vs not so free speech. i guess we're choosing "not so free"?

krapp 7y ago
1. How is allowing the unchecked proliferation of false and misleading media not also a slippery slope, and why is that never a concern? 2. False dichotomy - Twitter attempting to label deepfakes and deceptive media has no negative effect on critical thinking or education, just as Twitter not doing has no positive effect on either. Twitter is not setting education policy for any society or government, here. 3. There is no government or society in the world which does not place some limits on speech in some form, even the US has laws against terrorist threats, slander, libel, lying under oath, misleading labels on food and drugs, misleading contracts, etc ... demonstrating that one common and widely accepted basis for limiting speech is deception and misleading information. Also, merely classifying data is not preventing its spread, and is not, therefore, a limit on free speech even in theory. If that were the case, then every tag on Twitter would also somehow be "not free speech." Also, Twitter is a privately owned platform, not a government or a public square.
- kmlx 7y ago
  > 1. How is allowing the proliferation of false and misleading media not also a slippery slope, and why is that never a concern? we have had this for the past 400 years (since the birth of the modern press). why do we need it now? how did you, for example, simply decide you can't rely on other people's good judgement? what triggered this change?
  - krapp 7y ago
    >we have had this for the past 400 years (since the birth of the modern press). why do we need it now? We've also had experts and arbiters of truth for 400 years. Why do we need to abandon the premise that truth has value in civil society? >how did you, for example, simply decide you can't rely on other people's good judgement? what triggered this change? I personally have never made any such decision. Trust is unavoidable either way. Twitter wants people to trust their good judgement in determining what is and isn't a deepfake, fabricated of false information. If you choose not to trust Twitter, you inevitably choose to trust the source, or the community, or your own bias (created and fed from third party sources.) No one is able to independently and objectively verify every claim they encounter. The only (as yet undetermined) relevant factor here is whether algorithms are any better at determining truthiness than the web zeitgeist. If so, then I see no harm done. If not, the feature will probably be abandoned. I don't particularly buy into the implied third option that Twitter and other platforms will be using features like this to intentionally mislead the public in order to further some radical political agenda of misinformation and thought control.

ducttape12 7y ago

You don't need advanced AI to create hyper realistic deep fake news. You just need to take a picture of X candidate, stick a made up quote beneath them, and put it on Facebook. There's enough people on social media who will share that without verifying it.

45ure 7y ago
The fake-ness is even more effective, if the said candidate is the originator and controls the narrative.
- pferde 7y ago
  If a person makes up a quote and distributes it with their own picture, is it really a made-up quote, is it "fake"? :)
  - bilekas 7y ago
    "So proud to be the Man of the year 2020" - Me Its fake, just because I said it, doesn't make the content true.
    - dTal 7y ago
      This seems to be a language drift worth nipping in the bud. It's not "fake". It's a lie. When something purports to be something it's not, "fake" is an adjective describing that thing. "Lie" is the adjective that describes the claim. If you tweet that you are Man of the Year 2020, you are fake, a phony. The tweet, on the other hand, is simply a lie.
      - bilekas 7y ago
        But the content is the article in this case. The content is fake.. No ? Maybe I have messed up the semantics. I'm not trying to be difficult, but is it not valid in this context to correlate "lie" with "fake" ?

sschueller 7y ago

If we are going to mark news and tweets as "out of context" or "altered video" I would like a link to be included to the original full source so I can see it for myself.

deleted 7y ago
[deleted]

sascha_sl 7y ago

Based on twitter engineering reliability and how well Twitter deals with reports, I'd give every flagged image about a 50% change to be real. The other 50% will be obvious jokes.

ReptileMan 7y ago

Truth is too important to let the public decide what is true. I think that twitter doesn't have enough resources to bear this burden alone. So it is better if the state give a helping hand. A dedicated ministry of truth should be there when Twitter cannot hold the line against the ever more devious deceptions. Also since a lot of the speech online is hate speech we should probably also make a state organization that helps people to love each other. A ministry of love.

geofft 7y ago
This seems reasonable, but I'm a bit worried you're being sarcastic. Are you worried about this for some reason? Maybe scared of something you read in a fiction book once? Is this a good idea in reality?
- pequalsnp 7y ago
  He is paraphrasing parts of 1984 by George Orwell. Your comment must be sarcastic as well. Please tell me that you don't actually believe that the government should hold a monopoly on truth...
  - geofft 7y ago
    Yeah, I'm aware there's a fictional book being referenced. Back in the real world - what does "monopoly on truth" mean?
SkyBelow 7y ago
I like the general idea of where you are going, but you go way too fast towards the end and jump the tracks. I think this post would work much better if you dropped the ministry of love bit and were slower introducing the ministry of truth. I would also suggest avoiding the phrase "ministry of truth" and stick to something a bit more generic, such as "a ministry dedicated to uncovering falsehoods". Still very on point to the extent of having a feeling of lacking subtlety, but without the over the top forced feeling of using the exact phrase.

MisterTea 7y ago

Twitter ignites cat and mouse war with deepfakes and other deceptive media FTFY.

pjc50 7y ago

There are obvious cases of deceptive media, where things are simply made up, but the most invidious ones are just "differential spin" and very hard to pin down. The recent Harry/Megan fiasco has provided a great set of examples on how the same action can be reported as a positive or negative depending on the paper's preferred spin: https://www.buzzfeednews.com/article/ellievhall/meghan-markle-kate-middleton-double-standards-royal https://www.buzzfeednews.com/article/ellievhall/meghan-markl...

wyldfire 7y ago
Goodness! The avocado one was two stories fewer than eighteen months apart. It almost looks like they were intending to get caught.
- pjc50 7y ago
  More that they don't care if they get caught. You could run diametrically opposing articles a week apart and not expect the readership to care enough to stop buying the paper. Several papers run opposite stories locally: DMG Media (owners of the Mail) notoriously did this with the HPV vaccine. Irish Mail were in favour, UK Daily Mail were against, on the same day. The Sun vs Scottish Sun is another repeat offender.
  - dTal 7y ago
    Sounds a bit like A/B testing with public opinion.
    - dsfyu404ed 7y ago
      Or catering to two different groups. Like selling arms to both sides.
    - 45ure 7y ago
      I would say they are Red-teaming (any relation to red-tops is unintentional). They already know their audience, so they strive to reduce the unknowns or false-positives.

geofft 7y ago

Are they gonna label fake screenshots to? Because unlike deepfakes, that's an actual problem today.

dsfyu404ed 7y ago

By allowing us to combine audio, video and text on a whim modern computing technology has opened the door to a whole new kind of communication and like any new form of communication anyone who strongly believes in their cause uses it as a tool to further their cause. Right now we're going through a rough patch because the effort required to forge images/video is dropping and the old heuristic of "if it's a video it's probably legit" is going out the window. Simultaneously the internet is re-arranging who gets to gate keep mass communication and to what extent. Both these things are going to cause strife. I think it's useful to remember what the printing press did to Christianity in Europe in the 1500s and what mass literacy did to monarchies in the late 18th through 19th century. Some blood was spilled but I think we can agree the printing press and mass literacy have been massive net positives to civilization. I won't be losing any sleep over whether TwatBookGram does or doesn't censor anything. It will all work itself out in the end.

tsukurimashou 7y ago

So, where do we exactly draw the line with these things? Considering everything you see on media has been altered in some ways. To me it looks like the same problem as "fake news", how do you define "fake news" considering every piece of news has at least some bias from the person that wrote it. I think it would be a better idea to educate people about these things, just like when "Photoshop" and other similar software started to pick up, a lot of people started creating "fake images", nowadays I feel like most people are aware that you cannot trust a picture, because they can be so easily edited.

bilekas 7y ago
> I think it would be a better idea to educate people about these things This argument is nonsense an innefective. Try and explain to your elders about what a deep fake is, and how to identify one. The movie : "The Irishman" used a lot of the technology successfully to take years off the cast. Applying labels to known false media is not the same as an opinion based 'fact' or alternative facts. So don't come with this absolute strawman point of : "Ooo where is the line." - The line is at fake. Always has been. Whats been blurred is the idea that Facts are not Facts. They are. An opinion is not a fact. A deepFake is Fake. Hence the name. And its about time people grew up and used their intelligence to realise whats a FACT and whats not a FACT. Whats real and whats fake. It's a shame its up to companies like Twitter to do this for everyone, but clearly too many people believe everything they see & hear without question.
- blackearl 7y ago
  > and other deceptive media Ok where's the line on deceptive media? A deepfake obviously has concrete criteria to identify it. What about a little spin? Cropping out a piece of a picture or a few seconds of a video? Muting a crucial bit of audio? What about that silly scenario where a reporter goes out in a kayak to show flooded streets and a cheeky passerby walks behind them in knee-high galoshes? Those all seem deceptive. I don't trust Twitter to have my best interests at heart when deciding. Then again, I don't use Twitter so I don't really care.
  - 7y ago

jug 7y ago

I honestly think oversimplified clickbait titles which inspire people to not even read the article but rather generate rings in their echo chamber as they share it is way more dangerous than the occasional deepfake. When are we going to start shaming clickbait? Hey, another thing in this topic: It would be very cool if a social network could somehow indicate if a shared/retweeted link had not been read for a while in advance. Imagine if a network would also have this system implemented for a good while, like a year, to then retroactively show this information on all links. I wonder what it would look like...

izzydata 7y ago

It will become a good training tool for the deepfake software. It clearly isn't deepfake enough if twitter can label it fake or not.

nathias 7y ago

like whatever can cause harm is so vague that its just a legitimation of censorship for profit/ideology

option 7y ago

Yesterday, I searched for a children song from an old soviet movie (my childhood) to play to my kids. Nothing political whatsoever. Youtube promptly showed me the warning for that channel saying that “VGTRK is funded by the Russian government” (which is true). Out of curiosity, I browsed throw tons of (political) clips posted by BBC and did not see a similar warning (regarding UK goverment) or anything close to it.

jedieaston 7y ago
I’m getting the warning for the BBC on this video: https://youtu.be/dRggIb5Je_w https://youtu.be/dRggIb5Je_w And I just checked, the CBC gives a similar warning as well.
- option 7y ago
  yes, I see it now too. Though it doesn’t say “funded by British government”, it says “BBC is a British public broadcaster”

kyrieeschaton 7y ago

Of course they didn't restrict themselves to "deepfakes"; they made sure to encompass anything "deceptive", "manipulated" media, "tropes", and "unwanted attention" - which, given every video clip has a start and end point, seems tailor made to allow them to flag eg clips of Ralph Northam endorsing infanticide or Hillary Clinton collapsing in public as leading to an incorrect political conclusion rather than being actually false.

Nuzzerino 7y ago
Will they go as far as to censor statements by politicians such as this? https://dailycaller.com/2016/10/09/media-bias-nbc-news-fact-check-declares-false-that-hillary-acid-washed-her-server/ https://dailycaller.com/2016/10/09/media-bias-nbc-news-fact-...
- iscrewyou 7y ago
  Hopefully. Same as certain individuals yelling innocence while obstructing and tampering jury as well as evidence.
  - Nuzzerino 7y ago
    That was meant to be rhetorical, but I forgot that Poe's law was a thing.
    - iscrewyou 7y ago
      Looking at the rest of the thread, it was hard to get a rhetorical vibe from your comment. But I get it now.
Mangalor 7y ago
Personally, given 2016 and now Iowa, I would rather err on the side of allowing users to find "on the fence" content outside of Twitter, rather than letting the propaganda chips fall where they may. These are ultra-powerful tools, and our society clearly hasn't mastered them (yet).

golemotron 7y ago

They are setting themselves up to become the humor police.

nabnob 7y ago

Twitter and other social media companies already heavily censor users that fall outside of the overton window (left or right). And now they have another justificiation to keep doing this, with all the buzz around deepfakes. It amazes me that people who consider themselves leftists are ok with massive corporations deciding what information they're able to see.

ravenstine 7y ago
> It amazes me that people who consider themselves leftists are ok with massive corporations deciding what information they're able to see. I used to be amazed, but not anymore. I've come to accept that people will always practice tribalism and will defy any principles to achieve their end.

v7x 7y ago

Twitter labeled it, so it must be fake! Twitter didn't label it, so it must be real!

lwb 7y ago

Considering who Twitter gives blue check marks to these days, I'm not convinced that the "false" label will be applied with much more integrity.

friendlybus 7y ago

This problem is in the human mind, there will always be multiple ways to interpret information. It's fundamental. The move to force two different interpretations into one is a cultural and political shift, not a technological move. The fires of cognitive dissonance and a fixation on hate will be ended. Valid interpretations will be set by a handful of tech companies regardless of the value lost. This is the Thanos snap that has been coming, two halves of the populace into one. Hate is dead, long live the heartless.

Bartweiss 7y ago

Twitter has an actual post up announcing the policy, with a fair bit more detail on tiers of actions and reasons for actions. (The big one that alarms me is "reduced visibility", which seems far more manipulative than labeling or deletion.) Link: https://blog.twitter.com/en_us/topics/company/2020/new-approach-to-synthetic-and-manipulated-media.html https://blog.twitter.com/en_us/topics/company/2020/new-appro...

deleted 7y ago

[deleted]

SeanFerree 7y ago

Love it! Good job Twitter

deminature 7y ago

Facebook received enormous pressure after the 2016 election to get their platform under control and hired over 30,000 manual moderators to try to prevent their platform from being leveraged by malicious actors to spread misinformation. [1] It's reasonable that Twitter should begin to be less permissive on their platform too, to get ahead of controversies that might occur on their platform in the upcoming election. Twitter political threads are still lawless wastelands full of incendiary comments from suspiciously bot-like accounts, so clearly something has to be done. [1] https://www.telegraph.co.uk/technology/2019/12/17/google-facebooks-moderators-superheroes-need-protection/ https://www.telegraph.co.uk/technology/2019/12/17/google-fac...

sixstringtheory 7y ago

Glad to see they’ve solved the problem of harassment and fake accounts and are moving on to much harder challenges.

pradn 7y ago
Multiple teams can work on separate abuse patterns, you know.

zadkey 7y ago

What happens when content from mainstream media gets flagged as a deepfake? What kind of verification is used to ensure that there are not false positives, or misuse for nefarious purposes? You know twitter is a company, it has a CEO, a board of directors, and is beholden to the wishes of the shareholders. What happens if shareholders want certain types of content flagged in a way itself is deceptive or is conducive to false conclusions about the status or quality of content? What happens when governments apply pressure? What kinds of safeguards in place to prevent this kind of thing? How do you secure such a system from becoming a tool of manipulation?