8 ms·

A few weeks ago my girlfriend made me discover Instagram filters. Obviously they're not in "deep fake" territory but I was still massively impressed by the qual

by simias 7y ago

A few weeks ago my girlfriend made me discover Instagram filters. Obviously they're not in "deep fake" territory but I was still massively impressed by the quality of the face recognition and tracking coupled with some rather advanced 3D reconstructions on top. 15 years ago that would've been science fiction, today it's a free gimmicky feature in an app running on commodity hardware. There are also apps that will smooth your skin or change the shape of your eyes in real time. All these technologies are evolving at a ridiculously fast pace, I'm sure that a decade from now you'll be able to do a much more convincing fake in a fraction of the time. We're getting deeper and deeper into a post-truth world.

ryanwaggoner 7y ago
I don't really understand this argument at all. We've been in this "post truth" world for a long time already for every form of media other than video. Text quotes, photos, and audio can all be easily faked. If I post a ridiculous quote here and say it's from Obama, you won't believe me. But if the NYTimes does the same thing, it carries a lot more weight. We've been here for a long time already, now those standards will just apply to video as well. Doesn't seem that crazy. Also, I'm fascinated by the idea of using AI to detect the use of AI. How good will your deep fake have to be in order to fool a deep fake detection AI?
- siffland 7y ago
  Does this mean now we have to coin the phrase "deepfake news" I can just hear people saying it now. Joking aside, and not taking in ANY ethical/unethical misuse considerations this software is capable of, it is really cool tech.
- ceejayoz 7y ago
  > But if the NYTimes does the same thing, it carries a lot more weight. Sure, and that's part of the danger. Convincing deep fakes will be used to delegitimize mainstream news organizations, as with what happened with Dan Rather - a trusted source presented faked documents.
  - ryanwaggoner 7y ago
    But why is this the danger now in a way that it hasn't been previously? You could have presented fake documents for hundreds of years now, but it's not like we don't have any credible news organizations as a result.
    - adrianN 7y ago
      It gets increasingly cheap to manufacture convincing fakes, but it doesn't get any cheaper to do quality journalism. The risk as I see it is that there is such an overwhelming tsunami of manufactured reality that it becomes impossible to tell truth from fiction.
      - setr 7y ago
        It was never that expensive to manufacture fake text. Minorly expensive to manufacture fake images. And we've been hit by tabloids, spam and so on before. None of that is new. Overwhelmed by content is/was/will be a problem that is solved very simply: people limit the distribution lanes they consume, and naturally establish "trust networks" and hierarchies. Nyt didn't become "trusted" by accident. People aren't as dumb as you seem to think. Anyways the problem is of distribution, not that a lie can be told. If we can't trust our distribution lanes to actually reflect the institutions we're trusting, then we can't establish our trust networks. And this is a real problem: our distribution lanes are divorced now from our actual information distrubuters; medium, Facebook, Twitter, etc fuck around with our trust networks, randomly injecting their own bullshit into our feeds and messing around with feed-order based on non-trust metrics (eg money paid) such that they've become fairly unreliable. And our classical trustable institutions have become less trustworthy, as they flounder about trying to make sense of the "digital age", and have so far done so in a pathetic fashion
        deleted 7y ago
        [deleted]
        adrianN 7y ago
        In the past you had to pay a person to write that fake text. Wait a few more years and you'll see convincing text coming out of some algorithm at the cost of an EC2 instance.
- jakear 7y ago
  If you post a ridiculous quote, say it’s from Obama, then provide a link to a video where Obama says that quote, more people believe you. People already believe tabloid journalism. Just because you might exclusively trust NYTimes doesn’t mean easily accessible mechanisms for fabricating evidence aren’t a problem for a massive chunk of the world population. And what about when this happens at a state-sponsored level (This is absolutely already in development, if it hasn’t happened already...). “Did you see that Fox News exclusive video leaked from the Bernie Sanders campaign? Yeah he was holding a private dinner for a bunch of Billionaires. They say they got it from some Russian oligarch he invited. So much for grass roots change, I think I’ll just stay home this Election Day.” Or the corollary: “That smoking-gun tape the Do-Nothing Dems have of me at Putin’s personal sex trafficking center, where the two of us are laughing about having Epstein take the hit? Fake news! They’re called deepfakes, and they make these just absolutely incredible fake videos. Not real at all though. Only trust what the Government-sanctioned media says!” As per your other point, given the GANs that are used to make these are precisely a deep fake AI fooling a deep fake detection AI, aren’t we already there? Further, when the government is making the generator, a privately trained discriminator has no chance. How would you even train that? Get the government to give you a bunch of generated fake media so that you can develop a really spot-on model? I don’t think so. Two predictions: Deepfakes will play a roll in the 2020 election. Deepfakes will play a roll in every election thereafter. If this doesn’t concern you, I don’t know what would. The thing with the “post-truth world” is that once people think we’re in it, we are.
  - Gunax 7y ago
    I don't think the argument is that deepfakes won't matter. It's that they don't represent a new paradigm, but are instead just an additional medium (video) which must be more scrutinized than before. Long have documents and news been the subject of fraud. And for a few decades photo alterations have been common. Now we are adding video to the list. But there is nothing new about the faking of libelous material (eg Killian documents 2004)
    - jakear 7y ago
      Video is massively different for swaying public opinion. There’s a reason most news on Facebook/Instagram is in the form of 15-60 second video montages. People can tune in quickly, get information passively, and leave. Video isn’t just “another item on the list”. It’s the most powerful means of sharing information we have. How many people do you think actually read those “Killian documents”? How many people do you think a state-sponsored social media campaign built on the most accessible media would reach?
- jotm 7y ago
  Richard Stallman himself said: "With software there are only two outcomes: either the users control the program or the program controls the users. If the program is not under user control, and the developer controls everything, then the program is an instrument of unjust power, applied by the powers that be." Deepfakes are just a continuation of "fake news" (formerly known as propaganda), and the end outcome is we either succumb to those who use it, or adapt - filtering out the bs isn't easy, but it's not an insurmountable task, either.
- jml7c5 7y ago
  I don't think it's so much that people will be fooled by fake video, as that people can no longer rely on video as evidence. Up until now, video has been the gold standard of "proof" that an event happened. Quotes, photos, and (to an extent) audio have been fakeable for a long time, and they are mistrusted accordingly. And while CGI has slowly made it harder to spot fake events, videos of speech remained mostly safe. It's won't upend the world, but we are losing a hugely helpful tool that has been relied upon for 60 years.
  - mrec 7y ago
    It'll certainly be weakened, but I don't think it'll necessarily have to be discarded altogether. The need now will be for processes and tools to guarantee the provenance of a video. The technology to fake paper ballots is trivial, but they still (mostly) work fine for elections, because we've developed strict rules for how they need to be handled in order to retain credibility.
    - PavlovsCat 7y ago
      > The need now will be for processes and tools to guarantee the provenance of a video. Having a vague description of a need is not the same as having a workable solution, it doesn't even mean being on the way to it.
- krick 7y ago
  I was saying something like that for a long time now, but recently I started thinking that it's even more on an non-issue than I thought. I mean, what I was saying is that "yeah, this is all new and weird, but when everybody started using Photohop it was also weird, and yet we are moving on". But recently I read "Tlön, Uqbar, Orbis Tertius", and I thought that the point Borges was making could be easily described as critique of "fake-news post-realism of today", except it was 80 years ago. And while the notion of this problem not being new at first seemed weird to me, eventually I figured this is pretty natural: while 80 (or 800 for that matter) years ago there wasn't any such tools to manipulate information on that scale, nobody had the access to information on that scale in the first place. So anyone could be fooled by more powerful actor anyway. Offensive tools of any time are roughly the same as the defensive tools. It's just that these 20 years at the turn of the century were somewhat the sweet spot, when we had this "Internet for people like us", so it was us who was fooling the less knowledgeable people, while keeping the belief that we are relatively safe from being maleficently disinformed by somebody else. And now we can clearly see it's not necessarily the case anymore, ao it feels like something new. But it really isn't. As always.
- aianus 7y ago
  > How good will your deep fake have to be in order to fool a deep fake detection AI? The best generator and the best detector are actually part of the same model! If you create a better detector, you are making the generator better at the same time so you have not accomplished anything. "GANs are a clever way of training a generative model by framing the problem as a supervised learning problem with two sub-models: the generator model that we train to generate new examples, and the discriminator model that tries to classify examples as either real (from the domain) or fake (generated). The two models are trained together in a zero-sum game, adversarial, until the discriminator model is fooled about half the time, meaning the generator model is generating plausible examples." https://machinelearningmastery.com/what-are-generative-adversarial-networks-gans/ https://machinelearningmastery.com/what-are-generative-adver...
  - muttled 7y ago
    Is the generator always assumed to be able to keep up when the discriminator is improved algorithmically? I've always wondered that when I hear this counter-argument.
    - PeterisP 7y ago
      Well, from the pure GAN theory perspective, it is theoretically possible for a perfect generator to exist which generates samples that perfectly match the 'source data distribution' (the only question is how do we obtain one or close to it), and thus a perfect discriminator can't exist - a fake image could be realistic and indistinguishable in all aspects. And it does seem that the human-visible weaknesses of many currently published deepfake examples are those that would be hard for the current discriminator architectures to detect, not those which would be harder to generate than any other part of the fake.
- xamuel 7y ago
  >We've been in this "post truth" world for a long time already for every form of media other than video The film "Mr. Smith Goes to Washington" came out in 1939. It should be mandatory viewing.
- ojosilva 7y ago
  The problem of fake news is not actually that you (a rhetoric "you"), with a presumably above average intelligence, is able select or be skeptical about your sources of truth, but: 1) The researched human behavior where "1000 lies must be equal to 1 truth". https://www.researchgate.net/publication/256486791_Credibility_of_repeated_statements_Memory_for_trivia https://www.researchgate.net/publication/256486791_Credibili... 2) Monopolizing the information stream: the attention grabbing effectiveness of deepfake videos to convey fake messages and its novelty incentive to share them on social media, added to the fear-of-missing-out they generate on viewers and the fact that "the effort required to refute b* is 10 times greater than the effort required to produce it": https://www.telegraphindia.com/india/social-media-fatigue-and-fear-of-missing-out-drive-fake-news-forward-says-study/cid/1693562 https://www.telegraphindia.com/india/social-media-fatigue-an... 3) voter apathy caused by news fatigue due to the sheer volume of fake news, to which deepfake adds a new vector: https://www.nytimes.com/2019/11/18/us/polls-media-fake-news.html https://www.nytimes.com/2019/11/18/us/polls-media-fake-news...., https://www.datadriveninvestor.com/2019/07/04/chronicle-fatigue-syndrome-why-news-saturation-is-more-dangerous-than-fake-news/ https://www.datadriveninvestor.com/2019/07/04/chronicle-fati... 4) A majority of youngsters can't really spot (or care) about fake anymore: https://qz.com/1750839/most-young-2020-us-election-voters-cant-spot-fake-news/ https://qz.com/1750839/most-young-2020-us-election-voters-ca...
Pandabob 7y ago
It's my understanding (and experience) that Snapchat filters are even more impressive and that Snap CEO Evan Spiegel has pretty much bet the company on AR [1]. [1]: https://www.theverge.com/2019/10/22/20927521/snap-growth-earnings-ceo-user-ar-augmented-reality https://www.theverge.com/2019/10/22/20927521/snap-growth-ear...
- creaghpatr 7y ago
  The snapchat filters have been pretty mindblowing. I don't use them often but it's wild how far they've come in a few months.
  - cosmodisk 7y ago
    Agree.One of my slightly younger colleges introduced me to Snapchat filters: that's damn impressive!
FlyingAvatar 7y ago
Do you mean Snapchat filters?
- deleted 7y ago
  [deleted]
- Operyl 7y ago
  Nope, pretty sure Instagram has their own as well.
  - Nexxxeh 7y ago
    They may also be referring to the third party ones that people use before posting to Insta. Facetune possibly being the most well known.
thepete2 7y ago
What I find most worrying is the 2nd order consequence of not being able to trust pictures anymore.
- trianglem 7y ago
  I feel like that ship sailed a while ago with Photoshop being around. I think the problem now is you can do the same with videos and the technology is more commoditized so you don’t have to be very knowledgeable to do these things.
  - panarky 7y ago
    People predicted that Photoshop was the end of truth, that journalism could no longer be trusted, that photographic evidence at criminal trials would be inadmissible. None of that actually happened because people learned to be skeptical, and to care about who took the photo, the chain of custody, and the controls to prevent digital alteration. Maybe we need something similar but more rigorous for video. The camera itself could encode a PKI signature in every frame of the original. If subsequent edits preserve frames as is, then the viewer could determine how many frames were cut or duplicated. If subsequent edits crop, recolor or otherwise alter the frames, that editor could encode its own digital signature so the viewer could judge how much to trust their edits.
    - seriousaccount 7y ago
      I don't agree with your assessment. People didn't learn to be skeptical most of the time. We've seen what fake news does to people's opinions; debunking a video as fake after it was viewed by a large group of people is enough to do serious damage to people's viewpoints.
    - danShumway 7y ago
      > Maybe we need something similar but more rigorous for video. Is there evidence that deepfake videos will be worse than photos? My (naive) sense is that even for an AI, editing a photo will always be easier than editing a video with sound. If Photoshop and deep-fake images aren't a problem, shouldn't we be less concerned about video?
fizixer 7y ago
Please don't tell this to mainstream economists though (and their buddies in tech). They don't like to be challenged on the notion that this tech progress is business-as-usual, and that obsolete jobs will be replaced with new kinds of jobs.
everdrive 7y ago
I'm old enough to remember when people still attempted to claim that fashion magazines were forcing airbrushing on women. Clearly, there's a large amount of market interest for this sort of capability.
tux1968 7y ago
> We're getting deeper and deeper into a post-truth world. Are we though? Or is it just becoming much more obvious the limitations of relying on the media in general. This very well might lead to a better understanding of truth, and our personal responsibility in understanding it. At the very least, it's not a foregone conclusion that this will lead to worse outcomes than what we have today. It might be what tips us in a new and better direction.
- fernandopj 7y ago
  I do not agree this _might_ leads us in a better direction. What we'll continue to see are rising sects of flat-earthers, anti-vaxxers, cultists, polarization and deniying of anything not-in-my-bubble, a complete lack of trust in journalism and media, and futile attempts of fact-checking while we're all drowning in fake news shared endlessy on unsuspecting users. It takes a lot of effort to show something was manipulated digitally, and you'll have to fact-check by yourself, that doesn't scale at all.
  - tux1968 7y ago
    The problem is that we have always accepted what we see because we've assumed that things are more or less as they've been presented to us, we've mostly blindly accepted the story from the social truthmakers. We've had no reason to question them. This will lead to humanity doing a lot more questioning. I think this very well might lead to some very positive outcomes. The fact that we're more aware of flat-earthers today, is probably a good thing too. I'm not 100% sure that there are more of them today than before, or if they're just more noticeable. But in either case, we should be very careful to draw too many conclusions when we look around today because we're living in a time of great transition. We're basically in our teenage years of this technology. Just as you shouldn't judge an adult by who he was at 14, I don't think we should assume we know what the adult internet will look like because we see some acne on it today.
    - erikpukinskis 7y ago
      > we have always accepted what we see because we've assumed that things are more or less as they've been presented to us Even this is too simple. Many people have lived under propaganda regimes, yes, many have bought into them, and many people have seen through them. Each new wave of information technology provides new ways to “pull the wool” over peoples’ eyes. And though a propaganda machine has power... like all things it creates and equal an opposite power that eventually destroys it. The question for individuals is only: where am I in the cycle, how long will this cycle last, and what is my role to play?
goatinaboat 7y ago
I'm sure that a decade from now you'll be able to do a much more convincing fake in a fraction of the time. We're getting deeper and deeper into a post-truth world. Maybe I’m being paranoid but it seems that the public is being conditioned to believe deepfakes are so good prior to a certain cache of video recordings being made public
PavlovsCat 7y ago
I'd say a post-truth world wouldn't be a world at all so to speak, because if everybody lives in their own filter bubble, with no access to a common world, the public sphere, where humans can have experiences and communicate, ceases to exist. > You have to remember that in democratic societies citizens talking with each other is very important. We've lost a lot of that with the mass media. Now we have an opportunity for citizens to create their own communications with each other. So when these big deals with the big companies and the big governments carve up this new territory, I feel it's very important that we keep a kind of "social green belt", that we keep the ability for citizens to talk amongst each other. -- Howard Rheingold, https://www.youtube.com/watch?v=U_o8gerare0&t=22m14s https://www.youtube.com/watch?v=U_o8gerare0&t=22m14s > If everybody always lies to you, the consequence is not that you believe the lies, but rather that nobody believes anything any longer. This is because lies, by their very nature, have to be changed, and a lying government has constantly to rewrite its own history. On the receiving end you get not only one lie — a lie which you could go on for the rest of your days — but you get a great number of lies, depending on how the political wind blows. And a people that no longer can believe anything cannot make up its mind. It is deprived not only of its capacity to act but also of its capacity to think and to judge. And with such a people you can then do what you please. -- Hannah Arendt, Interview with Roger Errera (1974) https://www.nybooks.com/articles/1978/10/26/hannah-arendt-from-an-interview/ https://www.nybooks.com/articles/1978/10/26/hannah-arendt-fr...

by simias 7y ago

ryanwaggoner 7y ago

I don't really understand this argument at all. We've been in this "post truth" world for a long time already for every form of media other than video. Text quotes, photos, and audio can all be easily faked. If I post a ridiculous quote here and say it's from Obama, you won't believe me. But if the NYTimes does the same thing, it carries a lot more weight. We've been here for a long time already, now those standards will just apply to video as well. Doesn't seem that crazy. Also, I'm fascinated by the idea of using AI to detect the use of AI. How good will your deep fake have to be in order to fool a deep fake detection AI?

siffland 7y ago
Does this mean now we have to coin the phrase "deepfake news" I can just hear people saying it now. Joking aside, and not taking in ANY ethical/unethical misuse considerations this software is capable of, it is really cool tech.
ceejayoz 7y ago
> But if the NYTimes does the same thing, it carries a lot more weight. Sure, and that's part of the danger. Convincing deep fakes will be used to delegitimize mainstream news organizations, as with what happened with Dan Rather - a trusted source presented faked documents.
- ryanwaggoner 7y ago
  But why is this the danger now in a way that it hasn't been previously? You could have presented fake documents for hundreds of years now, but it's not like we don't have any credible news organizations as a result.
  - adrianN 7y ago
    It gets increasingly cheap to manufacture convincing fakes, but it doesn't get any cheaper to do quality journalism. The risk as I see it is that there is such an overwhelming tsunami of manufactured reality that it becomes impossible to tell truth from fiction.
    - setr 7y ago
      It was never that expensive to manufacture fake text. Minorly expensive to manufacture fake images. And we've been hit by tabloids, spam and so on before. None of that is new. Overwhelmed by content is/was/will be a problem that is solved very simply: people limit the distribution lanes they consume, and naturally establish "trust networks" and hierarchies. Nyt didn't become "trusted" by accident. People aren't as dumb as you seem to think. Anyways the problem is of distribution, not that a lie can be told. If we can't trust our distribution lanes to actually reflect the institutions we're trusting, then we can't establish our trust networks. And this is a real problem: our distribution lanes are divorced now from our actual information distrubuters; medium, Facebook, Twitter, etc fuck around with our trust networks, randomly injecting their own bullshit into our feeds and messing around with feed-order based on non-trust metrics (eg money paid) such that they've become fairly unreliable. And our classical trustable institutions have become less trustworthy, as they flounder about trying to make sense of the "digital age", and have so far done so in a pathetic fashion
      - deleted 7y ago
        [deleted]
      - adrianN 7y ago
        In the past you had to pay a person to write that fake text. Wait a few more years and you'll see convincing text coming out of some algorithm at the cost of an EC2 instance.
jakear 7y ago
If you post a ridiculous quote, say it’s from Obama, then provide a link to a video where Obama says that quote, more people believe you. People already believe tabloid journalism. Just because you might exclusively trust NYTimes doesn’t mean easily accessible mechanisms for fabricating evidence aren’t a problem for a massive chunk of the world population. And what about when this happens at a state-sponsored level (This is absolutely already in development, if it hasn’t happened already...). “Did you see that Fox News exclusive video leaked from the Bernie Sanders campaign? Yeah he was holding a private dinner for a bunch of Billionaires. They say they got it from some Russian oligarch he invited. So much for grass roots change, I think I’ll just stay home this Election Day.” Or the corollary: “That smoking-gun tape the Do-Nothing Dems have of me at Putin’s personal sex trafficking center, where the two of us are laughing about having Epstein take the hit? Fake news! They’re called deepfakes, and they make these just absolutely incredible fake videos. Not real at all though. Only trust what the Government-sanctioned media says!” As per your other point, given the GANs that are used to make these are precisely a deep fake AI fooling a deep fake detection AI, aren’t we already there? Further, when the government is making the generator, a privately trained discriminator has no chance. How would you even train that? Get the government to give you a bunch of generated fake media so that you can develop a really spot-on model? I don’t think so. Two predictions: Deepfakes will play a roll in the 2020 election. Deepfakes will play a roll in every election thereafter. If this doesn’t concern you, I don’t know what would. The thing with the “post-truth world” is that once people think we’re in it, we are.
- Gunax 7y ago
  I don't think the argument is that deepfakes won't matter. It's that they don't represent a new paradigm, but are instead just an additional medium (video) which must be more scrutinized than before. Long have documents and news been the subject of fraud. And for a few decades photo alterations have been common. Now we are adding video to the list. But there is nothing new about the faking of libelous material (eg Killian documents 2004)
  - jakear 7y ago
    Video is massively different for swaying public opinion. There’s a reason most news on Facebook/Instagram is in the form of 15-60 second video montages. People can tune in quickly, get information passively, and leave. Video isn’t just “another item on the list”. It’s the most powerful means of sharing information we have. How many people do you think actually read those “Killian documents”? How many people do you think a state-sponsored social media campaign built on the most accessible media would reach?
jotm 7y ago
Richard Stallman himself said: "With software there are only two outcomes: either the users control the program or the program controls the users. If the program is not under user control, and the developer controls everything, then the program is an instrument of unjust power, applied by the powers that be." Deepfakes are just a continuation of "fake news" (formerly known as propaganda), and the end outcome is we either succumb to those who use it, or adapt - filtering out the bs isn't easy, but it's not an insurmountable task, either.
jml7c5 7y ago
I don't think it's so much that people will be fooled by fake video, as that people can no longer rely on video as evidence. Up until now, video has been the gold standard of "proof" that an event happened. Quotes, photos, and (to an extent) audio have been fakeable for a long time, and they are mistrusted accordingly. And while CGI has slowly made it harder to spot fake events, videos of speech remained mostly safe. It's won't upend the world, but we are losing a hugely helpful tool that has been relied upon for 60 years.
- mrec 7y ago
  It'll certainly be weakened, but I don't think it'll necessarily have to be discarded altogether. The need now will be for processes and tools to guarantee the provenance of a video. The technology to fake paper ballots is trivial, but they still (mostly) work fine for elections, because we've developed strict rules for how they need to be handled in order to retain credibility.
  - PavlovsCat 7y ago
    > The need now will be for processes and tools to guarantee the provenance of a video. Having a vague description of a need is not the same as having a workable solution, it doesn't even mean being on the way to it.
krick 7y ago
I was saying something like that for a long time now, but recently I started thinking that it's even more on an non-issue than I thought. I mean, what I was saying is that "yeah, this is all new and weird, but when everybody started using Photohop it was also weird, and yet we are moving on". But recently I read "Tlön, Uqbar, Orbis Tertius", and I thought that the point Borges was making could be easily described as critique of "fake-news post-realism of today", except it was 80 years ago. And while the notion of this problem not being new at first seemed weird to me, eventually I figured this is pretty natural: while 80 (or 800 for that matter) years ago there wasn't any such tools to manipulate information on that scale, nobody had the access to information on that scale in the first place. So anyone could be fooled by more powerful actor anyway. Offensive tools of any time are roughly the same as the defensive tools. It's just that these 20 years at the turn of the century were somewhat the sweet spot, when we had this "Internet for people like us", so it was us who was fooling the less knowledgeable people, while keeping the belief that we are relatively safe from being maleficently disinformed by somebody else. And now we can clearly see it's not necessarily the case anymore, ao it feels like something new. But it really isn't. As always.
aianus 7y ago
> How good will your deep fake have to be in order to fool a deep fake detection AI? The best generator and the best detector are actually part of the same model! If you create a better detector, you are making the generator better at the same time so you have not accomplished anything. "GANs are a clever way of training a generative model by framing the problem as a supervised learning problem with two sub-models: the generator model that we train to generate new examples, and the discriminator model that tries to classify examples as either real (from the domain) or fake (generated). The two models are trained together in a zero-sum game, adversarial, until the discriminator model is fooled about half the time, meaning the generator model is generating plausible examples." https://machinelearningmastery.com/what-are-generative-adversarial-networks-gans/ https://machinelearningmastery.com/what-are-generative-adver...
- muttled 7y ago
  Is the generator always assumed to be able to keep up when the discriminator is improved algorithmically? I've always wondered that when I hear this counter-argument.
  - PeterisP 7y ago
    Well, from the pure GAN theory perspective, it is theoretically possible for a perfect generator to exist which generates samples that perfectly match the 'source data distribution' (the only question is how do we obtain one or close to it), and thus a perfect discriminator can't exist - a fake image could be realistic and indistinguishable in all aspects. And it does seem that the human-visible weaknesses of many currently published deepfake examples are those that would be hard for the current discriminator architectures to detect, not those which would be harder to generate than any other part of the fake.
xamuel 7y ago
>We've been in this "post truth" world for a long time already for every form of media other than video The film "Mr. Smith Goes to Washington" came out in 1939. It should be mandatory viewing.
ojosilva 7y ago
The problem of fake news is not actually that you (a rhetoric "you"), with a presumably above average intelligence, is able select or be skeptical about your sources of truth, but: 1) The researched human behavior where "1000 lies must be equal to 1 truth". https://www.researchgate.net/publication/256486791_Credibility_of_repeated_statements_Memory_for_trivia https://www.researchgate.net/publication/256486791_Credibili... 2) Monopolizing the information stream: the attention grabbing effectiveness of deepfake videos to convey fake messages and its novelty incentive to share them on social media, added to the fear-of-missing-out they generate on viewers and the fact that "the effort required to refute b* is 10 times greater than the effort required to produce it": https://www.telegraphindia.com/india/social-media-fatigue-and-fear-of-missing-out-drive-fake-news-forward-says-study/cid/1693562 https://www.telegraphindia.com/india/social-media-fatigue-an... 3) voter apathy caused by news fatigue due to the sheer volume of fake news, to which deepfake adds a new vector: https://www.nytimes.com/2019/11/18/us/polls-media-fake-news.html https://www.nytimes.com/2019/11/18/us/polls-media-fake-news...., https://www.datadriveninvestor.com/2019/07/04/chronicle-fatigue-syndrome-why-news-saturation-is-more-dangerous-than-fake-news/ https://www.datadriveninvestor.com/2019/07/04/chronicle-fati... 4) A majority of youngsters can't really spot (or care) about fake anymore: https://qz.com/1750839/most-young-2020-us-election-voters-cant-spot-fake-news/ https://qz.com/1750839/most-young-2020-us-election-voters-ca...

Pandabob 7y ago

It's my understanding (and experience) that Snapchat filters are even more impressive and that Snap CEO Evan Spiegel has pretty much bet the company on AR [1]. [1]: https://www.theverge.com/2019/10/22/20927521/snap-growth-earnings-ceo-user-ar-augmented-reality https://www.theverge.com/2019/10/22/20927521/snap-growth-ear...

creaghpatr 7y ago
The snapchat filters have been pretty mindblowing. I don't use them often but it's wild how far they've come in a few months.
- cosmodisk 7y ago
  Agree.One of my slightly younger colleges introduced me to Snapchat filters: that's damn impressive!

FlyingAvatar 7y ago

Do you mean Snapchat filters?

deleted 7y ago
[deleted]
Operyl 7y ago
Nope, pretty sure Instagram has their own as well.
- Nexxxeh 7y ago
  They may also be referring to the third party ones that people use before posting to Insta. Facetune possibly being the most well known.

thepete2 7y ago

What I find most worrying is the 2nd order consequence of not being able to trust pictures anymore.

trianglem 7y ago
I feel like that ship sailed a while ago with Photoshop being around. I think the problem now is you can do the same with videos and the technology is more commoditized so you don’t have to be very knowledgeable to do these things.
- panarky 7y ago
  People predicted that Photoshop was the end of truth, that journalism could no longer be trusted, that photographic evidence at criminal trials would be inadmissible. None of that actually happened because people learned to be skeptical, and to care about who took the photo, the chain of custody, and the controls to prevent digital alteration. Maybe we need something similar but more rigorous for video. The camera itself could encode a PKI signature in every frame of the original. If subsequent edits preserve frames as is, then the viewer could determine how many frames were cut or duplicated. If subsequent edits crop, recolor or otherwise alter the frames, that editor could encode its own digital signature so the viewer could judge how much to trust their edits.
  - seriousaccount 7y ago
    I don't agree with your assessment. People didn't learn to be skeptical most of the time. We've seen what fake news does to people's opinions; debunking a video as fake after it was viewed by a large group of people is enough to do serious damage to people's viewpoints.
  - danShumway 7y ago
    > Maybe we need something similar but more rigorous for video. Is there evidence that deepfake videos will be worse than photos? My (naive) sense is that even for an AI, editing a photo will always be easier than editing a video with sound. If Photoshop and deep-fake images aren't a problem, shouldn't we be less concerned about video?

fizixer 7y ago

Please don't tell this to mainstream economists though (and their buddies in tech). They don't like to be challenged on the notion that this tech progress is business-as-usual, and that obsolete jobs will be replaced with new kinds of jobs.

everdrive 7y ago

I'm old enough to remember when people still attempted to claim that fashion magazines were forcing airbrushing on women. Clearly, there's a large amount of market interest for this sort of capability.

tux1968 7y ago

> We're getting deeper and deeper into a post-truth world. Are we though? Or is it just becoming much more obvious the limitations of relying on the media in general. This very well might lead to a better understanding of truth, and our personal responsibility in understanding it. At the very least, it's not a foregone conclusion that this will lead to worse outcomes than what we have today. It might be what tips us in a new and better direction.

fernandopj 7y ago
I do not agree this _might_ leads us in a better direction. What we'll continue to see are rising sects of flat-earthers, anti-vaxxers, cultists, polarization and deniying of anything not-in-my-bubble, a complete lack of trust in journalism and media, and futile attempts of fact-checking while we're all drowning in fake news shared endlessy on unsuspecting users. It takes a lot of effort to show something was manipulated digitally, and you'll have to fact-check by yourself, that doesn't scale at all.
- tux1968 7y ago
  The problem is that we have always accepted what we see because we've assumed that things are more or less as they've been presented to us, we've mostly blindly accepted the story from the social truthmakers. We've had no reason to question them. This will lead to humanity doing a lot more questioning. I think this very well might lead to some very positive outcomes. The fact that we're more aware of flat-earthers today, is probably a good thing too. I'm not 100% sure that there are more of them today than before, or if they're just more noticeable. But in either case, we should be very careful to draw too many conclusions when we look around today because we're living in a time of great transition. We're basically in our teenage years of this technology. Just as you shouldn't judge an adult by who he was at 14, I don't think we should assume we know what the adult internet will look like because we see some acne on it today.
  - erikpukinskis 7y ago
    > we have always accepted what we see because we've assumed that things are more or less as they've been presented to us Even this is too simple. Many people have lived under propaganda regimes, yes, many have bought into them, and many people have seen through them. Each new wave of information technology provides new ways to “pull the wool” over peoples’ eyes. And though a propaganda machine has power... like all things it creates and equal an opposite power that eventually destroys it. The question for individuals is only: where am I in the cycle, how long will this cycle last, and what is my role to play?

goatinaboat 7y ago

I'm sure that a decade from now you'll be able to do a much more convincing fake in a fraction of the time. We're getting deeper and deeper into a post-truth world. Maybe I’m being paranoid but it seems that the public is being conditioned to believe deepfakes are so good prior to a certain cache of video recordings being made public

PavlovsCat 7y ago

I'd say a post-truth world wouldn't be a world at all so to speak, because if everybody lives in their own filter bubble, with no access to a common world, the public sphere, where humans can have experiences and communicate, ceases to exist. > You have to remember that in democratic societies citizens talking with each other is very important. We've lost a lot of that with the mass media. Now we have an opportunity for citizens to create their own communications with each other. So when these big deals with the big companies and the big governments carve up this new territory, I feel it's very important that we keep a kind of "social green belt", that we keep the ability for citizens to talk amongst each other. -- Howard Rheingold, https://www.youtube.com/watch?v=U_o8gerare0&t=22m14s https://www.youtube.com/watch?v=U_o8gerare0&t=22m14s > If everybody always lies to you, the consequence is not that you believe the lies, but rather that nobody believes anything any longer. This is because lies, by their very nature, have to be changed, and a lying government has constantly to rewrite its own history. On the receiving end you get not only one lie — a lie which you could go on for the rest of your days — but you get a great number of lies, depending on how the political wind blows. And a people that no longer can believe anything cannot make up its mind. It is deprived not only of its capacity to act but also of its capacity to think and to judge. And with such a people you can then do what you please. -- Hannah Arendt, Interview with Roger Errera (1974) https://www.nybooks.com/articles/1978/10/26/hannah-arendt-from-an-interview/ https://www.nybooks.com/articles/1978/10/26/hannah-arendt-fr...