5 ms·

I don’t think you should dismiss these concerns so quickly. It sounds like you have experience in this field. Perhaps that would make it easier for you to spot

I don’t think you should dismiss these concerns so quickly. It sounds like you have experience in this field. Perhaps that would make it easier for you to spot potential fakes? What about your grandma? How would she fare? And besides, the end game isn’t to fool everyone into believing a fake. No, the more insidious goal is to flood the zone with enough dis- and misinformation to overload our ability to filter it. It’s like gaslighting at scale- at some point you just stop being able to process information because it’s so voluminous and of dubious quality that you stop believing any of it.

qball 7y ago
>No, the more insidious goal is to flood the zone with enough dis- and misinformation to overload our ability to filter it. It’s like gaslighting at scale- at some point you just stop being able to process information because it’s so voluminous and of dubious quality that you stop believing any of it. This isn't new by any stretch of the imagination. It might as well outright be the job of the Fourth Estate ever since the Fourth Estate has existed (funded, as always, by ads). And when you look at the people complaining about this, which is for the most part people part of said political group (Facebook/Twitter/non-tech journalists), it makes sense that anything that could diminish their ability to control the narrative should be presented to the public as a new and novel threat. So this isn't going to result in widespread social chaos, because it's already happening. If anything, this technology will democratize what the old media is currently doing; which will likely result in driving political power towards crowdsourced content aggregation instead of the individual content creators themselves.
EGreg 7y ago
I would go further. Think of it like an ELO rating where most grandmas are 1200, and you pride yourself on being 2000. But eventually, the fakes become so good that the best humans can’t distinguish them, and half the time neither can ANY system, except with some probability. What then? An unlimited amount of glut that is indistinguishable from human writing, but can be produced with any kind of bias, by any number of fake identities and there is no reasoning with hem.
nitwit005 7y ago
There have been photographic fakes for a long time now, and they were quite good even in the pre-digital era. Have you seen it creating major social issues? I haven't observed much of anything. For the lies most people want to tell, there isn't any need to make a fake. Want to paint a protest in a bad light? Just use a real photo of a riot and falsely label it as the current protest.
- kevingadd 7y ago
  With how readily fake/modified content proliferates on the internet already and how much money tabloids make off made-up stories and faked photos, can you really argue that there is zero negative social impact from fakes? Even for someone with decent media literacy, proving 'this is faked' is a lot harder than 'this is some old content falsely labeled as new content', because you can't just look for the original source. You have to try to identify what the original source was pre-fake, and that's a LOT harder with deepfakes.
  - nitwit005 7y ago
    I didn't argue there was zero negative impact. I just said it wasn't major. If the tabloids were unable to make fake photos, would it affect anything at all? They can just write up a fake article without a photo. The actual act of making fake media just isn't necessary for any of this behavior.
    - andreilys 7y ago
      Pizza Gate led to an armed individual shooting up a pizza store because he thought it was an undercover pedo ring. So yeah, I imagine the fabrication, proliferation and automated spreading of conspiracy theories to have potential major societal impact
      - nitwit005 7y ago
        You're arguing against some sort of straw man argument I didn't make, but I'm not entirely sure what it is.
bransonf 7y ago
> What about your grandma? How would she fare? Grandma’s still falling for the phone and mail scams. No amount of legislation is going to fix the reality of technological illiteracy among the oldest adults. Deepfakes might fool some of today’s adults who don’t quite understand, but we are raising a generation that has turned into a meme: “Everything you read on the internet is true” -Abraham Lincoln I think the real silver lining here is that the internet is an alternate reality. Many of us refuse to believe that, but social media has created manufactured people. The only solution is to bring people back to the real world. The people are real here. Their opinion, no matter how controversial, comes from a real mouth, and the face you see is the one they were born with. If anyone forms their worldview based entirely on things they read on the internet, they probably would be just as susceptible to our real world forms of propaganda/gaslighting/ whatever you want to call it.
- skybrian 7y ago
  This essentially means the web is too difficult for some users and they need something else, like maybe an app store. Maybe some company will win big by providing a safer (or apparently safer) alternative? Previous examples: Gmail had a better spam filter. Apple and Google did a better (though not perfect) job of protecting users from arbitrary code execution, as did the web itself, way back when. This doesn't happen all that often, but if it succeeds, power users will scoff at how nerfed the new thing is. I'm reminded of an old story [1] about an early game for children: > I found myself unable to reconcile the idea of a virtual world, where kids would run around, play with objects, and chat with each other without someone saying or doing something that might upset another. Even in 1996, we knew that text-filters are no good at solving this kind of problem, so I asked for a clarification: "I’m confused. What standard should we use to decide if a message would be a problem for Disney?" > The response was one I will never forget: "Disney’s standard is quite clear: No kid will be harassed, even if they don’t know they are being harassed." But maybe text filters will be better if you throw enough machine learning at the problem? [1] http://habitatchronicles.com/2007/03/the-untold-history-of-toontowns-speedchat-or-blockchattm-from-disney-finally-arrives/ http://habitatchronicles.com/2007/03/the-untold-history-of-t...
  - 7y ago

qball 7y ago

>No, the more insidious goal is to flood the zone with enough dis- and misinformation to overload our ability to filter it. It’s like gaslighting at scale- at some point you just stop being able to process information because it’s so voluminous and of dubious quality that you stop believing any of it. This isn't new by any stretch of the imagination. It might as well outright be the job of the Fourth Estate ever since the Fourth Estate has existed (funded, as always, by ads). And when you look at the people complaining about this, which is for the most part people part of said political group (Facebook/Twitter/non-tech journalists), it makes sense that anything that could diminish their ability to control the narrative should be presented to the public as a new and novel threat. So this isn't going to result in widespread social chaos, because it's already happening. If anything, this technology will democratize what the old media is currently doing; which will likely result in driving political power towards crowdsourced content aggregation instead of the individual content creators themselves.

EGreg 7y ago

I would go further. Think of it like an ELO rating where most grandmas are 1200, and you pride yourself on being 2000. But eventually, the fakes become so good that the best humans can’t distinguish them, and half the time neither can ANY system, except with some probability. What then? An unlimited amount of glut that is indistinguishable from human writing, but can be produced with any kind of bias, by any number of fake identities and there is no reasoning with hem.

nitwit005 7y ago

There have been photographic fakes for a long time now, and they were quite good even in the pre-digital era. Have you seen it creating major social issues? I haven't observed much of anything. For the lies most people want to tell, there isn't any need to make a fake. Want to paint a protest in a bad light? Just use a real photo of a riot and falsely label it as the current protest.

kevingadd 7y ago
With how readily fake/modified content proliferates on the internet already and how much money tabloids make off made-up stories and faked photos, can you really argue that there is zero negative social impact from fakes? Even for someone with decent media literacy, proving 'this is faked' is a lot harder than 'this is some old content falsely labeled as new content', because you can't just look for the original source. You have to try to identify what the original source was pre-fake, and that's a LOT harder with deepfakes.
- nitwit005 7y ago
  I didn't argue there was zero negative impact. I just said it wasn't major. If the tabloids were unable to make fake photos, would it affect anything at all? They can just write up a fake article without a photo. The actual act of making fake media just isn't necessary for any of this behavior.
  - andreilys 7y ago
    Pizza Gate led to an armed individual shooting up a pizza store because he thought it was an undercover pedo ring. So yeah, I imagine the fabrication, proliferation and automated spreading of conspiracy theories to have potential major societal impact
    - nitwit005 7y ago
      You're arguing against some sort of straw man argument I didn't make, but I'm not entirely sure what it is.

bransonf 7y ago

> What about your grandma? How would she fare? Grandma’s still falling for the phone and mail scams. No amount of legislation is going to fix the reality of technological illiteracy among the oldest adults. Deepfakes might fool some of today’s adults who don’t quite understand, but we are raising a generation that has turned into a meme: “Everything you read on the internet is true” -Abraham Lincoln I think the real silver lining here is that the internet is an alternate reality. Many of us refuse to believe that, but social media has created manufactured people. The only solution is to bring people back to the real world. The people are real here. Their opinion, no matter how controversial, comes from a real mouth, and the face you see is the one they were born with. If anyone forms their worldview based entirely on things they read on the internet, they probably would be just as susceptible to our real world forms of propaganda/gaslighting/ whatever you want to call it.

skybrian 7y ago
This essentially means the web is too difficult for some users and they need something else, like maybe an app store. Maybe some company will win big by providing a safer (or apparently safer) alternative? Previous examples: Gmail had a better spam filter. Apple and Google did a better (though not perfect) job of protecting users from arbitrary code execution, as did the web itself, way back when. This doesn't happen all that often, but if it succeeds, power users will scoff at how nerfed the new thing is. I'm reminded of an old story [1] about an early game for children: > I found myself unable to reconcile the idea of a virtual world, where kids would run around, play with objects, and chat with each other without someone saying or doing something that might upset another. Even in 1996, we knew that text-filters are no good at solving this kind of problem, so I asked for a clarification: "I’m confused. What standard should we use to decide if a message would be a problem for Disney?" > The response was one I will never forget: "Disney’s standard is quite clear: No kid will be harassed, even if they don’t know they are being harassed." But maybe text filters will be better if you throw enough machine learning at the problem? [1] http://habitatchronicles.com/2007/03/the-untold-history-of-toontowns-speedchat-or-blockchattm-from-disney-finally-arrives/ http://habitatchronicles.com/2007/03/the-untold-history-of-t...
- 7y ago