5 ms·

Thinking bigger: I'm pretty sure the combo of a relatively free global Internet, liberal democracy on large (much bigger than city-state) scales, and cheap, cus

Thinking bigger: I'm pretty sure the combo of a relatively free global Internet, liberal democracy on large (much bigger than city-state) scales, and cheap, customized, on-demand generation of totally fake text + photo + video propaganda based on simple prompts, cannot all co-exist. At least one of these isn't going to survive alongside the others. If we just let things keep going the way they are, I expect "liberal democracy on large scales" is the one we'll lose—and whatever follows probably won't let the fairly-free, global Internet keep existing, either, so we'll lose that too.

blueflow 4y ago
Whats your reasoning on this? Because i dont see why liberal democracy would cease to exist... life would go on if we all know that all pictures can be fabricated. I think this is already the case without AI.
- smaudet 4y ago
  Apparently you missed the problem Deep Fakes posed... If you cannot distinguish reality (well), and in fact it becomes possible that most things you see do not exist, then there is nothing to stop a bad actor from producing a fake version of events in which they are elected, control everything, etc. So, democracy would cease to exist, because democracy relies ultimately on a choice - if you have no choice then you do not have democracy, only a dictatorship.
  - phpnode 4y ago
    This is already possible today and we don't need AI generated stock photos to do it. A bad actor can already spin events to fit their narrative, suppress dissent and control their population. Dictators have been doing it for centuries and we're seeing it in real time in the form of Putin's Russia right now.
    - mythrwy 4y ago
      If it were only Putin's Russia making stuff up we'd be in good shape. Otherwise, I fully agree with your point.
      - phpnode 4y ago
        indeed, it's just the most prominent example at the moment
    - corrral 4y ago
      Sure, but being able to do the same thing at 100,000x the scale for the same price seems like a pretty big difference. Throw in the ability to target narrow constituencies with custom messages via modern ad networks, automation-assisted astroturfing, et c., and the whole thing looks like a powderkeg to me.
      - nomel 4y ago
        > 100,000x the scale Then everyone quickly realizes it's all garbage. The bigger danger is when people think garbage isn't possible, which has been the case since Stalin erased people from photos 98 years ago.
  - vehementi 4y ago
    Right but today we have that problem already. We know that a bad actor journalist can write a fake story. We therefore require sources. If deepfakes come along we will know videos can be fake and so we will be skeptical, as we are today, and look to proper sources. We will easily come up with some way to validate sources via cryptography or org reputation (e.g. we might trust the NY Times to not just fabricate things)
    - corrral 4y ago
      This is already barely holding together with mostly human actors doing the astroturfing and creating bullshit "news organizations" expressly to spread propaganda. Automation is going to overwhelm a system that's already teetering.
      - vehementi 4y ago
        Yeah but we will get the word out that none of that is trustworthy, then. There will be countermeasures and reactions to this just like previous things. It will certainly be effective to some degree - propaganda is effective for sure - but it won't just be, oh, there are deepfakes, everyone will now just unthinkingly accept them.
        corrral 4y ago
        1) This effective backlash/education-campaign has not already happened despite there already being significant problems with this kind of thing, and most of it not being that hard to spot, even, and 2) I think the more likely effect is the destruction of shared trust in any set of news sources—we're already pretty damn close to this being the case, in fact. "It's all lies anyway" is a sentiment that favors dictators more than it does democracy.
        nomel 4y ago
        > This effective backlash/education-campaign has not already happened despite there already being significant problems with this kind of thing I think it could be claimed that we're clearly in the process of learning to distrust. People distrust media more now than ever [1], because they've demonstrated, again and again, that they can't be trusted. Trust in institutions are down to new lows [2], because they've also shown that they're untrustworthy and/or incompetent. Everyone is shouting "disinformation", "misinformation", and "fake news" constantly, because nobody trusts anything anymore, and rightfully so in many cases. 1. https://news.gallup.com/poll/355526/americans-trust-media-dips-second-lowest-record.aspx https://news.gallup.com/poll/355526/americans-trust-media-di... 2. https://news.gallup.com/poll/394283/confidence-institutions-down-average-new-low.aspx https://news.gallup.com/poll/394283/confidence-institutions-...
  - mythrwy 4y ago
    People already produce all kinds of fake news and doctored photos and false flags and all kinds of things. This has been going on since we developed language and photography I suspect. People already have trouble telling propaganda from fact. That has been going on since forever. At the end of the day I don't see this being a game changer. If anything, now video and photos are less evidence for/against something as the potential falseness becomes well known. Congressman X: "no, that wasn't me you saw leaving the hotel with the prostitute, my slimy opponent obviously is deep faking stuff". And people will continue to believe what they want to believe, in spite of all evidence to the contrary, just like they do right now.
    - corrral 4y ago
      There seems to me a huge difference between a few organizations being able to produce & distribute a total of X amount of self-serving bullshit with some limited reach, and anyone with a bit of money being able to produce 100,000 • X amount of self-serving bullshit and deliver it to exactly the people most likely to respond to it the way they want, anywhere in the world (save, notably, China and North Korea and such) while making it very hard to tell who it's coming from. An environment in which 90% of the information is adversarial is really bad. It's a severe problem and very challenging to navigate. An environment where 99.9999% of it's adversarial and it's even harder than before to sort truth from fiction, functionally no longer has any flow of real information whatsoever.
      - mythrwy 4y ago
        Another thought: Maybe liberal democracy is not the final outcome of human civilization. You like it and I like it (presumably we were both raised to believe this way) but perhaps it's not really true. Just to question a base assumption here. It seems to me, if all the things that are claimed to threaten liberal democracy actually do, liberal democracy might be much less robust and long lived then previously believed.
        corrral 4y ago
        Oh, absolutely. I've even come around to thinking that's likely. But one can hope. [EDIT] One thing I no longer think has any realistic future is the open, semi-anonymous Internet. We're either losing it because despots take over and definitely won't permit that threat to remain unfettered, or we're losing it (in perhaps a gentler-touch way) because we have to to prevent authoritarian take-over and vast civil strife. I don't think we're getting to keep that no matter what happens.
  - unfunco 4y ago
    Photoshop has existed for years and humans have been manipulating photos for longer, what's the difference, really? If I see a photo in the Guardian newspaper (or any other reputable news outfit) I'm going to presume it's real, and I expect journalists to verify that for me. If I see a random photo that doesn't look quite right on a 4chan, I'm not going to immediately assume it's news.
    - corrral 4y ago
      > Photoshop has existed for years and humans have been manipulating photos for longer, what's the difference, really? Scale, cost, and reach.
      - unfunco 4y ago
        Reach is no different, bots and humans are able to post to social media, and cost is probably no different at the moment either since AI isn't perfect, some human interaction is probably needed to make it believable, and because of that, scale is the same too. I think we're approaching all of those things but it's probably still quite some time away until a machine can be trusted to manipulate the public on its own.
    - gadtfly 4y ago
      > For a Linux user, you can already build such a system yourself quite trivially by getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem. From Windows or Mac, this FTP account could be accessed through built-in software.
konschubert 4y ago
You can put letters in any order you want and make them say any damn lie. This was not an impediment to liberal democracy. I am as concerned as the next guy but throwing the towel already seems a bit premature?
- corrral 4y ago
  > You can put letters in any order you want and make them say any damn lie. You can run a web server by responding to every request by hand-typing the response, too. But you couldn't realistically run one-one-millionth the modern Web that way. You can't have global-scale e-commerce that way, et c. Some things that technically could work that way, can't actually—it's too slow, too expensive. This is very much one of those "quantity has a quality all its own" things. Increase the productivity of every astroturf-poster or propaganda-front-news-site manager a few hundred times and that's a big difference. > I am as concerned as the next guy but throwing the towel already seems a bit premature? Where'd you get throwing in the towel? I do think we're (especially the US) really unlikely to do what we need to in time, in part because measures that are probably necessary to defend against this are themselves risky and rather unappealing. But we might.
  - stickfigure 4y ago
    Humans are smarter than you give them credit for. They will adapt with increasing skepticism. I don't see why this is a threat to democracy.
    - ithkuil 4y ago
      That assumes that democracy can survive overreacted skepticism. If you don't trust anything, you ... don't trust anything. How can institutions survive? We're already living through a moment where large swaths of the population cannot get a fix on the same reality as their neighbors. I would argue that this is happening precisely because the gullibles are being led into playing skeptics.
      - 4y ago
whatshisface 4y ago
I have heard this before, but I have several reasons to think it is not going to be a problem above what already happens: 1. AI lets you generate an enormous number of lies, but what is really dangerous is one well-placed lie within a trusted stream of many truths. CNN will retain a power to mislead far in excess of Twitter bots. 2. Democracy averages over everyone's confusion, which means lies are only dangerous when large numbers of people believe them at the same time. Hordes of bots generating spurious lies won't move a democracy in any specific direction, but again, mass media will retain its power to mislead everyone at once in the same, effectual, direction. 3. People have never respected the veracity of random tweets. In the same way that trust in mainstream media outlets is reaching record lows due to their consistently biased reporting (they might not all have the same bias, but I can't think of any I'd consider free of bias), everyone will learn to adjust their incredulity to match the true quality of random tweets. 4. Companies like Twitter and Google are known to be shaping their results and algorithms according to their own "political views" (broadly construed) so at worst this would represent a partial shift of power from the old masters to new masters quite like them (social media companies). In many ways trimming the front page to reflect editorial opinion is echoed in the way Twitter trims its feeds to reflect their own editorial opinions. All taken together, it seems like the media is afraid that equally large companies with similar business models (content, attention, advertising) might end up eclipsing them. The same old model where the TV station is afraid to upset its advertisers, thereby giving a voice to business interests, is well-recorded in the recent history of YouTube. Not so much will change, although seeing it in its old and newer forms might shed light on how it works.
- thr0wawayf00 4y ago
  > CNN will retain a power to mislead far in excess of Twitter bots. The major mistake here is thinking that these two things are mutually exclusive competition. They're not. In 2015, an internet rumor started spreading that Obama was going to invade Texas. This rumor was based on the fact that there was a routine military training exercise taking place in Bastrop called Jade Helm. It started on Facebook and was quickly picked up by the mass media. These rumors became so viral that the Texas governor activated the Texas State Guard "just in case".[1] Later, the US government alleged that this was a planned disinformation operation carried out by the Russians and was a precursor to later operations.[0] Twitter and Facebook started the conspiracy, but mass media laundered it and made it look real enough to get a governor to act. 0: https://www.kut.org/politics/2018-05-03/jade-helm-conspiracy-theories-were-part-of-russian-disinformation-campaign-former-cia-chief-says https://www.kut.org/politics/2018-05-03/jade-helm-conspiracy... 1: https://www.texasmonthly.com/news-politics/russians-sowed-divisions-texas-politics-says-u-s-senate-report/ https://www.texasmonthly.com/news-politics/russians-sowed-di...
hwers 4y ago
Lol imagine if someone said this as a prediction when photoshop was invented.
nomel 4y ago
I disagree completely. We've had this ability to create photorealistic fiction for decades now. You can go see how impressive the capability is by paying $12 at your local movie theater. The only thing that's changed is the ease and cost of doing it, but that has also been dropping for decades. Having people think that it's not possible right now is what is dangerous, because it's absolutely possible right now, no AI needed. Even if this is released quickly, the internet would be flooded with crazy images, and people would quickly learn to have a critical eye to the media that they see, which should already be the case, because photorealistic fiction is already possible, as everyone experiences with every blockbuster movie. Only a slight shift in perspective is required to realize that every image and video you see aren't any different than frames/clips of Thor: Love And Thunder.

blueflow 4y ago

Whats your reasoning on this? Because i dont see why liberal democracy would cease to exist... life would go on if we all know that all pictures can be fabricated. I think this is already the case without AI.

smaudet 4y ago
Apparently you missed the problem Deep Fakes posed... If you cannot distinguish reality (well), and in fact it becomes possible that most things you see do not exist, then there is nothing to stop a bad actor from producing a fake version of events in which they are elected, control everything, etc. So, democracy would cease to exist, because democracy relies ultimately on a choice - if you have no choice then you do not have democracy, only a dictatorship.
- phpnode 4y ago
  This is already possible today and we don't need AI generated stock photos to do it. A bad actor can already spin events to fit their narrative, suppress dissent and control their population. Dictators have been doing it for centuries and we're seeing it in real time in the form of Putin's Russia right now.
  - mythrwy 4y ago
    If it were only Putin's Russia making stuff up we'd be in good shape. Otherwise, I fully agree with your point.
    - phpnode 4y ago
      indeed, it's just the most prominent example at the moment
  - corrral 4y ago
    Sure, but being able to do the same thing at 100,000x the scale for the same price seems like a pretty big difference. Throw in the ability to target narrow constituencies with custom messages via modern ad networks, automation-assisted astroturfing, et c., and the whole thing looks like a powderkeg to me.
    - nomel 4y ago
      > 100,000x the scale Then everyone quickly realizes it's all garbage. The bigger danger is when people think garbage isn't possible, which has been the case since Stalin erased people from photos 98 years ago.
- vehementi 4y ago
  Right but today we have that problem already. We know that a bad actor journalist can write a fake story. We therefore require sources. If deepfakes come along we will know videos can be fake and so we will be skeptical, as we are today, and look to proper sources. We will easily come up with some way to validate sources via cryptography or org reputation (e.g. we might trust the NY Times to not just fabricate things)
  - corrral 4y ago
    This is already barely holding together with mostly human actors doing the astroturfing and creating bullshit "news organizations" expressly to spread propaganda. Automation is going to overwhelm a system that's already teetering.
    - vehementi 4y ago
      Yeah but we will get the word out that none of that is trustworthy, then. There will be countermeasures and reactions to this just like previous things. It will certainly be effective to some degree - propaganda is effective for sure - but it won't just be, oh, there are deepfakes, everyone will now just unthinkingly accept them.
      - corrral 4y ago
        1) This effective backlash/education-campaign has not already happened despite there already being significant problems with this kind of thing, and most of it not being that hard to spot, even, and 2) I think the more likely effect is the destruction of shared trust in any set of news sources—we're already pretty damn close to this being the case, in fact. "It's all lies anyway" is a sentiment that favors dictators more than it does democracy.
        nomel 4y ago
        > This effective backlash/education-campaign has not already happened despite there already being significant problems with this kind of thing I think it could be claimed that we're clearly in the process of learning to distrust. People distrust media more now than ever [1], because they've demonstrated, again and again, that they can't be trusted. Trust in institutions are down to new lows [2], because they've also shown that they're untrustworthy and/or incompetent. Everyone is shouting "disinformation", "misinformation", and "fake news" constantly, because nobody trusts anything anymore, and rightfully so in many cases. 1. https://news.gallup.com/poll/355526/americans-trust-media-dips-second-lowest-record.aspx https://news.gallup.com/poll/355526/americans-trust-media-di... 2. https://news.gallup.com/poll/394283/confidence-institutions-down-average-new-low.aspx https://news.gallup.com/poll/394283/confidence-institutions-...
- mythrwy 4y ago
  People already produce all kinds of fake news and doctored photos and false flags and all kinds of things. This has been going on since we developed language and photography I suspect. People already have trouble telling propaganda from fact. That has been going on since forever. At the end of the day I don't see this being a game changer. If anything, now video and photos are less evidence for/against something as the potential falseness becomes well known. Congressman X: "no, that wasn't me you saw leaving the hotel with the prostitute, my slimy opponent obviously is deep faking stuff". And people will continue to believe what they want to believe, in spite of all evidence to the contrary, just like they do right now.
  - corrral 4y ago
    There seems to me a huge difference between a few organizations being able to produce & distribute a total of X amount of self-serving bullshit with some limited reach, and anyone with a bit of money being able to produce 100,000 • X amount of self-serving bullshit and deliver it to exactly the people most likely to respond to it the way they want, anywhere in the world (save, notably, China and North Korea and such) while making it very hard to tell who it's coming from. An environment in which 90% of the information is adversarial is really bad. It's a severe problem and very challenging to navigate. An environment where 99.9999% of it's adversarial and it's even harder than before to sort truth from fiction, functionally no longer has any flow of real information whatsoever.
    - mythrwy 4y ago
      Another thought: Maybe liberal democracy is not the final outcome of human civilization. You like it and I like it (presumably we were both raised to believe this way) but perhaps it's not really true. Just to question a base assumption here. It seems to me, if all the things that are claimed to threaten liberal democracy actually do, liberal democracy might be much less robust and long lived then previously believed.
      - corrral 4y ago
        Oh, absolutely. I've even come around to thinking that's likely. But one can hope. [EDIT] One thing I no longer think has any realistic future is the open, semi-anonymous Internet. We're either losing it because despots take over and definitely won't permit that threat to remain unfettered, or we're losing it (in perhaps a gentler-touch way) because we have to to prevent authoritarian take-over and vast civil strife. I don't think we're getting to keep that no matter what happens.
- unfunco 4y ago
  Photoshop has existed for years and humans have been manipulating photos for longer, what's the difference, really? If I see a photo in the Guardian newspaper (or any other reputable news outfit) I'm going to presume it's real, and I expect journalists to verify that for me. If I see a random photo that doesn't look quite right on a 4chan, I'm not going to immediately assume it's news.
  - corrral 4y ago
    > Photoshop has existed for years and humans have been manipulating photos for longer, what's the difference, really? Scale, cost, and reach.
    - unfunco 4y ago
      Reach is no different, bots and humans are able to post to social media, and cost is probably no different at the moment either since AI isn't perfect, some human interaction is probably needed to make it believable, and because of that, scale is the same too. I think we're approaching all of those things but it's probably still quite some time away until a machine can be trusted to manipulate the public on its own.
  - gadtfly 4y ago
    > For a Linux user, you can already build such a system yourself quite trivially by getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem. From Windows or Mac, this FTP account could be accessed through built-in software.

konschubert 4y ago

You can put letters in any order you want and make them say any damn lie. This was not an impediment to liberal democracy. I am as concerned as the next guy but throwing the towel already seems a bit premature?

corrral 4y ago
> You can put letters in any order you want and make them say any damn lie. You can run a web server by responding to every request by hand-typing the response, too. But you couldn't realistically run one-one-millionth the modern Web that way. You can't have global-scale e-commerce that way, et c. Some things that technically could work that way, can't actually—it's too slow, too expensive. This is very much one of those "quantity has a quality all its own" things. Increase the productivity of every astroturf-poster or propaganda-front-news-site manager a few hundred times and that's a big difference. > I am as concerned as the next guy but throwing the towel already seems a bit premature? Where'd you get throwing in the towel? I do think we're (especially the US) really unlikely to do what we need to in time, in part because measures that are probably necessary to defend against this are themselves risky and rather unappealing. But we might.
- stickfigure 4y ago
  Humans are smarter than you give them credit for. They will adapt with increasing skepticism. I don't see why this is a threat to democracy.
  - ithkuil 4y ago
    That assumes that democracy can survive overreacted skepticism. If you don't trust anything, you ... don't trust anything. How can institutions survive? We're already living through a moment where large swaths of the population cannot get a fix on the same reality as their neighbors. I would argue that this is happening precisely because the gullibles are being led into playing skeptics.
    - 4y ago

whatshisface 4y ago

I have heard this before, but I have several reasons to think it is not going to be a problem above what already happens: 1. AI lets you generate an enormous number of lies, but what is really dangerous is one well-placed lie within a trusted stream of many truths. CNN will retain a power to mislead far in excess of Twitter bots. 2. Democracy averages over everyone's confusion, which means lies are only dangerous when large numbers of people believe them at the same time. Hordes of bots generating spurious lies won't move a democracy in any specific direction, but again, mass media will retain its power to mislead everyone at once in the same, effectual, direction. 3. People have never respected the veracity of random tweets. In the same way that trust in mainstream media outlets is reaching record lows due to their consistently biased reporting (they might not all have the same bias, but I can't think of any I'd consider free of bias), everyone will learn to adjust their incredulity to match the true quality of random tweets. 4. Companies like Twitter and Google are known to be shaping their results and algorithms according to their own "political views" (broadly construed) so at worst this would represent a partial shift of power from the old masters to new masters quite like them (social media companies). In many ways trimming the front page to reflect editorial opinion is echoed in the way Twitter trims its feeds to reflect their own editorial opinions. All taken together, it seems like the media is afraid that equally large companies with similar business models (content, attention, advertising) might end up eclipsing them. The same old model where the TV station is afraid to upset its advertisers, thereby giving a voice to business interests, is well-recorded in the recent history of YouTube. Not so much will change, although seeing it in its old and newer forms might shed light on how it works.

thr0wawayf00 4y ago
> CNN will retain a power to mislead far in excess of Twitter bots. The major mistake here is thinking that these two things are mutually exclusive competition. They're not. In 2015, an internet rumor started spreading that Obama was going to invade Texas. This rumor was based on the fact that there was a routine military training exercise taking place in Bastrop called Jade Helm. It started on Facebook and was quickly picked up by the mass media. These rumors became so viral that the Texas governor activated the Texas State Guard "just in case".[1] Later, the US government alleged that this was a planned disinformation operation carried out by the Russians and was a precursor to later operations.[0] Twitter and Facebook started the conspiracy, but mass media laundered it and made it look real enough to get a governor to act. 0: https://www.kut.org/politics/2018-05-03/jade-helm-conspiracy-theories-were-part-of-russian-disinformation-campaign-former-cia-chief-says https://www.kut.org/politics/2018-05-03/jade-helm-conspiracy... 1: https://www.texasmonthly.com/news-politics/russians-sowed-divisions-texas-politics-says-u-s-senate-report/ https://www.texasmonthly.com/news-politics/russians-sowed-di...

hwers 4y ago

Lol imagine if someone said this as a prediction when photoshop was invented.

nomel 4y ago

I disagree completely. We've had this ability to create photorealistic fiction for decades now. You can go see how impressive the capability is by paying $12 at your local movie theater. The only thing that's changed is the ease and cost of doing it, but that has also been dropping for decades. Having people think that it's not possible right now is what is dangerous, because it's absolutely possible right now, no AI needed. Even if this is released quickly, the internet would be flooded with crazy images, and people would quickly learn to have a critical eye to the media that they see, which should already be the case, because photorealistic fiction is already possible, as everyone experiences with every blockbuster movie. Only a slight shift in perspective is required to realize that every image and video you see aren't any different than frames/clips of Thor: Love And Thunder.