5 ms·

This feels like thinly veiled advocacy for censorship and breaking the internet. And soliciting federal grants to research how to do it. There is already a pr

by cwkoss 4y ago

This feels like thinly veiled advocacy for censorship and breaking the internet. And soliciting federal grants to research how to do it. There is already a practically infinite amount of misinformation on the internet. The misinformation problem is not currently constrained by lack of supply, so doubling or 10x'ing or 1000x'ing the amount of misinformation that can be produced doesn't seem significantly concerning to me. Misinformation is a problem rooted in the powerful having having a net economic incentive to deceive and the populace's lack of media literacy. We already have 'trusted news sources' repeating falsehoods they heard 'off the record' from the state department. Credibility of the institutions that used to be easy to trust has fallen for both justified and unjustified reasons. The people who already choose 'fake news' sources will only be affected by AI fake news to the extent that those who control the AI have different goals than the people who control those channels today. If people apply AI to the problem, I think AI could help us a lot with understanding how we should adapt our systems of trust to account for this. Humans are bad at remembering the most important betrayals of people who are notionally part of our tribe, so credibility reverts to mean without consequences too easily. It's also very hard for us to identify "this person has stuck to their word for 20 years" without it being lost in punditry noise. I think its important for people to think about how the populace can hold politicians and media more responsible, but AI language generation is almost entirely tangential to that (and stopping AI does nothing to solve the root issue).

georgeplusplus 4y ago
>>>This feels like thinly veiled advocacy for censorship and breaking the internet. And soliciting federal grants to research how to do it This is what I took away after reading it as well. I found it strange the author used such a poor example to prove their point such as; AI recommending porcelain in baby formula and readers may actually put crushed porcelain in their baby formula. This is such a poor example because you would literally have to be brain dead to do something like this. The bar needs to be set a lot higher to prove to me that AI misinformation is a threat.
- GloriousKoji 4y ago
  I don't see how AI porcelain baby formula is that far off from flesh brain ideas such as drinking bleach, eating tide pods and Nyquil marinated chicken.
  - georgeplusplus 4y ago
    The point was if you're dumb enough, marinating chicken in NyQuil and washing it down with bleach will seem like an award winning idea. No amount of filters and censorship will prevent people like this from existing and engaging in self harm. So why continue to try and prevent it? The tide pod teenage phase is similar. Teenagers will always find reckless things to do and share it to become popular. It's what teens do. Good parents make the difference here.
    - cwkoss 4y ago
      Tide pods thing was a failure of our current information systems. Teens were joking about eating tidepods. The MSM news heard the joke and reported it as factual, and only after the news reported it as factual did people actually eat them.
      - georgeplusplus 4y ago
        Teens do not listen to or watch msm in the first place. It's almost like it was shared on Reddit or TikTok and that combined with what I mentioned before about teens seeing other teens do it, made it viral. Should young popular influencers be held accountable? Because I could go for that.
- goatlover 4y ago
  Think of all the Covid-related conspiracy theories for dumb misinformation people will believe.
  - than3 4y ago
    Are you sure they were dumb? Maybe they were simply brain-damaged from exposure to various products that were certified as safe, but weren't. For some reason, that reminds me of https://en.wikipedia.org/wiki/The_Crazies_(1973_film) https://en.wikipedia.org/wiki/The_Crazies_(1973_film)
wwarner 4y ago
I mean Gary Marcus and Elon Musk seem to agree here. AI has very suddenly become incredibly powerful and will be further weaponized. It should fall under some kind of regulatory framework for safety and security. By way of comparison, AI is far more dangerous than cryptography, which is regulated in a national security context. In addition, we should not underestimate the increasing power of AI in the near future. AI is already using AI to improve AI faster than people can improve it. Witness all the educators figuring out ways to identify fake writing, without realizing that they are just feeding data to the next revision of GPT.
- cwkoss 4y ago
  How would you propose AI be regulated? I think any regulations would have to accurately encode our moral systems to accomplish safety and security without creating worse side effects, and since the law already does a pretty mediocre job of accurately encoding our moral systems, I'm skeptical that regulatory intervention can be fruitful. I fear that regulations' most likely outcome is increasing the chance of powerful AI being built as an amoral slave to capital rather than a societal good: it will be expensive to comply with regs, so it will kill mom+pop AI shops and only the megacorps will be allowed to play.
  - wwarner 4y ago
    Something along the lines of the norms and guidelines for CRISPR germ line editing
fanso99 4y ago
Somewhat agree with your take. A good example is Twitter: it’s trivial even for a non-state actor to automate a bot farm to post hundreds of thousands of disinfo tweets, create “connections”, retweet each other, etc. this is happening today at a massive scale. However, where I can see a model like ChatGPT making a difference is creating a tsunami of seemingly in-depth content both in public (Wikipedia) and private (buying 10K domain names) spaces. Generating fake research papers, interviews, blog posts, reviews… at a scale that until now required an unattainable amount of human supervision. How much content would it take to flood the more visible part of the internet?
- cwkoss 4y ago
  Fair point. But hasn't SEO blogspam already flooded every inch of the visible part of the internet already? Can't partisans already get 5-10 pundits to write articles advocating a questionable concept to add a veneer of credibility for a couple thousand bucks? I'm not sure what frontiers of the visible part of the internet remain. It will certainly become more economical to do this sort of nefarious stuff, but I suspect democratizing misinformation production will lead to the development of better media literacy at a rate that outpaces that misinformation drowning out people who are earnests trying their best to find and present the truth.
  - fanso99 4y ago
    In my experience blogspam has been primarily used for commercial purposes (selling stuff and/or ads). Political disinfo at a new scale could pose very interesting new challenges. In a similar way we saw a paradigm shift when Cambridge Analytica employed political targeting at a new scale. Before that anyone could hire a savvy social media manager to target specific groups, but automating this process proved to be a different beast. How soon our society will adapt to this, or how we will even react is yet to be seen. I don’t have high hopes for this to be a fast and painless process.
- commandlinefan 4y ago
  > to post hundreds of thousands of disinfo tweets If that's true, then they've been doing it for a long time now... has it actually made a difference? There have been a lot of news items that were accused of being disinfo that have since turned out to be absolutely true, but I have yet to have seen an example of actual disinfo that took hold and altered any meaningful course of events.
janalsncm 4y ago
> This feels like thinly veiled advocacy for censorship and breaking the internet. Marcus is arguing that the internet will be broken unless something is done. I think he is overreacting but he would argue there’s no going back from here. > The misinformation problem is not currently constrained by lack of supply, so doubling or 10x'ing or 1000x'ing the amount of misinformation that can be produced doesn't seem significantly concerning to me. Slightly flawed logic imo. A difference in quantity can definitely produce a qualitative shift in the internet. The best analogy I can think of is in surveillance. The Soviet Union had a huge surveillance network. Something like 2-3% of the population was employed to spy on the citizenry. That’s extremely expensive, which limits the amount of info you can reliably have. But what if you could cheaply surveil almost everyone? That’s the situation only made possible with modern technology. It’s not that surveillance didn’t exist before, but it was very expensive, which was self-limiting. Only the most suspect individuals could be surveilled.

by cwkoss 4y ago

georgeplusplus 4y ago

>>>This feels like thinly veiled advocacy for censorship and breaking the internet. And soliciting federal grants to research how to do it This is what I took away after reading it as well. I found it strange the author used such a poor example to prove their point such as; AI recommending porcelain in baby formula and readers may actually put crushed porcelain in their baby formula. This is such a poor example because you would literally have to be brain dead to do something like this. The bar needs to be set a lot higher to prove to me that AI misinformation is a threat.

GloriousKoji 4y ago
I don't see how AI porcelain baby formula is that far off from flesh brain ideas such as drinking bleach, eating tide pods and Nyquil marinated chicken.
- georgeplusplus 4y ago
  The point was if you're dumb enough, marinating chicken in NyQuil and washing it down with bleach will seem like an award winning idea. No amount of filters and censorship will prevent people like this from existing and engaging in self harm. So why continue to try and prevent it? The tide pod teenage phase is similar. Teenagers will always find reckless things to do and share it to become popular. It's what teens do. Good parents make the difference here.
  - cwkoss 4y ago
    Tide pods thing was a failure of our current information systems. Teens were joking about eating tidepods. The MSM news heard the joke and reported it as factual, and only after the news reported it as factual did people actually eat them.
    - georgeplusplus 4y ago
      Teens do not listen to or watch msm in the first place. It's almost like it was shared on Reddit or TikTok and that combined with what I mentioned before about teens seeing other teens do it, made it viral. Should young popular influencers be held accountable? Because I could go for that.
goatlover 4y ago
Think of all the Covid-related conspiracy theories for dumb misinformation people will believe.
- than3 4y ago
  Are you sure they were dumb? Maybe they were simply brain-damaged from exposure to various products that were certified as safe, but weren't. For some reason, that reminds me of https://en.wikipedia.org/wiki/The_Crazies_(1973_film) https://en.wikipedia.org/wiki/The_Crazies_(1973_film)

wwarner 4y ago

I mean Gary Marcus and Elon Musk seem to agree here. AI has very suddenly become incredibly powerful and will be further weaponized. It should fall under some kind of regulatory framework for safety and security. By way of comparison, AI is far more dangerous than cryptography, which is regulated in a national security context. In addition, we should not underestimate the increasing power of AI in the near future. AI is already using AI to improve AI faster than people can improve it. Witness all the educators figuring out ways to identify fake writing, without realizing that they are just feeding data to the next revision of GPT.

cwkoss 4y ago
How would you propose AI be regulated? I think any regulations would have to accurately encode our moral systems to accomplish safety and security without creating worse side effects, and since the law already does a pretty mediocre job of accurately encoding our moral systems, I'm skeptical that regulatory intervention can be fruitful. I fear that regulations' most likely outcome is increasing the chance of powerful AI being built as an amoral slave to capital rather than a societal good: it will be expensive to comply with regs, so it will kill mom+pop AI shops and only the megacorps will be allowed to play.
- wwarner 4y ago
  Something along the lines of the norms and guidelines for CRISPR germ line editing

fanso99 4y ago

Somewhat agree with your take. A good example is Twitter: it’s trivial even for a non-state actor to automate a bot farm to post hundreds of thousands of disinfo tweets, create “connections”, retweet each other, etc. this is happening today at a massive scale. However, where I can see a model like ChatGPT making a difference is creating a tsunami of seemingly in-depth content both in public (Wikipedia) and private (buying 10K domain names) spaces. Generating fake research papers, interviews, blog posts, reviews… at a scale that until now required an unattainable amount of human supervision. How much content would it take to flood the more visible part of the internet?

cwkoss 4y ago
Fair point. But hasn't SEO blogspam already flooded every inch of the visible part of the internet already? Can't partisans already get 5-10 pundits to write articles advocating a questionable concept to add a veneer of credibility for a couple thousand bucks? I'm not sure what frontiers of the visible part of the internet remain. It will certainly become more economical to do this sort of nefarious stuff, but I suspect democratizing misinformation production will lead to the development of better media literacy at a rate that outpaces that misinformation drowning out people who are earnests trying their best to find and present the truth.
- fanso99 4y ago
  In my experience blogspam has been primarily used for commercial purposes (selling stuff and/or ads). Political disinfo at a new scale could pose very interesting new challenges. In a similar way we saw a paradigm shift when Cambridge Analytica employed political targeting at a new scale. Before that anyone could hire a savvy social media manager to target specific groups, but automating this process proved to be a different beast. How soon our society will adapt to this, or how we will even react is yet to be seen. I don’t have high hopes for this to be a fast and painless process.
commandlinefan 4y ago
> to post hundreds of thousands of disinfo tweets If that's true, then they've been doing it for a long time now... has it actually made a difference? There have been a lot of news items that were accused of being disinfo that have since turned out to be absolutely true, but I have yet to have seen an example of actual disinfo that took hold and altered any meaningful course of events.

janalsncm 4y ago

> This feels like thinly veiled advocacy for censorship and breaking the internet. Marcus is arguing that the internet will be broken unless something is done. I think he is overreacting but he would argue there’s no going back from here. > The misinformation problem is not currently constrained by lack of supply, so doubling or 10x'ing or 1000x'ing the amount of misinformation that can be produced doesn't seem significantly concerning to me. Slightly flawed logic imo. A difference in quantity can definitely produce a qualitative shift in the internet. The best analogy I can think of is in surveillance. The Soviet Union had a huge surveillance network. Something like 2-3% of the population was employed to spy on the citizenry. That’s extremely expensive, which limits the amount of info you can reliably have. But what if you could cheaply surveil almost everyone? That’s the situation only made possible with modern technology. It’s not that surveillance didn’t exist before, but it was very expensive, which was self-limiting. Only the most suspect individuals could be surveilled.