6 ms·

Generative AI will make the world on general a worse place to be. They are not very good at writing truth, but they are very excellent at writing convincing bul

Generative AI will make the world on general a worse place to be. They are not very good at writing truth, but they are very excellent at writing convincing bullshit. It's already difficult to distinguish generated text/image/video from human responses / real footage, its only gonna get more difficult to do so and cheaper to generate. In other words, it's very likely generative AI will be very good at creating fake simulacra of reality, and very unlikely it will actually be good AGI. The worst possible outcome.

sausse 3y ago
Half of zoomers get their news from TikTok or Twitch streamers, neither of whom have any incentive for truthfulness over holistic narratives of right and wrong. The older generations are no better. While ProPublica or WSJ put effort into their investigative journalism, they can’t compete with the volume of trite commentary coming out of other MSM sources. Generative AI poses no unique threat; society’s capacity to “think once and cut twice” will remain in tact.
- lewhoo 3y ago
  > Generative AI poses no unique threat; While the threat isn't unique the magnitude of the threat is. This is why you can't argue in court the threat of a car crash is nothing unique even when you're speeding vs driving within limit.
  - sausse 3y ago
    Sure, if you presume organic propaganda is analogous to the level of danger driving within limit. But a car going into a stroller at 150mph versus 200mph is negligible. The democratization of generative AI would increase the number of bad agents, but with it would come a better awareness of their tactics; perhaps we push less strollers into the intersections known for drag racing.
    - FrontierProject 3y ago
      You underestimate the amount of people who don't at all care whether or not their stroller goes splat as long as they're on asphalt they like the feel of.
    - lewhoo 3y ago
      > But a car going into a stroller at 150mph versus 200mph is negligible. I guess when you distort every argument to an absurd you can claim you're right. > but with it would come a better awareness of their tactics I don't follow. Are you saying new and more sophisticated ways to scam people are actually good because we have a unique chance to know how they work ?
      - sausse 3y ago
        It’s not absurd. The bottleneck for additional predation is not the available toolkit, else we’d see a more obvious correlation between a society’s resource endowment and its callousness. Handwringing over the threat of AI without substantiating an argument beside “enabled volume” is just self-righteousness. AI isn’t posed to shift the balance of MFA versus phishers in a way that can’t be meaningfully corrected in the short and long term, so using “scamming” as a means to oppose disseminating tech feels reductive at best.
        lewhoo 3y ago
        > It’s not absurd. It is, because I wasn't directly comparing AI to traffic but only reaching for an example to illustrate how irrelevant is the case whether the threat is something completely unique or not. > Handwringing over the threat of AI without substantiating an argument beside “enabled volume” is just self-righteousness. Dismissing it as "meh, not new" is plain silliness. > AI isn’t posed to shift the balance of MFA versus phishers in a way that can’t be meaningfully corrected What on Earth makes you think that ? The beautiful way we're handling scams right now ? If you think it's irrelevant that phishing via phone call can now or soon be fully automated and the attack may even be conducted using a copy of someone's voice - well, we won't get anywhere here.
Al-Khwarizmi 3y ago
We will have to go back to using trust in the source as the main litmus test for credibility. Text from sources that are known to have humans write (or verify) everything they publish in a reasonably neutral way will be trusted, the rest will be assumed to be bullshit by default. It could be the return of real journalism. There is a lot to rebuild in this respect, as most journalism has gone to the dogs in the last few decades. In my country all major newspapers are political pamphlets that regularly publish fake news (without the need for any AI). But one can hope, maybe the lowering of the barrier of entry to generate fake content will make people more critical of what they read, hence incentivizing the creation of actually trustworthy sources.
- friendzis 3y ago
  If avalanche of generative content would tip the scales towards (blind) trust of human writers, these "journalists" pushing out propaganda and outright fake news will have increased incentive to do so, not lowered.
amarant 3y ago
Replace "AI" in your comment with "human journalists" and it still holds largely true though. It's not like AI invented clickbait, though it might have mastered the art of it. The convincing bullshit problem does not stem from AI, I'd argue it stems from the interaction between ad revenue and SEO and the weird and unexpected incentives created in when mixing those 2. To put it differently, the problem isn't that AI will be great at writing 100 pages of bullshit you'll need to scroll through to get to the actual recipie, the problem is that there was an incentive to write those pages in there first place. Personally I don't care if a human or a robot wrote the bs, in fact I'm glad one fewer human has to waste their time doing just that. Would be great if cutting the bs was a more profitable model though.
- friendzis 3y ago
  > I'd argue it stems from the interaction between ad revenue and SEO and the weird and unexpected incentives created in when mixing those 2. Personally, I highly dislike this handwaving of SEO. SEO is not some sinister agenda following secret cult trying to disseminate bullshit. SEO is just... following the rules set forth by search engines, which for quite a long time is effectively singlehandedly Google. Those "weird and unexpected incentives" are put forth by Google. If Google for whatever reason started ranking "vegetable growing articles > preparation technique articles > recipes > shops selling vegetables" we would see metaphorical explosion of home gardening in mere few years, only due to the relatively long lifecycles inherent in gardening.
  - lupire 3y ago
    The explosion would be in BS articles about gardening, plus ads for whatever the user's profile says they are susceptible to. SEO is gaming Google's heuristics. Google doest generate a perfect ranking according to Google human's values. SEO gaming is much older than Google. Back when "search" was just an alphabetical listing of everyone in a printed book, we had companies calling themselves "A A Aachen" to get to the front of the book.
    - friendzis 3y ago

sausse 3y ago

Half of zoomers get their news from TikTok or Twitch streamers, neither of whom have any incentive for truthfulness over holistic narratives of right and wrong. The older generations are no better. While ProPublica or WSJ put effort into their investigative journalism, they can’t compete with the volume of trite commentary coming out of other MSM sources. Generative AI poses no unique threat; society’s capacity to “think once and cut twice” will remain in tact.

lewhoo 3y ago
> Generative AI poses no unique threat; While the threat isn't unique the magnitude of the threat is. This is why you can't argue in court the threat of a car crash is nothing unique even when you're speeding vs driving within limit.
- sausse 3y ago
  Sure, if you presume organic propaganda is analogous to the level of danger driving within limit. But a car going into a stroller at 150mph versus 200mph is negligible. The democratization of generative AI would increase the number of bad agents, but with it would come a better awareness of their tactics; perhaps we push less strollers into the intersections known for drag racing.
  - FrontierProject 3y ago
    You underestimate the amount of people who don't at all care whether or not their stroller goes splat as long as they're on asphalt they like the feel of.
  - lewhoo 3y ago
    > But a car going into a stroller at 150mph versus 200mph is negligible. I guess when you distort every argument to an absurd you can claim you're right. > but with it would come a better awareness of their tactics I don't follow. Are you saying new and more sophisticated ways to scam people are actually good because we have a unique chance to know how they work ?
    - sausse 3y ago
      It’s not absurd. The bottleneck for additional predation is not the available toolkit, else we’d see a more obvious correlation between a society’s resource endowment and its callousness. Handwringing over the threat of AI without substantiating an argument beside “enabled volume” is just self-righteousness. AI isn’t posed to shift the balance of MFA versus phishers in a way that can’t be meaningfully corrected in the short and long term, so using “scamming” as a means to oppose disseminating tech feels reductive at best.
      - lewhoo 3y ago
        > It’s not absurd. It is, because I wasn't directly comparing AI to traffic but only reaching for an example to illustrate how irrelevant is the case whether the threat is something completely unique or not. > Handwringing over the threat of AI without substantiating an argument beside “enabled volume” is just self-righteousness. Dismissing it as "meh, not new" is plain silliness. > AI isn’t posed to shift the balance of MFA versus phishers in a way that can’t be meaningfully corrected What on Earth makes you think that ? The beautiful way we're handling scams right now ? If you think it's irrelevant that phishing via phone call can now or soon be fully automated and the attack may even be conducted using a copy of someone's voice - well, we won't get anywhere here.

Al-Khwarizmi 3y ago

We will have to go back to using trust in the source as the main litmus test for credibility. Text from sources that are known to have humans write (or verify) everything they publish in a reasonably neutral way will be trusted, the rest will be assumed to be bullshit by default. It could be the return of real journalism. There is a lot to rebuild in this respect, as most journalism has gone to the dogs in the last few decades. In my country all major newspapers are political pamphlets that regularly publish fake news (without the need for any AI). But one can hope, maybe the lowering of the barrier of entry to generate fake content will make people more critical of what they read, hence incentivizing the creation of actually trustworthy sources.

friendzis 3y ago
If avalanche of generative content would tip the scales towards (blind) trust of human writers, these "journalists" pushing out propaganda and outright fake news will have increased incentive to do so, not lowered.

amarant 3y ago

Replace "AI" in your comment with "human journalists" and it still holds largely true though. It's not like AI invented clickbait, though it might have mastered the art of it. The convincing bullshit problem does not stem from AI, I'd argue it stems from the interaction between ad revenue and SEO and the weird and unexpected incentives created in when mixing those 2. To put it differently, the problem isn't that AI will be great at writing 100 pages of bullshit you'll need to scroll through to get to the actual recipie, the problem is that there was an incentive to write those pages in there first place. Personally I don't care if a human or a robot wrote the bs, in fact I'm glad one fewer human has to waste their time doing just that. Would be great if cutting the bs was a more profitable model though.

friendzis 3y ago
> I'd argue it stems from the interaction between ad revenue and SEO and the weird and unexpected incentives created in when mixing those 2. Personally, I highly dislike this handwaving of SEO. SEO is not some sinister agenda following secret cult trying to disseminate bullshit. SEO is just... following the rules set forth by search engines, which for quite a long time is effectively singlehandedly Google. Those "weird and unexpected incentives" are put forth by Google. If Google for whatever reason started ranking "vegetable growing articles > preparation technique articles > recipes > shops selling vegetables" we would see metaphorical explosion of home gardening in mere few years, only due to the relatively long lifecycles inherent in gardening.
- lupire 3y ago
  The explosion would be in BS articles about gardening, plus ads for whatever the user's profile says they are susceptible to. SEO is gaming Google's heuristics. Google doest generate a perfect ranking according to Google human's values. SEO gaming is much older than Google. Back when "search" was just an alphabetical listing of everyone in a printed book, we had companies calling themselves "A A Aachen" to get to the front of the book.
  - friendzis 3y ago