7 ms·

The biggest real problem I’m experiencing right now isn’t controls on the AI, it’s weird spam emails that bypass spam filters because they look real enough, but

The biggest real problem I’m experiencing right now isn’t controls on the AI, it’s weird spam emails that bypass spam filters because they look real enough, but are just cold email marketing bullshit: https://x.com/tlalexander/status/1765122572067434857 https://x.com/tlalexander/status/1765122572067434857 These systems are making the internet a worse place to be.

andrepd 3y ago
Generative AI will make the world on general a worse place to be. They are not very good at writing truth, but they are very excellent at writing convincing bullshit. It's already difficult to distinguish generated text/image/video from human responses / real footage, its only gonna get more difficult to do so and cheaper to generate. In other words, it's very likely generative AI will be very good at creating fake simulacra of reality, and very unlikely it will actually be good AGI. The worst possible outcome.
- sausse 3y ago
  Half of zoomers get their news from TikTok or Twitch streamers, neither of whom have any incentive for truthfulness over holistic narratives of right and wrong. The older generations are no better. While ProPublica or WSJ put effort into their investigative journalism, they can’t compete with the volume of trite commentary coming out of other MSM sources. Generative AI poses no unique threat; society’s capacity to “think once and cut twice” will remain in tact.
  - lewhoo 3y ago
    > Generative AI poses no unique threat; While the threat isn't unique the magnitude of the threat is. This is why you can't argue in court the threat of a car crash is nothing unique even when you're speeding vs driving within limit.
    - sausse 3y ago
      Sure, if you presume organic propaganda is analogous to the level of danger driving within limit. But a car going into a stroller at 150mph versus 200mph is negligible. The democratization of generative AI would increase the number of bad agents, but with it would come a better awareness of their tactics; perhaps we push less strollers into the intersections known for drag racing.
      - FrontierProject 3y ago
        You underestimate the amount of people who don't at all care whether or not their stroller goes splat as long as they're on asphalt they like the feel of.
      - lewhoo 3y ago
        > But a car going into a stroller at 150mph versus 200mph is negligible. I guess when you distort every argument to an absurd you can claim you're right. > but with it would come a better awareness of their tactics I don't follow. Are you saying new and more sophisticated ways to scam people are actually good because we have a unique chance to know how they work ?
        sausse 3y ago
        It’s not absurd. The bottleneck for additional predation is not the available toolkit, else we’d see a more obvious correlation between a society’s resource endowment and its callousness. Handwringing over the threat of AI without substantiating an argument beside “enabled volume” is just self-righteousness. AI isn’t posed to shift the balance of MFA versus phishers in a way that can’t be meaningfully corrected in the short and long term, so using “scamming” as a means to oppose disseminating tech feels reductive at best.
        lewhoo 3y ago
        > It’s not absurd. It is, because I wasn't directly comparing AI to traffic but only reaching for an example to illustrate how irrelevant is the case whether the threat is something completely unique or not. > Handwringing over the threat of AI without substantiating an argument beside “enabled volume” is just self-righteousness. Dismissing it as "meh, not new" is plain silliness. > AI isn’t posed to shift the balance of MFA versus phishers in a way that can’t be meaningfully corrected What on Earth makes you think that ? The beautiful way we're handling scams right now ? If you think it's irrelevant that phishing via phone call can now or soon be fully automated and the attack may even be conducted using a copy of someone's voice - well, we won't get anywhere here.
- Al-Khwarizmi 3y ago
  We will have to go back to using trust in the source as the main litmus test for credibility. Text from sources that are known to have humans write (or verify) everything they publish in a reasonably neutral way will be trusted, the rest will be assumed to be bullshit by default. It could be the return of real journalism. There is a lot to rebuild in this respect, as most journalism has gone to the dogs in the last few decades. In my country all major newspapers are political pamphlets that regularly publish fake news (without the need for any AI). But one can hope, maybe the lowering of the barrier of entry to generate fake content will make people more critical of what they read, hence incentivizing the creation of actually trustworthy sources.
  - friendzis 3y ago
    If avalanche of generative content would tip the scales towards (blind) trust of human writers, these "journalists" pushing out propaganda and outright fake news will have increased incentive to do so, not lowered.
- amarant 3y ago
  Replace "AI" in your comment with "human journalists" and it still holds largely true though. It's not like AI invented clickbait, though it might have mastered the art of it. The convincing bullshit problem does not stem from AI, I'd argue it stems from the interaction between ad revenue and SEO and the weird and unexpected incentives created in when mixing those 2. To put it differently, the problem isn't that AI will be great at writing 100 pages of bullshit you'll need to scroll through to get to the actual recipie, the problem is that there was an incentive to write those pages in there first place. Personally I don't care if a human or a robot wrote the bs, in fact I'm glad one fewer human has to waste their time doing just that. Would be great if cutting the bs was a more profitable model though.
  - friendzis 3y ago
    > I'd argue it stems from the interaction between ad revenue and SEO and the weird and unexpected incentives created in when mixing those 2. Personally, I highly dislike this handwaving of SEO. SEO is not some sinister agenda following secret cult trying to disseminate bullshit. SEO is just... following the rules set forth by search engines, which for quite a long time is effectively singlehandedly Google. Those "weird and unexpected incentives" are put forth by Google. If Google for whatever reason started ranking "vegetable growing articles > preparation technique articles > recipes > shops selling vegetables" we would see metaphorical explosion of home gardening in mere few years, only due to the relatively long lifecycles inherent in gardening.
    - lupire 3y ago
      The explosion would be in BS articles about gardening, plus ads for whatever the user's profile says they are susceptible to. SEO is gaming Google's heuristics. Google doest generate a perfect ranking according to Google human's values. SEO gaming is much older than Google. Back when "search" was just an alphabetical listing of everyone in a printed book, we had companies calling themselves "A A Aachen" to get to the front of the book.
      - friendzis 3y ago
DrSiemer 3y ago
So you still believe the open internet has any chance of surviving what is coming? I admire your optimism. Reliable information and communication will soon be opt-in only. The "open internet" will become an eclectic collection of random experiences, where any real human output will be a pleasant, rare surprise, that stirs the pot for a short blip before it is assimilated and buried under the flood.
- flohofwoe 3y ago
  The internet will be fine, social media platforms will be flooded and killed by AI trash, but I can't see anything bad with that outcome. An actually 'open internet' for exchanging ideas with random strangers was a nice utopia from the early 90's that had been killed long ago (or arguably never existed).
  - BlueTemplar 3y ago
    E-mail has *not* been killed long ago (aside some issues trying to run your own server and not getting blocked by gmail/hotmail). It is under threat now, due to the increased sophistication of spam.
    - flohofwoe 3y ago
      Email might already be 'culturally dead' though. I guess my nephew might have an email address for the occasional password recovery, but the idea of actually communicating over email with other humans might be completely alien to him ;) Similar in my current job btw.
      - BlueTemplar 3y ago
        Ok, I get how one might use a variety of other tools for informal communication, I don't really use e-mail for that any more either, but I'm curious, what else can you possibly use for work ? With the requirement that it must be easy to back up and transfer (for yourself as well as the organization), so any platforms are immediately out of question.
eightman 3y ago
The use case for AI was, is and always will be spam.
- ben_w 3y ago
  For language models, spam creation/detection is kinda a GAN even when it isn't specifically designed to be: a faker and a discriminator each training on the other. But when that GAN passes the human threshold, suddenly you can use the faker to create interesting things and not just use the discriminator to reject fakes.
- hef19898 3y ago
  You forgot the initial use case for the internet: porn.
- AlienRobot 3y ago
  "The best minds of my generation are thinking about how to make people click ads. That sucks." I can't believe this went all the way to AI...
  - delichon 3y ago
    Civilization is an artifact of thermodynamics, not an exception to it. All life, including civilized life, is about acquiring energy and creating order out of it, primarily by replicating. Money is just one face of that. Ads are about money, which is about energy, which fuels life. AI is being created by these same forces, so is likely to go the same way. You might as well bemoan gravity.
    - lanstin 3y ago
      We might question the structural facets of the economy or the networking technology that made spam I mean ads a better investment than federated/distributed micropayments and reader-centric products. I would have kept using Facebook if they let me see the things my friends took the trouble to type in, rather than flooding me with stuff to sell more ads, and seeing the memes my friends like, which I already have too many cool memes, don't need yours.
lynx23 3y ago
Thanks Meta for releasing llama. One of the most questionable releases in the past years. Yes, I know, its fun to play with LocalLLM, and maybe reason enought o downvote this to hell. But there is also the other side, that free models like this enabled text pollution, which we now have. Did I already say "Thanks Meta"?
- fzzzy 3y ago
  What? How do OpenAI and Antrhropic and Mistral API access contribute less to text pollution?
- 123yawaworht456 3y ago
  neither of the big cloud models have any fucking guardrails against generating spam. I'd venture to guess that 99% of spam is either gpt3.5 (which is better, cheaper and easier to use than any local model) or gpt4 with scrapped keys or funded by stolen credit cards. you have no evidence whatsoever that llama models are being used for that purpose. meanwhile, twitter is full of bots posting GPT refusals.

andrepd 3y ago

Generative AI will make the world on general a worse place to be. They are not very good at writing truth, but they are very excellent at writing convincing bullshit. It's already difficult to distinguish generated text/image/video from human responses / real footage, its only gonna get more difficult to do so and cheaper to generate. In other words, it's very likely generative AI will be very good at creating fake simulacra of reality, and very unlikely it will actually be good AGI. The worst possible outcome.

sausse 3y ago
Half of zoomers get their news from TikTok or Twitch streamers, neither of whom have any incentive for truthfulness over holistic narratives of right and wrong. The older generations are no better. While ProPublica or WSJ put effort into their investigative journalism, they can’t compete with the volume of trite commentary coming out of other MSM sources. Generative AI poses no unique threat; society’s capacity to “think once and cut twice” will remain in tact.
- lewhoo 3y ago
  > Generative AI poses no unique threat; While the threat isn't unique the magnitude of the threat is. This is why you can't argue in court the threat of a car crash is nothing unique even when you're speeding vs driving within limit.
  - sausse 3y ago
    Sure, if you presume organic propaganda is analogous to the level of danger driving within limit. But a car going into a stroller at 150mph versus 200mph is negligible. The democratization of generative AI would increase the number of bad agents, but with it would come a better awareness of their tactics; perhaps we push less strollers into the intersections known for drag racing.
    - FrontierProject 3y ago
      You underestimate the amount of people who don't at all care whether or not their stroller goes splat as long as they're on asphalt they like the feel of.
    - lewhoo 3y ago
      > But a car going into a stroller at 150mph versus 200mph is negligible. I guess when you distort every argument to an absurd you can claim you're right. > but with it would come a better awareness of their tactics I don't follow. Are you saying new and more sophisticated ways to scam people are actually good because we have a unique chance to know how they work ?
      - sausse 3y ago
        It’s not absurd. The bottleneck for additional predation is not the available toolkit, else we’d see a more obvious correlation between a society’s resource endowment and its callousness. Handwringing over the threat of AI without substantiating an argument beside “enabled volume” is just self-righteousness. AI isn’t posed to shift the balance of MFA versus phishers in a way that can’t be meaningfully corrected in the short and long term, so using “scamming” as a means to oppose disseminating tech feels reductive at best.
        lewhoo 3y ago
        > It’s not absurd. It is, because I wasn't directly comparing AI to traffic but only reaching for an example to illustrate how irrelevant is the case whether the threat is something completely unique or not. > Handwringing over the threat of AI without substantiating an argument beside “enabled volume” is just self-righteousness. Dismissing it as "meh, not new" is plain silliness. > AI isn’t posed to shift the balance of MFA versus phishers in a way that can’t be meaningfully corrected What on Earth makes you think that ? The beautiful way we're handling scams right now ? If you think it's irrelevant that phishing via phone call can now or soon be fully automated and the attack may even be conducted using a copy of someone's voice - well, we won't get anywhere here.
Al-Khwarizmi 3y ago
We will have to go back to using trust in the source as the main litmus test for credibility. Text from sources that are known to have humans write (or verify) everything they publish in a reasonably neutral way will be trusted, the rest will be assumed to be bullshit by default. It could be the return of real journalism. There is a lot to rebuild in this respect, as most journalism has gone to the dogs in the last few decades. In my country all major newspapers are political pamphlets that regularly publish fake news (without the need for any AI). But one can hope, maybe the lowering of the barrier of entry to generate fake content will make people more critical of what they read, hence incentivizing the creation of actually trustworthy sources.
- friendzis 3y ago
  If avalanche of generative content would tip the scales towards (blind) trust of human writers, these "journalists" pushing out propaganda and outright fake news will have increased incentive to do so, not lowered.
amarant 3y ago
Replace "AI" in your comment with "human journalists" and it still holds largely true though. It's not like AI invented clickbait, though it might have mastered the art of it. The convincing bullshit problem does not stem from AI, I'd argue it stems from the interaction between ad revenue and SEO and the weird and unexpected incentives created in when mixing those 2. To put it differently, the problem isn't that AI will be great at writing 100 pages of bullshit you'll need to scroll through to get to the actual recipie, the problem is that there was an incentive to write those pages in there first place. Personally I don't care if a human or a robot wrote the bs, in fact I'm glad one fewer human has to waste their time doing just that. Would be great if cutting the bs was a more profitable model though.
- friendzis 3y ago
  > I'd argue it stems from the interaction between ad revenue and SEO and the weird and unexpected incentives created in when mixing those 2. Personally, I highly dislike this handwaving of SEO. SEO is not some sinister agenda following secret cult trying to disseminate bullshit. SEO is just... following the rules set forth by search engines, which for quite a long time is effectively singlehandedly Google. Those "weird and unexpected incentives" are put forth by Google. If Google for whatever reason started ranking "vegetable growing articles > preparation technique articles > recipes > shops selling vegetables" we would see metaphorical explosion of home gardening in mere few years, only due to the relatively long lifecycles inherent in gardening.
  - lupire 3y ago
    The explosion would be in BS articles about gardening, plus ads for whatever the user's profile says they are susceptible to. SEO is gaming Google's heuristics. Google doest generate a perfect ranking according to Google human's values. SEO gaming is much older than Google. Back when "search" was just an alphabetical listing of everyone in a printed book, we had companies calling themselves "A A Aachen" to get to the front of the book.
    - friendzis 3y ago

DrSiemer 3y ago

So you still believe the open internet has any chance of surviving what is coming? I admire your optimism. Reliable information and communication will soon be opt-in only. The "open internet" will become an eclectic collection of random experiences, where any real human output will be a pleasant, rare surprise, that stirs the pot for a short blip before it is assimilated and buried under the flood.

flohofwoe 3y ago
The internet will be fine, social media platforms will be flooded and killed by AI trash, but I can't see anything bad with that outcome. An actually 'open internet' for exchanging ideas with random strangers was a nice utopia from the early 90's that had been killed long ago (or arguably never existed).
- BlueTemplar 3y ago
  E-mail has *not* been killed long ago (aside some issues trying to run your own server and not getting blocked by gmail/hotmail). It is under threat now, due to the increased sophistication of spam.
  - flohofwoe 3y ago
    Email might already be 'culturally dead' though. I guess my nephew might have an email address for the occasional password recovery, but the idea of actually communicating over email with other humans might be completely alien to him ;) Similar in my current job btw.
    - BlueTemplar 3y ago
      Ok, I get how one might use a variety of other tools for informal communication, I don't really use e-mail for that any more either, but I'm curious, what else can you possibly use for work ? With the requirement that it must be easy to back up and transfer (for yourself as well as the organization), so any platforms are immediately out of question.

eightman 3y ago

The use case for AI was, is and always will be spam.

ben_w 3y ago
For language models, spam creation/detection is kinda a GAN even when it isn't specifically designed to be: a faker and a discriminator each training on the other. But when that GAN passes the human threshold, suddenly you can use the faker to create interesting things and not just use the discriminator to reject fakes.
hef19898 3y ago
You forgot the initial use case for the internet: porn.
AlienRobot 3y ago
"The best minds of my generation are thinking about how to make people click ads. That sucks." I can't believe this went all the way to AI...
- delichon 3y ago
  Civilization is an artifact of thermodynamics, not an exception to it. All life, including civilized life, is about acquiring energy and creating order out of it, primarily by replicating. Money is just one face of that. Ads are about money, which is about energy, which fuels life. AI is being created by these same forces, so is likely to go the same way. You might as well bemoan gravity.
  - lanstin 3y ago
    We might question the structural facets of the economy or the networking technology that made spam I mean ads a better investment than federated/distributed micropayments and reader-centric products. I would have kept using Facebook if they let me see the things my friends took the trouble to type in, rather than flooding me with stuff to sell more ads, and seeing the memes my friends like, which I already have too many cool memes, don't need yours.

lynx23 3y ago

Thanks Meta for releasing llama. One of the most questionable releases in the past years. Yes, I know, its fun to play with LocalLLM, and maybe reason enought o downvote this to hell. But there is also the other side, that free models like this enabled text pollution, which we now have. Did I already say "Thanks Meta"?

fzzzy 3y ago
What? How do OpenAI and Antrhropic and Mistral API access contribute less to text pollution?
123yawaworht456 3y ago
neither of the big cloud models have any fucking guardrails against generating spam. I'd venture to guess that 99% of spam is either gpt3.5 (which is better, cheaper and easier to use than any local model) or gpt4 with scrapped keys or funded by stolen credit cards. you have no evidence whatsoever that llama models are being used for that purpose. meanwhile, twitter is full of bots posting GPT refusals.