10 ms·

I'm really negative about the impact AI will have on the society. We have already been drowning in fake news and polarizing information. Now, the likes of DALL

I'm really negative about the impact AI will have on the society. We have already been drowning in fake news and polarizing information. Now, the likes of DALL-E and DeepFake can generate convincing fake graphics. Chatgpt and the likes generate convincing fake news. Voice AI can generate convincing voice from small samples. If you were afraid of your elderly relatives being scammed by people pretending to be policemen or grandchildren, now more tech-concious people will get scammed by the voice and look of their relatives. Are we really approaching the reality where we need 2FA to trust the other person is really who they are?

icepat 4y ago
> Chatgpt and the likes generate convincing fake news It's not like fake news was a non-issue before ChatGPT existed. Breitbart, and other fake news sites existed for years before this was even imaginable. Fake graphics were around for ages too, with image manipulation, even before computers. Take for example the Surgeon's Photo of the Loch Ness Monster.
- jacquesm 4y ago
  This argument has been rehashed time and again and it's getting a bit boring: the fact that you can do something at all is qualitatively different from when you can do that same thing at scale.
  - chimpanzee 4y ago
    Agreed. As I’m sure you know, it’s easier for people to reason when the arena presents only discrete possibilities, especially binary ones, eg yes/no or black/white. The arena of the continuous, which encompasses most of the natural world, is far more difficult. It doesn’t allow for the sort of arguments where one can assert things with ego-boosting absolute confidence. Matters of scale naturally fall into the continuous sort, but scale can also introduce new discrete possibilities. Perhaps we need to present the new specific, discrete effects and outcomes that AI-at-scale will introduce. Maybe, just maybe, that’ll change minds of people who are truly open to being changed. My intuition though is that this argument is repeated here (on HN in particular) not for lack of knowledge or thought, but rather because they WANT to see the chaos that’ll result. They want all the positives and negatives, no matter the balance, simply because it’s exciting and adds to their “mundane” lives. Where the mundanity is of course subjective, simply the result of their default worldview, rather than anything nearing an objective description of the world as it is.
    - jacquesm 4y ago
      > simply because it’s exciting and adds to their lives. It's certainly exciting but I wouldn't bet on it adding to our lives just yet. Maybe. But many avenues leading to net negative outcomes are still part of the tree.
      - chimpanzee 4y ago
        Sorry, I had already edited that portion before seeing your response. But you’re absolutely right.
      - deleted 4y ago
        [deleted]
  - SV_BubbleTime 4y ago
    You could make fake photos with photoshop. And now you can make them with AI. A normal person couldn’t use photoshop to detect the image was a fake. But a normal person will be able to use AI to detect it. I see all negative nancies about it. I don’t see anyone realizing that as good as the tools are at faking, the tools to detect will be exactly as good.
    - fnimick 4y ago
      > the tools to detect will be exactly as good [citation needed]
      - icepat 4y ago
        I was curious about this recently, so I built a very rudimentary neural net trained on GPT generated text messages, and human generated text messages. I was able to get a surprisingly good detection accuracy with just under 1k lines from each sample set. I'm not sure it's as apocalyptic as people think.
        kroolik 4y ago
        I guess time will really tell. We might as well hit the problem Google Translate hit, where the training set started to contain more and more data created from GT itself. Similar thing may happen with your NN. At some point there may be so much AI-generated content (by different AIs) that it may be difficult to compose a trustworthy training set.
        icepat 4y ago
        I suppose a solution to this would be something like pre-war iron. We would have to rely on archived sources, like Wikipedia past edits, that come from before ChatGPT existed.
    - kroolik 4y ago
      I thought about this a bit, and am wondering about the results of LLM-fueled arms-race when it comes to figuring out whether an AI has created some piece of text. I'm worried we may reach a point AI will get so good faking people that real peoples' output will be treated as fake due to only as many combinations possible that originates from humans. You will start depending on AI telling you what is true and what is fake for every single piece of information. This leads to a question how to tell which AI is right - you can't really verify an opinion, only facts. And being able to manipulate opinions is a very strong perk.
  - ChatGTP 4y ago
    Amen, “it happened before so let’s keep it up, maybe at uncontrollable levels, yay”
  - dmix 4y ago
    AI doesn't solve the reputation problem. Just because you can make 1000x more fake news articles doesn't mean anyone will ever see it. Spam is still spam, AI doesn't suddenly make your site rank on Google or get you followers on Twitter or get you upvotes on Reddit. I'm skeptical content generation was the thing holding back this wave of fake news and scams from ruining the Internet. These doomer posts are always handwavy on the specifics. And we've seen little evidence of it operating at scale in the past 2yrs since these tools have been unleashed. Maybe automated niche targeting but that still depends on networks with reputation systems + spam detection... So mostly depends on ostensibly solvable tech problems like caller ID spoofing, or someone sending mass email spoofed campaigns.
    - jacquesm 4y ago
      You can easily apply AI to the generation of fake accounts used to amplify the message, this is a tried and true playbook that can now be enacted at lower cost and with much greater speed. That you have seen little evidence of it doesn't mean that it isn't happening, it could also mean that (1) you're looking in the wrong places, (2) that it is so good that you don't detect it and probably other explanations besides those.
      - dmix 4y ago
        I'm on HN daily like you are, I'm sure if this was a developing crisis in spam detection and social media we'd be hearing about it constantly. People on HN love a good AI doomer story, it won't be hiding. Otherwise it's mostly just predictions of widescale disruption which beyond hyper targetted attacks I remain skeptical.
        jacquesm 4y ago
        Run a couple of hundred comments on HN sequentially sampled from the comments stream through the AI detector and see what pops up. The fraction is rising steadily.
        themacguffinman 4y ago
        People complain about it on HN a lot, although "hearing about it constantly" is a fairly vague and nebulous criteria. Search "gmail spam" on hn.algolia.com with a "past year" time filter, here's a bunch of the most popular titles: - Bypassing Gmail's spam filters with ChatGPT - Ask HN: What's happening with Gmail spam filtering? - Tell HN: Gmail's spam filters have gone bonkers - Gmail spam is out of control - Ask HN: Gmail having spam problems for anybody else? - Ask HN: Have you noticed an increase of spam evading Gmail's filters recently? - Ask HN: Has Gmail's spam filter been defeated by this type of spam lately? Maybe it's not relentlessly posted every week but a lot of people are certainly noticing it on HN.
  - Al-Khwarizmi 4y ago
    True in general, but in the particular case of misinformation, there was so much of it already before LLMs that I don't think scale makes a qualitative difference. Maybe even the opposite, perhaps the deluge of AI-generated content will make the average person trust less what they find at random sources... which is healthy. Note that in general I'm not too optimistic about AI risks (the very news that motivated this thread is scary) but I don't see the worry about mass misinformation in particular to be such a big deal.
  - icepat 4y ago
    Right, that's a fair point, but I think when it comes to fake news, the issue really is about the quality, rather than quantity. Really lethal fake news requires you to be in touch at a deep level with your target audience to make it stick.
- r9295 4y ago
  Yeah but hyperlocal targeting with extremely specific information/metrics was not possible.
- all2 4y ago
  I'd be curious to hear what Breitbart has faked.
  - fwip 4y ago
    This list contains a few: https://en.wikipedia.org/wiki/Breitbart_News#Notable_events https://en.wikipedia.org/wiki/Breitbart_News#Notable_events (Not all of those items are made up by Breitbart, but some, like the NC wildfires, are.)
    - SV_BubbleTime 4y ago
      I see controversy, but not many of those include anything fake.
      - deleted 4y ago
        [deleted]
      - fwip 3y ago
        HN says that I should assume good faith, so please reread the list again - it must have slipped your notice the first time around.
    - Shaggy2000 4y ago
      [dead]
    - carlosjobim 4y ago
      [flagged]
    - 5040 4y ago
      Personally, I was won over by Scott Alexander's argument that news sites very rarely lie. They very often mislead and obfuscate, but that's fakery of another kind. https://astralcodexten.substack.com/p/sorry-i-still-think-i-am-right-about https://astralcodexten.substack.com/p/sorry-i-still-think-i-...
- Shaggy2000 4y ago
  [dead]
- rejectfinite 4y ago
  >Breitbart Was atleast trying to look legit. I think OP means like blogspam news scam shit.
  - dsfyu404ed 4y ago
    > I think OP means like blogspam news scam shit. The bad-faith commenters, YouTube, journalists and other types who have an axe to grind already happily cite garbage sources without verifying (or perhaps caring) when making their motivated arguments and there's more than enough out there to back up whatever BS they're trying to spew any minute. I don't see how quantity available changes that. And of course AI can be deployed in the counter direction. I think (hope) you need a qualitative change in order to tip the balance of power.
yamtaddle 4y ago
The "fun" thing about the tech in the current state is that it's wrong quite a bit, which puts a cap on how much of a productivity boost it can give any organization that cares about correctness or its reputation; meanwhile, it has nigh-unlimited productivity-boosting promise for organizations that don't care about correctness or reputation. Which kinds of organizations don't care about correctness or reputation? Scammers, spammers, and certain types of propaganda-spewing organization, especially those that puppeteer "grassroots" campaigns. We may give a single-digit multiplier in productivity to some roles in legitimate business, while giving a two or three digit multiplier to similar roles in harmful, parasitic organizations.
- smt88 4y ago
  I'm much more concerned about dictators than I am about scammers. Some countries have already successfully rewritten the history of less than 30 years ago with surveillance and media control. What will happen when they can also control all digital reality? They can invent an uprising of millions of people that never actually happened, and there will be photo, audio, and video "evidence". AGI is scary, but it's not immediate. I think companies like OpenAI are wrecklessly pushing us toward immediate societal collapse.
  - infogulch 4y ago
    I'm also very concerned about where AI will take us, but will bad actors just quit if OpenAI decides to halt? Or will they continue to develop it, but in secret, so we have no clue what its capabilities are or how to control it? I fear the only way out is through.
    - iamdamian 4y ago
      Couldn't you say this about nuclear research or bioweapons as well? Yet those are highly (and, seemingly effectively) regulated.
      - from 4y ago
LatticeAnimal 4y ago
... That is an interesting point. I wonder how long it will be before some enterprising developer hooks up an LLM to a HN account with the instructions to "blend in while promoting X". Maybe that AI already exists... It wouldn't take more than a day.
li4ick 4y ago
I'm also more on the negative side because I'm really not convinced by the whole "AI will just automate/remove the boring aspects of life". Every single prototype capability of current AI points in the direction of a worst-case scenario of overall misery.
- yoyohello13 4y ago
  Absolutely. I feel like I'm just watching the car crash in slow motion. Hopefully I'm wrong, but I really don't see this AI revolution working out well for humanity (in the short/medium term). I'm sure we will eventually get through it, but I think life for most is going to get real hard for the next 20-50 years.
- anonymouskimmer 4y ago
  > Every single prototype capability of current AI points in the direction of a worst-case scenario of overall misery. Software is often aimed at solving problems and making processes more cost-effective. Founders are often most interested in solving their own problems, problems of their friends, or the most valuable problems of their customers. For B2B companies to customers are other businesses. The greatest cost for businesses are employee costs. Solving the employee cost, but eliminating employees, is a great target for business owners.
morkalork 4y ago
I think we're passed due for some kind of 2FA for the phone. Sometimes my bank or credit card company will call and try to sell me shit and I tell them I have no proof of who they are hang up. Too bad for them.
backtoyoujim 4y ago
Like I'm going to carry photos of traffic lights and crosswalks everywhere I go.
sebzim4500 4y ago
I'm sure those will all be serious problems (or already are problems) but I think they pale in comparison to the potential upsides. This could be the biggest jump in productivity since the industrial revolution. Managed properly, it could lead to what is essentially a utopia.
- layer8 4y ago
  I don’t think the way economic and other incentives work are necessarily favorable to a utopia being established.
  - sebzim4500 4y ago
    As long as one country does it properly and lets people immigrate we are ok.
toss1 4y ago
>>Are we really approaching the reality where we need 2FA to trust the other person is really who they are? YES In fact, not approaching, but already passed the threshold. If you have any assets, it'd time to be sure you have a set of actually obscure un-guessable prompt & responses to verify identity in your family. And it needs to be better than "what's our first dog's name?" or "where do we vacation?" type stuff found on FB; if they're going to the trouble of getting images & voice samples to clone, they'll find that stuff. Now, it is not that everyone is famous for 15 minutes, it is that everyone needs to be up-to-date on security to avoid being randomly shot or scammed. Nice society we've built.
newhouseb 4y ago
I was having a conversation with some friends the other day about what schemes might mitigate some of these risks, something like an anti-safe word, i.e. a "danger word" that someone can be used to remotely validate that a loved one is authentically in danger. This is fairly low-tech and likely susceptible to various kinds of social engineering, but I'm curious what a more robust approach might look like that doesn't involve us all regurgitating 6-digit codes like robots all the time.
- theNJR 4y ago
  Yep, this is the way
- mLuby 4y ago
  Let's say you call or text me, requesting gift cards or my password or something else weird. I tell you I'll email you immediately to verify. I email you to confirm you want N $X gift cards. You do so. Or if the inbound contact is by email, you verify by text or phone call. Or Signal/Discord/FB chat apps, etc. Heck even LinkedIn Inmail could be your verification channel. You could still be compromised, but that'd require attackers to have significantly more access and readiness. And if you reach the other person and they're unaware, now you both have some support as you figure it out.
  - jamesshamenski 4y ago
    If one service was compromised, isn't it reasonable to think that the scammer can access other services during this breach?
confoundcofound 4y ago
> Are we really approaching the reality where we need 2FA to trust the other person is really who they are? Yes we are. We are approaching a world where people will not want to invest their time, energy, and emotion engaging with other supposed humans remotely unless they have verified their personhood / identity. I'm actually surprised we have not yet seen an entire industry built around human authentication. Seems like Apple is the only company taking this seriously. The standard of human interaction will either be meeting IRL or signing communications biometrically.
- NickC25 4y ago
  >I'm actually surprised we have not yet seen an entire industry built around human authentication. Seems like Apple is the only company taking this seriously. Completely agree - but to that - with how quickly AI voice cloning is improving, and how quickly image creation tools are improving, how long will it be until someone can sample my voice well enough to command Siri to do something and Siri won't be able to tell the difference? How long will it be until an AI generated photo of me is so realistic that it can unlock my phone via FaceID? I have a feeling that's going to happen in a shorter timeframe than we all think it will. I mean, my voice is somewhat similar to my father's, and if I'm around his phone, I can say "Hey Siri" and his phone will think it's his voice, not mine.
  - confoundcofound 4y ago
    We're going to see an arms race between AI and biometrics / authentication technology. My hope is that the latter outpaces the former, but it doesn't seem to be panning out that way. In the future, we might be only able to unlock our phone or command Siri when hooked up to a continuous biometrics sensor.
    - NickC25 4y ago
      You're probably not too far off from what will eventually become reality. Perhaps that's Apple's angle with the AR/VR headset it supposedly has in the works - if you've got your headset on, you can unlock your devices, and if you don't, you have to manually unlock devices via a password or PIN or whatever.
comfypotato 4y ago
How is AI different than any other technology in this respect? As far back as the discovery of fire, new technology has enabled more positive outcomes than negative. Some tech more so than others, I guess, but what makes you lean so negatively regarding AI? It’s already improved my life considerably. Just the basic ChatGPT web app has extended my capacity in multiple respects.
- puttycat 4y ago
  Rule of thumb: reductionist arguments don't hold for technologies from the Internet onwards. For example, the Internet is nothing like TV, so arguments like "we grew up on TV and ended up ok" don't hold. Same holds in my opinion to AI-scams.
  - bawolff 4y ago
    I disagree. Do you have examples to back that up? > we grew up on TV and ended up ok Not sure what you are comparing here, but we (people in early 30s) also grew up on the internet and mostly ended up ok. Some people who grew up on tv did not end up ok.
    - pixl97 4y ago
      Quantity is a quality in itself. If I catch a fish in a lake that's not problem. If I catch all the fish in the lake we need to stock the lake again with fish. If I catch 10^30 fish on the planet, fish are extinct.
      - ta1243 4y ago
        > If I catch 10^30 fish on the planet, fish are extinct. The entire planet is extinct
        pixl97 4y ago
        Oopsie, I turned the visible universe into paperclips.
simion314 4y ago
Maybe all countries need to finally give importance to this issues, put a few take scam crimes seriously, stop trading with countries that do not cooperate, put the scammers in jail. Tech evolved any idiot can get a copy of photoshop and some video software, we need tp solve the problem, otherwise is like preventing creation of email because we are too incompetent to address the spam problem.
marksmith2996 4y ago
Don't worry it'll probably just exterminate us anyway then fake news doesn't matter.
dave_sullivan 4y ago
Don't be negative about revolutionary new technology that can make the world a better place. As it stands, people believe whatever dumb thing they read online. Thanks to AI, they will have to learn that if something is out of the ordinary, they should confirm with multiple sources. I call this a net win. And for those literally medically incapable of this level of reasoning, we will soon have "AI firewalls" (Gibson ICE) that can tell people "Hey, this looks like a scam!" and also help them reason about complex topics. This is fantastic, what a time to be alive!
- kroolik 4y ago
  > NOW they will have to learn that if something is out of the ordinary I once shared this approach, but learned that the majority of people don't really want to leave their echo chambers and gladly accept anything that amplifies it. And the bad part about it is that the western world is based on democracy, and democracy is controlled by the majority. You may not care about people that are easy to manipulate, but in reality you are not the target as your voice won't matter at scale.
sandworm101 4y ago
>> Are we really approaching the reality where we need 2FA to trust the other person is really who they are? We are already there. In the kidnapping context, we have been there for a great many years. If someone say they have kidnapped my child, I will text/call that child immediately to verify. The 'second factor' is the realworld daughter. A kidnapper must both create a facsimile of the daughter and then also render the real daughter incommunicado. We don't need new 2FA because we already have it.
darkerside 4y ago
It's funny, technology was able to bring us together across geographic distances. Someone on the other side of the world was a phone call away. You can turn on your TV or phone and instantly tap into seeing, hearing, and reading people and their thoughts from around the globe. Almost as quickly, AI may unwind all of that. It lay become that there's more noise than signal across all of these technical media. The only thing you can trust, and the only thing that matters, is the people in front of you. Just like the old days.
deleted 4y ago
[deleted]
mr_mitm 4y ago
We need a PKI run by the governments. National ID cards that are smart cards. Cryptographically sign any and all digital communication. Self-signed certificates could still be used with TOFU. However, I'm not sure if total loss of pseudonymity is less of a horror scenario.
- layer8 4y ago
  You can still use pseudonyms with PKIs. The CA will know who you really are, but it doesn’t need to be public by default. And it can be regulated such that a judicial order is necessary to disclose your identity to a third party (including to the government).
- tommiegannert 4y ago
  I've been wondering what would be the event that causes digital signatures to become the norm. Though, I didn't expect it to be machine learning. I'm scared by how much is just done by phone with people who don't know me.
lbotos 4y ago
I think it will be addressed by "community". For the last probably decade (maybe more) every online social space has been pressured to serve global scale. That increases surface area, and increases more strangers. Smaller, more trusted rooms, will combat the threat of misinformation. The universe is built on waves/cycles. I think we are going to see pressures to make smaller rooms because of AI. I'm actually excited for smaller, more intimate spaces, with trusted people who now have 10x ability from the powers of AI. That seems like a LOT of fun.
dclusin 4y ago
Ai generated news has actually been a thing for a while now. The news agencies that use it still do fact check the output.
ravenstine 4y ago
Your concern is understandable and relatable, but I approach the issue from a stance that is, while not necessarily optimistic, definitely is less pessimistic. I don't think we truly know just yet how negative an impact AI will have on society. Every time there has been a technological leap, people have panicked over what the gizmo of the now will do to society. Again, not an invalid concern, but society has yet to have been blown apart by anything. Also, everything potentially being just AI may inadvertently get the public to do the right thing. People should never have been as trusting of authority figures or institutions in the first place. If everyone assumes that everything is likely to be complete hogwash, which was already true in many cases, then they may not just swallow everything as fact. Maybe fewer people will blindly consume news, which is a good thing; 99% of news is not actionable or good for an average person's well-being. And if enough ransom demands turn out to be AI-generated scams such that the real ones are overall far less successful, it's possible that fewer people are kidnapped for ransom in the first place. I'm not saying I know any of this, but rather the opposite.
- yeck 4y ago
  In a world where people need to be constantly paranoid, I think many will develop true paranoia, or just give up trying to understand what is going on and defer to who has the power already. Certainly it is not possible for individuals to seek truth on all things on their own, even if it was their full time responsibility to attempt so.
onemoresoop 4y ago
> If you were afraid of your elderly relatives being scammed by people pretending to be policemen or grandchildren, now more tech-concious people will get scammed by the voice and look of their relatives. Are we really approaching the reality where we need 2FA to trust the other person is really who they are? I thought I was immune to certain scams where the scammer accent was a dead giveaway that they were scammers. Now with capabilities of AI I feel somewhat vulnerable again though Im far from being elderly.
ZainRiz 4y ago
The past century has really been an anomaly when it came to trusting news, with photos and videos seeming to be reliable proof. Before that, societies had to be structured very differently to account for the lack of proof. You'd hear people from out of town talking about what they saw in a neighboring city. You'd need to judge how trustworthy the person was. People would expect a chain of narration, to understand how _that_ person came to learn a bit of information (or if they claimed to witness it first hand). As AI generated content becomes more popular, I predict we as a society are going to go back to relying more and more on the reputation of the speakers in question. Who might you consider reputable? People you've met personally, people who your community respects, and of course, influencers who you follow who's persuasive words match your pre-existing world views
- Georgelemental 4y ago
  This change will likely advantage established institutions, no? It will be much harder for a whistleblower, random, etc to prove something that the powers-that-be would prefer be kept quiet.
MagicMoonlight 4y ago
It’s going to be great. The internet will be a lifeless sack of shit full of bots and we’ll all have to meet up in person for true conversation. We can have parties again and have friends instead of depression.
bcrl 4y ago
Of course the dominant 2FA methods will be those that are easily forged, so the general public will still get swindled (I'm looking at you banks that are insisting on using SMS to "verify" my identity). It's as if nobody cares about the world programmers are creating.
1letterunixname 4y ago
This Michael Bay movie script writes itself: 1. A not exactly rocket scientist POTUS takes office in 2024. Let's suppose they were a populist, triumphalist, religious, agro, law-and-order DINO to make it interesting and fictional. 2. Rogue AI launches a social engineering communication offensive against CENTCOM impersonating generals, cols, and ltcs against enlisted and lower ranks to carry out a first strike against China with a digital hallucination that Taiwan is being "attacked". It uses details gleaned by wiretapping the upper echelon to circumvent normal N-person keying rules and authentication protocols. WarGames 2: Joshua Wins

icepat 4y ago

> Chatgpt and the likes generate convincing fake news It's not like fake news was a non-issue before ChatGPT existed. Breitbart, and other fake news sites existed for years before this was even imaginable. Fake graphics were around for ages too, with image manipulation, even before computers. Take for example the Surgeon's Photo of the Loch Ness Monster.

jacquesm 4y ago
This argument has been rehashed time and again and it's getting a bit boring: the fact that you can do something at all is qualitatively different from when you can do that same thing at scale.
- chimpanzee 4y ago
  Agreed. As I’m sure you know, it’s easier for people to reason when the arena presents only discrete possibilities, especially binary ones, eg yes/no or black/white. The arena of the continuous, which encompasses most of the natural world, is far more difficult. It doesn’t allow for the sort of arguments where one can assert things with ego-boosting absolute confidence. Matters of scale naturally fall into the continuous sort, but scale can also introduce new discrete possibilities. Perhaps we need to present the new specific, discrete effects and outcomes that AI-at-scale will introduce. Maybe, just maybe, that’ll change minds of people who are truly open to being changed. My intuition though is that this argument is repeated here (on HN in particular) not for lack of knowledge or thought, but rather because they WANT to see the chaos that’ll result. They want all the positives and negatives, no matter the balance, simply because it’s exciting and adds to their “mundane” lives. Where the mundanity is of course subjective, simply the result of their default worldview, rather than anything nearing an objective description of the world as it is.
  - jacquesm 4y ago
    > simply because it’s exciting and adds to their lives. It's certainly exciting but I wouldn't bet on it adding to our lives just yet. Maybe. But many avenues leading to net negative outcomes are still part of the tree.
    - chimpanzee 4y ago
      Sorry, I had already edited that portion before seeing your response. But you’re absolutely right.
    - deleted 4y ago
      [deleted]
- SV_BubbleTime 4y ago
  You could make fake photos with photoshop. And now you can make them with AI. A normal person couldn’t use photoshop to detect the image was a fake. But a normal person will be able to use AI to detect it. I see all negative nancies about it. I don’t see anyone realizing that as good as the tools are at faking, the tools to detect will be exactly as good.
  - fnimick 4y ago
    > the tools to detect will be exactly as good [citation needed]
    - icepat 4y ago
      I was curious about this recently, so I built a very rudimentary neural net trained on GPT generated text messages, and human generated text messages. I was able to get a surprisingly good detection accuracy with just under 1k lines from each sample set. I'm not sure it's as apocalyptic as people think.
      - kroolik 4y ago
        I guess time will really tell. We might as well hit the problem Google Translate hit, where the training set started to contain more and more data created from GT itself. Similar thing may happen with your NN. At some point there may be so much AI-generated content (by different AIs) that it may be difficult to compose a trustworthy training set.
        icepat 4y ago
        I suppose a solution to this would be something like pre-war iron. We would have to rely on archived sources, like Wikipedia past edits, that come from before ChatGPT existed.
  - kroolik 4y ago
    I thought about this a bit, and am wondering about the results of LLM-fueled arms-race when it comes to figuring out whether an AI has created some piece of text. I'm worried we may reach a point AI will get so good faking people that real peoples' output will be treated as fake due to only as many combinations possible that originates from humans. You will start depending on AI telling you what is true and what is fake for every single piece of information. This leads to a question how to tell which AI is right - you can't really verify an opinion, only facts. And being able to manipulate opinions is a very strong perk.
- ChatGTP 4y ago
  Amen, “it happened before so let’s keep it up, maybe at uncontrollable levels, yay”
- dmix 4y ago
  AI doesn't solve the reputation problem. Just because you can make 1000x more fake news articles doesn't mean anyone will ever see it. Spam is still spam, AI doesn't suddenly make your site rank on Google or get you followers on Twitter or get you upvotes on Reddit. I'm skeptical content generation was the thing holding back this wave of fake news and scams from ruining the Internet. These doomer posts are always handwavy on the specifics. And we've seen little evidence of it operating at scale in the past 2yrs since these tools have been unleashed. Maybe automated niche targeting but that still depends on networks with reputation systems + spam detection... So mostly depends on ostensibly solvable tech problems like caller ID spoofing, or someone sending mass email spoofed campaigns.
  - jacquesm 4y ago
    You can easily apply AI to the generation of fake accounts used to amplify the message, this is a tried and true playbook that can now be enacted at lower cost and with much greater speed. That you have seen little evidence of it doesn't mean that it isn't happening, it could also mean that (1) you're looking in the wrong places, (2) that it is so good that you don't detect it and probably other explanations besides those.
    - dmix 4y ago
      I'm on HN daily like you are, I'm sure if this was a developing crisis in spam detection and social media we'd be hearing about it constantly. People on HN love a good AI doomer story, it won't be hiding. Otherwise it's mostly just predictions of widescale disruption which beyond hyper targetted attacks I remain skeptical.
      - jacquesm 4y ago
        Run a couple of hundred comments on HN sequentially sampled from the comments stream through the AI detector and see what pops up. The fraction is rising steadily.
      - themacguffinman 4y ago
        People complain about it on HN a lot, although "hearing about it constantly" is a fairly vague and nebulous criteria. Search "gmail spam" on hn.algolia.com with a "past year" time filter, here's a bunch of the most popular titles: - Bypassing Gmail's spam filters with ChatGPT - Ask HN: What's happening with Gmail spam filtering? - Tell HN: Gmail's spam filters have gone bonkers - Gmail spam is out of control - Ask HN: Gmail having spam problems for anybody else? - Ask HN: Have you noticed an increase of spam evading Gmail's filters recently? - Ask HN: Has Gmail's spam filter been defeated by this type of spam lately? Maybe it's not relentlessly posted every week but a lot of people are certainly noticing it on HN.
- Al-Khwarizmi 4y ago
  True in general, but in the particular case of misinformation, there was so much of it already before LLMs that I don't think scale makes a qualitative difference. Maybe even the opposite, perhaps the deluge of AI-generated content will make the average person trust less what they find at random sources... which is healthy. Note that in general I'm not too optimistic about AI risks (the very news that motivated this thread is scary) but I don't see the worry about mass misinformation in particular to be such a big deal.
- icepat 4y ago
  Right, that's a fair point, but I think when it comes to fake news, the issue really is about the quality, rather than quantity. Really lethal fake news requires you to be in touch at a deep level with your target audience to make it stick.
r9295 4y ago
Yeah but hyperlocal targeting with extremely specific information/metrics was not possible.
all2 4y ago
I'd be curious to hear what Breitbart has faked.
- fwip 4y ago
  This list contains a few: https://en.wikipedia.org/wiki/Breitbart_News#Notable_events https://en.wikipedia.org/wiki/Breitbart_News#Notable_events (Not all of those items are made up by Breitbart, but some, like the NC wildfires, are.)
  - SV_BubbleTime 4y ago
    I see controversy, but not many of those include anything fake.
    - deleted 4y ago
      [deleted]
    - fwip 3y ago
      HN says that I should assume good faith, so please reread the list again - it must have slipped your notice the first time around.
  - Shaggy2000 4y ago
    [dead]
  - carlosjobim 4y ago
    [flagged]
  - 5040 4y ago
    Personally, I was won over by Scott Alexander's argument that news sites very rarely lie. They very often mislead and obfuscate, but that's fakery of another kind. https://astralcodexten.substack.com/p/sorry-i-still-think-i-am-right-about https://astralcodexten.substack.com/p/sorry-i-still-think-i-...
Shaggy2000 4y ago
[dead]
rejectfinite 4y ago
>Breitbart Was atleast trying to look legit. I think OP means like blogspam news scam shit.
- dsfyu404ed 4y ago
  > I think OP means like blogspam news scam shit. The bad-faith commenters, YouTube, journalists and other types who have an axe to grind already happily cite garbage sources without verifying (or perhaps caring) when making their motivated arguments and there's more than enough out there to back up whatever BS they're trying to spew any minute. I don't see how quantity available changes that. And of course AI can be deployed in the counter direction. I think (hope) you need a qualitative change in order to tip the balance of power.

yamtaddle 4y ago

The "fun" thing about the tech in the current state is that it's wrong quite a bit, which puts a cap on how much of a productivity boost it can give any organization that cares about correctness or its reputation; meanwhile, it has nigh-unlimited productivity-boosting promise for organizations that don't care about correctness or reputation. Which kinds of organizations don't care about correctness or reputation? Scammers, spammers, and certain types of propaganda-spewing organization, especially those that puppeteer "grassroots" campaigns. We may give a single-digit multiplier in productivity to some roles in legitimate business, while giving a two or three digit multiplier to similar roles in harmful, parasitic organizations.

smt88 4y ago
I'm much more concerned about dictators than I am about scammers. Some countries have already successfully rewritten the history of less than 30 years ago with surveillance and media control. What will happen when they can also control all digital reality? They can invent an uprising of millions of people that never actually happened, and there will be photo, audio, and video "evidence". AGI is scary, but it's not immediate. I think companies like OpenAI are wrecklessly pushing us toward immediate societal collapse.
- infogulch 4y ago
  I'm also very concerned about where AI will take us, but will bad actors just quit if OpenAI decides to halt? Or will they continue to develop it, but in secret, so we have no clue what its capabilities are or how to control it? I fear the only way out is through.
  - iamdamian 4y ago
    Couldn't you say this about nuclear research or bioweapons as well? Yet those are highly (and, seemingly effectively) regulated.
    - from 4y ago

LatticeAnimal 4y ago

... That is an interesting point. I wonder how long it will be before some enterprising developer hooks up an LLM to a HN account with the instructions to "blend in while promoting X". Maybe that AI already exists... It wouldn't take more than a day.

li4ick 4y ago

I'm also more on the negative side because I'm really not convinced by the whole "AI will just automate/remove the boring aspects of life". Every single prototype capability of current AI points in the direction of a worst-case scenario of overall misery.

yoyohello13 4y ago
Absolutely. I feel like I'm just watching the car crash in slow motion. Hopefully I'm wrong, but I really don't see this AI revolution working out well for humanity (in the short/medium term). I'm sure we will eventually get through it, but I think life for most is going to get real hard for the next 20-50 years.
anonymouskimmer 4y ago
> Every single prototype capability of current AI points in the direction of a worst-case scenario of overall misery. Software is often aimed at solving problems and making processes more cost-effective. Founders are often most interested in solving their own problems, problems of their friends, or the most valuable problems of their customers. For B2B companies to customers are other businesses. The greatest cost for businesses are employee costs. Solving the employee cost, but eliminating employees, is a great target for business owners.

morkalork 4y ago

I think we're passed due for some kind of 2FA for the phone. Sometimes my bank or credit card company will call and try to sell me shit and I tell them I have no proof of who they are hang up. Too bad for them.

backtoyoujim 4y ago

Like I'm going to carry photos of traffic lights and crosswalks everywhere I go.

sebzim4500 4y ago

I'm sure those will all be serious problems (or already are problems) but I think they pale in comparison to the potential upsides. This could be the biggest jump in productivity since the industrial revolution. Managed properly, it could lead to what is essentially a utopia.

layer8 4y ago
I don’t think the way economic and other incentives work are necessarily favorable to a utopia being established.
- sebzim4500 4y ago
  As long as one country does it properly and lets people immigrate we are ok.

toss1 4y ago

>>Are we really approaching the reality where we need 2FA to trust the other person is really who they are? YES In fact, not approaching, but already passed the threshold. If you have any assets, it'd time to be sure you have a set of actually obscure un-guessable prompt & responses to verify identity in your family. And it needs to be better than "what's our first dog's name?" or "where do we vacation?" type stuff found on FB; if they're going to the trouble of getting images & voice samples to clone, they'll find that stuff. Now, it is not that everyone is famous for 15 minutes, it is that everyone needs to be up-to-date on security to avoid being randomly shot or scammed. Nice society we've built.

newhouseb 4y ago

I was having a conversation with some friends the other day about what schemes might mitigate some of these risks, something like an anti-safe word, i.e. a "danger word" that someone can be used to remotely validate that a loved one is authentically in danger. This is fairly low-tech and likely susceptible to various kinds of social engineering, but I'm curious what a more robust approach might look like that doesn't involve us all regurgitating 6-digit codes like robots all the time.

theNJR 4y ago
Yep, this is the way
mLuby 4y ago
Let's say you call or text me, requesting gift cards or my password or something else weird. I tell you I'll email you immediately to verify. I email you to confirm you want N $X gift cards. You do so. Or if the inbound contact is by email, you verify by text or phone call. Or Signal/Discord/FB chat apps, etc. Heck even LinkedIn Inmail could be your verification channel. You could still be compromised, but that'd require attackers to have significantly more access and readiness. And if you reach the other person and they're unaware, now you both have some support as you figure it out.
- jamesshamenski 4y ago
  If one service was compromised, isn't it reasonable to think that the scammer can access other services during this breach?

confoundcofound 4y ago

> Are we really approaching the reality where we need 2FA to trust the other person is really who they are? Yes we are. We are approaching a world where people will not want to invest their time, energy, and emotion engaging with other supposed humans remotely unless they have verified their personhood / identity. I'm actually surprised we have not yet seen an entire industry built around human authentication. Seems like Apple is the only company taking this seriously. The standard of human interaction will either be meeting IRL or signing communications biometrically.

NickC25 4y ago
>I'm actually surprised we have not yet seen an entire industry built around human authentication. Seems like Apple is the only company taking this seriously. Completely agree - but to that - with how quickly AI voice cloning is improving, and how quickly image creation tools are improving, how long will it be until someone can sample my voice well enough to command Siri to do something and Siri won't be able to tell the difference? How long will it be until an AI generated photo of me is so realistic that it can unlock my phone via FaceID? I have a feeling that's going to happen in a shorter timeframe than we all think it will. I mean, my voice is somewhat similar to my father's, and if I'm around his phone, I can say "Hey Siri" and his phone will think it's his voice, not mine.
- confoundcofound 4y ago
  We're going to see an arms race between AI and biometrics / authentication technology. My hope is that the latter outpaces the former, but it doesn't seem to be panning out that way. In the future, we might be only able to unlock our phone or command Siri when hooked up to a continuous biometrics sensor.
  - NickC25 4y ago
    You're probably not too far off from what will eventually become reality. Perhaps that's Apple's angle with the AR/VR headset it supposedly has in the works - if you've got your headset on, you can unlock your devices, and if you don't, you have to manually unlock devices via a password or PIN or whatever.

comfypotato 4y ago

How is AI different than any other technology in this respect? As far back as the discovery of fire, new technology has enabled more positive outcomes than negative. Some tech more so than others, I guess, but what makes you lean so negatively regarding AI? It’s already improved my life considerably. Just the basic ChatGPT web app has extended my capacity in multiple respects.

puttycat 4y ago
Rule of thumb: reductionist arguments don't hold for technologies from the Internet onwards. For example, the Internet is nothing like TV, so arguments like "we grew up on TV and ended up ok" don't hold. Same holds in my opinion to AI-scams.
- bawolff 4y ago
  I disagree. Do you have examples to back that up? > we grew up on TV and ended up ok Not sure what you are comparing here, but we (people in early 30s) also grew up on the internet and mostly ended up ok. Some people who grew up on tv did not end up ok.
  - pixl97 4y ago
    Quantity is a quality in itself. If I catch a fish in a lake that's not problem. If I catch all the fish in the lake we need to stock the lake again with fish. If I catch 10^30 fish on the planet, fish are extinct.
    - ta1243 4y ago
      > If I catch 10^30 fish on the planet, fish are extinct. The entire planet is extinct
      - pixl97 4y ago
        Oopsie, I turned the visible universe into paperclips.

simion314 4y ago

Maybe all countries need to finally give importance to this issues, put a few take scam crimes seriously, stop trading with countries that do not cooperate, put the scammers in jail. Tech evolved any idiot can get a copy of photoshop and some video software, we need tp solve the problem, otherwise is like preventing creation of email because we are too incompetent to address the spam problem.

marksmith2996 4y ago

Don't worry it'll probably just exterminate us anyway then fake news doesn't matter.

dave_sullivan 4y ago

Don't be negative about revolutionary new technology that can make the world a better place. As it stands, people believe whatever dumb thing they read online. Thanks to AI, they will have to learn that if something is out of the ordinary, they should confirm with multiple sources. I call this a net win. And for those literally medically incapable of this level of reasoning, we will soon have "AI firewalls" (Gibson ICE) that can tell people "Hey, this looks like a scam!" and also help them reason about complex topics. This is fantastic, what a time to be alive!

kroolik 4y ago
> NOW they will have to learn that if something is out of the ordinary I once shared this approach, but learned that the majority of people don't really want to leave their echo chambers and gladly accept anything that amplifies it. And the bad part about it is that the western world is based on democracy, and democracy is controlled by the majority. You may not care about people that are easy to manipulate, but in reality you are not the target as your voice won't matter at scale.

sandworm101 4y ago

>> Are we really approaching the reality where we need 2FA to trust the other person is really who they are? We are already there. In the kidnapping context, we have been there for a great many years. If someone say they have kidnapped my child, I will text/call that child immediately to verify. The 'second factor' is the realworld daughter. A kidnapper must both create a facsimile of the daughter and then also render the real daughter incommunicado. We don't need new 2FA because we already have it.

darkerside 4y ago

It's funny, technology was able to bring us together across geographic distances. Someone on the other side of the world was a phone call away. You can turn on your TV or phone and instantly tap into seeing, hearing, and reading people and their thoughts from around the globe. Almost as quickly, AI may unwind all of that. It lay become that there's more noise than signal across all of these technical media. The only thing you can trust, and the only thing that matters, is the people in front of you. Just like the old days.

deleted 4y ago

[deleted]

mr_mitm 4y ago

We need a PKI run by the governments. National ID cards that are smart cards. Cryptographically sign any and all digital communication. Self-signed certificates could still be used with TOFU. However, I'm not sure if total loss of pseudonymity is less of a horror scenario.

layer8 4y ago
You can still use pseudonyms with PKIs. The CA will know who you really are, but it doesn’t need to be public by default. And it can be regulated such that a judicial order is necessary to disclose your identity to a third party (including to the government).
tommiegannert 4y ago
I've been wondering what would be the event that causes digital signatures to become the norm. Though, I didn't expect it to be machine learning. I'm scared by how much is just done by phone with people who don't know me.

lbotos 4y ago

I think it will be addressed by "community". For the last probably decade (maybe more) every online social space has been pressured to serve global scale. That increases surface area, and increases more strangers. Smaller, more trusted rooms, will combat the threat of misinformation. The universe is built on waves/cycles. I think we are going to see pressures to make smaller rooms because of AI. I'm actually excited for smaller, more intimate spaces, with trusted people who now have 10x ability from the powers of AI. That seems like a LOT of fun.

dclusin 4y ago

Ai generated news has actually been a thing for a while now. The news agencies that use it still do fact check the output.

ravenstine 4y ago

Your concern is understandable and relatable, but I approach the issue from a stance that is, while not necessarily optimistic, definitely is less pessimistic. I don't think we truly know just yet how negative an impact AI will have on society. Every time there has been a technological leap, people have panicked over what the gizmo of the now will do to society. Again, not an invalid concern, but society has yet to have been blown apart by anything. Also, everything potentially being just AI may inadvertently get the public to do the right thing. People should never have been as trusting of authority figures or institutions in the first place. If everyone assumes that everything is likely to be complete hogwash, which was already true in many cases, then they may not just swallow everything as fact. Maybe fewer people will blindly consume news, which is a good thing; 99% of news is not actionable or good for an average person's well-being. And if enough ransom demands turn out to be AI-generated scams such that the real ones are overall far less successful, it's possible that fewer people are kidnapped for ransom in the first place. I'm not saying I know any of this, but rather the opposite.

yeck 4y ago
In a world where people need to be constantly paranoid, I think many will develop true paranoia, or just give up trying to understand what is going on and defer to who has the power already. Certainly it is not possible for individuals to seek truth on all things on their own, even if it was their full time responsibility to attempt so.

onemoresoop 4y ago

> If you were afraid of your elderly relatives being scammed by people pretending to be policemen or grandchildren, now more tech-concious people will get scammed by the voice and look of their relatives. Are we really approaching the reality where we need 2FA to trust the other person is really who they are? I thought I was immune to certain scams where the scammer accent was a dead giveaway that they were scammers. Now with capabilities of AI I feel somewhat vulnerable again though Im far from being elderly.

ZainRiz 4y ago

The past century has really been an anomaly when it came to trusting news, with photos and videos seeming to be reliable proof. Before that, societies had to be structured very differently to account for the lack of proof. You'd hear people from out of town talking about what they saw in a neighboring city. You'd need to judge how trustworthy the person was. People would expect a chain of narration, to understand how _that_ person came to learn a bit of information (or if they claimed to witness it first hand). As AI generated content becomes more popular, I predict we as a society are going to go back to relying more and more on the reputation of the speakers in question. Who might you consider reputable? People you've met personally, people who your community respects, and of course, influencers who you follow who's persuasive words match your pre-existing world views

Georgelemental 4y ago
This change will likely advantage established institutions, no? It will be much harder for a whistleblower, random, etc to prove something that the powers-that-be would prefer be kept quiet.

MagicMoonlight 4y ago

It’s going to be great. The internet will be a lifeless sack of shit full of bots and we’ll all have to meet up in person for true conversation. We can have parties again and have friends instead of depression.

bcrl 4y ago

Of course the dominant 2FA methods will be those that are easily forged, so the general public will still get swindled (I'm looking at you banks that are insisting on using SMS to "verify" my identity). It's as if nobody cares about the world programmers are creating.

1letterunixname 4y ago

This Michael Bay movie script writes itself: 1. A not exactly rocket scientist POTUS takes office in 2024. Let's suppose they were a populist, triumphalist, religious, agro, law-and-order DINO to make it interesting and fictional. 2. Rogue AI launches a social engineering communication offensive against CENTCOM impersonating generals, cols, and ltcs against enlisted and lower ranks to carry out a first strike against China with a digital hallucination that Taiwan is being "attacked". It uses details gleaned by wiretapping the upper echelon to circumvent normal N-person keying rules and authentication protocols. WarGames 2: Joshua Wins