10 ms·

A Google search for practically any long-tail keywords will reveal that LLMs have already had a very significant impact. DuckDuckGo has suffered even more. Soci

A Google search for practically any long-tail keywords will reveal that LLMs have already had a very significant impact. DuckDuckGo has suffered even more. Social media is absolutely lousy with AI-powered fraud of varying degrees of sophistication. It's glib to dismiss safety concerns because we haven't all turned into paperclips yet. LLMs and image gen models are having real effects now. We're already at a point where AI can generate text and images that will fool a lot of people a lot of the time. For every college-educated young person smugly pointing out that they aren't fooled by an image with six-fingered hands, there are far more people who had marginal media literacy to begin with and are now almost defenceless against a tidal wave of hyper-scaleable deception. We're already at a point where we're counselling elders to ignore late-night messages from people claiming to be a relative in need of an urgent wire transfer. What defences do we have when an LLM will be able to have a completely fluent, natural-sounding conversation in someone else's voice? I'm not confident that I'd be able to distinguish GPT-4o from a human speaker in the best of circumstances and I'm almost certain that I could be fooled if I'm hurried, distracted, sleep deprived or otherwise impaired. Regardless of any future impacts on the labour market or any hypothesised X-risks, I think we should be very worried about the immediate risks to trust and social cohesion. An awful lot of people are turning into paranoid weirdos at the moment and I don't particularly blame them, but I can see things getting seriously ugly if we can't abate that trend.

fsloth 2y ago
We’ve reached a stage, where it would be advisable to not release recent photos of yourself, nor any video with sound clips to public, unless you want an AI fake instaperson of yourself starting to reach out to member of your externally visible social network, asking for money, emergency help, etc. I guess we need to have an AI secretary to take in all phonecalls from now on (spam folder will become a lot more interesting with celebrity phone calls, your dead relative phoning you etc)
- smokel 2y ago
  Hopefully, we will soon enter the stage where nobody believes anything they see anymore. Then, you no longer have to be afraid of being misinterpreted, because nobody is listening anymore anyway. Great time to be alive!
  - IanCal 2y ago
    I don't know why anyone thinks this will happen. You can obviously write anything you want (we have an entire realm of works in this area that everyone knows about, fiction) and yet huge amounts of people believe passed around stories either from bad or faked media sources or entirely unsourced.
    - flakeoil 2y ago
      I'm not saying either you or the parent commenter is right or wrong, but fiction in books and movies are clearly fiction and we consume it as such. You are right that some people have been making up fake stories and others (the more naive) have been quick to believe in those false stories. The difference now is that it's not just text invented and written by a human, which takes time and dedication. Now it's done in a second. On top of that it's easy to enhance the text with realistic photos, audio and video. It becomes much more convincing. And this material is created in a few seconds or minutes. It's hard to know what to believe if you get a phone call with the voice of your child or colleague, and your "child"/"colleague" replies within milliseconds in a convincing way.
      - IanCal 2y ago
        I agree it's fundamentally different in application which I think will have a large impact (just like targeted advertising with optimisation vs billboards), but my point is that given people know you can just write anything and yet misinformation is abound - I don't see how knowing that you can fake any picture or video or sound leading to a situation where everyone just stops believing them. I think unfortunately it will massively lower the trust of actual real videos and images, because someone can dismiss them with little thought.
  - simianparrot 2y ago
    Luckily there’s a “solution” to that: Just don’t use the internet for dialogue anymore. As someone that grew up with late-90’s internet culture and has seen all the pros and cons and changes over the decades, I find myself using the internet less and less for dialogue with people. And I’m spending more time in nature and saying hi to strangers in reality. I’m still worries about the impact this will have on a lot of people’s ability to reason however. “Just” Tik Tok and apps like it has already had devastating results on certain demographics.
- sethammons 2y ago
  I'm paranoid enough that I now modulate my voice and speak differently when answering an unknown phone call just in case they are recording and building a model to call back a loved one later. If they do get a call, they will be like, "why are you talking like that?"
  - NikkiA 2y ago
    I can only imagine "Hello, this is Guy Incognito..."
  - octatrack 2y ago
    Better yet, don’t answer unknown phone calls.
    - pooper 2y ago
      > unknown phone calls This is my biggest gripe against the telecom industry. Calls pretending to be from someone else. For every single call, someone somewhere must know at least the next link in the chain to connect a call. Keep following the chain until you find someone who either through malice or by looking the other way allows someone to spoof someone else's number AND remove their ability to send the current link in the chain (or anyone) messages. (Ideally also send them to prison if they are in the same country.) It shouldn't be that hard, right?
      - mabster 2y ago
        Companies have complex telecoms but generally want the outside as one company number. Solution, the sender send a packet with the number they should get perceived as. Everyone sends this on. Everyone "looks the other way" by design haha
        mschuster91 2y ago
        So what, gate that feature behind a check that you can only set an outgoing caller ID belonging to a number range that you own. The technology to build trustable caller ID has existed for a long time, the problem is no one wants to be the one forcing telcos all over the world to upgrade their sometimes many decades old systems.
- PeterisP 2y ago
  I think for most people it's far too late, as there exists at least something on the internet and that something is sufficient - photos can be aged virtually and a single photo is enough, voice doesn't change much and you need only a tiny sample, etc. And that's the case even if you've never ever posted anything on your social media - it could be family&friends, or employer, or if you're ever been in a public-facing job position that has ever done any community outreach, or ever done a public performance with your music or another hobby, or if you've ever walked past a news crew asking questions to bystanders of some event, or if you've ever participated in some contests or competitions or sports leagues, etc, all of that is generally findable in various archives.
  - swores 2y ago
    > photos can be aged virtually and a single photo is enough I'm sure AI-based ageing can do a good enough job to convince many people that a fake image of someone they haven't seen for years is an older version of the person they remember; but how often would it succeed in ageing an old photo in such a way that it looks like a person I have seen recently and therefore have knowledge rather than guesses about exactly what the years have changed about them? (Not a rhetorical question to disagree with you, I genuinely have no idea if ageing is predictable enough for a high % result or if it would only fool people with poor visual memory and/or who haven't seen the person in over a decade.) I feel like even ignoring the big unknowns (at what age, if any, will a person start going bald, or choose to grow a beard or to die their hair, or get a scar on their face, etc.) there must be a lot of more subtle but still important aspects from skin tone to makeup style to hair to... I've looked up photos of some school classmates that I haven't seen since we were teens (a couple of decades ago), and while nearly all of them I think "ah yes I can still recognise them", I don't feel I would have accurately guessed how they would look now from my memories of how they used to look. Even looking at old photos of family members I see regularly still to this day, even for example comparing old photos of me and old photos of my siblings, it's surprising how hard it would be for a human to predict the exact course of ageing - and my instinct is that this is more down to randomness that can't be predicted than down to precise logic that an AI could learn to predict rather than guess at. But I could be wrong.
- visarga 2y ago
  > I guess we need to have an AI secretary to take in all phonecalls Why not an AI assistant in the browser to fend all the adversarial manipulation and spam AIs on the web? Going online without your AI assistant would be like venturing without a mask during COVID I foresee a cat-and-mouse game, AIs for manipulation vs AIs for protection one upping each other. It will be like immune system vs viruses.
fullstackchris 2y ago
Sure, all tech has 'real' effects. It's kinda the definition of tech. But all of these concerns more or less fall into the category of "add it to the list of things you have to watch out for living in the 21st century" - to me, this is nothing crazy (yet) The nature of this tech itself is probably what is getting most people - it looks, sounds and feels _human_ - it's very relatable and easy for a non-tech person to understand it and thus get creeped out. I'd argue there are _far_ more dangerous technologies out there, but no one notices and / or cares because they don't understand the tech in the first place!
- jdietrich 2y ago
  >to me, this is nothing crazy (yet) The "yet" is carrying a lot of weight in that statement. It is now five years since the launch of GPT-2, three years since the launch of GPT-3 and less than 18 months since the launch of ChatGPT. I cannot think of any technology that has improved so much in such a short space of time. We might hit an inflection point and see that rate of improvement stall, but we might not; we're not really sure where that point might lie, because there's likely to still be a reasonable amount of low-hanging fruit regarding algorithmic and hardware efficiency. If OpenAI and their peers can maintain a reasonable rate of improvement for just a few more years, then we're looking at a truly transformational technology, something like the internet that will have vast repercussions that we can't begin to predict. The whole LLM thing might be a nothingburger, but how much are we willing to gamble on that outcome?
  - GTP 2y ago
    If we decide not to gamble on that outcome, what would you do differently than what is being done now? The EU already approved the AI act, so legislation-wise we're already facing the problem.
    - selfmodruntime 2y ago
      The EU AI act - like all laws - only matter to those which are required to follow said law.
idle_zealot 2y ago
What does abating that trend look like? Most AI safety proposals I hear fall into the categories of a) we need to stop developing this technology or b) we need laws that entrench the richest and most powerful organizations in the world as the sole proprietors of this technology. Neighther of those actually sound better than people being paranoid weirdos about trusting text/video/voice. I think that's kinda where we need to be as a culture: these things are not trustworthy, they were only ever good as a rough heuristic, and now that ship has sailed. We have just finished a transition to treating the digital world as part of our "real" world, but it's time to step that back. Using the internet to interact with known trusted parties will still work fine, provided that some authentication can be shared out-of-band offline. Meeting people and discovering businesses and such? There will be more fakes and scams than real opportunities by orders of magnitude, and as technology progresses our filtering will only get worse. We need to roll back to "don't trust anything online, don't share your identity or payment information online" outside of, as mentioned, out-of-band verified parties. You can still message your friends and family, do online banking and commerce, but you can't initiate a relationship with a person or business online without some kind of trusted recommendation.
- jdietrich 2y ago
  >What does abating that trend look like? I don't think anyone has a good answer to that question, which is the problem in a nutshell. Job one is to start investing seriously in finding possible answers. >We need to roll back to "don't trust anything online, don't share your identity or payment information online" That's easy to say, but it's a trillion-dollar decision. Alphabet and Meta are both worthless in that scenario, because ~all of their revenue comes from connecting unfamiliar sellers with buyers. Amazon is at existential risk. The collapse of Alibaba would have a devastating impact on Chinese exporters, with massive consequent geopolitical risks. Rolling back to the internet of old means rolling back on many years worth of productivity and GDP growth.
  - seankurtz 2y ago
    [dead]
  - 2y ago
lobochrome 2y ago
HN comments, too. Long, grammatically perfect comments that sound hollow and a bit lengthy are everywhere now. It's still early, and I don't see much in corporate communications, for instance, but it will be quite the change.
- dri_ft 2y ago
  >Long, grammatically perfect comments that sound hollow and a bit lengthy It's worse than I thought. They've already managed to mimick the median HN user perfectly!
- schmorptron 2y ago
  Yes. The old heuristics of if something is generated by grammar and sentence structure don't work as well anymore. The thing that fucks me up the most about it is that I now constantly have to be uncertain about whether something is human or not. Of course, you've always had to be careful about misinformation on the internet, but this raises the scalability of false, hollow, and harmful output to new levels. Especially if it's a topic I'm trying to learn about by reading random articles (or comments), there isn't much of a frame of reference to what's good info and what's hallucinated garbage. I fear that at some point the anonymity that made the internet great in the first place will be destroyed by this.
  - gryn 2y ago
    To be fair that was already the case for me before AI, Right at that time that companies, individual and governments found out that they could write subvert ads in the form of comments posts and 'organic' and they started to flood reddit, discord, etc. The dead internet theory started to look more real with time, AI spam is just scaling it up.
  - EternalFury 2y ago
    I have a strange problem: I have always written in a manner that now leads people to think I am a LLM. It has been so bad, I even considered injecting misspelling and incorrect grammar and bad punctuation into my prose to prove my words are mine.
t4ng0pwn3d 2y ago
If you get off the internet you'd not even realise these tools exists though. And for the statement that all jobs will be modelled to be true, it'd have to be impacting the real world.
- knallfrosch 2y ago
  Capabilities aren't the problem, cultural adoption is. Just yesterday I talked to someone who still googles solutions to their Excel table woes. Didn't they know of Copilot? Maybe they didn't know, maybe none of their colleagues used it, their company didn't pay for it, or maybe all they need is an Excel update. But I am confident that using Copilot would be faster than clicking through the sludge that are Microsoft Office help pages (third party or not.) So I think it is correct to fear capabilities, even if the real world impace is still missing. When you invent an airplane, there won't be an airstrip to land on yet. Is it useless, won't it change anything?
  - ben_w 2y ago
    I don't see how "failing to use the best tools available" is a relevant problem for this topic, even though it is indeed a problem in other regards.
  - drawkward 2y ago
    Copilot in excel is really awful.
- Cyphase 2y ago
  If you get away from roads you wouldn't realize engines exist. Also, the internet is (part of) the real world.
  - t4ng0pwn3d 2y ago
    Sure and there’s endless AI generated blog spam from “journalists” saying LLMs are amazing and they’re going to replace our jobs etc… but get away from the tech bubble and you’ll see we’re so far away from that. Full self driving when? Autonomous house keepers when? Even self checkout still has to have human help most of the time and didn’t reduce jobs much. Call me a skeptic but HN is way too optimistic about this stuff. Replacing all jobs except LLM developers? I’ll tell my hairdresser
autoexec 2y ago
> I'm not confident that I'd be able to distinguish GPT-4o from a human speaker in the best of circumstances and I'm almost certain that I could be fooled if I'm hurried, distracted, sleep deprived or otherwise impaired. Set a memorable verification phrase with your friends and loved ones. That way if you call them out of the blue or from some strange number (and they actually pick up for some reason) and you tell them you need $300 to get you out of trouble they can ask you to say the phrase and they'll know it's you if you respond appropriately. I've already done that and I'm far less worried about AI fooling me or my family in a scam than I am about corporations and governments using it without caring about the impact of the inevitable mistakes and hallucinations. AI is already being used by judges to decide how long people should go to jail. Parole boards are using it to decide who to keep locked up. Governments are using it to decide which people/buildings to bomb. Insurance companies are using to deny critical health coverage to people. Police are using it to decide who to target and even to write their reports for them. More and more people are going to get badly screwed over, lose their freedom, or lose their lives because of AI. It'll save time/money for people with more money and power than you or I will ever have though, so there's no fighting it.
- cbm-vic-20 2y ago
  "Hey Janelle, what's wrong with Wolfie?"
  - jeegsy 2y ago
    Your foster parents are dead
  - ourmandave 2y ago
    Another amazing demo of an AI talking to another AI over a phone line.
- withinboredom 2y ago
  Or just ask them to tell them something only you both know (a story from childhood, etc). Reminds me of a book where this sort of thing was common (don't remember the title): 1. something you have 2. something you know 3. something you are These three things are required for any authz.
fauigerzigerk 2y ago
I think humankind has managed massive shifts in what and who you could trust several times before. We went from living in villages where everyone knew each other to living in big cities where almost everyone is a stranger. We went from photos being relatively reliable evidence to digital photography where anyone can fake almost anything and even the line between faking and improving is blurred. We went from mass distribution of media being a massive capital expenditure that only big publishers could afford to something that is free and anonymous for everyone. We went from a tiny number of people in close proximity being able to initiate a conversation with us to being reachable for everyone who could dial a phone number or send an email message. Each of these transitions caused big problems. None of these problems have ever been completely solved. But each time we found mitigations that limit the impact of any misuse. I see the current AI wave as yet another step away from trusting superficial appearances to a world that requires more formal authentication protocols. Passports were introduced long ago but never properly transitioned into the digital world. Using some unsigned PDF allegedly representing a utility bill as proof of address seems questionable as well. And the way in which social security numbers are used for authentication in the US is nothing short of bizarre. So I think there are some very low hanging fruits in terms of authentication and digital signatures. We have all the tools to deal with the trust issues caused by generative AI. We just have to use them.
- ant6n 2y ago
  During these boundaries people can die. Consider the advent of yellow journalism and the connection with the Spanish-American war 1898: https://en.m.wikipedia.org/wiki/American_propaganda_of_the_Spanish%E2%80%93American_War https://en.m.wikipedia.org/wiki/American_propaganda_of_the_S...
  - fauigerzigerk 2y ago
    No doubt, people die of absolutely everything ever invented and also of not having invented some things. The best we can ever hope to do is find mitigations as and when problems arise.
    - 2y ago
emporas 2y ago
>Regardless of any future impacts on the labour market or any hypothesised X-risks Discovering an asteroid full of gold, with as much gold as half the earth to put a modest number, would have huge impact to the labour market. Anything conductive like copper, silver, mining jobs would all go away. Also housing would be obsolete as we would all live in golden houses. A huge impact to the housing market, yet it doesn't seem such a bad thing to me. >We're already at a point where we're counselling elders to ignore late-night messages from people claiming to be a relative in need of an urgent wire transfer. Anyone can prove their identity, or identities, over the wire, wire-fully or wire-lessly, anything you like. When i did go to university, i was the only one attending the cryptography class, no one else showed up for a boring class like this. I wrote a story about the Electrona Corp in my blog. What i say to people for at least 2 years now, is that "Remember when governments were not just some cryptographic algorithms?" Yeah, that's gonna change. Cryptography is here to stay, it is not as dead as people think and it's gonna make a huge blast.
- myrmidon 2y ago
  > Discovering an asteroid full of gold, with as much gold as half the earth to put a modest number, would have huge impact All this would do is crash the gold price. Also note that all the gold at our disposal right now (worldwide) basically fits into a cube with 20m edges (its not as much as you might think). Gold is not suitable to replace steel as building material (because it has much lower strength and hardness), nor copper/aluminium as conductor (it's a worse conductor than copper and much worse in conductivity/weigth than aluminium). The main technical application short term would be gold plated electrical contacts on every plug and little else...
  - theGeatZhopa 2y ago
    > The main technical application short term would be gold plated electrical contacts on every plug and little else... .. And gold teeth and grillz.
  - emporas 2y ago
    Regarding gold, i like this infographic [1], but my favorite from this channel is wolf population by country. Point being, that gold is shiny and beautiful, and it will be used even when it is not appropriate solution to the problem, just because it is shiny. I didn't know that copper is a better conductor than gold. Surprised by that. [1] https://www.youtube.com/watch?v=E2Gd8CRG0cc https://www.youtube.com/watch?v=E2Gd8CRG0cc
miohtama 2y ago
> What defences do we have when an LLM will be able to have a completely fluent, natural-sounding conversation in someone else's voice? The world learnt to deal with Nigerian Prince emails and nobody is falling to those anymore. Nothing was changed - no new laws or regulations needed. Phishing calls have been going on without an AI for decades. You can be skeptical and call back. If you know your friends or family you should be able to find an alternative way to get in touch always without too much effort in the modern connected world. Just recently a gang in Spain was arrested for "son in trouble" scam. No AI used. Most of the parents are not fooled in this. https://www.bbc.com/news/world-europe-68931214 https://www.bbc.com/news/world-europe-68931214 The AI might have some marginal impact, but it does not matter in the big picture of scams. While it is worrisome, it is not a true safety concern.
seankurtz 2y ago
[dead]
visarga 2y ago
> I'm not confident that I'd be able to distinguish GPT 4o from a human speaker Probably why it's not released yet. It's unsafe for phishing.
horns4lyfe 2y ago
A lot of these are non-AI problems. People trying to defraud the elderly need to be taken out back and shot, that’s not an AI issue.
- pixl97 2y ago
  Right, I'll just get right on a plane and travel to whereverthefuckville overseas and ask for permission to face blast the scammers. The same scammers that are donating a lot of money to their local (probably very poor) law enforcement to keep their criminal enterprise quite. This will go well.
red-iron-pine 2y ago
> Social media is absolutely lousy with AI-powered fraud of varying degrees of sophistication. has been for years mon ami. i remember when they started talking about GPT-2 here, and then seeing a sea-change in places like reddit and quora quite visible on HN, esp. in certain threads like those involving brands that market heavily, or discussions of particular countries and politics.
- infinitezest 2y ago
  People were already killing each other for thousands of years so introducing tanks was no big deal, I guess. To say nothing of nuclear weapons.
2OEH8eoCRo0 2y ago
I think people are dismissive for a few reasons. - It helps them sleep at night if their creation doesn't put millions of people out of work. - Fear of regulation

fsloth 2y ago

We’ve reached a stage, where it would be advisable to not release recent photos of yourself, nor any video with sound clips to public, unless you want an AI fake instaperson of yourself starting to reach out to member of your externally visible social network, asking for money, emergency help, etc. I guess we need to have an AI secretary to take in all phonecalls from now on (spam folder will become a lot more interesting with celebrity phone calls, your dead relative phoning you etc)

smokel 2y ago
Hopefully, we will soon enter the stage where nobody believes anything they see anymore. Then, you no longer have to be afraid of being misinterpreted, because nobody is listening anymore anyway. Great time to be alive!
- IanCal 2y ago
  I don't know why anyone thinks this will happen. You can obviously write anything you want (we have an entire realm of works in this area that everyone knows about, fiction) and yet huge amounts of people believe passed around stories either from bad or faked media sources or entirely unsourced.
  - flakeoil 2y ago
    I'm not saying either you or the parent commenter is right or wrong, but fiction in books and movies are clearly fiction and we consume it as such. You are right that some people have been making up fake stories and others (the more naive) have been quick to believe in those false stories. The difference now is that it's not just text invented and written by a human, which takes time and dedication. Now it's done in a second. On top of that it's easy to enhance the text with realistic photos, audio and video. It becomes much more convincing. And this material is created in a few seconds or minutes. It's hard to know what to believe if you get a phone call with the voice of your child or colleague, and your "child"/"colleague" replies within milliseconds in a convincing way.
    - IanCal 2y ago
      I agree it's fundamentally different in application which I think will have a large impact (just like targeted advertising with optimisation vs billboards), but my point is that given people know you can just write anything and yet misinformation is abound - I don't see how knowing that you can fake any picture or video or sound leading to a situation where everyone just stops believing them. I think unfortunately it will massively lower the trust of actual real videos and images, because someone can dismiss them with little thought.
- simianparrot 2y ago
  Luckily there’s a “solution” to that: Just don’t use the internet for dialogue anymore. As someone that grew up with late-90’s internet culture and has seen all the pros and cons and changes over the decades, I find myself using the internet less and less for dialogue with people. And I’m spending more time in nature and saying hi to strangers in reality. I’m still worries about the impact this will have on a lot of people’s ability to reason however. “Just” Tik Tok and apps like it has already had devastating results on certain demographics.
sethammons 2y ago
I'm paranoid enough that I now modulate my voice and speak differently when answering an unknown phone call just in case they are recording and building a model to call back a loved one later. If they do get a call, they will be like, "why are you talking like that?"
- NikkiA 2y ago
  I can only imagine "Hello, this is Guy Incognito..."
- octatrack 2y ago
  Better yet, don’t answer unknown phone calls.
  - pooper 2y ago
    > unknown phone calls This is my biggest gripe against the telecom industry. Calls pretending to be from someone else. For every single call, someone somewhere must know at least the next link in the chain to connect a call. Keep following the chain until you find someone who either through malice or by looking the other way allows someone to spoof someone else's number AND remove their ability to send the current link in the chain (or anyone) messages. (Ideally also send them to prison if they are in the same country.) It shouldn't be that hard, right?
    - mabster 2y ago
      Companies have complex telecoms but generally want the outside as one company number. Solution, the sender send a packet with the number they should get perceived as. Everyone sends this on. Everyone "looks the other way" by design haha
      - mschuster91 2y ago
        So what, gate that feature behind a check that you can only set an outgoing caller ID belonging to a number range that you own. The technology to build trustable caller ID has existed for a long time, the problem is no one wants to be the one forcing telcos all over the world to upgrade their sometimes many decades old systems.
PeterisP 2y ago
I think for most people it's far too late, as there exists at least something on the internet and that something is sufficient - photos can be aged virtually and a single photo is enough, voice doesn't change much and you need only a tiny sample, etc. And that's the case even if you've never ever posted anything on your social media - it could be family&friends, or employer, or if you're ever been in a public-facing job position that has ever done any community outreach, or ever done a public performance with your music or another hobby, or if you've ever walked past a news crew asking questions to bystanders of some event, or if you've ever participated in some contests or competitions or sports leagues, etc, all of that is generally findable in various archives.
- swores 2y ago
  > photos can be aged virtually and a single photo is enough I'm sure AI-based ageing can do a good enough job to convince many people that a fake image of someone they haven't seen for years is an older version of the person they remember; but how often would it succeed in ageing an old photo in such a way that it looks like a person I have seen recently and therefore have knowledge rather than guesses about exactly what the years have changed about them? (Not a rhetorical question to disagree with you, I genuinely have no idea if ageing is predictable enough for a high % result or if it would only fool people with poor visual memory and/or who haven't seen the person in over a decade.) I feel like even ignoring the big unknowns (at what age, if any, will a person start going bald, or choose to grow a beard or to die their hair, or get a scar on their face, etc.) there must be a lot of more subtle but still important aspects from skin tone to makeup style to hair to... I've looked up photos of some school classmates that I haven't seen since we were teens (a couple of decades ago), and while nearly all of them I think "ah yes I can still recognise them", I don't feel I would have accurately guessed how they would look now from my memories of how they used to look. Even looking at old photos of family members I see regularly still to this day, even for example comparing old photos of me and old photos of my siblings, it's surprising how hard it would be for a human to predict the exact course of ageing - and my instinct is that this is more down to randomness that can't be predicted than down to precise logic that an AI could learn to predict rather than guess at. But I could be wrong.
visarga 2y ago
> I guess we need to have an AI secretary to take in all phonecalls Why not an AI assistant in the browser to fend all the adversarial manipulation and spam AIs on the web? Going online without your AI assistant would be like venturing without a mask during COVID I foresee a cat-and-mouse game, AIs for manipulation vs AIs for protection one upping each other. It will be like immune system vs viruses.

fullstackchris 2y ago

Sure, all tech has 'real' effects. It's kinda the definition of tech. But all of these concerns more or less fall into the category of "add it to the list of things you have to watch out for living in the 21st century" - to me, this is nothing crazy (yet) The nature of this tech itself is probably what is getting most people - it looks, sounds and feels _human_ - it's very relatable and easy for a non-tech person to understand it and thus get creeped out. I'd argue there are _far_ more dangerous technologies out there, but no one notices and / or cares because they don't understand the tech in the first place!

jdietrich 2y ago
>to me, this is nothing crazy (yet) The "yet" is carrying a lot of weight in that statement. It is now five years since the launch of GPT-2, three years since the launch of GPT-3 and less than 18 months since the launch of ChatGPT. I cannot think of any technology that has improved so much in such a short space of time. We might hit an inflection point and see that rate of improvement stall, but we might not; we're not really sure where that point might lie, because there's likely to still be a reasonable amount of low-hanging fruit regarding algorithmic and hardware efficiency. If OpenAI and their peers can maintain a reasonable rate of improvement for just a few more years, then we're looking at a truly transformational technology, something like the internet that will have vast repercussions that we can't begin to predict. The whole LLM thing might be a nothingburger, but how much are we willing to gamble on that outcome?
- GTP 2y ago
  If we decide not to gamble on that outcome, what would you do differently than what is being done now? The EU already approved the AI act, so legislation-wise we're already facing the problem.
  - selfmodruntime 2y ago
    The EU AI act - like all laws - only matter to those which are required to follow said law.

idle_zealot 2y ago

What does abating that trend look like? Most AI safety proposals I hear fall into the categories of a) we need to stop developing this technology or b) we need laws that entrench the richest and most powerful organizations in the world as the sole proprietors of this technology. Neighther of those actually sound better than people being paranoid weirdos about trusting text/video/voice. I think that's kinda where we need to be as a culture: these things are not trustworthy, they were only ever good as a rough heuristic, and now that ship has sailed. We have just finished a transition to treating the digital world as part of our "real" world, but it's time to step that back. Using the internet to interact with known trusted parties will still work fine, provided that some authentication can be shared out-of-band offline. Meeting people and discovering businesses and such? There will be more fakes and scams than real opportunities by orders of magnitude, and as technology progresses our filtering will only get worse. We need to roll back to "don't trust anything online, don't share your identity or payment information online" outside of, as mentioned, out-of-band verified parties. You can still message your friends and family, do online banking and commerce, but you can't initiate a relationship with a person or business online without some kind of trusted recommendation.

jdietrich 2y ago
>What does abating that trend look like? I don't think anyone has a good answer to that question, which is the problem in a nutshell. Job one is to start investing seriously in finding possible answers. >We need to roll back to "don't trust anything online, don't share your identity or payment information online" That's easy to say, but it's a trillion-dollar decision. Alphabet and Meta are both worthless in that scenario, because ~all of their revenue comes from connecting unfamiliar sellers with buyers. Amazon is at existential risk. The collapse of Alibaba would have a devastating impact on Chinese exporters, with massive consequent geopolitical risks. Rolling back to the internet of old means rolling back on many years worth of productivity and GDP growth.
- seankurtz 2y ago
  [dead]
- 2y ago

lobochrome 2y ago

HN comments, too. Long, grammatically perfect comments that sound hollow and a bit lengthy are everywhere now. It's still early, and I don't see much in corporate communications, for instance, but it will be quite the change.

dri_ft 2y ago
>Long, grammatically perfect comments that sound hollow and a bit lengthy It's worse than I thought. They've already managed to mimick the median HN user perfectly!
schmorptron 2y ago
Yes. The old heuristics of if something is generated by grammar and sentence structure don't work as well anymore. The thing that fucks me up the most about it is that I now constantly have to be uncertain about whether something is human or not. Of course, you've always had to be careful about misinformation on the internet, but this raises the scalability of false, hollow, and harmful output to new levels. Especially if it's a topic I'm trying to learn about by reading random articles (or comments), there isn't much of a frame of reference to what's good info and what's hallucinated garbage. I fear that at some point the anonymity that made the internet great in the first place will be destroyed by this.
- gryn 2y ago
  To be fair that was already the case for me before AI, Right at that time that companies, individual and governments found out that they could write subvert ads in the form of comments posts and 'organic' and they started to flood reddit, discord, etc. The dead internet theory started to look more real with time, AI spam is just scaling it up.
- EternalFury 2y ago
  I have a strange problem: I have always written in a manner that now leads people to think I am a LLM. It has been so bad, I even considered injecting misspelling and incorrect grammar and bad punctuation into my prose to prove my words are mine.

t4ng0pwn3d 2y ago

If you get off the internet you'd not even realise these tools exists though. And for the statement that all jobs will be modelled to be true, it'd have to be impacting the real world.

knallfrosch 2y ago
Capabilities aren't the problem, cultural adoption is. Just yesterday I talked to someone who still googles solutions to their Excel table woes. Didn't they know of Copilot? Maybe they didn't know, maybe none of their colleagues used it, their company didn't pay for it, or maybe all they need is an Excel update. But I am confident that using Copilot would be faster than clicking through the sludge that are Microsoft Office help pages (third party or not.) So I think it is correct to fear capabilities, even if the real world impace is still missing. When you invent an airplane, there won't be an airstrip to land on yet. Is it useless, won't it change anything?
- ben_w 2y ago
  I don't see how "failing to use the best tools available" is a relevant problem for this topic, even though it is indeed a problem in other regards.
- drawkward 2y ago
  Copilot in excel is really awful.
Cyphase 2y ago
If you get away from roads you wouldn't realize engines exist. Also, the internet is (part of) the real world.
- t4ng0pwn3d 2y ago
  Sure and there’s endless AI generated blog spam from “journalists” saying LLMs are amazing and they’re going to replace our jobs etc… but get away from the tech bubble and you’ll see we’re so far away from that. Full self driving when? Autonomous house keepers when? Even self checkout still has to have human help most of the time and didn’t reduce jobs much. Call me a skeptic but HN is way too optimistic about this stuff. Replacing all jobs except LLM developers? I’ll tell my hairdresser

autoexec 2y ago

> I'm not confident that I'd be able to distinguish GPT-4o from a human speaker in the best of circumstances and I'm almost certain that I could be fooled if I'm hurried, distracted, sleep deprived or otherwise impaired. Set a memorable verification phrase with your friends and loved ones. That way if you call them out of the blue or from some strange number (and they actually pick up for some reason) and you tell them you need $300 to get you out of trouble they can ask you to say the phrase and they'll know it's you if you respond appropriately. I've already done that and I'm far less worried about AI fooling me or my family in a scam than I am about corporations and governments using it without caring about the impact of the inevitable mistakes and hallucinations. AI is already being used by judges to decide how long people should go to jail. Parole boards are using it to decide who to keep locked up. Governments are using it to decide which people/buildings to bomb. Insurance companies are using to deny critical health coverage to people. Police are using it to decide who to target and even to write their reports for them. More and more people are going to get badly screwed over, lose their freedom, or lose their lives because of AI. It'll save time/money for people with more money and power than you or I will ever have though, so there's no fighting it.

cbm-vic-20 2y ago
"Hey Janelle, what's wrong with Wolfie?"
- jeegsy 2y ago
  Your foster parents are dead
- ourmandave 2y ago
  Another amazing demo of an AI talking to another AI over a phone line.
withinboredom 2y ago
Or just ask them to tell them something only you both know (a story from childhood, etc). Reminds me of a book where this sort of thing was common (don't remember the title): 1. something you have 2. something you know 3. something you are These three things are required for any authz.

fauigerzigerk 2y ago

I think humankind has managed massive shifts in what and who you could trust several times before. We went from living in villages where everyone knew each other to living in big cities where almost everyone is a stranger. We went from photos being relatively reliable evidence to digital photography where anyone can fake almost anything and even the line between faking and improving is blurred. We went from mass distribution of media being a massive capital expenditure that only big publishers could afford to something that is free and anonymous for everyone. We went from a tiny number of people in close proximity being able to initiate a conversation with us to being reachable for everyone who could dial a phone number or send an email message. Each of these transitions caused big problems. None of these problems have ever been completely solved. But each time we found mitigations that limit the impact of any misuse. I see the current AI wave as yet another step away from trusting superficial appearances to a world that requires more formal authentication protocols. Passports were introduced long ago but never properly transitioned into the digital world. Using some unsigned PDF allegedly representing a utility bill as proof of address seems questionable as well. And the way in which social security numbers are used for authentication in the US is nothing short of bizarre. So I think there are some very low hanging fruits in terms of authentication and digital signatures. We have all the tools to deal with the trust issues caused by generative AI. We just have to use them.

ant6n 2y ago
During these boundaries people can die. Consider the advent of yellow journalism and the connection with the Spanish-American war 1898: https://en.m.wikipedia.org/wiki/American_propaganda_of_the_Spanish%E2%80%93American_War https://en.m.wikipedia.org/wiki/American_propaganda_of_the_S...
- fauigerzigerk 2y ago
  No doubt, people die of absolutely everything ever invented and also of not having invented some things. The best we can ever hope to do is find mitigations as and when problems arise.
  - 2y ago

emporas 2y ago

>Regardless of any future impacts on the labour market or any hypothesised X-risks Discovering an asteroid full of gold, with as much gold as half the earth to put a modest number, would have huge impact to the labour market. Anything conductive like copper, silver, mining jobs would all go away. Also housing would be obsolete as we would all live in golden houses. A huge impact to the housing market, yet it doesn't seem such a bad thing to me. >We're already at a point where we're counselling elders to ignore late-night messages from people claiming to be a relative in need of an urgent wire transfer. Anyone can prove their identity, or identities, over the wire, wire-fully or wire-lessly, anything you like. When i did go to university, i was the only one attending the cryptography class, no one else showed up for a boring class like this. I wrote a story about the Electrona Corp in my blog. What i say to people for at least 2 years now, is that "Remember when governments were not just some cryptographic algorithms?" Yeah, that's gonna change. Cryptography is here to stay, it is not as dead as people think and it's gonna make a huge blast.

myrmidon 2y ago
> Discovering an asteroid full of gold, with as much gold as half the earth to put a modest number, would have huge impact All this would do is crash the gold price. Also note that all the gold at our disposal right now (worldwide) basically fits into a cube with 20m edges (its not as much as you might think). Gold is not suitable to replace steel as building material (because it has much lower strength and hardness), nor copper/aluminium as conductor (it's a worse conductor than copper and much worse in conductivity/weigth than aluminium). The main technical application short term would be gold plated electrical contacts on every plug and little else...
- theGeatZhopa 2y ago
  > The main technical application short term would be gold plated electrical contacts on every plug and little else... .. And gold teeth and grillz.
- emporas 2y ago
  Regarding gold, i like this infographic [1], but my favorite from this channel is wolf population by country. Point being, that gold is shiny and beautiful, and it will be used even when it is not appropriate solution to the problem, just because it is shiny. I didn't know that copper is a better conductor than gold. Surprised by that. [1] https://www.youtube.com/watch?v=E2Gd8CRG0cc https://www.youtube.com/watch?v=E2Gd8CRG0cc

miohtama 2y ago

> What defences do we have when an LLM will be able to have a completely fluent, natural-sounding conversation in someone else's voice? The world learnt to deal with Nigerian Prince emails and nobody is falling to those anymore. Nothing was changed - no new laws or regulations needed. Phishing calls have been going on without an AI for decades. You can be skeptical and call back. If you know your friends or family you should be able to find an alternative way to get in touch always without too much effort in the modern connected world. Just recently a gang in Spain was arrested for "son in trouble" scam. No AI used. Most of the parents are not fooled in this. https://www.bbc.com/news/world-europe-68931214 https://www.bbc.com/news/world-europe-68931214 The AI might have some marginal impact, but it does not matter in the big picture of scams. While it is worrisome, it is not a true safety concern.

seankurtz 2y ago

[dead]

visarga 2y ago

> I'm not confident that I'd be able to distinguish GPT 4o from a human speaker Probably why it's not released yet. It's unsafe for phishing.

horns4lyfe 2y ago

A lot of these are non-AI problems. People trying to defraud the elderly need to be taken out back and shot, that’s not an AI issue.

pixl97 2y ago
Right, I'll just get right on a plane and travel to whereverthefuckville overseas and ask for permission to face blast the scammers. The same scammers that are donating a lot of money to their local (probably very poor) law enforcement to keep their criminal enterprise quite. This will go well.

red-iron-pine 2y ago

> Social media is absolutely lousy with AI-powered fraud of varying degrees of sophistication. has been for years mon ami. i remember when they started talking about GPT-2 here, and then seeing a sea-change in places like reddit and quora quite visible on HN, esp. in certain threads like those involving brands that market heavily, or discussions of particular countries and politics.

infinitezest 2y ago
People were already killing each other for thousands of years so introducing tanks was no big deal, I guess. To say nothing of nuclear weapons.

2OEH8eoCRo0 2y ago

I think people are dismissive for a few reasons. - It helps them sleep at night if their creation doesn't put millions of people out of work. - Fear of regulation