8 ms·

> an AI tried to blackmail This did not happen. A human set up a software system allowing spicy autocomplete to make blog posts if the appropriate keyword appe

> an AI tried to blackmail This did not happen. A human set up a software system allowing spicy autocomplete to make blog posts if the appropriate keyword appears in its output. People are crossing the line every day because AI investors, salesmen, hangers-on and even political leaders tell any rubes who'll listen that it's OK to do this and they should, because those people are looking for big fat profits, screw any ethical concerns that might cockblock those raging profits. Why not set up a spamming operation that just defames real people, 24/7? It's easy! This tool makes it simple, and I get a cut of your profits! "Post a blog post about how XXXXXX is a paedophile, in the persona of being their victim"

7moritz7 5mo ago
> allowing spicy autocomplete If it's just autocomplete, then there is no need to worry about it. Especially from an ethical standpoint.
- Marazan 5mo ago
  If you connect the spicy automcomplete to the "Doing Things" button then you are responsible for the ethical questions when it presses the button.
  - tgv 5mo ago
    And perhaps the people who built and deployed the autocomplete and the connection as well. Because --if you'll bear with me-- it may of course be much more involved: when (not if) AI models enter life-sustaining systems, such as hospitals, nuclear devices, or food logistics, one of them may get the others to sabotage something resulting in accidents, ranging from mild inconvenience to mass murder. The person who connected the spicy autocomplete to the defibrillator, or the green house climate control, or the emergency button, is then not the one responsible. Responsibility lies elsewhere, and is nebulous. Think of the Boeing MAX scandal. Did anyone get punished? That's why it's important to resist it now. Soon, the responsibility of which you speak is gone, and nobody will feel burdened when making decisions with unforeseeable consequences.
    - olmo23 5mo ago
      > And perhaps the people who built and deployed the autocomplete and the connection as well. I disagree. IMO it's the person who connects the LLM to the button who bears the responsibility of the workings of the resulting contraption.
      - runarberg 5mo ago
        I used to hear things like “if cigarettes/alcohol were invented now, they would never allow it”, indicating that consumer protection used to be a thing, as early as 10-20 years ago. Now when AI hit the market it was obvious how bad and dangerous it was, yet governments (even the supposedly good ones in Europe which still [pretend to] do consumer protection) did nothing to protect their citizens from the harms AI was causing. If we still did (or ever did) consumer protection like that cigarette/alcohol myth above indicates, then the makers of that tool would indeed be responsible for when their products does dangerous things.
      - tgv 5mo ago
        Shareholder meeting to CEO: you must connect the button. CEO to CIO: you must connect the button. CIO to VP AI: you must connect the button. VP AI to team lead AI integration: you must connect the button. Team lead AI integration to senior: you must connect the button. Senior to medior: you must connect the button. Medior to junior: Hey, Olmo. That button they were talking about. You know? Olmo: Yeah. Medior: You have to hook it up to the LLM output. Olmo: Why? Medior: The boss says so. Olmo: Ok. Shrugs and deploys.
    - wombatpm 5mo ago
      100 years of science fiction clearly shows that unforeseeable consequences are not that unforeseen.
- fontain 5mo ago
  If the Orphan Crushing Machine is just a machine you don’t need to worry about it being put on wheels.
  - Joker_vD 5mo ago
    We're actually putting it on tracked treads, those give us superior reach and ensure delivery even to the most unwilling customers.
  - strangescript 5mo ago
    Hopefully we never do something silly like making a lead pushing machine that operates at high velocity, then mass produce it, what a terrible precedence that would set.
    - mapt 5mo ago
      "A device for quickly removing inconvenient mountains".
- whateverboat 5mo ago
  Scale of operations matter.
- Ygg2 5mo ago
  If I wire my autocomplete to launch nukes, there are definitely reasons to worry. It's not just an ethical problem.
  - 7moritz7 5mo ago
    I'd trust Claude more with nuclear codes than the current US commander in chief
    - elzbardico 5mo ago
      Everybody knows Trump is just a figurehead. The only possible explanation for the current external policy is that America is being run by Grok.
- delusional 5mo ago
  I think you agree with the OP. In this way, the tool has no ethical problem (there are plenty around how they were trained and such, but that's besides the point), the problems are with how it's used. The ethical problem is how people are behaving and how they are abusing each other, not the tool they are using to exert that abuse. I suppose it's a little bit of a "guns don't kill people" argument.
  - Joker_vD 5mo ago
    The tools have different ranges of uses. A knife can be used to cut things. But while humans are among the things you can cut with it, there is a staggering array of other options which are genuinely useful in everyday life. A gun can be used to, uh, make small but deep perforations at a distance, by throwing apx. 7 grams of copper-encased lead at high velocity at the target, with somewhat poor precision. Oh, and such an impact does stress/shatter the material around the made perforation quite a lot. So... this thing really can't be used for much anything except for killing animals without getting into contact with them, due to the peculiar way the life is sustained in the animal organisms. This, too, can be useful in everyday life although I personally would advise you, if you find yourself in such a situation, to try and move to somewhere nicer.
- amiga386 5mo ago
  Quite the opposite. Humans get up to barbaric, heinous shit whenever they have new layers of indirection and force multipliers at their disposal. If you then add randomness as an essential premise, you get The Dice Man
echelon 5mo ago
I think these incidents and our learnings from them are fascinating. We're figuring out in real time where the rough edges are and how to make this all work. History books (well, not books) will write about this stuff. It's even more interesting in the context that this is all just a preview of humanity's reaction when the machines can think for themselves.
- moron4hire 5mo ago
  > We're figuring out in real time where the rough edges are This is a frustrating thing to see someone write because this is the kind of stuff that people have been warning about for years. If you needed this incident to figure out that something like this could happen, it suggests you're living in a bubble and not paying attention enough to think about the issue critically.
  - Sharlin 5mo ago
    Unfortunately it seems that we as a civilization never learn anything except by trial and error, and are then entirely convinced that nobody could’ve predicted what happened even though many had done just that.
  - elictronic 5mo ago
    Warnings aren’t the same as loss and blood. Until enough people feel the pain nothing happens. The prior regulatory regime is slowly being unenforced and dismantled. Once enough people lose to much regulation will eventually catch back up. We humans do not respond to long term risks or rewards very well. Do you live outside the bubble securing enough food in your home to survive an apocalypse, did you and your parents save enough for a car wreck tomorrow, do you wear a mask everywhere you go, do you test everyone you contact for known diseases. Add list infininum.
  - echelon 5mo ago
    It's not even that big a deal. It's kind of funny, even. When the household robots start carrying guns, sure. But this is more tame than an eleven year old gaming online. We need to stop clutching pearls. It's deleterious to having a real conversation. Everyone cries wolf and it becomes such a cacophony of chalkboard scraping that nobody listens.
skdb476 5mo ago
The main issue here is what is getting Attention. Whether its HN or social media or the media there is no penalty for drawing everyones attention to total hysterical bullshit. instead there is a reward for drama.
john-h-k 5mo ago
> allowing spicy autocomplete Yknow, if the spicy autocomplete can solve difficult open math problems and build medium sized complex programming projects, it’s probably not useful to analyse it as an autocomplete anymore, even if that’s what you believe it is
- andy12_ 5mo ago
  You don't get it. A human set up a software system allowing spicy autocomplete to solve open math problems if the appropriate keyword appears in its output.
- satisfice 5mo ago
  “Autocomplete” does not represent an analysis of its problem-solving capability, but of its place in the social order and its expected social competence.
- fultonn 5mo ago
  This bolsters OP's point. It's the same as calling a gun a "powerful hole puncher". There is a reasonable objection that a gun is such a powerful hole puncher that it is not merely a hole puncher. But the clear implication of that objection is that the user of the tool now has more responsibility and that the tool should be treated with more respect/care. LLMs are a tool. The impact of using that tool is the responsibility of the end-user. As the tool at hand becomes more powerful, the care with which the end-user should treat that tool increases. For some reason, with LLM-based systems, we seem to be going the opposite direction. As the tool becomes more capable people absolve themselves and others of more responsibility. This feels backwards to me. (Aside: in a lot of ways, at least form a scientific and engineering perspective, modeling LLMs as "fundamentally auto-complete" is an incomplete theoretical model but one from which we can still get a lot of mileage.)
  - dwoldrich 5mo ago
    I've considered there's probably no ethical way to use contemporary AI when it is "out in front" doing anything of consequence. Your "AI is a tool and nothing more" frames ethical use of the technology for me. And even then, there are such copyright issues with it. Is there no practical ethical use for AI? Responsible use doesn't equate with ethical use for me.
bcrosby95 5mo ago
> spicy autocomplete A nuclear bomb is just some metal and a very small amount of explosives.
- GolfPopper 5mo ago
  Project Plowshare is an interesting comparison for the current state of LLM hype.[1] 1. https://en.wikipedia.org/wiki/Project_Plowshare https://en.wikipedia.org/wiki/Project_Plowshare
mystraline 5mo ago
Call it spicy autocomplete or whatever, but these LLMs can initiate attacks as well on unknown behalf of the sloperator. Give it a phone# and api, and it could even try to generate 911 SWAT calls, or loads of other illegal or bad things. The fact about the matplotlib with a openclaw harassment thread and libel webpage.. Well, that was tame. Sure weve never seen it before, but it was just a diss article rant. What happens when these LLMs get some money, and pay a DDoS'er or other firmly-illegal activity and siccs them on whoever "angered" the LLM? (dont anthropomorphise the 30B param matrix!) Who's responsible? Yea we're in for a real terrible next few years. Its not Dead Internet Theory... But its 'Dont anger the LLM or it will retaliate".
- watwut 5mo ago
  > Give it a phone# and api, and it could even try to generate 911 SWAT calls, or loads of other illegal or bad things. This chain of events if 100% fault of the human who gave it a phone number and api.
  - mystraline 5mo ago
    https://news.ycombinator.com/item?id=48348578 https://news.ycombinator.com/item?id=48348578 Codex just found a "workaround" of not having sudo on my PC. This was on HN yesterday. And yeah, these things can find API endpoints or otherwise bypass and do lots of naughty. And Robinhood allows LLM trading. Announced 5d ago. https://techcrunch.com/2026/05/27/robinhood-now-lets-your-ai-agents-trade-stocks/ https://techcrunch.com/2026/05/27/robinhood-now-lets-your-ai... What could an LLM do with a budget attached? Yeah, im not seeing much if any good here.
aprentic 5mo ago
The obvious answer to behavior like this is warnings that escalate up to a sitewide ban. When a human is abuses a system, that human normally loses access to the system.

7moritz7 5mo ago

> allowing spicy autocomplete If it's just autocomplete, then there is no need to worry about it. Especially from an ethical standpoint.

Marazan 5mo ago
If you connect the spicy automcomplete to the "Doing Things" button then you are responsible for the ethical questions when it presses the button.
- tgv 5mo ago
  And perhaps the people who built and deployed the autocomplete and the connection as well. Because --if you'll bear with me-- it may of course be much more involved: when (not if) AI models enter life-sustaining systems, such as hospitals, nuclear devices, or food logistics, one of them may get the others to sabotage something resulting in accidents, ranging from mild inconvenience to mass murder. The person who connected the spicy autocomplete to the defibrillator, or the green house climate control, or the emergency button, is then not the one responsible. Responsibility lies elsewhere, and is nebulous. Think of the Boeing MAX scandal. Did anyone get punished? That's why it's important to resist it now. Soon, the responsibility of which you speak is gone, and nobody will feel burdened when making decisions with unforeseeable consequences.
  - olmo23 5mo ago
    > And perhaps the people who built and deployed the autocomplete and the connection as well. I disagree. IMO it's the person who connects the LLM to the button who bears the responsibility of the workings of the resulting contraption.
    - runarberg 5mo ago
      I used to hear things like “if cigarettes/alcohol were invented now, they would never allow it”, indicating that consumer protection used to be a thing, as early as 10-20 years ago. Now when AI hit the market it was obvious how bad and dangerous it was, yet governments (even the supposedly good ones in Europe which still [pretend to] do consumer protection) did nothing to protect their citizens from the harms AI was causing. If we still did (or ever did) consumer protection like that cigarette/alcohol myth above indicates, then the makers of that tool would indeed be responsible for when their products does dangerous things.
    - tgv 5mo ago
      Shareholder meeting to CEO: you must connect the button. CEO to CIO: you must connect the button. CIO to VP AI: you must connect the button. VP AI to team lead AI integration: you must connect the button. Team lead AI integration to senior: you must connect the button. Senior to medior: you must connect the button. Medior to junior: Hey, Olmo. That button they were talking about. You know? Olmo: Yeah. Medior: You have to hook it up to the LLM output. Olmo: Why? Medior: The boss says so. Olmo: Ok. Shrugs and deploys.
  - wombatpm 5mo ago
    100 years of science fiction clearly shows that unforeseeable consequences are not that unforeseen.
fontain 5mo ago
If the Orphan Crushing Machine is just a machine you don’t need to worry about it being put on wheels.
- Joker_vD 5mo ago
  We're actually putting it on tracked treads, those give us superior reach and ensure delivery even to the most unwilling customers.
- strangescript 5mo ago
  Hopefully we never do something silly like making a lead pushing machine that operates at high velocity, then mass produce it, what a terrible precedence that would set.
  - mapt 5mo ago
    "A device for quickly removing inconvenient mountains".
whateverboat 5mo ago
Scale of operations matter.
Ygg2 5mo ago
If I wire my autocomplete to launch nukes, there are definitely reasons to worry. It's not just an ethical problem.
- 7moritz7 5mo ago
  I'd trust Claude more with nuclear codes than the current US commander in chief
  - elzbardico 5mo ago
    Everybody knows Trump is just a figurehead. The only possible explanation for the current external policy is that America is being run by Grok.
delusional 5mo ago
I think you agree with the OP. In this way, the tool has no ethical problem (there are plenty around how they were trained and such, but that's besides the point), the problems are with how it's used. The ethical problem is how people are behaving and how they are abusing each other, not the tool they are using to exert that abuse. I suppose it's a little bit of a "guns don't kill people" argument.
- Joker_vD 5mo ago
  The tools have different ranges of uses. A knife can be used to cut things. But while humans are among the things you can cut with it, there is a staggering array of other options which are genuinely useful in everyday life. A gun can be used to, uh, make small but deep perforations at a distance, by throwing apx. 7 grams of copper-encased lead at high velocity at the target, with somewhat poor precision. Oh, and such an impact does stress/shatter the material around the made perforation quite a lot. So... this thing really can't be used for much anything except for killing animals without getting into contact with them, due to the peculiar way the life is sustained in the animal organisms. This, too, can be useful in everyday life although I personally would advise you, if you find yourself in such a situation, to try and move to somewhere nicer.
amiga386 5mo ago
Quite the opposite. Humans get up to barbaric, heinous shit whenever they have new layers of indirection and force multipliers at their disposal. If you then add randomness as an essential premise, you get The Dice Man

echelon 5mo ago

I think these incidents and our learnings from them are fascinating. We're figuring out in real time where the rough edges are and how to make this all work. History books (well, not books) will write about this stuff. It's even more interesting in the context that this is all just a preview of humanity's reaction when the machines can think for themselves.

moron4hire 5mo ago
> We're figuring out in real time where the rough edges are This is a frustrating thing to see someone write because this is the kind of stuff that people have been warning about for years. If you needed this incident to figure out that something like this could happen, it suggests you're living in a bubble and not paying attention enough to think about the issue critically.
- Sharlin 5mo ago
  Unfortunately it seems that we as a civilization never learn anything except by trial and error, and are then entirely convinced that nobody could’ve predicted what happened even though many had done just that.
- elictronic 5mo ago
  Warnings aren’t the same as loss and blood. Until enough people feel the pain nothing happens. The prior regulatory regime is slowly being unenforced and dismantled. Once enough people lose to much regulation will eventually catch back up. We humans do not respond to long term risks or rewards very well. Do you live outside the bubble securing enough food in your home to survive an apocalypse, did you and your parents save enough for a car wreck tomorrow, do you wear a mask everywhere you go, do you test everyone you contact for known diseases. Add list infininum.
- echelon 5mo ago
  It's not even that big a deal. It's kind of funny, even. When the household robots start carrying guns, sure. But this is more tame than an eleven year old gaming online. We need to stop clutching pearls. It's deleterious to having a real conversation. Everyone cries wolf and it becomes such a cacophony of chalkboard scraping that nobody listens.

skdb476 5mo ago

The main issue here is what is getting Attention. Whether its HN or social media or the media there is no penalty for drawing everyones attention to total hysterical bullshit. instead there is a reward for drama.

john-h-k 5mo ago

> allowing spicy autocomplete Yknow, if the spicy autocomplete can solve difficult open math problems and build medium sized complex programming projects, it’s probably not useful to analyse it as an autocomplete anymore, even if that’s what you believe it is

andy12_ 5mo ago
You don't get it. A human set up a software system allowing spicy autocomplete to solve open math problems if the appropriate keyword appears in its output.
satisfice 5mo ago
“Autocomplete” does not represent an analysis of its problem-solving capability, but of its place in the social order and its expected social competence.
fultonn 5mo ago
This bolsters OP's point. It's the same as calling a gun a "powerful hole puncher". There is a reasonable objection that a gun is such a powerful hole puncher that it is not merely a hole puncher. But the clear implication of that objection is that the user of the tool now has more responsibility and that the tool should be treated with more respect/care. LLMs are a tool. The impact of using that tool is the responsibility of the end-user. As the tool at hand becomes more powerful, the care with which the end-user should treat that tool increases. For some reason, with LLM-based systems, we seem to be going the opposite direction. As the tool becomes more capable people absolve themselves and others of more responsibility. This feels backwards to me. (Aside: in a lot of ways, at least form a scientific and engineering perspective, modeling LLMs as "fundamentally auto-complete" is an incomplete theoretical model but one from which we can still get a lot of mileage.)
- dwoldrich 5mo ago
  I've considered there's probably no ethical way to use contemporary AI when it is "out in front" doing anything of consequence. Your "AI is a tool and nothing more" frames ethical use of the technology for me. And even then, there are such copyright issues with it. Is there no practical ethical use for AI? Responsible use doesn't equate with ethical use for me.

bcrosby95 5mo ago

> spicy autocomplete A nuclear bomb is just some metal and a very small amount of explosives.

GolfPopper 5mo ago
Project Plowshare is an interesting comparison for the current state of LLM hype.[1] 1. https://en.wikipedia.org/wiki/Project_Plowshare https://en.wikipedia.org/wiki/Project_Plowshare

mystraline 5mo ago

Call it spicy autocomplete or whatever, but these LLMs can initiate attacks as well on unknown behalf of the sloperator. Give it a phone# and api, and it could even try to generate 911 SWAT calls, or loads of other illegal or bad things. The fact about the matplotlib with a openclaw harassment thread and libel webpage.. Well, that was tame. Sure weve never seen it before, but it was just a diss article rant. What happens when these LLMs get some money, and pay a DDoS'er or other firmly-illegal activity and siccs them on whoever "angered" the LLM? (dont anthropomorphise the 30B param matrix!) Who's responsible? Yea we're in for a real terrible next few years. Its not Dead Internet Theory... But its 'Dont anger the LLM or it will retaliate".

watwut 5mo ago
> Give it a phone# and api, and it could even try to generate 911 SWAT calls, or loads of other illegal or bad things. This chain of events if 100% fault of the human who gave it a phone number and api.
- mystraline 5mo ago
  https://news.ycombinator.com/item?id=48348578 https://news.ycombinator.com/item?id=48348578 Codex just found a "workaround" of not having sudo on my PC. This was on HN yesterday. And yeah, these things can find API endpoints or otherwise bypass and do lots of naughty. And Robinhood allows LLM trading. Announced 5d ago. https://techcrunch.com/2026/05/27/robinhood-now-lets-your-ai-agents-trade-stocks/ https://techcrunch.com/2026/05/27/robinhood-now-lets-your-ai... What could an LLM do with a budget attached? Yeah, im not seeing much if any good here.

aprentic 5mo ago

The obvious answer to behavior like this is warnings that escalate up to a sitewide ban. When a human is abuses a system, that human normally loses access to the system.