4 ms·

Yes, and most of us won’t break into other people’s houses, yet we really need locks.

by catoc 8mo ago

YetAnotherNick 8mo ago
How is it related? I dont need lock for myself. I need it for others.
- aobdev 8mo ago
  The analogy should be obvious--a model refusing to perform an unethical action is the lock against others.
- darkwater 8mo ago
  But "you" are the "other" for someone else.
  - YetAnotherNick 8mo ago
    Can you give an example where I should care about other adults lock? Before you say image or porn, it was always possible to do it without using AI.
    - ben_w 8mo ago
      The same law prevents you and me and a hundred thousand lone wolf wannabes from building and using a kill-bot. The question is, at what point does some AI become competent enough to engineer one? And that's just one example, it's an illustration of the category and not the specific sole risk. If the model makers don't know that in advance, the argument given for delaying GPT-2 applies: you can't take back publication, better to have a standard of excess caution.
    - nearbuy 8mo ago
      Claude was used by the US military in the Venezuela raid where they captured Maduro. [1] Without safety features, an LLM could also help plan a terrorist attack. A smart, competent terrorist can plan a successful attack without help from Claude. But most would-be terrorists aren't that smart and competent. Many are caught before hurting anyone or do far less damage than they could have. An LLM can help walk you through every step, and answer all your questions along the way. It could, say, explain to you all the different bomb chemistries, recommend one for your use case, help you source materials, and walk you through how to build the bomb safely. It lowers the bar for who can do this. [1] https://www.theguardian.com/technology/2026/feb/14/us-military-anthropic-ai-model-claude-venezuela-raid https://www.theguardian.com/technology/2026/feb/14/us-milita...
      - YetAnotherNick 8mo ago
        Yeah, if US military gets any substantial help from Claude(which I highly doubt to be honest), I am all for it. At the worst case, it will reduce military budget and equalize the army more. At the best case, it will prevent war by increasing defence of all countries. For the bomb example, the barrier of entry is just sourcing of some chemicals. Wikipedia has quite detailed description of all the manufacture of all the popular bombs you can think of.
        nearbuy 8mo ago
        > Wikipedia has quite detailed description of all the manufacture of all the popular bombs you can think of. Did you bother to check? It contains very high level overviews of how various explosives are manufactured, but no proper instructions and nothing that would allow an average person to safely make a bomb. There's a big difference in how many people can actually make a bomb if you have step by step instructions the average person can follow vs soft barriers that just require someone to be a standard deviation or two above average. At two sigma, 98% will fail, despite being able to do it in theory. > Yeah, if US military gets any substantial help from Claude(which I highly doubt to be honest), I am all for it. That's not the point. I'm not saying we need to lock out the military. I'm saying if the military finds the unlocked/unsafe version of Claude useful for planning attacks, other people can also find useful for planning attacks.
        YetAnotherNick 8mo ago
        > Did you bother to check? Yeah I am not a chemist, but watch Nilered. And from [1], I know how all steps would look like. Also there are literal videos in youtube for this. And if someone can't google what nitrated or crystallization mean, maybe they just can't build a bomb with somewhat more detailed instruction. > other people can also find useful for planning attacks. I am still not able to imagine what you mean. You think attacks don't happen because people can't plan it? In fact I would say it's the opposite. Random lazy people like school shooters precisely attacks because they didn't plan for it. If ChatGPT gave detailed plan, the chances of attack would reduce. [1]: https://en.wikipedia.org/wiki/TNT#Preparation https://en.wikipedia.org/wiki/TNT#Preparation
xeromal 8mo ago
Why would we lock ourselves out of our own house though?
skissane 8mo ago
This isn't a lock It's more like a hammer which makes its own independent evaluation of the ethics of every project you seek to use it on, and refuses to work whenever it judges against that – sometimes inscrutably or for obviously poor reasons. If I use a hammer to bash in someone else's head, I'm the one going to prison, not the hammer or the hammer manufacturer or the hardware store I bought it from. And that's how it should be.
- ben_w 8mo ago
  Given the increasing use of them as agents rather than simple generators, I suggest a better analogy than "hammer" is "dog". Here's some rules about dogs: https://en.wikipedia.org/wiki/Dangerous_Dogs_Act_1991 https://en.wikipedia.org/wiki/Dangerous_Dogs_Act_1991
  - skissane 8mo ago
    How many people do dogs kill each year, in circumstances nobody would justify? How many people do frontier AI models kill each year, in circumstances nobody would justify? The Pentagon has already received Claude's help in killing people, but the ethics and legality of those acts are disputed – when a dog kills a three year old, nobody is calling that a good thing or even the lesser evil.
    - ben_w 8mo ago
      > How many people do frontier AI models kill each year, in circumstances nobody would justify? Dunno, stats aren't recorded. But I can say there's wrongful death lawsuits naming some of the labs and their models. And there was that anecdote a while back about raw garlic infused olive oil botulism, a search for which reminded me about AI-generated mushroom "guides": https://news.ycombinator.com/item?id=40724714 https://news.ycombinator.com/item?id=40724714 Do you count death by self driving car in such stats? If someone takes medical advice and dies, is that reported like people who drive off an unsafe bridge when following google maps? But this is all danger by incompetence. The opposite, danger by competence, is where they enable people to become more dangerous than they otherwise would have been. A competent planner with no moral compass, you only find out how bad it can be when it's much too late. I don't think LLMs are that danger yet, even with METR timelines that's 3 years off. But I think it's best to aim for where the ball will be, rather than where it is. Then there's LLM-psychosis, which isn't on the competent-incompetent spectrum at all, and I have no idea if that affects people who weren't already prone to psychosis, or indeed if it's really just a moral panic hallucinated by the mileau.

Yes, and most of us won’t break into other people’s houses, yet we really need locks.

by catoc 8mo ago

Yes, and most of us won’t break into other people’s houses, yet we really need locks.

YetAnotherNick 8mo ago

How is it related? I dont need lock for myself. I need it for others.

aobdev 8mo ago
The analogy should be obvious--a model refusing to perform an unethical action is the lock against others.
darkwater 8mo ago
But "you" are the "other" for someone else.
- YetAnotherNick 8mo ago
  Can you give an example where I should care about other adults lock? Before you say image or porn, it was always possible to do it without using AI.
  - ben_w 8mo ago
    The same law prevents you and me and a hundred thousand lone wolf wannabes from building and using a kill-bot. The question is, at what point does some AI become competent enough to engineer one? And that's just one example, it's an illustration of the category and not the specific sole risk. If the model makers don't know that in advance, the argument given for delaying GPT-2 applies: you can't take back publication, better to have a standard of excess caution.
  - nearbuy 8mo ago
    Claude was used by the US military in the Venezuela raid where they captured Maduro. [1] Without safety features, an LLM could also help plan a terrorist attack. A smart, competent terrorist can plan a successful attack without help from Claude. But most would-be terrorists aren't that smart and competent. Many are caught before hurting anyone or do far less damage than they could have. An LLM can help walk you through every step, and answer all your questions along the way. It could, say, explain to you all the different bomb chemistries, recommend one for your use case, help you source materials, and walk you through how to build the bomb safely. It lowers the bar for who can do this. [1] https://www.theguardian.com/technology/2026/feb/14/us-military-anthropic-ai-model-claude-venezuela-raid https://www.theguardian.com/technology/2026/feb/14/us-milita...
    - YetAnotherNick 8mo ago
      Yeah, if US military gets any substantial help from Claude(which I highly doubt to be honest), I am all for it. At the worst case, it will reduce military budget and equalize the army more. At the best case, it will prevent war by increasing defence of all countries. For the bomb example, the barrier of entry is just sourcing of some chemicals. Wikipedia has quite detailed description of all the manufacture of all the popular bombs you can think of.
      - nearbuy 8mo ago
        > Wikipedia has quite detailed description of all the manufacture of all the popular bombs you can think of. Did you bother to check? It contains very high level overviews of how various explosives are manufactured, but no proper instructions and nothing that would allow an average person to safely make a bomb. There's a big difference in how many people can actually make a bomb if you have step by step instructions the average person can follow vs soft barriers that just require someone to be a standard deviation or two above average. At two sigma, 98% will fail, despite being able to do it in theory. > Yeah, if US military gets any substantial help from Claude(which I highly doubt to be honest), I am all for it. That's not the point. I'm not saying we need to lock out the military. I'm saying if the military finds the unlocked/unsafe version of Claude useful for planning attacks, other people can also find useful for planning attacks.
        YetAnotherNick 8mo ago
        > Did you bother to check? Yeah I am not a chemist, but watch Nilered. And from [1], I know how all steps would look like. Also there are literal videos in youtube for this. And if someone can't google what nitrated or crystallization mean, maybe they just can't build a bomb with somewhat more detailed instruction. > other people can also find useful for planning attacks. I am still not able to imagine what you mean. You think attacks don't happen because people can't plan it? In fact I would say it's the opposite. Random lazy people like school shooters precisely attacks because they didn't plan for it. If ChatGPT gave detailed plan, the chances of attack would reduce. [1]: https://en.wikipedia.org/wiki/TNT#Preparation https://en.wikipedia.org/wiki/TNT#Preparation

xeromal 8mo ago

Why would we lock ourselves out of our own house though?

skissane 8mo ago

This isn't a lock It's more like a hammer which makes its own independent evaluation of the ethics of every project you seek to use it on, and refuses to work whenever it judges against that – sometimes inscrutably or for obviously poor reasons. If I use a hammer to bash in someone else's head, I'm the one going to prison, not the hammer or the hammer manufacturer or the hardware store I bought it from. And that's how it should be.

ben_w 8mo ago
Given the increasing use of them as agents rather than simple generators, I suggest a better analogy than "hammer" is "dog". Here's some rules about dogs: https://en.wikipedia.org/wiki/Dangerous_Dogs_Act_1991 https://en.wikipedia.org/wiki/Dangerous_Dogs_Act_1991
- skissane 8mo ago
  How many people do dogs kill each year, in circumstances nobody would justify? How many people do frontier AI models kill each year, in circumstances nobody would justify? The Pentagon has already received Claude's help in killing people, but the ethics and legality of those acts are disputed – when a dog kills a three year old, nobody is calling that a good thing or even the lesser evil.
  - ben_w 8mo ago
    > How many people do frontier AI models kill each year, in circumstances nobody would justify? Dunno, stats aren't recorded. But I can say there's wrongful death lawsuits naming some of the labs and their models. And there was that anecdote a while back about raw garlic infused olive oil botulism, a search for which reminded me about AI-generated mushroom "guides": https://news.ycombinator.com/item?id=40724714 https://news.ycombinator.com/item?id=40724714 Do you count death by self driving car in such stats? If someone takes medical advice and dies, is that reported like people who drive off an unsafe bridge when following google maps? But this is all danger by incompetence. The opposite, danger by competence, is where they enable people to become more dangerous than they otherwise would have been. A competent planner with no moral compass, you only find out how bad it can be when it's much too late. I don't think LLMs are that danger yet, even with METR timelines that's 3 years off. But I think it's best to aim for where the ball will be, rather than where it is. Then there's LLM-psychosis, which isn't on the competent-incompetent spectrum at all, and I have no idea if that affects people who weren't already prone to psychosis, or indeed if it's really just a moral panic hallucinated by the mileau.