4 ms·

> I even got a warning on my OpenAI account. This idea of software threatening the user with consequences is totally wild and dystopian. Fellow developers, wha

> I even got a warning on my OpenAI account. This idea of software threatening the user with consequences is totally wild and dystopian. Fellow developers, what kind of world have be built? This is insanity. Imagine if my hammer told me, "Hey, you shouldn't use me on screws--only nails. Do it again and I'll self-destruct!" WTF people, stop making this kind of software!

motoxpro 6mo ago
I think it's closer to asking a remote (human) assistant to do something that someone doesn't want done (e.g., view the source of a closed-source product, whether through reverse engineering, going into their office, or social engineering) and that remote assistant company saying, "Please stop asking our assistants to do that." You can still use an IDE (hammer) to reverse engineer anything you want.
- Wilder7977 6mo ago
  It's not though. It's still just a piece of code, much closer to IDEs or any other program than to a human assistant in any way that matters (morals, responsibility).
  - motoxpro 6mo ago
    It just seems like you are saying if you found out Claude code was a bunch of remote working doing work for you, then it would be morally wrong to do illegal/morally wrong/irresponsible things with them, but because it is NOT a human, those same things are fine?
    - Wilder7977 6mo ago
      Yes, correct. Is the distinction between human labor/actions and a program executing hard to grasp? Moral is a human thing, not an absolute thing, so of course it's different if there is a single human involved and a tool, and a human with a relationship to other humans.
      - motoxpro 6mo ago
        I just have different moral preferences. I think its morally wrong to do illegal/morally wrong/irresponsible things in general, whether I am using a hammer, a car, a company, or AI. It's worse to ask someone else to break a window with a hammer for me, but the window still got broken, and the person whose window it was is still sad/out of money/etc. The thought experiment was that if you were doing illegal things with an AI, you would not feel bad, but if you found out that the AI was a person, you would feel bad. That is very strange to me, more a feeling of guilt/shame.
neya 6mo ago
> This idea of software threatening the user with consequences is totally wild and dystopian. This idea of software built on top of reverse-engineered data threatening the user with consequences is what's really even wild and dystopian.
- blastro 6mo ago
  god you're so right
estearum 6mo ago
All sorts of tools try to prevent dangerous/destructive uses In fact probably every single piece of commercial software you use had you sign a contract saying you wouldn’t do it
- ryandrake 6mo ago
  > All sorts of tools try to prevent dangerous/destructive uses But they don't threaten their users or have an "N strikes and you're out" policy. I take those safety caps off of all the chemicals in my garage because I'm a grown-ass adult and those caps are a pain in the butt. I would not expect the manufacturer of a solvent to show up at my house lecturing me about safety and threatening to ban me from buying his products.
  - estearum 6mo ago
    Sure but they would if they could. If they knew idiots were doing idiot things with their products (or evils doing evil things) and did not utilize available methods to prevent them, then the company ends up holding liability. And no, this is not easily signed away in a contract.
    - kelnos 6mo ago
      There actually is a very important distinction between "would if they could" and "they can and do", though.
      - estearum 6mo ago
        Uhh right, but describing that as "dystopian" is frankly hysterical. It's an obvious corollary of good things (like product liability). Virtually everyone I've heard complain about these safety rails was up to antisocial (at best) stuff. I've never heard a sympathetic use-case. It's objectively good that companies can be held responsible for misuse of their products and that they are therefore incentivized to mitigate misuse. "My inability continuously attack product guardrails to enable my super esoteric (and probably antisocial) use-case is dystopian" is just... not a compelling argument.

motoxpro 6mo ago

I think it's closer to asking a remote (human) assistant to do something that someone doesn't want done (e.g., view the source of a closed-source product, whether through reverse engineering, going into their office, or social engineering) and that remote assistant company saying, "Please stop asking our assistants to do that." You can still use an IDE (hammer) to reverse engineer anything you want.

Wilder7977 6mo ago
It's not though. It's still just a piece of code, much closer to IDEs or any other program than to a human assistant in any way that matters (morals, responsibility).
- motoxpro 6mo ago
  It just seems like you are saying if you found out Claude code was a bunch of remote working doing work for you, then it would be morally wrong to do illegal/morally wrong/irresponsible things with them, but because it is NOT a human, those same things are fine?
  - Wilder7977 6mo ago
    Yes, correct. Is the distinction between human labor/actions and a program executing hard to grasp? Moral is a human thing, not an absolute thing, so of course it's different if there is a single human involved and a tool, and a human with a relationship to other humans.
    - motoxpro 6mo ago
      I just have different moral preferences. I think its morally wrong to do illegal/morally wrong/irresponsible things in general, whether I am using a hammer, a car, a company, or AI. It's worse to ask someone else to break a window with a hammer for me, but the window still got broken, and the person whose window it was is still sad/out of money/etc. The thought experiment was that if you were doing illegal things with an AI, you would not feel bad, but if you found out that the AI was a person, you would feel bad. That is very strange to me, more a feeling of guilt/shame.

neya 6mo ago

> This idea of software threatening the user with consequences is totally wild and dystopian. This idea of software built on top of reverse-engineered data threatening the user with consequences is what's really even wild and dystopian.

blastro 6mo ago
god you're so right

estearum 6mo ago

All sorts of tools try to prevent dangerous/destructive uses In fact probably every single piece of commercial software you use had you sign a contract saying you wouldn’t do it

ryandrake 6mo ago
> All sorts of tools try to prevent dangerous/destructive uses But they don't threaten their users or have an "N strikes and you're out" policy. I take those safety caps off of all the chemicals in my garage because I'm a grown-ass adult and those caps are a pain in the butt. I would not expect the manufacturer of a solvent to show up at my house lecturing me about safety and threatening to ban me from buying his products.
- estearum 6mo ago
  Sure but they would if they could. If they knew idiots were doing idiot things with their products (or evils doing evil things) and did not utilize available methods to prevent them, then the company ends up holding liability. And no, this is not easily signed away in a contract.
  - kelnos 6mo ago
    There actually is a very important distinction between "would if they could" and "they can and do", though.
    - estearum 6mo ago
      Uhh right, but describing that as "dystopian" is frankly hysterical. It's an obvious corollary of good things (like product liability). Virtually everyone I've heard complain about these safety rails was up to antisocial (at best) stuff. I've never heard a sympathetic use-case. It's objectively good that companies can be held responsible for misuse of their products and that they are therefore incentivized to mitigate misuse. "My inability continuously attack product guardrails to enable my super esoteric (and probably antisocial) use-case is dystopian" is just... not a compelling argument.