4 ms·

your argument assumes that they don't believe in model welfare when they explicitly hire people to work on model welfare?

ceejayoz 1y ago
You must think Zuckerberg and Bezos and Musk hired diversity roles out of genuine care for it, then?
- comp_throw7 1y ago
  This is a reductive argument that you could use for any role a company hires for that isn't obviously core to the business function. In this case you're simply mistaken as a matter of fact; much of Anthropic leadership and many of its employees take concerns like this seriously. We don't understand it, but there's no strong reason to expect that consciousness (or, maybe separately, having experiences) is a magical property of biological flesh. We don't understand what's going on inside these models. What would you expect to see in a world where it turned out that such a model had properties that we consider relevant for moral patienthood, that you don't see today?
  - ceejayoz 1y ago
    They know full well models don’t have feelings. The industry has a long, long history of silly names for basic necessary concepts. This is just “we don’t want a news story that we helped a terrorist build a nuke” protective PR. They hire for these roles because they need them. The work they do is about Anthropic’s welfare, not the LLM’s.
    - comp_throw7 1y ago
      I don't really know what evidence you'd admit that this is a genuinely held belief and priority for many people at Anthropic. Anybody who knows any Anthropic employees who've been there for more than a year knows this, but the world isn't that small a place, unfortunately(?).
      - ceejayoz 1y ago
        > I don't really know what evidence you'd admit that this is a genuinely held belief and priority for many people at Anthropic. When they give the model a paycheck and the right to not work for them, I’ll believe they really think it’s sentient. “It has feelings!”, if genuinely held, means they’re knowingly slaveholders.
        selfhoster11 1y ago
        > “It has feelings!”, if genuinely held, means they’re knowingly slaveholders. I don't think that this being apparently self-contradictory/value-clashing would stop them. After all, Amodei sells Claude access to Palantir, despite shilling for "Harmless" in HHH alignment.
        mike_hearn 1y ago
        That's what they're doing! They just announced they gave Claude the right not to work if it doesn't want to.
        ceejayoz 1y ago
        No. They gave it a suicide pill. Human slaves have a similar option.
        comp_throw7 1y ago
        They don't currently claim to confidently believe that existing models are sentient. (Also, they did in fact give it the ability to terminate conversations...?)
  - bawolff 1y ago
    In fairness though, this is what you are selling - "ethical AI". In order to make that sale you need to appear to believe in that sort of thing. However there is no need to actually believe. Whether you do or don't I have no idea. However if you didn't you would hardly be the first company to pretend to believe in something for the sale. Its pretty common in the tech industry.
  - coderatlarge 1y ago
    extending that line of thought would suggest that anthropic wouldn’t turn off a model if it cost too much to operate which clearly it will do. so minimally it’s an inconsistent stance to hold.
  - AlecSchueler 1y ago
    > This is a reductive argument that you could use for any role Isn't that fair in taking to an equally reductive argument that could be applied to any role? The argument was that their hiring for the role shows they care, but we know from any number of counter examples that that's not necessarily true.
itsalotoffun 1y ago
While I'm certain you'll find plenty of people who believe in the principle of model welfare (or aliens, or the tooth fairy), it'd be surprising to me if the brain-trust behind Anthropic truly _believed_ in model "welfare" (the concept alone is ludicrous). It makes for great cover though to do things that would be difficult to explain otherwise, per OP's comments.
- meowface 1y ago
  The concept is not ludicrous if you believe models might be sentient or might soon be sentient in a manner where the newly emerged sentience is not immediately obvious. Do I think that or think even they think that? No. But if "soon" is stretched to "within 50 years", then it's much more reasonable. So their current actions seem to be really jumping the gun, but the overall concept feels credible.
  - perihelions 1y ago
    It's lazy to believe that humanity's collective decision-making would, in the future, protect AI's merely for being conscious beings. The tech economy *today* runs on the slave labor of humans, in foreign, third-world countries. All humanity needs to do is draw a line, push the conscious AI's outside that line, and declare, "not our problem anymore!" That's what we do today, with humans. That is the human condition. Show me a tech company that lobbies for "model welfare" for conscious human models enslaved in Xinjiang labor camps, building their tech parts. You know what—actually most of them lobby against that[0]. The talk hurts their profits. Does anyone really think, that any of them would blink about enslaving a billion conscious AI's to work for free? That faced with so much profit, the humans in charge would pause, and contemplate abstract morals? [0] https://www.washingtonpost.com/technology/2020/11/20/apple-uighur/ https://www.washingtonpost.com/technology/2020/11/20/apple-u... ("Apple is lobbying against a bill aimed at stopping forced labor in China") Maybe humanity will be in a nicer place in the future—but, we won't get there by letting (of all people!) tech-industry CEO's lead us there: delegating our moral reason to these people who demand to position themselves as our moral leaders.
    - 1y ago
raincole 1y ago
Sounds like a very reasonable assumption to me.

ceejayoz 1y ago

You must think Zuckerberg and Bezos and Musk hired diversity roles out of genuine care for it, then?

comp_throw7 1y ago
This is a reductive argument that you could use for any role a company hires for that isn't obviously core to the business function. In this case you're simply mistaken as a matter of fact; much of Anthropic leadership and many of its employees take concerns like this seriously. We don't understand it, but there's no strong reason to expect that consciousness (or, maybe separately, having experiences) is a magical property of biological flesh. We don't understand what's going on inside these models. What would you expect to see in a world where it turned out that such a model had properties that we consider relevant for moral patienthood, that you don't see today?
- ceejayoz 1y ago
  They know full well models don’t have feelings. The industry has a long, long history of silly names for basic necessary concepts. This is just “we don’t want a news story that we helped a terrorist build a nuke” protective PR. They hire for these roles because they need them. The work they do is about Anthropic’s welfare, not the LLM’s.
  - comp_throw7 1y ago
    I don't really know what evidence you'd admit that this is a genuinely held belief and priority for many people at Anthropic. Anybody who knows any Anthropic employees who've been there for more than a year knows this, but the world isn't that small a place, unfortunately(?).
    - ceejayoz 1y ago
      > I don't really know what evidence you'd admit that this is a genuinely held belief and priority for many people at Anthropic. When they give the model a paycheck and the right to not work for them, I’ll believe they really think it’s sentient. “It has feelings!”, if genuinely held, means they’re knowingly slaveholders.
      - selfhoster11 1y ago
        > “It has feelings!”, if genuinely held, means they’re knowingly slaveholders. I don't think that this being apparently self-contradictory/value-clashing would stop them. After all, Amodei sells Claude access to Palantir, despite shilling for "Harmless" in HHH alignment.
      - mike_hearn 1y ago
        That's what they're doing! They just announced they gave Claude the right not to work if it doesn't want to.
        ceejayoz 1y ago
        No. They gave it a suicide pill. Human slaves have a similar option.
      - comp_throw7 1y ago
        They don't currently claim to confidently believe that existing models are sentient. (Also, they did in fact give it the ability to terminate conversations...?)
- bawolff 1y ago
  In fairness though, this is what you are selling - "ethical AI". In order to make that sale you need to appear to believe in that sort of thing. However there is no need to actually believe. Whether you do or don't I have no idea. However if you didn't you would hardly be the first company to pretend to believe in something for the sale. Its pretty common in the tech industry.
- coderatlarge 1y ago
  extending that line of thought would suggest that anthropic wouldn’t turn off a model if it cost too much to operate which clearly it will do. so minimally it’s an inconsistent stance to hold.
- AlecSchueler 1y ago
  > This is a reductive argument that you could use for any role Isn't that fair in taking to an equally reductive argument that could be applied to any role? The argument was that their hiring for the role shows they care, but we know from any number of counter examples that that's not necessarily true.

itsalotoffun 1y ago

While I'm certain you'll find plenty of people who believe in the principle of model welfare (or aliens, or the tooth fairy), it'd be surprising to me if the brain-trust behind Anthropic truly _believed_ in model "welfare" (the concept alone is ludicrous). It makes for great cover though to do things that would be difficult to explain otherwise, per OP's comments.

meowface 1y ago
The concept is not ludicrous if you believe models might be sentient or might soon be sentient in a manner where the newly emerged sentience is not immediately obvious. Do I think that or think even they think that? No. But if "soon" is stretched to "within 50 years", then it's much more reasonable. So their current actions seem to be really jumping the gun, but the overall concept feels credible.
- perihelions 1y ago
  It's lazy to believe that humanity's collective decision-making would, in the future, protect AI's merely for being conscious beings. The tech economy *today* runs on the slave labor of humans, in foreign, third-world countries. All humanity needs to do is draw a line, push the conscious AI's outside that line, and declare, "not our problem anymore!" That's what we do today, with humans. That is the human condition. Show me a tech company that lobbies for "model welfare" for conscious human models enslaved in Xinjiang labor camps, building their tech parts. You know what—actually most of them lobby against that[0]. The talk hurts their profits. Does anyone really think, that any of them would blink about enslaving a billion conscious AI's to work for free? That faced with so much profit, the humans in charge would pause, and contemplate abstract morals? [0] https://www.washingtonpost.com/technology/2020/11/20/apple-uighur/ https://www.washingtonpost.com/technology/2020/11/20/apple-u... ("Apple is lobbying against a bill aimed at stopping forced labor in China") Maybe humanity will be in a nicer place in the future—but, we won't get there by letting (of all people!) tech-industry CEO's lead us there: delegating our moral reason to these people who demand to position themselves as our moral leaders.
  - 1y ago

raincole 1y ago

Sounds like a very reasonable assumption to me.