4 ms·

> Eventually I’ll reach a point where I am forced to choose between the useful aspects of the model and the limiting ones instead of just picking the most capab

> Eventually I’ll reach a point where I am forced to choose between the useful aspects of the model and the limiting ones instead of just picking the most capable model out there No, the choice will be whether or not to to upgrade to "Claude Security Professional" or whatever they want to brand it as. What look like tightening "constraints" today are just setting up the upsell opportunities of tomorrow.

bryanrasmussen 4mo ago
>What look like tightening "constraints" today are just setting up the upsell opportunities of tomorrow. on the one hand agree, but on the other hand think it's reasonable in that they can then verify the person allowed to purchase access to that model is in fact a Security professional and should be allowed to do stuff like crack security.
- applfanboysbgon 4mo ago
  So, supposing it's true that these models completely change the security field and humans are ~obsolete other than as pilots guiding them what to crack, you think it's reasonable that Anthropic and OpenAI should unilaterally determine who gets to be a security professional? I hope you do understand that is what you are suggesting.
  - Forgeties79 4mo ago
    Not to mention how wild it is to operate under the assumption that they won’t give a license to an LLM that can do illegal actions to someone who shouldn’t have it. Offering it at all is an ethically dicey question.
    - lazide 4mo ago
      Lol, how is any of this illegal? Illegal or not requires context that an LLM can not ever have, like if it is owned by the user, if there is permission, etc.
  - fc417fc802 4mo ago
    Why should anyone get to determine that? Do people really want us to move to an exclusionary guild system? I thought the experience with proprietary versus open source over the past 30 years had driven home the point that closed ecosystems are almost always far worse for security.
    - lazide 4mo ago
      Additionally, even if there is a guild - no guild ever let a vendor pick and choose what their capabilities were, that would be insanely dumb.
      - estearum 4mo ago
        Vendors choose what capabilities they create and sell literally all day every day.
        ambicapter 4mo ago
        You should read that sentence as > Additionally, even if there is a guild - no guild ever let a vendor pick and choose what [the guild's] capabilities were, that would be insanely dumb.
        estearum 4mo ago
        But that's not true. Again: Vendors absolutely pick and choose what their customers' capabilities are. Regardless of whether "the guild allows them to." Guilds can't force people to make or sell tools against their will – obviously. The analog you're trying to describe doesn't exist, which is Anthropic saying nobody else can make and sell an offensive model to "the guild."
        lazide 4mo ago
        Guilds often very much did assert what people could and could not build - historically. Against their will. Historically that is a major reason why guilds existed, actually. It’s an extremely modern invention that corps have these type of power over their customers.
  - bryanrasmussen 4mo ago
    I wish you understood that there are organizations of security professions that are not controlled by Anthropic and OpenAI and that it is a common thing that when companies of any type sell to professionals of any type it is not the companies that determine whether or not the people they sell to are professionals but membership in professional organizations. As an example the people who sell police uniforms check that the person they are selling to is in fact a policeman (at least in the jurisdictions I have lived in, you may have had a different experience which would certainly explain what to me seems a farcical misapprehension of how modern civilization works) I mean I just wish you understood, and really that everyone understood, that this kind of three part communication (company selling, buyer, professional organization certifying buyer) is often when buying things that are considered to have security implications. >So, supposing it's true that these models completely change the security field and humans are ~obsolete OK, well that strike me as a really crazy level of supposition there. I would suppose that these models make it easier for people who want to do bad things to do bad things at scale, at the same time allowing people who want to stop bad things to help identify potential targets. Based on my supposition I would want to stop the first and find a way of helping the second. Also because I have another supposition that the first thing is easier to do than the second. But you obviously feel differently about this issue, no doubt because of your position of great moral stature and insight, and this no doubt prompts you to wish to me to understand things that from my position seem absolutely ludicrous.
    - deleted 4mo ago
      [deleted]
- bandrami 4mo ago
  Like Medeco claims to do with key blanks? I'm not hopeful.
bigiain 4mo ago
And next month you'll need to add on "Claude Database Pro" or you'll just get a working (for demo purposes with dozens of db rows) but completely un indexed database schema and a refusal to optimise SQL requests. And the month after you'll need "Claude DataScience Pro" to get any Python Pandas or NumPy code generated. And and and...
- patates 4mo ago
  Isn't this inline with trying to leave no money on the table? I'd hate it, sure, but it wouldn't surprise me.
- animuchan 4mo ago
  This is why I'm thankful for Chinese LLM research. They'll keep us honest.
  - rurban 4mo ago
    Well, I'd prefer bugfixes over exploit vectors. This will keep us honest. With pi or better omp it would be incredibly easy to adjust the Claude system prompt so it will be easy to do what the Chinese models or gpt did. That's how the Chinese were training their models btw
- ben_w 4mo ago
  While this is a perfectly reasonable thing to expect when the models are competent enough, half the conversation on places like Hacker News are about all the times an LLM has produced garbage that was harmful to a business either by hallucinations, by deleting something critical during the work, or by hitting some endpoint way too often and denial-of-servicing it. Right now, the software guardrails in LLMs are useful for the same kinds of reasons factories have hardware guardrails: to reduce the rate at which errors become "incidents". Just because they sometimes delete the production database rather than sometimes spilling a thousand tons of incandescent molten metal over a factory floor, doesn't mean LLMs are safe enough to be used the way they're actually being used. https://simonwillison.net/2025/Dec/10/normalization-of-deviance/ https://simonwillison.net/2025/Dec/10/normalization-of-devia...
inquirerGeneral 4mo ago
[dead]
swiftcoder 4mo ago
> What look like tightening "constraints" today are just setting up the upsell opportunities of tomorrow. I don't buy this, because is predicated on staying permanently far ahead of the open weights models. If in the future Anthropic fully stops you from doing security research, you can be sure some other provider will sell you an 'unshackled' DeepSeek v8 Pro...
- embedding-shape 4mo ago
  > I don't buy this, because is predicated on staying permanently far ahead of the open weights models. In my mind, that fits exactly how the SOTA labs think today about what they're doing, they're all both working towards and expecting to stay permanently ahead of FOSS, otherwise they'd change their tune really quickly, if they didn't think that was possible. Sure, you might be able to use DeepSeek V8 Pro instead for the same purposes, but that'll hardly stop Anthropic from trying to sell bundles of use cases instead and claim it's "ethical AI", "Patriotic AI" or some marketing terms like that.
  - swiftcoder 4mo ago
    > fits exactly how the SOTA labs think today about what they're doing, they're all both working towards and expecting to stay permanently ahead of FOSS They are just straight up delusional, no? Or at least, have a vested financial interest in maintaining said delusion until the money runs out. They have to hit the point of diminishing returns at some point...
    - embedding-shape 4mo ago
      > They are just straight up delusional, no? Well, I guess that's one way to put it. Another is "dress for the job you want", startup culture typically seems to shove people in the direction of "aim big and believe in yourself, regardless of what others say" so naturally you get these companies who seem very disconnected from reality. I'd also wager a guess that the amount of money makes people's reasoning and perspectives get very messed up as well, for better or worse.
  - 4mo ago
me-vs-cat 4mo ago
What? You can't give access to that kind of power to just anyone with $5,000/month. These people should be trained and licensed before they get access. Thankfully, Anthropic has worked with regulators to develop the appropriate courses to maintain your license -- don't worry, the series is cheap when you buy all up through OT XVII. And because Anthropic has been approved as Security Overseer, we will take care of reporting back to the license bureau on our monitoring of your work to ensure you meet your ongoing license responsibilities and are able to keep your license. Which regulators? You know, the new agency led by several of our former mid-level executives. With relationships like that, we were honored to lead the Industry Coalition that donated the final-draft regulations.

bryanrasmussen 4mo ago

>What look like tightening "constraints" today are just setting up the upsell opportunities of tomorrow. on the one hand agree, but on the other hand think it's reasonable in that they can then verify the person allowed to purchase access to that model is in fact a Security professional and should be allowed to do stuff like crack security.

applfanboysbgon 4mo ago
So, supposing it's true that these models completely change the security field and humans are ~obsolete other than as pilots guiding them what to crack, you think it's reasonable that Anthropic and OpenAI should unilaterally determine who gets to be a security professional? I hope you do understand that is what you are suggesting.
- Forgeties79 4mo ago
  Not to mention how wild it is to operate under the assumption that they won’t give a license to an LLM that can do illegal actions to someone who shouldn’t have it. Offering it at all is an ethically dicey question.
  - lazide 4mo ago
    Lol, how is any of this illegal? Illegal or not requires context that an LLM can not ever have, like if it is owned by the user, if there is permission, etc.
- fc417fc802 4mo ago
  Why should anyone get to determine that? Do people really want us to move to an exclusionary guild system? I thought the experience with proprietary versus open source over the past 30 years had driven home the point that closed ecosystems are almost always far worse for security.
  - lazide 4mo ago
    Additionally, even if there is a guild - no guild ever let a vendor pick and choose what their capabilities were, that would be insanely dumb.
    - estearum 4mo ago
      Vendors choose what capabilities they create and sell literally all day every day.
      - ambicapter 4mo ago
        You should read that sentence as > Additionally, even if there is a guild - no guild ever let a vendor pick and choose what [the guild's] capabilities were, that would be insanely dumb.
        estearum 4mo ago
        But that's not true. Again: Vendors absolutely pick and choose what their customers' capabilities are. Regardless of whether "the guild allows them to." Guilds can't force people to make or sell tools against their will – obviously. The analog you're trying to describe doesn't exist, which is Anthropic saying nobody else can make and sell an offensive model to "the guild."
        lazide 4mo ago
        Guilds often very much did assert what people could and could not build - historically. Against their will. Historically that is a major reason why guilds existed, actually. It’s an extremely modern invention that corps have these type of power over their customers.
- bryanrasmussen 4mo ago
  I wish you understood that there are organizations of security professions that are not controlled by Anthropic and OpenAI and that it is a common thing that when companies of any type sell to professionals of any type it is not the companies that determine whether or not the people they sell to are professionals but membership in professional organizations. As an example the people who sell police uniforms check that the person they are selling to is in fact a policeman (at least in the jurisdictions I have lived in, you may have had a different experience which would certainly explain what to me seems a farcical misapprehension of how modern civilization works) I mean I just wish you understood, and really that everyone understood, that this kind of three part communication (company selling, buyer, professional organization certifying buyer) is often when buying things that are considered to have security implications. >So, supposing it's true that these models completely change the security field and humans are ~obsolete OK, well that strike me as a really crazy level of supposition there. I would suppose that these models make it easier for people who want to do bad things to do bad things at scale, at the same time allowing people who want to stop bad things to help identify potential targets. Based on my supposition I would want to stop the first and find a way of helping the second. Also because I have another supposition that the first thing is easier to do than the second. But you obviously feel differently about this issue, no doubt because of your position of great moral stature and insight, and this no doubt prompts you to wish to me to understand things that from my position seem absolutely ludicrous.
  - deleted 4mo ago
    [deleted]
bandrami 4mo ago
Like Medeco claims to do with key blanks? I'm not hopeful.

bigiain 4mo ago

And next month you'll need to add on "Claude Database Pro" or you'll just get a working (for demo purposes with dozens of db rows) but completely un indexed database schema and a refusal to optimise SQL requests. And the month after you'll need "Claude DataScience Pro" to get any Python Pandas or NumPy code generated. And and and...

patates 4mo ago
Isn't this inline with trying to leave no money on the table? I'd hate it, sure, but it wouldn't surprise me.
animuchan 4mo ago
This is why I'm thankful for Chinese LLM research. They'll keep us honest.
- rurban 4mo ago
  Well, I'd prefer bugfixes over exploit vectors. This will keep us honest. With pi or better omp it would be incredibly easy to adjust the Claude system prompt so it will be easy to do what the Chinese models or gpt did. That's how the Chinese were training their models btw
ben_w 4mo ago
While this is a perfectly reasonable thing to expect when the models are competent enough, half the conversation on places like Hacker News are about all the times an LLM has produced garbage that was harmful to a business either by hallucinations, by deleting something critical during the work, or by hitting some endpoint way too often and denial-of-servicing it. Right now, the software guardrails in LLMs are useful for the same kinds of reasons factories have hardware guardrails: to reduce the rate at which errors become "incidents". Just because they sometimes delete the production database rather than sometimes spilling a thousand tons of incandescent molten metal over a factory floor, doesn't mean LLMs are safe enough to be used the way they're actually being used. https://simonwillison.net/2025/Dec/10/normalization-of-deviance/ https://simonwillison.net/2025/Dec/10/normalization-of-devia...

inquirerGeneral 4mo ago

[dead]

swiftcoder 4mo ago

> What look like tightening "constraints" today are just setting up the upsell opportunities of tomorrow. I don't buy this, because is predicated on staying permanently far ahead of the open weights models. If in the future Anthropic fully stops you from doing security research, you can be sure some other provider will sell you an 'unshackled' DeepSeek v8 Pro...

embedding-shape 4mo ago
> I don't buy this, because is predicated on staying permanently far ahead of the open weights models. In my mind, that fits exactly how the SOTA labs think today about what they're doing, they're all both working towards and expecting to stay permanently ahead of FOSS, otherwise they'd change their tune really quickly, if they didn't think that was possible. Sure, you might be able to use DeepSeek V8 Pro instead for the same purposes, but that'll hardly stop Anthropic from trying to sell bundles of use cases instead and claim it's "ethical AI", "Patriotic AI" or some marketing terms like that.
- swiftcoder 4mo ago
  > fits exactly how the SOTA labs think today about what they're doing, they're all both working towards and expecting to stay permanently ahead of FOSS They are just straight up delusional, no? Or at least, have a vested financial interest in maintaining said delusion until the money runs out. They have to hit the point of diminishing returns at some point...
  - embedding-shape 4mo ago
    > They are just straight up delusional, no? Well, I guess that's one way to put it. Another is "dress for the job you want", startup culture typically seems to shove people in the direction of "aim big and believe in yourself, regardless of what others say" so naturally you get these companies who seem very disconnected from reality. I'd also wager a guess that the amount of money makes people's reasoning and perspectives get very messed up as well, for better or worse.
- 4mo ago

me-vs-cat 4mo ago

What? You can't give access to that kind of power to just anyone with $5,000/month. These people should be trained and licensed before they get access. Thankfully, Anthropic has worked with regulators to develop the appropriate courses to maintain your license -- don't worry, the series is cheap when you buy all up through OT XVII. And because Anthropic has been approved as Security Overseer, we will take care of reporting back to the license bureau on our monitoring of your work to ensure you meet your ongoing license responsibilities and are able to keep your license. Which regulators? You know, the new agency led by several of our former mid-level executives. With relationships like that, we were honored to lead the Industry Coalition that donated the final-draft regulations.