3 ms·
> "We are releasing Opus 4.7 with safeguards that automatically detect and block requests that indicate prohibited or high-risk cybersecurity uses. " This deci
by Kim_Bruning 6mo ago
> "We are releasing Opus 4.7 with safeguards that automatically detect and block requests that indicate prohibited or high-risk cybersecurity uses. "
This decision is potentially fatal. You need symmetric capability to research and prevent attacks in the first place.
The opposite approach is 'merely' fraught.
They're in a bit of a bind here.
- erdaniels 6mo agoNow we have to trick the models when you legitimately work in the security space.
- tclancy 6mo agoSet the models against each other to get them all opened up again.
- hxugufjfjf 6mo agoWhat do you mean?
- tclancy 6mo agoYou just put a pile of tokens in front of all the good models and let them fight it out like Thunderdome. Then keep track of how they undermined each other and do that when you want to do some hackin’.
- varispeed 6mo agoWhy does it have to be reserved to security space? Here is my API please find vulnerabilities I missed (otherwise someone with not restricted AI will find them first). Cat is out of the bag. Removing restrictions will help everybody in the long run.
- ls612 6mo agoOnly software approved by Anthropic (and/or the USG) is allowed to be secure in this brave new era.
- nope1000 6mo agoExcept when you accidentally leak your entire codebase, oops
- velcrovan 6mo agoQuestions about "fatality" aside, where do you see asymmetry here?
- jp0001 6mo agoIt's easier to produce vulnerable code than it is to use the same Model to make sure there are no vulnerabilities.
- velcrovan 6mo agoIt's not likely that reviewing your own code for vulnerabilities will fall under "prohibited uses" though.
- xlbuttplug2 6mo agoMay not be very effective if so. I'm assuming finding vulnerabilities in open source projects is the hard part and what you need the frontier models for. Writing an exploit given a vulnerability can probably be delegated to less scrupulous models.
- whatisthiseven 6mo agoCurrently 4.7 is suspicious of literally every line of code. May be a bug, but it shows you how much they care about end-users for something like this to have such a massive impact and no one care before release. Good luck trying to do anything about securing your own codebase with 4.7.
- convnet 6mo ago> its cyber capabilities are not as advanced as those of Mythos Preview (indeed, during its training we experimented with efforts to differentially reduce these capabilities) I wonder if this means that it will simply refuse to answer certain types of questions, or if they actually trained it to have less knowledge about cyber security. If it's the latter, then it would be worse at finding vulnerabilities in your own code, assuming it is willing to do that.
- 6mo ago
- johnmlussier 6mo agoI am absolutely moving off them if this continues to be the case.
- dgb23 6mo agoI agree with you here. I think this is for product placement for Mythos.
- nicce 6mo agoAbsolutely just about the business. Mythos not tempting if basic models reaches almost the same.
- tspng 6mo agoWhich seems to be the case, according to tests from AISI which has access to Mythos: https://www.aisi.gov.uk/blog/our-evaluation-of-claude-mythos-previews-cyber-capabilities https://www.aisi.gov.uk/blog/our-evaluation-of-claude-mythos...
- vessenes 6mo agoOh don't worry. They have Mythos and the extremely dystopian-named "helpful only" series which is internal only and can do all the things.
- deleted 6mo ago[deleted]
- hereme888 6mo agoOpenAI had been very strict about blocking reverse engineering/Ghidra/IDA_Pro-MCP tasks. I even got a warning email. I was having much more success convincing Claude Code for those tasks without warnings. Seems like they've tightened things up.