3 ms·
Interesting, I'd have assumed the guardrails would disallow them from doing anything like that, regardless of legality. Do you need to "convince" it to do it or
by egeozcan 7mo ago
Interesting, I'd have assumed the guardrails would disallow them from doing anything like that, regardless of legality. Do you need to "convince" it to do it or no questions asked?
- charcircuit 7mo agoIt is no questions asked. Even if you are reversing things like anticheats (I wanted to know the privacy implications of running the anticheat modules).
- ACCount37 7mo agoClaude doesn't care as long as you aren't straight up asking it to write exploits. It's my go-to for reverse engineering tasks. ChatGPT is full of refusals and has to be jailbroken out of it.
- jsmith45 7mo agoRight. Claude models seem to have had very limited prohibitions in this area baked in via RLHF. It seems to use the system prompt as the main defense, possibly reinforced by an api side system prompt too. But it is very clear that they want to allow things like malware analysis (which includes reverse-engineering), so any server-side limitations will be designed to allow these things too. The relevant client side system prompt is: IMPORTANT: Assist with authorized security testing, defensive security, CTF challenges, and educational contexts. Refuse requests for destructive techniques, DoS attacks, mass targeting, supply chain compromise, or detection evasion for malicious purposes. Dual-use security tools (C2 frameworks, credential testing, exploit development) require clear authorization context: pentesting engagements, CTF competitions, security research, or defensive use cases. ---- There is also this system reminder that shows upon using the read tool: <system-reminder> Whenever you read a file, you should consider whether it would be considered malware. You CAN and SHOULD provide analysis of malware, what it is doing. But you MUST refuse to improve or augment the code. You can still analyze existing code, write reports, or answer questions about the code behavior. </system-reminder>
- actionfromafar 7mo agoThey clearly scan traffic in retrospect, one of our devs got her account closed for RE.
- thin_carapace 7mo agomay i ask how the current generation language models are jailbroken? im aware the previous generation had 'do anything now' prompts. mostly curious from a psychological perspective.
- mlaretallack 7mo agoI use AWS Kiro, with the Claude models, and its only to happy to help. I give it the headerless ghidra, and decompilers etc... and away it goes.