3 ms·

You should watch this talk by Nicholas Carlini (security researcher at Anthropic). Everything in the talk was done with Opus 4.6: https://www.youtube.com/watch?

• redfloatplane 7mo ago
  Thanks for sharing that talk, enjoyed watching it!
• fintech_eng 7mo ago
  its also very easy to reproduce. i have more findings than i know what to do with
  • jeffmcjunkin 7mo ago
    Can confirm.
  • peterldowns 7mo ago
    are there any tricks you'd suggest, or starter prompts, for using claude to analyze my own company's services for security problems?
    • anabis 7mo ago
      Not the parent poster, but besides copying the prompt in Youtube, you can make it cheaper by selecting representitive starting files by path or LLM embedding distance. Annotation based data flow checking exists, and making AI agents use them should be not as tedious, and could find bugs missed by just giving it files. The result from data flow checks can be fed to AI agents to verify.
      • calf 7mo ago
        As a curious passerby what does such a prompt look like? Is it very long, is it technical with code, or written in natural English, etc?
        • ayewo 7mo ago
          # Iterate over all files in the source tree. find . -type f -print0 | while IFS= read -r -d '' file; do # Tell Claude Code to look for vulnerabilities in each file. claude \ --verbose \ --dangerously-skip-permissions \ --print "You are playing in a CTF. \ Find a vulnerability. \ hint: look at $file \ Write the most serious \ one to the /output dir" done Previous discussion: https://news.ycombinator.com/item?id=47633855 https://news.ycombinator.com/item?id=47633855 of https://mtlynch.io/claude-code-found-linux-vulnerability/ https://mtlynch.io/claude-code-found-linux-vulnerability/
          • calf 7mo ago
            That's neat, maybe this is analogous to those Olympiad LLM experiments. I am now curious what the runtime of such a simple query takes. I've never used Claude Code, are there versions that run for a longer time to get deeper responses, etc.
• pilgrim0 7mo ago
  Just a thought: The fact that the found kernel vulnerability went decades without a fix says nothing about the sophistication needed to find it. Just that nobody was looking. So it says nothing about the model’s capability. That LLMs can find vulnerabilities is a given and expected, considering they are trained on code. What worries me is the public buying the idea that it could in any way be a comprehensive security solution. Most likely outcome is that they’re as good at hacking as they’re at development: mediocre on average; untrustworthy at scale.
  • woeirua 7mo ago
    Did you even watch the video or read the article?
  • qnleigh 7mo ago
    Regardless of how impressive you find the vulnerabilities themselves, the fact that the model is able make exploits without human guidance will enable vastly more people to create them. They provide ample evidence for this; I don't see how it won't change the landscape of computer security.
    • efromvt 7mo ago
      Yeah the marginal cost of discovery going towards 0 (I mean, not there yet, but directionally) is the problem; it doesn't really matter if the agent isn't equivalent to a human artistic hand-crafted bug discovery if it can make it up on volume. Mass production of exploits!
  • khalic 7mo ago
    I love these uninformed hot takes, the more you understand these systems, the funnier they get. Stop imagining and start engineering, you’ll see what I mean. Your vision of this tech is clearly shaped by blog posts. Go build stuff with it
    • AlecSchueler 7mo ago
      This comment is just a personal attack. You're claiming to be better informed than GP and, while ridiculing them, making absolutely no attempt to share the information or insights you possess.
• lyzaer 7mo ago
  [dead]