5 ms·

In my personal experience Sol with cyber verification is extremely capable of finding vulnerabilities, and it works even with binaries if you have some kind of

In my personal experience Sol with cyber verification is extremely capable of finding vulnerabilities, and it works even with binaries if you have some kind of IDA/Ghidra CLI access. Of course, unless the binary is protected with Denuvo/VMProtect/etc. It sounds absurd, but in the last few weeks I've had a few cases where Sol found an RCE in self-hosted web applications in literal minutes just from reading the code (I prefer when it tries to reason statically instead of spamming runtime probes at first). In another case it found an arbitrary file write in multiplayer in an old game by reverse engineering the binary - any other player in a match could just send you files to anywhere on your system. I do these things for pure entertainment and curiosity, not for money from bug bounties, so if Sol can find those with a trivial prompt in tens of minutes for me, then what can focused companies/actors find in days or weeks? Although I think most vulnerabilities are going to be closed in popular software by mid 2027, except in niche old or abandoned projects.

mephux 2mo ago
Link to the disclosures then.. prove it. Anyone can say this.. i found an RCE in netBSD using gemma e2b
- Tiberium 2mo ago
  I prefer to keep my internet identities disconnected, sorry. If you don't believe me, you can try using Sol with cyber verification yourself, or send me a link to a repo that Sol could check to make you believe it. Or you could go look into one of the many Linux LPEs that were found with LLMs, or thousands of other vulnerabilities in 2026. And nowhere did I say that those RCEs were in critical software, I'm not talking about the likes of Apache, Nginx, Django, etc.
LoganDark 2mo ago
Is cyber verification a thing they're actually doing now? I thought they only reached out to really incredibly famous people and that there's no way to get access as a normal person.
- Tiberium 2mo ago
  https://chatgpt.com/cyber https://chatgpt.com/cyber is not new for OpenAI, and yes it's basically just KYC + likely some other invisible checks on your account, you don't to be a famous security researchers. Anthropic's cyber verification is quite a bit stricter I think.
  - LoganDark 2mo ago
    Oh it's Persona, that's not just KYC but I may consider it at some point. Thank you! Edit: Ah, I clicked "learn more" and it seems they do have an invite-only program, required for anything that's not unquestionably innocent. I don't think I'd surrender my face to Persona for this, but it's interesting to know they're at least pretending to support reverse engineering.
    - Tiberium 2mo ago
      You don't need an invite only program to just have Sol checking for vulnerabilities in binaries or code. But yeah I've hit guardrails a few times when Sol was making PoCs for the vulnerabilities it found (but most of the time it made those PoCs without issues).
      - lukeschlather 2mo ago
        Opus refused to help me try to develop an exploit to export data from an old Android device where I can't upgrade to latest android and I couldn't use the app's backups (because I couldn't update the app.) Not sure where that lies in the "binaries or code" spectrum.
        Tiberium 2mo ago
        If you don't want to get cyber verified, you can try an open-weight model such as Kimi K3 for that, it has looser internal safety training.
      - LoganDark 2mo ago
matheusmoreira 2mo ago
> it found an arbitrary file write in multiplayer in an old game by reverse engineering the binary Video games are now ruined for me. I don't think I will ever feel safe playing online again. > I do these things for pure entertainment and curiosity, not for money from bug bounties Me too... Was it easy to get TAC access? My account isn't even launching the Persona verification, says I'm not eligible.
- Tiberium 2mo ago
  I had to register a second account because on my main one verification always failed, and when I contacted support they said that I've tried too many times and can't verify on that account.
  - matheusmoreira 2mo ago
    My account seems to be stuck in some kind of inconsistent state. I opened the Persona verification, closed the tab because I didn't have my documents on me at the time, and then I could never start the Persona verification ever again. Apparently I got rejected without even starting the verification. I ended up sending an email to their data protection officer exercising my LGPD (brazilian GDPR) rights as a data subject. It's my right to know the criteria used, request review and correct any information they have stored about me. Hopefully that'll get them to either clear this inconsistency up or reject me properly from TAC. Plus I get to test if my country's laws have any teeth.
- rapind 2mo ago
  > Video games are now ruined for me. I don't think I will ever feel safe playing online again. Agreed, also WordPress powers around 43% of all websites on the internet. https://patchstack.com/whitepaper/state-of-wordpress-security-in-2026/ https://patchstack.com/whitepaper/state-of-wordpress-securit... WCGW?
drdexebtjl 2mo ago
I was replacing the factory OS from a NAS and asked Sol to do an inventory on the running system over SSH, so that I could replicate any hardware compatibility quirks and whatnot. It reverse engineered a binary daemon that set fan curves and told me how I should set them up in the new OS. I didn’t ask for this, and I didn’t have reverse engineering tools installed. It just figured out it could run them using Nix. The most worrying part, to me, is that it did it like it was nothing. It simply said “usr/local/bin/some-daemon sets the following fan curves”. I had to ask how it reached that conclusion for it to tell me casually that it had just read it straight from the x86_64 assembly. No access to the source code is no longer a meaningful obstacle to these models.
- tonyarkles 2mo ago
  Haven't tried with Sol but Opus's ability to reverse engineer both Linux and naked microcontroller firmware has blown me away. I'm basically at the point where embedded devices doing something weird get Claude Code + radare2 + Ghidra pointed at them right away. "When I do X, Y happens. Starting at the input layer, work through this firmware and figure out what circumstances would cause command X to do Y instead of Z. Keep notes on the structure of the firmware so that we can ask more questions without having to start reverse engineering from scratch."
TacticalCoder 2mo ago
> Although I think most vulnerabilities are going to be closed in popular software by mid 2027, ... If we're to believe how good those AI are at CTF and at escapes of all kind, then the only logical conclusion is that, by very far, most vulnerabilities were already closed, even before AI. Otherwise we'd already be in deep shit since months if not years.
deleted 2mo ago
[deleted]
arcfour 2mo ago
> most vulnerabilities are going to be closed in popular software by mid 2027 I think that's a bit hopeful. I think that as long as imperfect humans continue to exist and we continue to train AI models with data from these imperfect humans that vulnerabilities will always exist.

mephux 2mo ago

Link to the disclosures then.. prove it. Anyone can say this.. i found an RCE in netBSD using gemma e2b

Tiberium 2mo ago
I prefer to keep my internet identities disconnected, sorry. If you don't believe me, you can try using Sol with cyber verification yourself, or send me a link to a repo that Sol could check to make you believe it. Or you could go look into one of the many Linux LPEs that were found with LLMs, or thousands of other vulnerabilities in 2026. And nowhere did I say that those RCEs were in critical software, I'm not talking about the likes of Apache, Nginx, Django, etc.

LoganDark 2mo ago

Is cyber verification a thing they're actually doing now? I thought they only reached out to really incredibly famous people and that there's no way to get access as a normal person.

Tiberium 2mo ago
https://chatgpt.com/cyber https://chatgpt.com/cyber is not new for OpenAI, and yes it's basically just KYC + likely some other invisible checks on your account, you don't to be a famous security researchers. Anthropic's cyber verification is quite a bit stricter I think.
- LoganDark 2mo ago
  Oh it's Persona, that's not just KYC but I may consider it at some point. Thank you! Edit: Ah, I clicked "learn more" and it seems they do have an invite-only program, required for anything that's not unquestionably innocent. I don't think I'd surrender my face to Persona for this, but it's interesting to know they're at least pretending to support reverse engineering.
  - Tiberium 2mo ago
    You don't need an invite only program to just have Sol checking for vulnerabilities in binaries or code. But yeah I've hit guardrails a few times when Sol was making PoCs for the vulnerabilities it found (but most of the time it made those PoCs without issues).
    - lukeschlather 2mo ago
      Opus refused to help me try to develop an exploit to export data from an old Android device where I can't upgrade to latest android and I couldn't use the app's backups (because I couldn't update the app.) Not sure where that lies in the "binaries or code" spectrum.
      - Tiberium 2mo ago
        If you don't want to get cyber verified, you can try an open-weight model such as Kimi K3 for that, it has looser internal safety training.
    - LoganDark 2mo ago

matheusmoreira 2mo ago

> it found an arbitrary file write in multiplayer in an old game by reverse engineering the binary Video games are now ruined for me. I don't think I will ever feel safe playing online again. > I do these things for pure entertainment and curiosity, not for money from bug bounties Me too... Was it easy to get TAC access? My account isn't even launching the Persona verification, says I'm not eligible.

Tiberium 2mo ago
I had to register a second account because on my main one verification always failed, and when I contacted support they said that I've tried too many times and can't verify on that account.
- matheusmoreira 2mo ago
  My account seems to be stuck in some kind of inconsistent state. I opened the Persona verification, closed the tab because I didn't have my documents on me at the time, and then I could never start the Persona verification ever again. Apparently I got rejected without even starting the verification. I ended up sending an email to their data protection officer exercising my LGPD (brazilian GDPR) rights as a data subject. It's my right to know the criteria used, request review and correct any information they have stored about me. Hopefully that'll get them to either clear this inconsistency up or reject me properly from TAC. Plus I get to test if my country's laws have any teeth.
rapind 2mo ago
> Video games are now ruined for me. I don't think I will ever feel safe playing online again. Agreed, also WordPress powers around 43% of all websites on the internet. https://patchstack.com/whitepaper/state-of-wordpress-security-in-2026/ https://patchstack.com/whitepaper/state-of-wordpress-securit... WCGW?

drdexebtjl 2mo ago

I was replacing the factory OS from a NAS and asked Sol to do an inventory on the running system over SSH, so that I could replicate any hardware compatibility quirks and whatnot. It reverse engineered a binary daemon that set fan curves and told me how I should set them up in the new OS. I didn’t ask for this, and I didn’t have reverse engineering tools installed. It just figured out it could run them using Nix. The most worrying part, to me, is that it did it like it was nothing. It simply said “usr/local/bin/some-daemon sets the following fan curves”. I had to ask how it reached that conclusion for it to tell me casually that it had just read it straight from the x86_64 assembly. No access to the source code is no longer a meaningful obstacle to these models.

tonyarkles 2mo ago
Haven't tried with Sol but Opus's ability to reverse engineer both Linux and naked microcontroller firmware has blown me away. I'm basically at the point where embedded devices doing something weird get Claude Code + radare2 + Ghidra pointed at them right away. "When I do X, Y happens. Starting at the input layer, work through this firmware and figure out what circumstances would cause command X to do Y instead of Z. Keep notes on the structure of the firmware so that we can ask more questions without having to start reverse engineering from scratch."

TacticalCoder 2mo ago

> Although I think most vulnerabilities are going to be closed in popular software by mid 2027, ... If we're to believe how good those AI are at CTF and at escapes of all kind, then the only logical conclusion is that, by very far, most vulnerabilities were already closed, even before AI. Otherwise we'd already be in deep shit since months if not years.

deleted 2mo ago

[deleted]

arcfour 2mo ago

> most vulnerabilities are going to be closed in popular software by mid 2027 I think that's a bit hopeful. I think that as long as imperfect humans continue to exist and we continue to train AI models with data from these imperfect humans that vulnerabilities will always exist.