2 ms·

If the author is here: thanks for that, interesting read, and also nice to note the absence of a marketing name for the bugs. If you have time, couple (edit: th

If the author is here: thanks for that, interesting read, and also nice to note the absence of a marketing name for the bugs. If you have time, couple (edit: three) questions: 1. Could you expand on this? "That human-in-the-loop step still matters a lot, because AI candidate findings are cheap while trustworthy reports are not." Roughly how many candidate reports did the LLMs create vs the eventual 7 true vulnerabilities? 2. As I was reading "CP-ABE access-control break via AND-share bug" I thought "why wasn't this caught with a test?", which was going to be my question but clicking through to the commit (thanks for that too btw) I see there was a regression test added: https://github.com/cloudflare/circl/commit/def2fd35b8535b0b8fe84f904936ebfd84b5552b#diff-532d7feadfcd42eec36535b59c53abf291227a6507a43d0124727741eb73126d https://github.com/cloudflare/circl/commit/def2fd35b8535b0b8... but I'm wondering why there isn't a test further up the stack that is simply checking "can't decrypt if the required attribute isn't present"? Seems similar to those situations where nobody thought to test an auth system for "user can't log in when they present the wrong password"!! Perhaps I'm missing some subtlety though. 3. This is probably a dumb question, but I wasn't sure (even after reading the linked article on zkao) exactly what zkao actually is. One description seems to be "a system for continuously running an LLM audit pass on a codebase". But that can't be right because this article talks about running it on the LLM-found vulnerability reports. Is it an LLM? (but better than the frontier LLMs?) Anyway, bit confused and would appreciate some clarity.

baby 3mo ago
I work on www.zkao.io so I can expand on this: it's basically like an audit, you click a button and ~9h later you get a report with findings. It's supposed to be better at finding bugs (especially cryptographic bugs) than frontier models AND it's supposed to be better at discarding false positives.
wbl 3mo ago
The ability to decrypt depends on figuring out that the sharing scheme has your pieces covered, and knowing that you have in fact covered the right thing because of a bug like this is hard to do. There are some ways to test the sharing scheme that might have caught this, and looking at example shares carefully should have, but it's a bit more subtle to make generic tests.
- dboreham 3mo ago
  Ok but my understanding from the article was that the bug is "it always allows what shouldn't be allowed", therefore any negative outcome test should have failed (or at least a very very simple one). Again I'm sure I'm misunderstanding something about the context here.

baby 3mo ago

I work on www.zkao.io so I can expand on this: it's basically like an audit, you click a button and ~9h later you get a report with findings. It's supposed to be better at finding bugs (especially cryptographic bugs) than frontier models AND it's supposed to be better at discarding false positives.

wbl 3mo ago

The ability to decrypt depends on figuring out that the sharing scheme has your pieces covered, and knowing that you have in fact covered the right thing because of a bug like this is hard to do. There are some ways to test the sharing scheme that might have caught this, and looking at example shares carefully should have, but it's a bit more subtle to make generic tests.

dboreham 3mo ago
Ok but my understanding from the article was that the bug is "it always allows what shouldn't be allowed", therefore any negative outcome test should have failed (or at least a very very simple one). Again I'm sure I'm misunderstanding something about the context here.