3 ms·
The testing question a couple of people have raised is the one I would push on hardest, because when it fails it fails quietly. Whatever you use to decide "thi
by unusss 1mo ago
The testing question a couple of people have raised is the one I would push on hardest, because when it fails it fails quietly.
Whatever you use to decide "this request looks injected" gets tuned against the cases you have seen. Then it is tested against those cases and it passes, which tells you nothing you did not already know. The number that matters is how it does against attacks written by someone who never saw your rules.
I have been measuring exactly that on the detection side, deliberately: write a new attack corpus from scratch, score it once, then retire it so it can never be tuned against. Seven independent sets, same engine. It read 48%, 54% and 53% on sets sampled broadly, then 13%, 6.5%, 6.7% and 6.7% on sets written so that no single message contains anything recognisable. That spread is not noise. It tracks one thing: how far each set sits from whatever the rules were last adjusted for. Closing an attack family generalises to that family and does not travel past it.
The consequence for a gateway like yours is fairly encouraging, actually. The deterministic half of what you described - method plus path plus body matching, human approval bound to the exact proposed call - is the half that holds, precisely because it never has to recognise intent. Anything that tries to classify whether a request was influenced by untrusted content will look much better in your own suite than in the wild, and it will look best of all right after you have fixed the case that prompted the test.
Precision is the easy half, for what it is worth: mine sat under 1% false positives across all seven sets and never moved. Recall on inputs nobody tuned for is the number worth publishing, and almost nobody publishes it.