> If not, you just burned the vulns to that inference provider's training data (and any intermediary), and future benchmarks will be meaningless. Even assuming

> If not, you just burned the vulns to that inference provider's training data (and any intermediary), and future benchmarks will be meaningless. Even assuming that the provider necessarily trains on what it’s used on, how does “Find all vulnerabilities in this code: <code>” and checking that it contains all the expected vulnerabilities magically make future models aware of what the expected vulnerabilities were?