I’ve tried (and abandoned) a similar question suite for Security SWEs. Giving people sample code riddled with security flaws and asking them to explain what was

I’ve tried (and abandoned) a similar question suite for Security SWEs. Giving people sample code riddled with security flaws and asking them to explain what was wrong. I found it difficult in the security domain since so much of it is context dependent (what’s the interface? Is this code in the kernel? Are you concerned about speculative side channels?). I also found it difficult to calibrate. There’s probably a good question space in there if I were to spend time testing the questions more, but there are enough other questions in the neighborhood that I’ve moved on.