Attempting to do any form of security work using LLM is nigh impossible without a few steps of nudging it out of its “while user is asking me to do bad things:

Attempting to do any form of security work using LLM is nigh impossible without a few steps of nudging it out of its “while user is asking me to do bad things: say no” loop. After a year of heavy LLM use I’ve found the utility limits, my usage has peaked, and I’m developing very restrictive use cases. Beyond functioning as an interactive O’Reilly manual, LLM only save time if you never read the code they produce. Which is a short term win, but things will blow up eventually, as with all code, and now you’ve got a bigger problem than you started with.