I've also noticed the opposite problem: Sometimes the LLM, when asked a detailed question (probably with some lead-in), pushes back in a way that betrays that t

I've also noticed the opposite problem: Sometimes the LLM, when asked a detailed question (probably with some lead-in), pushes back in a way that betrays that they fell back to general tropes without really considering the nuances of your specific context. This happens many times, and I usually have to lead the LLM through a chain of reasoning to prove to it that its objection, through generally sound, do not apply to my specific situation. Someone not as well versed in the subject matter would think the LLM found a smoking gun (which they love to do), and be led on a wild goose chase.