That might be a valid thing to defend against by using specific algorithms that protect against it, but that's distinct from what you wrote about it being easy

That might be a valid thing to defend against by using specific algorithms that protect against it, but that's distinct from what you wrote about it being easy to forget to implement a crucial part of it. Everything in cryptography is easy to get wrong, that's why we use tested libraries. But if there is new advice on protecting against attacks like spectre, rowhammer, and others that exploit hardware problems, I'd be interested to learn more. If you have a blog or paper where someone explores the fragility of different algorithms in that regard, let me know!