That's kind of like saying that a backend developer has managed to avoid needing a frontend component frameworks for 20 years, so the need for such a framework

That's kind of like saying that a backend developer has managed to avoid needing a frontend component frameworks for 20 years, so the need for such a framework is rare. It's a self fulfilling prophecy. The OP didn't pick jobs completely randomly, I assume, so they're mostly going to be working around their specific skillset, which may just not require performance critical work. If they worked at Nvidia, Intel, or Epic Games on low level engines, they could flip that around and say they haven't been in a situation where perf wasn't one of their top priorities.