I'd like to take this opportunity to encourage people to learn about sampling profilers. What they do is periodically record the program counter and call stack

I'd like to take this opportunity to encourage people to learn about sampling profilers. What they do is periodically record the program counter and call stack of a process, then aggregate these to get a picture of where the program was spending its time. With a low enough sampling frequency they can have negligible impact on the speed of the software being profiled, they require no changes to production code, and they operate with fine enough granularity that they can easily tell you if (for example) a particular polymorphic function call is adding significantly to the runtime of a hot loop. Often all you need to do to find out what the real bottlenecks are is to point one of these profilers at a running program and look through the results. Super low effort, potentially big payoff.