As an example, usually there is a default C++ implementation because C++ makes things easier for the programmer so it makes sense if you're at the end of the fo

As an example, usually there is a default C++ implementation because C++ makes things easier for the programmer so it makes sense if you're at the end of the food chain that requires perfomance (ie you're not using a memory managed language in the first place because that's easier still). So then you benchmark it thorougly. Get stuck into all the perf numbers and analysing those. You note why you think you can shave X off because the latency is due to i, j and k measures that look as though they could be improved so that's probably worth a go. Now you can (at least sometimes) in your lab, replace one part with (say in this example, C code) and see if your perf stat ratios improve the way you think they should. Yes it is difficult. Yes there are plenty of times where you see nothing because the slow from the code you changed is actually manifesting in cache misses elsewhere. Micro benchmarks sadly have a way of showing nothing much at that end of things. Showing it's no worse/better doesn't even mean it's no worse if the additional/reduced pressure on caches doesn't result in much in the way of additional/reduced cache spills. So you end up doing mirco optimisations that each, taken alone don't do much. But when you get them all together you cross a threshold that has a considerable improvement. I believe SQLite did something like this although probably in a more restrained fashion as they have other goals than being the lowest latency solution against similarly motivated competitors. Then of course at some point you get a reputation for success so you show the problem and the numbers identifying it and just say "I want to invest X of my time trying Y which I believe has a reasonable chance of helping" And you are told to focus on that and please let them know if you have any more good news. ;-) Sometimes your ideas are genuinely a wash and not worth merging but it is noted that it was worth trying and you're thanked for your efforts. As somebody else noted in this thread the game has moved beyond much of this kind of thing in some, possibly many instances.