In 2012, I was doing NLP stuff with the Stanford NLP POS tagger. They claimed it was highly optimized, and hey, who was I to doubt them? But man, it sure _felt_

In 2012, I was doing NLP stuff with the Stanford NLP POS tagger. They claimed it was highly optimized, and hey, who was I to doubt them? But man, it sure _felt_ slow. So I did some _very minimal_ profiling and, as a complete outsider, found small, simple changes that increased throughput by 4x without affecting maintainability. I think this fits your "low N" example -- I'm much more likely to believe something is optimal if you can tell me that multiple (experienced!) people have tried to optimize something and failed. If it's just 1 person, that 1 person better have amazing domain expertise for me to believe it.