It's relevant that more people in software tend not to understand this either, despite our entire profession being rife with these sorts of problems. The non-l

It's relevant that more people in software tend not to understand this either, despite our entire profession being rife with these sorts of problems. The non-linear scaling of sorting algorithms is a classic one, and even the different optimization points is baked into the "optimal" solution - i.e. quicksort when your number of elements is below a threshold is better replaced with an insertion-sort, and indeed this is what most implementations do. Which is an algorithmic example of solution overhead dwarfing problem size (and yet if you're above that threshold, the more complex solution will always win).