That's what's often done in mission-critical software with critical latency elements. Problem is, you have to make sure those values are directly traced to eith

That's what's often done in mission-critical software with critical latency elements. Problem is, you have to make sure those values are directly traced to either specific customer requirements, whole system spec or normative reference. It's too easy for systems engineers to come up with hard-to-reach numbers with no grounding on the Real customer need... You also have to specify and measure exactly the correct element. Sometimes the latency budget is split in not clearly-cut parts, and on events or categories of events that are not clearly defined. Saying 'all requests should give a complete result in < 20ms' just isn't realistic and creates a world of hurt. What kind of request? Always, always? In what load conditions? What about error conditions? Fail-over conditions? Hot/cold start?... It's not simple like specifying a clear-cut feature. Performance budget should often not be all black and white. Especially when trying to push the envelope of what modern PC hardware (with OoO execution, complex memory hierarchies, multicore CPU/GPU rapid-busy-bused hybrid archs) can do. It should probably be a risk law, and at least a critical spec, and thoroughly tested and regression-tested as such. And here, sometimes premature optimization will be necessary. If someone gives you hard latency requirements, you'll have to say, very quick, 'I'll need a soft/hard real-time OS it won't do extreme high-throughput'. You'll have to bench... Premature /micro/-optimization is a problem, sure. Make it work, make it right, make it fast. But global optimization must start at the system design level. How much money do you have? What can you do for $X?