Sure, but you can just check the high precision wall-clock timer at the end of your computation and makes sure you took at least X nanoseconds, and pad that out

Sure, but you can just check the high precision wall-clock timer at the end of your computation and makes sure you took at least X nanoseconds, and pad that out so that X is always greater than the amount of wall-clock nanoseconds the actual computation takes. Then, following a computation, you sleep until X. While this won't fool timing attacks that are operating on the same machine as your process, the computation time becomes completely opaque to the network which is what the "remote" attacks are built on.