First of all the difficulty doesn't need to be set perfectly. It just can't be way too high or way too low. Even if the algorithm is slightly off the optimal di

First of all the difficulty doesn't need to be set perfectly. It just can't be way too high or way too low. Even if the algorithm is slightly off the optimal difficulty everything still works fine. Secondly, because other nodes need to validate the submitted blocks for the miners get any reward they are strongly incentivized to fill out all the information in the block as correctly as possible, including the time. Imagine mining a block and getting a 12.5 BTC reward (currently valued at around $100,000) and having it get rejected because you put the wrong timestamps -- ouch. I don't know what logic they use to see if the timestamp is valid, but even if there was a bug in this logic that was used for nefarious purposes other peers would wise up and patch the software if it hurt anyone in any way. So yes, the difficulty calculation may not be perfect and could even be subject to manipulation but it seems good enough for now to me.