TPM 2.0 Extended Access policies can be very complex. You can make it so that in order to unlock key objects in the TPM you have to have more than matching PCR

TPM 2.0 Extended Access policies can be very complex. You can make it so that in order to unlock key objects in the TPM you have to have more than matching PCRs, but also a PIN, and the software on the host shouldn't provide the PIN if the PCRs don't match what it expects. Of course, TPM-using software is still too simple in this regard so that you're correct right now, but it doesn't have to be that way.