WoT and CA systems are both problematic since they can be altered on the fly and thus 'hijacked'. I wonder if we wouldn't be better of with something similar t

WoT and CA systems are both problematic since they can be altered on the fly and thus 'hijacked'. I wonder if we wouldn't be better of with something similar to what SSH does. Accept trust the first time and verify that the signature doesn't change on every subsequent connection attempt. This way one would be immune to hijacks. It wouldn't solve first time verification, but how likely is a first time spoof? And for really sensitive communications you could use pre-shared keys. I could for instance get a hardware token from my bank containing their public key.