As far as I know, you can't trivially decrypt traffic as a passive observer, even if you have the private key of the website or the CA. The sides still perform

As far as I know, you can't trivially decrypt traffic as a passive observer, even if you have the private key of the website or the CA. The sides still perform key exchange using something like Diffie-Hellman. The asymmetric keys are used to verify that you're performing the key exchange with the real server and not with a MITM. If you have the CA's private you must sign your own fake certificate and MITM the connection with the fake certificate. In this case, as you pointed out, Certificate Transparency does help because the fake certificate won't be present in the certificate logs.