At least in Let's Encrypt's case, I thought the free SSL cert was handled as a normal CSR request; the private key should never leave the server the certificate

At least in Let's Encrypt's case, I thought the free SSL cert was handled as a normal CSR request; the private key should never leave the server the certificate is issued for, assuming the client performing the CSR isn't compromised in some way. All Let's Encrypt's validator service should get is a challenge signed by the private key, not the key itself. A CA compromise is still very bad, but mostly because it allows an attacker to issue valid certificates and set themselves up as a MitM more easily. Let's Encrypt is still subject to this kind of attack, but so is every other CA; it's not a new threat. That can be reasonably worked around with certificate pinning and other techniques.