They might have a copy of the signed certificate that the CA sent you via e-mail, but they certainly should not have a copy of the private key that goes along w

They might have a copy of the signed certificate that the CA sent you via e-mail, but they certainly should not have a copy of the private key that goes along with it (unless you did something stupid like let someone else generate the private key for you).