What gets transmitted during key exchange is not the final symmetric key, but a kind of work-in-progress that both parties finish off to generate the final key,

What gets transmitted during key exchange is not the final symmetric key, but a kind of work-in-progress that both parties finish off to generate the final key, either by password, PK or any other method. No secure info is ever transmitted.