You are only trusting the half of the security which is better IMHO. The evil person would need to hack/steal both you and the (likely more careful) third party

You are only trusting the half of the security which is better IMHO. The evil person would need to hack/steal both you and the (likely more careful) third party in order to access your accounts. In your scenario it would only need to hack you to get everything.