The problem with the pure "email me a link" approach seems to be that breaking into an email account gets only more valuable: The rightful user can't do anythin

The problem with the pure "email me a link" approach seems to be that breaking into an email account gets only more valuable: The rightful user can't do anything if an attacker gets access to his mail account (or even just his mails), logs into the service secured by "email me a link" and then changes the associated email address for that account. How would the rightful user ever get access to the service again that was secured by "email a link" if there is no additional secret necessary for logging in and authorizing changes to the user data?