I think the idea was that if an attacker sophisticated enough to overcome your reasonable defenses protecting your local network and potentially hack your Home

I think the idea was that if an attacker sophisticated enough to overcome your reasonable defenses protecting your local network and potentially hack your Home Assistant installation or whatever else, then they're probably sophisticated enough to do something much more damaging in the long term than unlock your front door and steal your TV or whatever—eg hack your laptop/NAS/etc and clear out your bank account or steal your identity or similar. Though I just realized that I'm privileged enough to live in an area with a low violent crime rate, so physical violence isn't part of my threat model... if someone has broken into your home and you are home at the same time, you're right, the security of your local network is probably the least of your concerns.