Right, so ideally I would like to keep something in a safe deposit box (or similar), but the point is I don't want to keep going back and forth to the bank ever

Right, so ideally I would like to keep something in a safe deposit box (or similar), but the point is I don't want to keep going back and forth to the bank every time I sign up for a new account with 2fa. That's why I want one long-lived secret that I can put there, and have the TOTP secrets encrypted with that one.