I think it depends on the carrier and the adversary. For example, a government could ask the carrier to give them a copy of traffic to your number, or to cancel

I think it depends on the carrier and the adversary. For example, a government could ask the carrier to give them a copy of traffic to your number, or to cancel your number, or to let them send data as you. This is already "practical" if you have a capable enough adversary; for example, there have been reports of attackers subverting SMS-based 2FA. https://www.google.com/search?q=attacks+on+sms+2fa https://www.google.com/search?q=attacks+on+sms+2fa This could be based on cooperation of the carrier, or perhaps SS7 attacks, or IMSI catchers.