"That is always allowed." The last sentence you wrote is the killer. You are technically correct that changing the number of circuits in the panel is not itse

"That is always allowed." The last sentence you wrote is the killer. You are technically correct that changing the number of circuits in the panel is not itself disallowed. But .... you would also be hard pressed to get an AHJ to pass a number-of-circuits upgrade without knowing how many circuits you are planning on adding, and doing the feed sizing calculation to make sure you don't have to upgrade the incoming feed to support it.