That's easy to do for a new line but pretty much impossible for existing lines, because the clearances are not tall enough to fit a normal double decker train w

That's easy to do for a new line but pretty much impossible for existing lines, because the clearances are not tall enough to fit a normal double decker train where the first level is at existing platform height. (In the current ones, you step down to the first level.) It would be hard to retrofit tunnel heights and bridges after the fact. You also run into the problem where these special trains are only permitted on parts of the network and stations that can handle them, which severely limits their utilization. It's why even basic iterations of technologies requiring modification are often market failures, like the A380 requiring special gates to handle it. Boutique technologies like monorail or maglev fail even harder. For most cases it is profitable to use the less optimal but more flexible solution.