Some systems even combine PSK and ASK (like QAM) but asymmetrically (called APSK), obviously though you start getting into diminishing returns; noise limiting t

Some systems even combine PSK and ASK (like QAM) but asymmetrically (called APSK), obviously though you start getting into diminishing returns; noise limiting the gains. Of course you are limited by Shannon's law. But depending on the transmission medium lower energy signals can be more resistant to noise, so you can use either probabilistic or dynamic symbol selection to reduce the effect noise has on the signal and hence increase the available error free bandwidth (error free symbol rate)[0]. [0] https://en.wikipedia.org/wiki/Constellation_shaping https://en.wikipedia.org/wiki/Constellation_shaping ; check the references and external links for more in depth explanations.