As an interested layman I'd have thought, the higher, the better, because less friction. Now I'm wondering why. Because of lower outside air temperature, making

As an interested layman I'd have thought, the higher, the better, because less friction. Now I'm wondering why. Because of lower outside air temperature, making some material expand/shrink too fast? Now I'm puzzled. Does this mean it can't go (high) supersonic at lower altitudes either? Because denser air causing even more friction? I mean something like Mach 2 at 1000 feet. Mission impossible?