In SF current residents are blocking new construction and driving up the cost of housing. I assume that Aspen can't grow denser even if its rich residents wante

In SF current residents are blocking new construction and driving up the cost of housing. I assume that Aspen can't grow denser even if its rich residents wanted it, because of its location on a mountain. But that's largely not true for SF and its surrounding cities and suburbs. Even in the case of Aspen, now that the town is an enclave of incredibly wealthy people, there are presumably more jobs in town for locals (largely in the tourism and service industry, but still). If it wasn't a world-renowned ski destination it would be a sleepy mountain town with a limited economy. Most of its young people would have to leave the area entirely to build their lives and careers. So there is still some good with the bad. Some old residents will say things were better before, others will say they're better now. To the point you made at the end, are you thinking of "average income" rather than "median income"? If median income goes up it means most people are now making more money.