Yep, we're in agreement. I'll concede that if you replace crappy housing with nicer housing (and don't increase the number of units overall) some of the demand

Yep, we're in agreement. I'll concede that if you replace crappy housing with nicer housing (and don't increase the number of units overall) some of the demand that wouldn't consider it before now will so the market price goes up (and the market price of other fancier housing likely reduces a bit). This is the crux of the problem. There is just so much pent up housing demand that the market is totally out-of-kilter and below-average units get replaced with above-average units. And even if that nudges the average down marginally, the residents of the below-average units are now priced out of town.