The point is if nobody builds the nicer dwelling, the person with the money that could have afforded the higher price will just turf out a lower-earning person

The point is if nobody builds the nicer dwelling, the person with the money that could have afforded the higher price will just turf out a lower-earning person living in a worse house, because the main feature of the house is that it provides access to the occupant's job.