If some people are willing to live in those kinds of houses, it will decrease the cost of living in the kinds of houses you personally want to live in. This is

If some people are willing to live in those kinds of houses, it will decrease the cost of living in the kinds of houses you personally want to live in. This is an allocation problem, not a production problem. If you want or need those things, then building alternatives for people who don't will free up space for people who do want those things. This isn't a zero-sum game.