The problem is that there always is some objection. Building out in a wilderness area will have environmental impact. Building in a farm area will have a huge i

The problem is that there always is some objection. Building out in a wilderness area will have environmental impact. Building in a farm area will have a huge impact on the local, formerly rural community. Building in a former industrial area might have cleanup issues. Projects don't have "enough" affordability, or are too dense, or don't match the local style, or will require more investment in public services. There are many impacts. But collectively if all taken seriously, it means we can't build anything. Or the costs to build just get really high. And then we don't have enough housing and the housing we do have is either very costly or in undesirable areas. Something must give, and it really might impose costs on locals. But the alternatives ensure housing scarcity and its many ills.