But you don't invite them to your house. What makes you believe that somehow you have more right to land that doesn't directly belong to you than another human

But you don't invite them to your house. What makes you believe that somehow you have more right to land that doesn't directly belong to you than another human being from Earth? America is a big country with plenty of land where many more people could live not bothering each other.