Why should people born outside the imaginary lines of my country be allowed to come inside it? The power structure resides inside the described imaginary lines

Why should people born outside the imaginary lines of my country be allowed to come inside it? The power structure resides inside the described imaginary lines, thus the outsiders need to make this argument to the insiders, not the opposite way, realistically. If the outsiders can't convince the insiders, they simply won't be let in. With that framing, it is easy to see how it isn't (necessarily) bigotry, but just that the outsiders appear to offer little or negative value to many people. By saying "They are taking our jobs", people are expressing that they believe the outsiders being let has a negative effect on them, thus it is quite rational that they want to strengthen those imaginary lines.