And then you might just end up using setcap on it anyhow, perhaps to load privileged ports, which is treated like suid, making it so that the linker won't look

And then you might just end up using setcap on it anyhow, perhaps to load privileged ports, which is treated like suid, making it so that the linker won't look at LD_LIBRARY_PATH, in which case you have to create a conf file with your paths and feed it to ldconfig after using ldd on the program because you're confused about why it suddenly doesn't load its libraries. And there's another fun security restriction in there where chown erases any setcap permissions you had applied, just in case you had to adjust something and didn't understand why you suddenly lost your setcap permissions.