Unless the AI was formed under a selection mechanism that rewarded something like kin preference, there's no reason to think it would care. The orthogonality p

Unless the AI was formed under a selection mechanism that rewarded something like kin preference, there's no reason to think it would care. The orthogonality principle is that you could have any level of intelligence furthering any particular goal/utility function; there isn't some level of computational complexity where e.g. a paperclip maximiser's utility function magically changes. In the particular case of humans, our utility function(s) is(are) so complex that what we think of as our 'true values' can change (because our True values are some inscrutable tug-of-war between all the parts of the brain), and also we're hacked together by evolution, so just blindly trying to make a human smarter might change their values (or turn them mad, or cause seizures, or...). But this doesn't have to be the case in general for intelligent agents. In principle, you can build an AI whose terminal values remain stable as it improves its intelligence. (If this is impossible, we're doomed.) So unless you explicitly built an AI to care about all humans, there's no reason to think it magically would.