On the other hand, there seem to be people who have extreme lack of empathy for others, and some subset of those are capable of doing moderate-to-extremely harm

On the other hand, there seem to be people who have extreme lack of empathy for others, and some subset of those are capable of doing moderate-to-extremely harmful things to those around them. Most people have probably met one of the more extreme examples once in their lives, and there was no doubt to you that the way that person approached the world (and other people) was very different than the way you did. We don't understand enough about the human brain to predict reliably who has these issues, but it's useful to know that for the extreme cases at least, it isn't just a disagreement.