One tweak to this can be to segregate karma points by category (or sub-reddit). That way, any downvote points that you need will need to be "earned" in the same

One tweak to this can be to segregate karma points by category (or sub-reddit). That way, any downvote points that you need will need to be "earned" in the same topic.