I think a major part of the issue is the psychological need for moral consistency. The Holy Madman donates almost all of his money and time to helping those in

I think a major part of the issue is the psychological need for moral consistency. The Holy Madman donates almost all of his money and time to helping those in the most need. Almost all people would hate to be a Holy Madman - he is a bound slave forced by moral law to voluntarily make his quality of life hell forever for as long as there are people in a worse kind of hell. If you accept opting into charity to a moderate degree out of genuine moral concern (as opposed to status seeking or genuine enjoyment), then usually you also ought indulge more fully.