The confusion comes from what posesses the information, you or the object. If a distribution is spread out, I know very little about it. If I learn something a

The confusion comes from what posesses the information, you or the object. If a distribution is spread out, I know very little about it. If I learn something about it, and make it less spread out, then I have more information about it, and there is less information left in it. Learning about something is taking information out of it. So something has a lot of information if you know nothing about it. If you know a lot about it, you've taken all the information out of it, and there's nothing left.