Another thing that contributes to the confusion which you have noted but not fully realized is that there are two different concepts that use the same equation

Another thing that contributes to the confusion which you have noted but not fully realized is that there are two different concepts that use the same equation and the same word: "entropy". Information entropy and statistical entropy are two different things.