My favorite physical interpretation (discrete version): How many extra bits per character your data compression algorithm would need to store text from distrib

My favorite physical interpretation (discrete version): How many extra bits per character your data compression algorithm would need to store text from distribution P if it (mistakenly) assumed it were drawn from Q. That is, reserve the shortest “words” for the most common characters based on the assumption that the data will be drawn from Q. Then KL(P||Q) is how much bigger the compressed data will be (per input character) if the data is actually drawn from P.