2 ms·
Hey author, would you mind explaining the conversion from compression length to probability please? Namely this line: scipy.special.log_softmax(-code_lengthsse
by usgroup 3y ago
Hey author, would you mind explaining the conversion from compression length to probability please? Namely this line:
scipy.special.log_softmax(-code_lengthsself.conversion(1/temperature))
Your codes are K-ary but this doesn't look like its taken into account ala the README. What is the log(256) conversion factor? What is 1/temperature for?
- canjobear 3y agoI’m measuring the length of the gzipped string in bytes, so K=256. The temperature parameter is there in case anyone wants to play around with it.
- usgroup 3y agoFor anyone else wondering how this works out: 1. You want: p(x) ~ K^(-|x|), where K=256. 2. log p(x) ~ log K^(-|x|) = -|x|log K 3. he is using log(softmax) ~ log(e^x) 4. and log(e^(-|x|log(K))) = -|x|*log K as required.