3 ms·
For anyone else wondering how this works out: 1. You want: p(x) ~ K^(-|x|), where K=256. 2. log p(x) ~ log K^(-|x|) = -|x|log K 3. he is using log(softmax) ~
by usgroup 3y ago
For anyone else wondering how this works out:
1. You want: p(x) ~ K^(-|x|), where K=256.
2. log p(x) ~ log K^(-|x|) = -|x|log K
3. he is using log(softmax) ~ log(e^x)
4. and log(e^(-|x|log(K))) = -|x|*log K as required.