2 ms·
So I went on a bit of a rabbit hole on this one. The above code is wrong. I have since fixed it. It's the same idea with some added terms to normalize for da
by willis936 7y ago
So I went on a bit of a rabbit hole on this one. The above code is wrong. I have since fixed it. It's the same idea with some added terms to normalize for data size.
Word size matters when calculating entropy. Calculating entropy on this file using 2 to 64 bits nets a minimum information of 860 bits using a 64 bit word size. Moving past a 64 bit word size requires rethinking some things. MATLAB doesn't have great support for data sizes greater than 64 bits (and I'm certain the performance impact would be great).
words = double(unique(X))';
binEdges = [words-0.5; words+0.5];
binEdges = unique(binEdges(:));
P = histcounts(X, binEdges) / M;
P(P == 0) = [];
M = numel(X);
H = -sum(P .* log(P) ./ log(max([nBits, 2]))) / nBits * log2(max([nBits, 2]));
E = H * M * nBits;