3 ms·
I presume you have a regression background. If so: "minimizing entropy" is the same paradigm as "minimizing the sum of squares", in a different context. If not
by tomrod 9y ago
I presume you have a regression background. If so: "minimizing entropy" is the same paradigm as "minimizing the sum of squares", in a different context.
If not: reducing entropy means finding weights/coefficients in a supplied functional form that minimize some objective function applied to the problem.
Usually the jargon applies to Shannon entropy from signal theory, or some derivation thereof like transfer entropy.
Entropic estimates take a form similar to
$$ -\sum(j) {p(x_j) log(p(x_j))}$$
where j is the event space (e.g. heads or tails on a coin flip).