4 ms·
>A distribution over strings has entropy. You have to limit yourself to each string being independent for that to be true which is a limitation that leads to t
by AnotherGoodName 3mo ago
>A distribution over strings has entropy.
You have to limit yourself to each string being independent for that to be true which is a limitation that leads to the impossibility of calculating entropy generically.
- amluto 3mo agoIf you want to get correct answers out of math, it helps to start with correct definitions. A random variable is a random variable and is not the same thing as a stochastic process. You now seem to be talking about stochastic processes, which are a different thing. A stochastic process can have an entropy rate: https://en.wikipedia.org/wiki/Entropy_rate https://en.wikipedia.org/wiki/Entropy_rate If you want to calculate entropy "generically" without defining what kind of entropy you're talking about, I think you are not likely to succeed.