3 ms·

This seems like an unnecessary ad hominem. Just because something is factually incorrect doesn’t mean it isn’t true. Surprise is a measure of new Information

by ukj 6y ago

This seems like an unnecessary ad hominem. Just because something is factually incorrect doesn’t mean it isn’t true. Surprise is a measure of new Information (ref: Shannon). It is simply a tautology. To be unsurprised is to provide the answer before the computer produces it.

magicalhippo 6y ago
> This seems like an unnecessary ad hominem. I didn't mean it as such. I just found it wrong, and thus not very interesting, for the reasons I stated. > Surprise is a measure of new Information (ref: Shannon). Ah. Not read Shannon himself, so wasn't aware he used his own special definition of surprise. All the work I've seen has talked about information or entropy. > To be unsurprised is to provide the answer before the computer produces it. I'm still not buying that definition of surprise...
- _jal 6y ago
  > so wasn't aware he used his own special definition of surprise. All the work I've seen has talked about information or entropy. It isn't 'special', and directly relates to the definition of entropy.
  - magicalhippo 6y ago
    Well guess I've either read sources which avoided that word when defining entropy, or I mentally erased it because of the poor fit.
    - ukj 6y ago
      "Poor fit" is an ironic choice of words. https://en.wikipedia.org/wiki/Goodness_of_fit https://en.wikipedia.org/wiki/Goodness_of_fit Fundamentally statistics is Information theory. https://en.wikipedia.org/wiki/Principle_of_maximum_entropy#History https://en.wikipedia.org/wiki/Principle_of_maximum_entropy#H...
- ukj 6y ago
  >Ah. Not read Shannon himself, so wasn't aware he used his own special definition of surprise. You must have missed it then... https://en.wikipedia.org/wiki/Entropy_(information_theory) https://en.wikipedia.org/wiki/Entropy_(information_theory) https://en.wikipedia.org/wiki/Information_content https://en.wikipedia.org/wiki/Information_content https://plus.maths.org/content/information-surprise https://plus.maths.org/content/information-surprise >I'm still not buying that definition of surprise... You don't have to "buy it". I am just telling you that it's the definition used by every person who knows what a bit is. https://en.wikipedia.org/wiki/Bit https://en.wikipedia.org/wiki/Bit

by ukj 6y ago

magicalhippo 6y ago

> This seems like an unnecessary ad hominem. I didn't mean it as such. I just found it wrong, and thus not very interesting, for the reasons I stated. > Surprise is a measure of new Information (ref: Shannon). Ah. Not read Shannon himself, so wasn't aware he used his own special definition of surprise. All the work I've seen has talked about information or entropy. > To be unsurprised is to provide the answer before the computer produces it. I'm still not buying that definition of surprise...

_jal 6y ago
> so wasn't aware he used his own special definition of surprise. All the work I've seen has talked about information or entropy. It isn't 'special', and directly relates to the definition of entropy.
- magicalhippo 6y ago
  Well guess I've either read sources which avoided that word when defining entropy, or I mentally erased it because of the poor fit.
  - ukj 6y ago
    "Poor fit" is an ironic choice of words. https://en.wikipedia.org/wiki/Goodness_of_fit https://en.wikipedia.org/wiki/Goodness_of_fit Fundamentally statistics is Information theory. https://en.wikipedia.org/wiki/Principle_of_maximum_entropy#History https://en.wikipedia.org/wiki/Principle_of_maximum_entropy#H...
ukj 6y ago
>Ah. Not read Shannon himself, so wasn't aware he used his own special definition of surprise. You must have missed it then... https://en.wikipedia.org/wiki/Entropy_(information_theory) https://en.wikipedia.org/wiki/Entropy_(information_theory) https://en.wikipedia.org/wiki/Information_content https://en.wikipedia.org/wiki/Information_content https://plus.maths.org/content/information-surprise https://plus.maths.org/content/information-surprise >I'm still not buying that definition of surprise... You don't have to "buy it". I am just telling you that it's the definition used by every person who knows what a bit is. https://en.wikipedia.org/wiki/Bit https://en.wikipedia.org/wiki/Bit