I didn't see this Sabine video mentioned in the comments. Maybe of interest to someone interested in entropy in physics. "I don't believe the 2nd law of thermo

I didn't see this Sabine video mentioned in the comments. Maybe of interest to someone interested in entropy in physics. "I don't believe the 2nd law of thermodynamics. (The most uplifting video I'll ever make.)" https://m.youtube.com/watch?v=89Mq6gmPo0s https://m.youtube.com/watch?v=89Mq6gmPo0s I come to entropy from Machine Learning, Information Theory and probability. For me it's fairly straightforwad. Of interest and useful - but nothing mysterious there. The p.d.f. aka a fancy histogram is my current best knowledge of how many times an outcome is expected to happen. When I do one experiment - I can't tell the outcome. I can only count the number of different outcomes, without being certain when will any of them appear exactly. A flat p.d.f. means my knowledge is poor: every outcome is about equaly possible. I'm very ignorant (high entropy). A spiky p.d.f. means I have good knowledge: some outcome is much more likely (low entropy). In extremis the p.d.f. is a Dirac impulse - that's deterministic knowledge. The only mildly interesting thing is when a new observation reduces my knowledge. Say right now I'm fairly certain I have not got cancer. My chances are 90:10 for my age. Tomorrow I take a test, the test comes back positive. Of people of my age that test positive, aboout half have cancer for real. After the test my chances are 50:50. Now I am perfectly ignorant whether I have cancer or not. Whereas before I took the test and got a positive result, I was very certain I have not got cancer. The new information (positive test result), transformed my probability of cancer from spiky marginal P_Y(y)={0.9,0.1} to perfectly ignorant conditional P_Y(y|+ve test)={0.5,0.5}. This example is from "How to measure the information gained from one symbol" by DeWeese and Meister (https://pubmed.ncbi.nlm.nih.gov/10695762/ https://pubmed.ncbi.nlm.nih.gov/10695762/).