4 ms·

It's disheartening to see that the content of this isn't just "well, duh!" and that some people actually need to be convinced. That said, this article is weird

It's disheartening to see that the content of this isn't just "well, duh!" and that some people actually need to be convinced. That said, this article is weird ("theoretical probability" vs. "experimental probability"... huh?) and is not exactly something I would want to share around. It almost seems like the author wrote it for their own benefit, after themselves recently realizing that probability is important and useful/essential in data science, and wanting to share what they learned. I'm not sure it's worth anyone's time to read it.

n4r9 4y ago
> "theoretical probability" vs. "experimental probability"... huh? That raised my eyebrows as well. It looks like it's a Thing They Teach In Schools These Days: https://www.statisticshowto.com/theoretical-probability/ https://www.statisticshowto.com/theoretical-probability/ My guess is they're trying to get kids to think about inductive Vs deductive reasoning. And it may set a foundation for understanding the difference between a posterior and a priori distributions. But it comes across as if they're a different type of object, which feels a little off.
- epgui 4y ago
  I'm not sure if this is related, but I believe there are very deep open questions in probability (eg.: the interpretation of probabilities based on counter-factuals) and it's what came to mind: https://en.wikipedia.org/wiki/Probability_interpretations https://en.wikipedia.org/wiki/Probability_interpretations
- AstixAndBelix 4y ago
  Wait, isn't "experimental probability" literally what statistics is?
  - n4r9 4y ago
    In some sense statistics is like "applied" probability, which does sound a bit like "experimental" probability but that's not what I understand the linked article to mean. My understanding is: I flip a coin twice. It lands heads-up both times. Then the experimental probability of this coin landing heads-up is 1. You give me a coin which you guarantee has a 50/50 chance of landing heads-up. The theoretical probability of it landing heads-up twice is 1/4. Either type of probability can be used in statistics.
    - AstixAndBelix 4y ago
      Dunno, at my college course we were taught that probability is about knowing the probabilistic properties of an experiment and predicting its outcomes, whereas statistic is about knowing the outcomes and predicting the probabilistic properties of the experiment
      - kqr 4y ago
        This is close to correct. Statistics (the old school kind) is just about computing summary metrics for large amounts of data. Probability is about deriving properties of a sample from the population, based on population parameters. Statistical inference (what is usually meant by "statistics" these days) is about figuring out what sort of population parameters could have produced the sample at hand -- i.e. probability but backwards.
- mhh__ 4y ago
  Subjectivity in probability theory goes back quite a long time.
hcks 4y ago
Well I indeed expected more than a few basic definitions for sure. For instance I was expecting a few case studies where a blind application of sklearn defaults led to issues fixed with proper probabilistic thinking. I remain unconvinced about the importance of probabilities for data science, even more so when I doubt 1% of practicing data scientists could explain what a measure is.
qsort 4y ago
> this article is weird Yes, it's very weird. I can't tell if the author can't express themselves well enough in English, doesn't understand the topic, or something else happened (editorialization, SEO garbage...) > Likelihood is applied when you want to increase the chances of a specific event or outcome occurring. ??? > if 12 cars go down a particular road at 11 am every day, we can use Poisson distribution to figure out how many cars go down that road at 11 am in a month. ??? > if we’re looking at how many chocolate bars were sold in a day, we would use the normal distribution. However, if we want to look into how many were sold in a specific hour, we will use t-distribution ???
- isoprophlex 4y ago
  I too suffered from a very high "wtf per minute" rate when reading this article. GPTZero thinks it's written by a human, for what its worth... i guess the author is a bit confused or unable to clearly articulate ideas.
- parpfish 4y ago
  I suffered whiplash for how quickly I went from "awesome, this person is going to explain some nuanced distinction I've never considered!" to "i don't think this makes any sense"
bitL 4y ago
The intuitive probability we have built-in in our brains doesn't match the mathematical concept of probability, leading to certain paradoxical outcomes that don't make sense in regular language but are perfectly normal for trained minds. This pops up when data scientists discuss their findings with untrained managers all the time. The longer a mind is used to the mathematical notion of probability, the less it remembers early intuitive notion and loses the contact with regular folks and their perception.
keithalewis 4y ago
If only there were people who lived before us who put some ink on dead trees and made an effort to think clearly about probability. We could just read what smart people figured out. Let's start a new field called Human Learning!

n4r9 4y ago

> "theoretical probability" vs. "experimental probability"... huh? That raised my eyebrows as well. It looks like it's a Thing They Teach In Schools These Days: https://www.statisticshowto.com/theoretical-probability/ https://www.statisticshowto.com/theoretical-probability/ My guess is they're trying to get kids to think about inductive Vs deductive reasoning. And it may set a foundation for understanding the difference between a posterior and a priori distributions. But it comes across as if they're a different type of object, which feels a little off.

epgui 4y ago
I'm not sure if this is related, but I believe there are very deep open questions in probability (eg.: the interpretation of probabilities based on counter-factuals) and it's what came to mind: https://en.wikipedia.org/wiki/Probability_interpretations https://en.wikipedia.org/wiki/Probability_interpretations
AstixAndBelix 4y ago
Wait, isn't "experimental probability" literally what statistics is?
- n4r9 4y ago
  In some sense statistics is like "applied" probability, which does sound a bit like "experimental" probability but that's not what I understand the linked article to mean. My understanding is: I flip a coin twice. It lands heads-up both times. Then the experimental probability of this coin landing heads-up is 1. You give me a coin which you guarantee has a 50/50 chance of landing heads-up. The theoretical probability of it landing heads-up twice is 1/4. Either type of probability can be used in statistics.
  - AstixAndBelix 4y ago
    Dunno, at my college course we were taught that probability is about knowing the probabilistic properties of an experiment and predicting its outcomes, whereas statistic is about knowing the outcomes and predicting the probabilistic properties of the experiment
    - kqr 4y ago
      This is close to correct. Statistics (the old school kind) is just about computing summary metrics for large amounts of data. Probability is about deriving properties of a sample from the population, based on population parameters. Statistical inference (what is usually meant by "statistics" these days) is about figuring out what sort of population parameters could have produced the sample at hand -- i.e. probability but backwards.
mhh__ 4y ago
Subjectivity in probability theory goes back quite a long time.

hcks 4y ago

Well I indeed expected more than a few basic definitions for sure. For instance I was expecting a few case studies where a blind application of sklearn defaults led to issues fixed with proper probabilistic thinking. I remain unconvinced about the importance of probabilities for data science, even more so when I doubt 1% of practicing data scientists could explain what a measure is.

qsort 4y ago

> this article is weird Yes, it's very weird. I can't tell if the author can't express themselves well enough in English, doesn't understand the topic, or something else happened (editorialization, SEO garbage...) > Likelihood is applied when you want to increase the chances of a specific event or outcome occurring. ??? > if 12 cars go down a particular road at 11 am every day, we can use Poisson distribution to figure out how many cars go down that road at 11 am in a month. ??? > if we’re looking at how many chocolate bars were sold in a day, we would use the normal distribution. However, if we want to look into how many were sold in a specific hour, we will use t-distribution ???

isoprophlex 4y ago
I too suffered from a very high "wtf per minute" rate when reading this article. GPTZero thinks it's written by a human, for what its worth... i guess the author is a bit confused or unable to clearly articulate ideas.
parpfish 4y ago
I suffered whiplash for how quickly I went from "awesome, this person is going to explain some nuanced distinction I've never considered!" to "i don't think this makes any sense"

bitL 4y ago

The intuitive probability we have built-in in our brains doesn't match the mathematical concept of probability, leading to certain paradoxical outcomes that don't make sense in regular language but are perfectly normal for trained minds. This pops up when data scientists discuss their findings with untrained managers all the time. The longer a mind is used to the mathematical notion of probability, the less it remembers early intuitive notion and loses the contact with regular folks and their perception.

keithalewis 4y ago

If only there were people who lived before us who put some ink on dead trees and made an effort to think clearly about probability. We could just read what smart people figured out. Let's start a new field called Human Learning!