3 ms·

I think I have settled for ML = gets better with more examples ("learns"). So I'd say Bayesian models of all flavours are definitely in the ML umbrella. AI is s

I think I have settled for ML = gets better with more examples ("learns"). So I'd say Bayesian models of all flavours are definitely in the ML umbrella. AI is super fuzzy though. I think of that as, does something you'd think a computer shouldn't be able to do but a human can, and is thus a forever moving goalpost.

disgruntledphd2 7y ago
I probably wouldn't use that definition, as all statistical techniques will improve as the number of examples increase (i.e. the estimate will be more precise). If you mean that ML normally estimates more parameters, and as such improves more with more examples, I would agree, but it's very difficult to draw a dividing line then (what about splines - loads of parameters, very flexible, but not normally classified as an ML technique).
- gbrown 7y ago
  Depending on how you look at "improvement", that's not strictly true. Often, large data sources are slightly biased relative to the population we want to generalize to (say, a single regional company's customers, trying to generalize to a national population). Working with larger and larger such data sets, that bias can become large relative to standard errors/credible interval width. Xiao-Li Meng has had some interesting talks/papers about this and related ideas: https://dash.harvard.edu/handle/1/10886849 https://dash.harvard.edu/handle/1/10886849
  - ska 7y ago
    That applies to ML techniques also, mutatis mutandis.
  - disgruntledphd2 7y ago
    I was never convinced by that paper, to be honest. I almost always use MLE so that's not really an issue. It just struck me as a pedantic distinction without a difference. But if you use GEE, it's probably great to know.
- salty_biscuits 7y ago
  Splines are totally ML if gaussian processes or markov random fields are ML... Also, not saying that I agree with the definition or any sensible cut between statistics and ML, just my mental model for interpretation of what people are talking about when they talk about ML

disgruntledphd2 7y ago

I probably wouldn't use that definition, as all statistical techniques will improve as the number of examples increase (i.e. the estimate will be more precise). If you mean that ML normally estimates more parameters, and as such improves more with more examples, I would agree, but it's very difficult to draw a dividing line then (what about splines - loads of parameters, very flexible, but not normally classified as an ML technique).

gbrown 7y ago
Depending on how you look at "improvement", that's not strictly true. Often, large data sources are slightly biased relative to the population we want to generalize to (say, a single regional company's customers, trying to generalize to a national population). Working with larger and larger such data sets, that bias can become large relative to standard errors/credible interval width. Xiao-Li Meng has had some interesting talks/papers about this and related ideas: https://dash.harvard.edu/handle/1/10886849 https://dash.harvard.edu/handle/1/10886849
- ska 7y ago
  That applies to ML techniques also, mutatis mutandis.
- disgruntledphd2 7y ago
  I was never convinced by that paper, to be honest. I almost always use MLE so that's not really an issue. It just struck me as a pedantic distinction without a difference. But if you use GEE, it's probably great to know.
salty_biscuits 7y ago
Splines are totally ML if gaussian processes or markov random fields are ML... Also, not saying that I agree with the definition or any sensible cut between statistics and ML, just my mental model for interpretation of what people are talking about when they talk about ML