4 ms·
If the linear / logistic regression stuff is improved through Bayesian methods and hierarchical modeling, and if it grows large enough that there is a need to u
by mlthoughts2018 7y ago
If the linear / logistic regression stuff is improved through Bayesian methods and hierarchical modeling, and if it grows large enough that there is a need to use MCMC sampling techniques for approximate posterior inference, then even though the model specification itself would be very simple, I’d absolutely say this is “ML”.
- disgruntledphd2 7y agoI really, really wouldn't. That's Andrew Gelman's gig, and he's been doing that since the 80's, when ML/AI was all about expert systems. I think of that as a statistical technique, not an ML one (and for some weird reason, many quantitative people in insurance don't like Bayesian techniques). But then, our disagreement here just highlights the difficulties involved in getting consensus on what ml vs statistics actually are.
- salty_biscuits 7y agoI think I have settled for ML = gets better with more examples ("learns"). So I'd say Bayesian models of all flavours are definitely in the ML umbrella. AI is super fuzzy though. I think of that as, does something you'd think a computer shouldn't be able to do but a human can, and is thus a forever moving goalpost.
- disgruntledphd2 7y agoI probably wouldn't use that definition, as all statistical techniques will improve as the number of examples increase (i.e. the estimate will be more precise). If you mean that ML normally estimates more parameters, and as such improves more with more examples, I would agree, but it's very difficult to draw a dividing line then (what about splines - loads of parameters, very flexible, but not normally classified as an ML technique).
- gbrown 7y agoDepending on how you look at "improvement", that's not strictly true. Often, large data sources are slightly biased relative to the population we want to generalize to (say, a single regional company's customers, trying to generalize to a national population). Working with larger and larger such data sets, that bias can become large relative to standard errors/credible interval width. Xiao-Li Meng has had some interesting talks/papers about this and related ideas: https://dash.harvard.edu/handle/1/10886849 https://dash.harvard.edu/handle/1/10886849
- ska 7y agoThat applies to ML techniques also, mutatis mutandis.
- disgruntledphd2 7y agoI was never convinced by that paper, to be honest. I almost always use MLE so that's not really an issue. It just struck me as a pedantic distinction without a difference. But if you use GEE, it's probably great to know.
- salty_biscuits 7y agoSplines are totally ML if gaussian processes or markov random fields are ML... Also, not saying that I agree with the definition or any sensible cut between statistics and ML, just my mental model for interpretation of what people are talking about when they talk about ML
- sampo 7y ago> I think of that as a statistical technique, not an ML one Many ML techniques are statistical techniques, not ML. If we go down this road.
- disgruntledphd2 7y agoI completely agree, and have been banging this drum for years :)
- ska 7y agoThere is a real risk of venturing into narcissism of small differences with this.
- disgruntledphd2 7y agoLike, personally, I regard statistics and machine learning as the same subject from different perspectives (mathematics vs computer science). Their differences are primarily driven by the context of the time of their development. Back when we had very little data, we needed strong assumptions to make inferences. As compute increased, this became less necessary and we could just bootstrap instead of needing normal theory confidence intervals. But apparently, this is a controversial view.
- bart_spoon 7y ago> I think of that as a statistical technique, not an ML one Perhaps not all, but most of the distinctions between "statistical" techniques and "ML" techniques are fairly arbitrary.
- sgt101 7y agoAnd do we care if they do the job? Especially if they share the same trait of being so complex and dependent on Byzantine treatment of data that they need to be regulated and governed in a different way from something that can be groked via a single ppt slide? edited for the second clause.
- mlthoughts2018 7y agoMachine learning is a superset of all of statistics.
- disgruntledphd2 7y agoI would probably have said the opposite. Can you clarify why you think the scope of ML is greater than that of statistics?
- mlthoughts2018 7y agoRule based systems
- disgruntledphd2 7y agoOK, but I guess I could argue that asymptotic theory and design of experiments are proof that statistics is a super-set of ML (for the record, I'm not sure either of those is true).