5 ms·
An Introduction to Model-Based Machine Learning
- vertis 10y agoThe authors initial feelings around ML are similar to how I feel. It's such a broad subject, it feels like you could read/study for years and not cover everything. Worse still it's ever changing.
- amelius 10y agoI'd like to see a high-level overview of the kind of problems that can be solved by the different kinds of ML, and their applications.
- denzil_correa 10y agoThe only material I have seen which does this or comes to this is the Andrew Ng Machine Learning class on Coursera and Academic Earth.
- jimbokun 10y agoTry Pedro Domingos "The Master Algorithm". Good high level overview of the various "schools" of Machine Learning. Not sure if it identifies which problems are best solved by which approach, though. More the history of how they have taken turns as the most successful paradigm.
- chestervonwinch 10y agoI don't know about that. It might appear that way, if you only read news articles on ML since these usually are announcements of the state-of-the-art. However, compare the topics covered in Duda and Hart's text (1st edition, 1973; 2nd edition 1995) to the more recent text by Hastie et al. (current edition ~2013). There isn't a huge difference in subject matter. The latter is slightly more advanced and has more of a statistics perspective, but the foundations are there: Bayes' decision theory, linear methods for classification / regression, naive bayes, neural networks, decision trees, ensembles, clustering, etc... There is a range of topics that are foundational to ML, and thus, relatively stable. These topics are built upon the even more solid foundations of probability theory and statistics. The biggest advances in ML in the last decade (I would argue) were not due to advances in theory.
- Florin_Andrei 10y ago> the more recent text by Hastie et al. Is that 'The elements of statistical learning'?
- chestervonwinch 10y agoThat's what I was referring to, yes.
- apathy 10y agoYou just described all fields worth doing research in. If we knew the right answers we wouldn't waste our time implementing the wrong ones. It's not a bad thing: it means there is room for you to discover something genuinely new and understand something, however small, that nobody else ever completely understood. I think that's great. But YMMV.
- blahi 10y agoWhat is a good textbook that will take me from 0 to practical proficiency with Bayesian Nets, if I have experience with regression models, hand built and machine learned?
- apathy 10y agoDo what the author did: take Koller's course and follow the research that interests you. In this case, the propagation through factor graphs is also a good analogy for when you start looking at back propagation in NNs and the autoencoder.
- blahi 10y agoThis is a very fragmented approach and is very frustrating to go down that road. Every field is filled with hype and people pushing their own agenda and publicity to further their careers. Speaking from experience with self learning discriminatve modeling, the overwhelming majority of the books is either superficial or needlessly complicated. In both of those situations the books are badly written as well. Finding a book that hit the sweet spot for regressions wasn't easy but was doable. I was hoping there would be something similar with Bayesian Nets/Generative Models.
- Florin_Andrei 10y ago> Finding a book that hit the sweet spot for regressions wasn't easy but was doable. Could you provide an example?
- blahi 10y agoRegression Modeling Strategies. You need at least some notion of what regressions and probability are all about, but if you have the basics covered, this book will take you through 80% of the journey and the rest is some googling to figure out some concepts that might be murky. This is a book that emphasizes practical applications without getting bent on the math details too much. If on the other hand you are a math whiz, Elements of Statistical Learning is THE book but it expects you to be very proficient in math. Both books are seriously underrated, which is kind of funny to say because you will find only praises about them, but they deserve even more.
- et2o 10y agoSo what did he actually do with this model? Why is it better to do it this way? What is the point? There's no evidence or output here.
- dj-wonk 10y agoJust to touch on one part of your question -- did you see the case study section? There are five conclusions: > 1. This approach provides a systematic process of developing bespoke models tailored to our specific problem. > 2. It provides transparency to our model as we explicitly defined our model assumptions by leveraging prior knowledge about traffic congestion. > 3. The approach allows handling of uncertainty in a principled manner using probability theory. > 4. It does not suffer from overfitting as the model parameters are learned using Bayesian inference and not optimization. > 5. Finally, MBML separates the model development from inference which allows us to build several models and use the same inference algorithm to learn the model parameters. This in turn helps to quickly compare several alternative models and select the best model that is explained by the observed data.
- jimbokun 10y agoIt's just a blog article clearly titled "An Introduction to Model-Based Machine Learning". While the questions you ask are certainly useful, there was no indication the answers would be found in the article.
- plusepsilon 10y agoAuthor calls it "Model-Based" in place for Bayesian. I transitioned from using Bayesian models in academia to using machine learning models in industry. One of the core differences in the two paradigms is the "feel" when constructing models. For a Bayesian model, you feel like you're constructing the model from first principles. You set your conditional probabilities and priors and see if it fits the data. I'm sure probabilistic programming languages facilitated that feeling. For machine learning models, it feels like you're starting from the loss function and working back to get the best configuration. Much of the underlying machinery behind Bayesian vs. machine learning models is the same. Hidden Markov Models are Hidden Markov Models whether they have a prior or not. But this difference in feel influences how you build models and hence, the results. Now that optimization algos for Bayesian models are catching up, Bayesian ML might become a thing. Cool stuff.
- jimbokun 10y agoWhat do you mean by "machine learning models", as distinct from Bayesian models? In the literature, Bayesian models are usually considered a type of machine learning model.
- apathy 10y agoThink Bayesian vs frequentist. Obviously from the perspective of "optimize against a loss function" they're both special cases, but one is mathematically and philosophically parsimonious, the other... Less so.
- dj-wonk 10y agoThe blog post author (Daniel Emaasit) wasn't the first to use the "model-based machine learning" phrase. He cites "Model-based machine learning" by Christopher M. Bishop.
- deleted 10y ago[deleted]
- tartakovsky 10y agoIt seems this is a rebranding of probabilistic graphical methods.