3 ms·
(I'm assuming that we are talking about the original post http://sharpsightlabs.com/blog/machine-learning-prerequisite-isnt-math/ http://sharpsightlabs.com/blog
by mwexler 9y ago
(I'm assuming that we are talking about the original post http://sharpsightlabs.com/blog/machine-learning-prerequisite-isnt-math/ http://sharpsightlabs.com/blog/machine-learning-prerequisite... reposted on r-bloggers, focusing on business/company use of data science, not necessarily on the many other uses).
Lots of folks like to slag these articles and talk about how "If it's not real Maths, it's crap" (http://www.dailymotion.com/video/xgzfxs http://www.dailymotion.com/video/xgzfxs), but if you've worked in larger orgs, you recognize that there is a need for more than just advanced math. There really are myriad needs, and the failure of AI/ML (yes, I'm combining them for simplicity here) in an org is usually the lack of understanding these needs. Here are some I've seen:
1) What is the problem that AI/ML is being applied to? What is it optimizing, deciding, predicting, forecasting, categorizing? How will this decision be used in a process or flow? This requires a business analytic approach, understanding available data, _what it means_, how it's generated, and how the business might evaluate the impact of the ML/AI. These folks need to interact with the business folks as well, so some communication skills are helpful... though that's true for every role these days.
2) How will said decision be implemented both in tech build, test/QA/FUT/UAT/etc, and prod? This technical architecture and approach is data engineering, but some folks in the data science world are amazing at this. BTW, a model that works in dev may not scale in prod. The fact that so many folks keep "re-discovering" this is scary to me. There is a whole class of amazing folks who can re-implement models to scale them, and if you know them, reward them well.
3) How will models/algos/systems be built? This workflow is often pretty sloppy, just a bunch of jupyter notebooks or a tonload scripts (but they're in Git, so it's ok), and so replication and scaling becomes painful... esp. in regulated industries. Again, data engineering approach, but needs a more nuanced understanding of the vagaries of ML/AI. I find that solving business problems may not always fit a traditional software workflow (call it "agile" all you want, it doesn't always fit) but ymmv. So, you may need to create new workflows for your org's needs, and this tooling may not be off the shelf, but like automated testing, this tech debt will need to be paid sooner or later.
4) What ML/AI approach should I use, esp. if I'm using pre-existing approaches? This becomes more of the data science analytic approach, mixing the understanding of how ML/AI works with the data landscape (how was my internal data generated? What does it mean? Is it stable? What's available at score time, and what's it's latency?) and how to build various working predictive models. Note that this is traditionally where data scientists/model builders/analysts spend their time, from data cleansing to preliminary analysis to generating/training various models to crying when none of them predict well to stumbling onto a fascinating and amazing combination of models and approaches at 3am. Yes, math is defn helpful here, but _understanding_ the underlying math is often helpful enough, vs. _mastery_. A good understanding of the levers affecting each model/algo/DL architecture/etc. and how to diagnose them can get you pretty far, though you may violate assumptions or overfit if you aren't careful.
5) Real Algo design: Using all that math that underlie the models to not just diagnosing a pre-designed package but making your own optimizer, or your minimizers, or your own way of computing the Hessian, or a new weighting approaches, or whatever new approach your expertise is in. From writing your own primitives to making your own deep learning architecture, this is often the real wizardry, but it's not needed in EVERY case. But the effort can really pay off, from optimized prediction, improved use of resources, and unique IP which can be a competitive advantage. Remember, custom work is great, but maintenance of the resulting product can be painful, esp. if done in a language few folks in the org understand and if few resources are available outside.
Finding skills to meet all of these needs in one person is certainly possible, but somewhat unicornish. And you may not need all of these either. But the best orgs that have to scale have at least a "Data Prep/Data Engineering" role, a "Data Science" role, and if needed, a "Data Analyst" role who can help translate specific business needs into analytic problems, and also carve off the easy stuff (ad-hoc pulls and simpler analyses).
So, to original post: yeah, sort of. You need to be able to analyze, but you also need to be able to do some of the math, and understand the impact of the rest. If you are awesome at the math, go up to making your own magic. But if you aren't, at least try to understand as much as you can, while trying to also be familiar with the other areas. I do think that the emphasis on visualization is somewhat glossy-ily ignoring real analysis skills, but it's one of many helpful approaches to understanding the problem. (Pet peeve: Just drawing graphs/charts is usually not analysis. Sorry, but true.)
If you are hiring for a data scientist, at least be clear with yourself on what of these needs you are looking for, and if you need them all. Also be clear about where you hope to grow, and if you are hiring for now, near future, or the year 3000.
(Ok, one more: do yourself a favor and learn experimental design. I am sort of shocked at how many data scientists I chat with who haven't really tested champion/challenger models, or compared 2 or more processes or approaches in a randomly assigned test, or gotten as close as they can with quasi-experimental or matched-groups approaches. The simple ones are indeed really Excel simple, but if it's so easy, why haven't you at least tried it? And the causal-modeling stuff is a bayesian's dream come true, so it's worth learning.)