4 ms·
I had the same moment of realization a few years ago when my workplace decided to hold a kaggle-style competition to select an ML consultancy to prioritize inco
by cjlars 8y ago
I had the same moment of realization a few years ago when my workplace decided to hold a kaggle-style competition to select an ML consultancy to prioritize incoming prospective customers. The consultants had given us their sales pitch about how they had the best algorithms, credentialed specialists, proprietary data and so forth. Meanwhile I was just an Excel Monkey who convinced his boss to let me teach myself ML on the company clock.
So off I go to pull my data and spend about half a day building a script to score the leads with a randomforest package, literally teaching myself the syntax as I go. A couple weeks later, the scores come back and I'm second of seven, beating six companies with multi-million dollar funding rounds.
My secret? Nothing really, I knew the data well so I created a few features that the outside consultants apparently missed, which apparently made up for my lack of ML skills. Turns out the 'proprietary data' they were touting was basically just census data and none of their engineers could be bothered to think through the real world situation on the ground and just fed the data into their system raw.
I think there are some really interesting opportunities for people who are working on the cutting edge of ML with things like self driving cars, AlphaGo, and others, but there are an awful lot of business problems can be solved with 'good enough' solutions with relatively simple, undergrad level algorithms and good domain knowledge.
- Ragib_Zaman 8y agoVery much agreed. See my comment in this thread. The value of being able to build decent ML models is decreasing rapidly, so the difference maker will be acquiring better/more data (which you often have little control over) or better feature engineering (which is basically gaining domain knowledge through better skills in mathematics, problem solving and reasoning). You showed that you were the better problem solver against 5 other consulting companies with teams that became used to wow-ing naive companies with little effort.
- shoo 8y agowell done. good on your workplace for setting up a competition to benchmark competing solutions & good on you for learning how to do it yourself. > there are an awful lot of business problems can be solved with 'good enough' solutions with relatively simple, undergrad level algorithms and good domain knowledge i agree. also, the subsystem making predictions is but one part of the overall system, and the performance of the overall system is often most impacted by the weakest link in the chain. this might not be the predictions, it might be data quality, or how well the system can execute on the output of the predictions. in some sense this is freeing --- it can be incredibly challenging - or perhaps impossible - to make "optimal" decisions, but in a lot of places there's value in just doing a good enough job.