4 ms·
I think the "natural" structure of the big-data/data-science/data-analytics/machine-learning market is that of a number of small vertical-focused consultancies,
by nostrademons 11y ago
I think the "natural" structure of the big-data/data-science/data-analytics/machine-learning market is that of a number of small vertical-focused consultancies, each using a handful of very narrowly-focused tools to solve specific problems.
Data science is an interesting market for three main reasons:
1.) The client owns the data.
2.) The skills needed to effectively manipulate the data are very tied up with domain knowledge of the data itself. This tends to throw people accustomed to the rest of the software industry for a loop: if you're a frontend engineer, your effectiveness is much more based on your knowledge of JS/iOS/Android than on your knowledge of the clients' problem. But when I watched the best data-scientists at Google work (and did some unstructured data-mining and machine-learning myself), I found that the majority of your time is actually spent looking at data - it's pulling examples, collecting golden sets, determining outliers, graphing data, etc. And it's often non-transferrable: people who were very effective with the News corpus were often totally ineffective with Social, people familiar with the Web corpus might be ineffective with News, etc.
3.) You often need an interdisciplinary grab bag of tools to get useful insights. The most effective data scientists at Google were the ones who knew some basic HTML and JS or Flash charting libraries, because they could really quickly graph their data and send the graphs around for comments. Deep knowledge of machine-learning is usually less effective than pragmatic knowledge of machine-learning combined with a basic knowledge of stats combined with an intuitive sense of the users who're generating all this data combined with basic presentation skills.
This makes it very difficult to develop effective all-in-one tools that are broadly applicable. Unfortunately, many VCs follow the hype machine and aren't interested in consulting businesses, which means that a lot of money has chased the last wave and not gone into efficiently solving problems. There's probably a tradeable business opportunity here, but I have little interest in going into consulting, so unfortunately I'm not the one that can take advantage of it...
- deleted 11y ago[deleted]
- jamii 11y ago> There's probably a tradeable business opportunity here, but I have little interest in going into consulting, so unfortunately I'm not the one that can take advantage of it... If you were, how might you trade in on it?
- deleted 11y ago[deleted]
- arvinsim 11y agoAs a web developer who is interested in exploring machine learning, this is what I needed to know.