4 ms·
As a current data scientist, the problem is that the concepts behind data science/big data are legitimate solutions to business problems, but the current market
by minimaxir 8y ago
As a current data scientist, the problem is that the concepts behind data science/big data are legitimate solutions to business problems, but the current marketing of the field is either misleading (e.g. it's not all about building models/AI, and I wrote a post complaining about that: https://minimaxir.com/2018/10/data-science-protips/ https://minimaxir.com/2018/10/data-science-protips/ ), or it's business upsell for an enterprise product that may not be the optimal solution to a given problem.
It's one of the reasons I've switched more to publishing open-source tooling for data science/AI instead of blog posts about data science.
- bunderbunder 8y agoThe obsession with "AI" is particularly fascinating. 10 years ago, before it was a household term, people could mention without blushing that their production implementation ended up being a simple if-statement. If anything, being able to come up with an insight that was clear and well-focused enough to be implemented in such a straightforward way was a feather in one's cap. Nowadays, if you aren't building a model with at least a few million parameters (though billions would be better), that ends up being so complex that the only hope for productionizing it is to stick it into a container and ask the engineering and ops folks not to peek inside it too carefully - sort of like a parent leaving the door to their teenager's bedroom closed - then nobody wants to take it seriously. Call me a curmudgeon, but it doesn't half remind me of a while back when I was being directed to process 12MB datasets on a distributed computing cluster. At some point the tools stopped being a means to an end, and started being the only thing anybody cares about.
- deleted 8y ago[deleted]
- salty_biscuits 8y agoIt is funny, what you are usually supposed to be replacing /automating with ML is the nasty 100 line conditional statement that forms some bit of "business logic" that nobody really understands. This thing usually being the time pressured fever dream of some dev that left the company years ago. Now it is replaced by something that is even more incomprehensible. If it is done well then at least you get performance guarantees and the ability to retrain. If it is done badly, then you don't know what the features are, which model belongs with which dataset, how to regenerate training sets, where the original training set went, how to set the hyperparameters to get reasonable training performance, etc, etc.
- reactspa 8y ago> the analysis often doesn’t quantify the insight/value of the results Can you clarify what you mean by this?
- bane 8y agoIBM's attempted marketing grab of the space with Watson has probably set back significant portions of the market by a decade at least.