3 ms·

Whenever people make this comment I have to seriously question if they've ever worked with any actually data scientists or at a company doing any data related w

Whenever people make this comment I have to seriously question if they've ever worked with any actually data scientists or at a company doing any data related work at scale? Data science is a unique skill set that is quite different than any previously existing role (most of which, for the most part, still exist). "Big data" is not some myth. I've worked at companies big and small that are easily ingesting terabytes of useable data per day and queries consume multiple days worth of this semi-structured data. So starting out you need people with the basic software proficiency to efficiently query and work with this data. Virtually all data science team meetings that I've been in have assumed a pretty strong familiarity with probability/statistics, linear algebra, and calculus. White board solutions that come up everyday involve a non-trivial amount of quantitative reasoning. It's incredibly rare to see non-DS people with these skills outside of what were traditionally research roles in larger companies. Likewise the job requires not just knowing the math but knowing how to implement it when necessary. Sure there are plenty of black box libraries out there, but these will only get you so far. Diving into the software implementation details of these things is a pretty common issue. What decades old methodologies do you think are being merely rebranded in the data scientists you've worked with? Nearly all of the pure stats people I've worked with in the years past did most of their work on the white board (or SAS) but left the code up to someone else. Business analysts still exist and honestly have none of the technical skills necessary. Software engineers rarely have the quantitive background to quickly prototype. In all the places I've worked DS is a genuinely unique intersection of skills that requires a pretty reasonable proficiency in across the board to be effective.

disgruntledphd2 7y ago
I think that both of you are right, to be honest. There have always been stats people that code, and coding people that do stats. Most of the useful (for structured-data problems) models are pretty old, and you can solve an awful lot of business problems with linear models with carefully chosen features. (it is perhaps worth pointing out, that with the exception of ReLU, CNN's in their modern form have existed since the 80s). But on the other hand, the large-scale hiring of people to do this work is pretty new, as well as the volume of data available to even moderately sized services, so there's a quantitative change that is somewhat qualitative. I'm also interested in the fact that there is no mention of A-B tests in the OP, given that the need for this is relatively new, and something that probably pushed a lot of companies towards needing more quantitative people, which led to the current nature of "modern" data science.

disgruntledphd2 7y ago

I think that both of you are right, to be honest. There have always been stats people that code, and coding people that do stats. Most of the useful (for structured-data problems) models are pretty old, and you can solve an awful lot of business problems with linear models with carefully chosen features. (it is perhaps worth pointing out, that with the exception of ReLU, CNN's in their modern form have existed since the 80s). But on the other hand, the large-scale hiring of people to do this work is pretty new, as well as the volume of data available to even moderately sized services, so there's a quantitative change that is somewhat qualitative. I'm also interested in the fact that there is no mention of A-B tests in the OP, given that the need for this is relatively new, and something that probably pushed a lot of companies towards needing more quantitative people, which led to the current nature of "modern" data science.