7 ms·

I dislike that "substantive domain knowledge" has here been replaced with "communications skills." Science stands on the shoulders of giants, and ignorance of w

I dislike that "substantive domain knowledge" has here been replaced with "communications skills." Science stands on the shoulders of giants, and ignorance of what has come before doesn't do that. Being able to spin a good yarn isn't really enough here. If data science just becomes a code word for brogramming your way through a set of black-box ML algorithms, then I will welcome the inevitable crash of data science. If insight is the goal, then classic applied statistics plus reproducibility feels like a much better story. At least if insight rather than "making it go" is the goal.

kerbalspacepro 10y ago
>If computer science becomes code word for brogramming your way through a set of black-box algorithms, then I will welcome the inevitable crash of computer science. huh. Hasn't crashed yet.
nickdavidhaynes 10y ago
>If data science just becomes a code word for brogramming your way through a set of black-box ML algorithms, then I will welcome the inevitable crash of data science. A fundamental challenge I see here is how bottom-heavy data science feels now. There are tons of people out there trying to "get into data science" from other fields, but the number of people with substantive domain knowledge, strong programming skills, and the math background to be able to understand the ML black boxes is quite small relative to the number of people calling themselves data scientists. In other words, real insight definitely is (or should be) the goal, but real insight is really hard, and scikit-learn is so easy. My hope is that this improves over the next 5-10 years - the more mature data science becomes as a discipline/career, the better the education will be and the more experienced people there will be. There is a risk in the mean time, though, that a flood of relatively inexperienced people causes a collapse in expectations for data science, making businesses less eager to hire them in the future.
- jackgolding 10y ago
  From my experience the biggest hinder to the future of data science is how crappy it is to learn statistics. And I think this is why a lot of data science courses stop at Z-tests and p values or super basic Bayes theorem. I think mathematicans and statisticians have a lot of work to do to make more advanced parts of the field more accessible, otherwise we will end up with people ignoring important assumptions and using tools like a black box.
  - nickdavidhaynes 10y ago
    To be fair, learning statistics is hard for the same reason that doing statistics is hard - any statistic involves assumptions, and the different assumptions underlying different models can be very subtle. There's a lot of disagreement among even professional, academic statisticians about fundamental concepts like p values [1] and how to quantify uncertainty under multiple hypothesis testing [2]. Unfortunately, I don't see any of this getting easier any time soon (although I would love to be proven wrong). [1] http://www.tandfonline.com/doi/full/10.1080/00031305.2016.1154108 http://www.tandfonline.com/doi/full/10.1080/00031305.2016.11... [2] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1112991/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1112991/
  - claytonjy 10y ago
    I completely agree; I've found it much harder to self-learn the stats than the software side of things. Sibling post makes a good point, but I think the history of stats vs. comp sci bears weight here too; having many people want to learn stats outside academia is a much newer phenomenon than people doing the same with programming. Anyone have any good resources for self-teaching stats? I have a BS in math but only took one stats course, and it was as terrible as all intro-stats classes are. I have a strong, proof-based understanding of probability theory, but haven't found a similar approach to stats. It all seems to be "if data looks like this, use this test, watch for these pitfalls" which is terrible for building intuition.
    - parul 10y ago
      Try the Khan Academy stats resources - https://www.khanacademy.org/math/statistics-probability https://www.khanacademy.org/math/statistics-probability Datacamp also launched a bunch of new stats courses recently. I haven't checked them out yet, but their courses are usually good quality. https://www.datacamp.com/courses/topic:probablity_and_statistics https://www.datacamp.com/courses/topic:probablity_and_statis...
- laxatives 10y ago
  Strongly disagree. Maybe thats the case the a huge company, but most small organizations I've worked with are extremely top-heavy, filled with STEM PhD's who are very capable, but require 1-3 years to get a useful result and aren't often familiar with programming best practices or how to turn their results into a product. You need a larger team of engineers to make that happen and if there's a large overlap between engineers familiar with machine learning, that transition is much easier. Furthermore, there's a number of practitioners that expect their data to be ready for them in some perfect state. Probably a majority of the task is create a pipeline for acquiring data and labeling it appropriately if necessary, which may require developing some ontology or classification with rigid guidelines such that someone in India can delegate the task to a large team. Then the practitioner spends an inordinate time optimizing some heuristic that has a meaning that drifts over time, or is completely inconsistent with the goals of the product. These are both problems outside the realm of domain knowledge or experience.
  - nickdavidhaynes 10y ago
    Sorry, I might not have been clear about what I meant by "bottom-heavy". I think we actually agree - as someone who's hiring for DS roles right now, I've seen a ton of exactly what you're talking about. -Some candidates can write great code, but don't have the math background to understand what ML black boxes are doing. -Then there are STEM PhDs that have never written non-research (i.e. maintainable) code or had to formulate a qualitative business problem into a quantitative problem they can solve. Both types of candidates need to come in at a "junior" level and do some on-the-job learning in order to be fully successful data scientists. IMO it appears to be easier to teach STEM PhDs how to code than programmers how to do math, but that might be personal bias (since I came from the former group).
    - deleted 10y ago
      [deleted]
    - deleted 10y ago
      [deleted]
    - peatmoss 10y ago
jupiter90000 10y ago
I think another thing that could contribute to a crash is companies hiring data scientist roles without really being sure what type of problems they need to have them solve. "We have TBs of data, maybe they can turn it into money." However if the 'data scientists' weren't/aren't even involved in what type of data to collect to solve certain problems (or the problems to solve aren't even known), it makes for very ambiguously drawn up goals and expectations. Also, depending on the political priorities of the organization, data science may not even be really used. Executives/management may look for analysis results to support their ideas, and just throw out the ones that don't align with what 'they already knew to be right.' After all, who wants to be proven wrong? EDIT: One anecdote -- I worked for a company and showed pretty plainly that the length of customer engagement had fallen since the previous year. My boss basically said "why did you point that out?" because it made them look bad to the owner of the business.
dworin 10y ago
It's very hard to find people with both deep domain knowledge and deep math/statistics knowledge, in the same way that it's often hard to find people with deep programming knowledge and deep business knowledge. We solve the latter problem by having business analysts or product managers that "get" the technology enough to provide direction, even if they wouldn't be effective implementing it themselves. I think there's a next phase where, as we try to do data-science at scale, we look for a similar role that deeply understands the business and knows enough about the analytical techniques to define the problem and work with a team of specialists to figure out the best analytical approach. People talk about data science teams being multifunctional - with programmers, data engineers, data scientists, and designers - but we always leave out the role for someone with deep business expertise and shallow but meaningful data science expertise.
- plusbzz 10y ago
  As the author of the OP, I must say this is very well put. Part of the problem is that there is no fixed 'role' for the person with the 'deep business expertise and shallow but meaningful data science expertise'. In my experience, it could be a bunch of different people. When I was in a network security startup, this expert would typically be a malware analyst. In other companies, depending on the project, it could be someone from Product, Sales or Marketing. Similar to designers, a data scientist is expected to figure out who the main stakeholders are and get them engaged in the process, instead of the business stakeholder being part of the data science team per se.

kerbalspacepro 10y ago

>If computer science becomes code word for brogramming your way through a set of black-box algorithms, then I will welcome the inevitable crash of computer science. huh. Hasn't crashed yet.

nickdavidhaynes 10y ago

>If data science just becomes a code word for brogramming your way through a set of black-box ML algorithms, then I will welcome the inevitable crash of data science. A fundamental challenge I see here is how bottom-heavy data science feels now. There are tons of people out there trying to "get into data science" from other fields, but the number of people with substantive domain knowledge, strong programming skills, and the math background to be able to understand the ML black boxes is quite small relative to the number of people calling themselves data scientists. In other words, real insight definitely is (or should be) the goal, but real insight is really hard, and scikit-learn is so easy. My hope is that this improves over the next 5-10 years - the more mature data science becomes as a discipline/career, the better the education will be and the more experienced people there will be. There is a risk in the mean time, though, that a flood of relatively inexperienced people causes a collapse in expectations for data science, making businesses less eager to hire them in the future.

jackgolding 10y ago
From my experience the biggest hinder to the future of data science is how crappy it is to learn statistics. And I think this is why a lot of data science courses stop at Z-tests and p values or super basic Bayes theorem. I think mathematicans and statisticians have a lot of work to do to make more advanced parts of the field more accessible, otherwise we will end up with people ignoring important assumptions and using tools like a black box.
- nickdavidhaynes 10y ago
  To be fair, learning statistics is hard for the same reason that doing statistics is hard - any statistic involves assumptions, and the different assumptions underlying different models can be very subtle. There's a lot of disagreement among even professional, academic statisticians about fundamental concepts like p values [1] and how to quantify uncertainty under multiple hypothesis testing [2]. Unfortunately, I don't see any of this getting easier any time soon (although I would love to be proven wrong). [1] http://www.tandfonline.com/doi/full/10.1080/00031305.2016.1154108 http://www.tandfonline.com/doi/full/10.1080/00031305.2016.11... [2] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1112991/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1112991/
- claytonjy 10y ago
  I completely agree; I've found it much harder to self-learn the stats than the software side of things. Sibling post makes a good point, but I think the history of stats vs. comp sci bears weight here too; having many people want to learn stats outside academia is a much newer phenomenon than people doing the same with programming. Anyone have any good resources for self-teaching stats? I have a BS in math but only took one stats course, and it was as terrible as all intro-stats classes are. I have a strong, proof-based understanding of probability theory, but haven't found a similar approach to stats. It all seems to be "if data looks like this, use this test, watch for these pitfalls" which is terrible for building intuition.
  - parul 10y ago
    Try the Khan Academy stats resources - https://www.khanacademy.org/math/statistics-probability https://www.khanacademy.org/math/statistics-probability Datacamp also launched a bunch of new stats courses recently. I haven't checked them out yet, but their courses are usually good quality. https://www.datacamp.com/courses/topic:probablity_and_statistics https://www.datacamp.com/courses/topic:probablity_and_statis...
laxatives 10y ago
Strongly disagree. Maybe thats the case the a huge company, but most small organizations I've worked with are extremely top-heavy, filled with STEM PhD's who are very capable, but require 1-3 years to get a useful result and aren't often familiar with programming best practices or how to turn their results into a product. You need a larger team of engineers to make that happen and if there's a large overlap between engineers familiar with machine learning, that transition is much easier. Furthermore, there's a number of practitioners that expect their data to be ready for them in some perfect state. Probably a majority of the task is create a pipeline for acquiring data and labeling it appropriately if necessary, which may require developing some ontology or classification with rigid guidelines such that someone in India can delegate the task to a large team. Then the practitioner spends an inordinate time optimizing some heuristic that has a meaning that drifts over time, or is completely inconsistent with the goals of the product. These are both problems outside the realm of domain knowledge or experience.
- nickdavidhaynes 10y ago
  Sorry, I might not have been clear about what I meant by "bottom-heavy". I think we actually agree - as someone who's hiring for DS roles right now, I've seen a ton of exactly what you're talking about. -Some candidates can write great code, but don't have the math background to understand what ML black boxes are doing. -Then there are STEM PhDs that have never written non-research (i.e. maintainable) code or had to formulate a qualitative business problem into a quantitative problem they can solve. Both types of candidates need to come in at a "junior" level and do some on-the-job learning in order to be fully successful data scientists. IMO it appears to be easier to teach STEM PhDs how to code than programmers how to do math, but that might be personal bias (since I came from the former group).
  - deleted 10y ago
    [deleted]
  - deleted 10y ago
    [deleted]
  - peatmoss 10y ago

jupiter90000 10y ago

I think another thing that could contribute to a crash is companies hiring data scientist roles without really being sure what type of problems they need to have them solve. "We have TBs of data, maybe they can turn it into money." However if the 'data scientists' weren't/aren't even involved in what type of data to collect to solve certain problems (or the problems to solve aren't even known), it makes for very ambiguously drawn up goals and expectations. Also, depending on the political priorities of the organization, data science may not even be really used. Executives/management may look for analysis results to support their ideas, and just throw out the ones that don't align with what 'they already knew to be right.' After all, who wants to be proven wrong? EDIT: One anecdote -- I worked for a company and showed pretty plainly that the length of customer engagement had fallen since the previous year. My boss basically said "why did you point that out?" because it made them look bad to the owner of the business.

dworin 10y ago

It's very hard to find people with both deep domain knowledge and deep math/statistics knowledge, in the same way that it's often hard to find people with deep programming knowledge and deep business knowledge. We solve the latter problem by having business analysts or product managers that "get" the technology enough to provide direction, even if they wouldn't be effective implementing it themselves. I think there's a next phase where, as we try to do data-science at scale, we look for a similar role that deeply understands the business and knows enough about the analytical techniques to define the problem and work with a team of specialists to figure out the best analytical approach. People talk about data science teams being multifunctional - with programmers, data engineers, data scientists, and designers - but we always leave out the role for someone with deep business expertise and shallow but meaningful data science expertise.

plusbzz 10y ago
As the author of the OP, I must say this is very well put. Part of the problem is that there is no fixed 'role' for the person with the 'deep business expertise and shallow but meaningful data science expertise'. In my experience, it could be a bunch of different people. When I was in a network security startup, this expert would typically be a malware analyst. In other companies, depending on the project, it could be someone from Product, Sales or Marketing. Similar to designers, a data scientist is expected to figure out who the main stakeholders are and get them engaged in the process, instead of the business stakeholder being part of the data science team per se.