11 ms·

Beware the data science pin factory: The power of the data science generalist

195 points by ericcolson 8y ago

deleted 8y ago
[deleted]
sgt101 8y ago
A very good article, but I think that there is a missing concept - which is organisational maturity. In a fully mature data driven organisation (like... errm Google I guess - reading Jeff Deans papers anyway) there is a well developed data fabric, polished processes for providing credentials and authority, right sized resourcing pools and also substantial diversity of specialisation coupled with experience and domain insight. Specialists can flourish and deliver value out of proportion to their costs. In other, less developed, organisations there's no chance this will happen and specialists will be left floundering looking for the setting in which they can do their thang.
- zwieback 8y ago
  > A very good article, but I think that there is a missing concept - which is organisational maturity. Maturity and also scale - I suppose a small or even one-man shop requiring a generalist could be mature. Once you get to a certain size specialization happens automatically.
  - sgt101 8y ago
    I agree that a one man shop can be "mature"; but there are many very large scale operations that have cultures that absolutely preclude speciality.
jonathankoren 8y ago
I’m not sure this is entirely true. The author is arguing for full stack scientists, and I prefer those people, but they’re hard to find, and even then you don’t necessarily want them doing everything. Worse yet, if you put someone in a full stack position, and they’re not already full stack, you need to budget a lot of mentoring, because if you don’t, you’re going to get a big pile of unmaintainable code. The author kind of builds a strawman of super specialized data scientists that constantly throw code over the wall to someone else. That doesn’t work, and you simply can’t do that unless your headcount is in the thousands. You have to have people that can productionize their work. At the same time, he’s arguing that scientists should should be maintaining their own data infrastructure, but that’s not good either. The best advice I was given was to hire people either to make you smarter, or to make you stronger/faster. You hire data scientists and ML experts to make you smarter. They should be working on problems that you can’t solve today. Infrastructure on the other hand, isn’t your product. It’s overhead. It’s a tool. Comparatively, it’s easier to hire people to build and maintain your infrastructure. Hire people to do that. All the time your scientists are dealing with infrastructure, is time they could be doing useful work. All that said, know when you should just shove the infrapeople aside and do it yourself.
- gowld 8y ago
  Infrastructure isn't your product. Why build an infrastructure instead of buying it?
  - perpetualpatzer 8y ago
    Not OP, but given the context, it seems OP is using infrastructure to mean "all prerequisites to doing ML/data analysis work." Some of that (e.g. datawarehousing, etc.) is easier to outsource; other parts (data acquisition from your product, ETL design, etc.) are necessarily bespoke to your company an thus not readily "buyable." I understand OP to be arguing roughly "you can get a good DBA for much cheaper than you can get a good ML Engineer (much less a good ML Engineer who's ALSO a good DBA), so there's no sense in making Database management part of the Data Scientist role."
    - jonathankoren 8y ago
      You have correctly understood what I am saying.
mmsimanga 8y ago
Article's sentiments are also true for Business Intelligence. The most effective (I deliberately used work effective) BI developers have the following qualities interested in the business, able to chat to clients (emotional intelligence) and also able to code. The best BI people end up being generalists. Talkative nerds who can converse with business types and from the business end, you get the business people who are genuinely curious and willing to learn some SQL. Being able to communicate is key in BI because this enables you to focus on the right business problems.
natalyarostova 8y ago
I generally agree with this article, and I am, and continue to aspire to be, a strong generalist data scientist. However, I do still enjoy/need to have 1 or 2 really really strong quants/statistician types on my team, since they are able to solve certain problems at a level of depth I can't reach. However, if they aren't supported by generalists, they also struggle to make impact.
- UncleEntity 8y ago
  Yes, indeed, the main issue TFA missed out on is comparative advantage. Theoretically speaking...it's much more efficient to have the specialists doing what they do best instead of trying to learn how to optimize SQL queries or whatever.
reilly3000 8y ago
I really wish hiring managers read this. I am a data generalist, and have had no traction with obtaining even an interview for a data science job. I’ve setup a private JupyterHub where I run python ETL, interactive models, and dashboards. I deployed Metabase several times and have written hundreds of SQL queries. I’ve used Tableau with gigantic datasets. I built a front end serverless analytics pipeline from scratch with AWS that handles 30M events/mo. I've demonstrably grown revenue and margins in multiple contexts with my data products. I’m working on making a fully dynamic frontend for content recommendations. I have self-taught all of these skills in the past 3 years after a decade in sales, marketing, and entrepreneurship. What I haven’t done: a CS/math degree (mine was music), graduate work, or tech work at a household name. Lived in the Bay Area. Gotten an interview for any data job. Sigh.
- walterbell 8y ago
  Can you create a public demo site to showcase your multi-disciplinary skill set? Perhaps make it specific to sales/marketing, where you have domain knowledge.
- 5874-4b22-a4e0 8y ago
  Are you assuming people with technical degrees don't have these same skills? You can also train entry level for a good amount less too.
- johns 8y ago
  You should put some contact info in your bio
- estsauver 8y ago
  If you're interested in relocating to Amsterdam, send an email to earl at apolloagriculture dot com.
- deleted 8y ago
  [deleted]
- scarejunba 8y ago
  Depends where you're applying, right? If you don't know what hyper-parameters, regularization, or cross-validation are and how they affect your work, then some jobs just won't make sense until you're able to talk about those coherently. The skills you have are useful. I know we wouldn't hire you in for a data science role on my team. I could imagine many other places they're useful, though, so perhaps it's the places you're searching.
bpyne 8y ago
This sounds suspiciously like the battle software developers have been waging with people who want to run software development in a manufacturing model. The battle itself really sucks the love of making something right out of you.
mempko 8y ago
The author points this out at the end but I want to highlight it. Adam Smith also said that division of labor makes a person "as stupid and ignorant" as a person can become. https://www.pitt.edu/~syd/ASIND.html https://www.pitt.edu/~syd/ASIND.html
mlthoughts2018 8y ago
This article is terrible. You can’t make a case by putting a bunch of unsupported assertions into section-heading fonts and then just filling in paragraphs. This reads like a desperate business person wrote it, who wishes that one full-stack set of drives made sense and coexisted in a single person to make that labor cheaper and more commidity, despite the reality that it’s simply not true. The person who spent the time to master web service frameworks, query languages and product engineering necessarily did not also master professional level knowledge of deep learning or MCMC sampling or natural language processing. The two types of people need to coexist and work symbiotically, but it’s just asinine wishful thinking to pretend like they are the same person, let alone to write a baseless essay full of assertions that if they aren’t the same person it somehow results in first principles economic inefficiency.
- gk1 8y ago
  You'll have to do better than an ad hominem + "the opposite is true." Author is Chief Algorithms Officer at Stitchfix and former VP Data Science & Engineering at Netflix.
  - mlthoughts2018 8y ago
    No, sorry. Argument from authority doesn’t mean the original article has a cogent point. There’s no burden on anyone to refute anything from this piece, as the piece itself has not met any basic requirement of presenting facts or evidence in the first place. It’s merely a matter of fact to point out this deficiency of the article. The premises of the article could still be accurate (though I think that is fleetingly unlikely), but even if so, this article does not justify any of those claims, so nobody could know one way or the other from this article. Again, this is just a matter of observation of the justifications given. This author would personally find it more convenient if the skillset of data scientists and data platform engineers coexisted in one person who also happened to have the drive to undertake employment spanning all those skill sets, and wouldn’t become unhappy if the employer did not respect specializations. So this author has decided to read tea leaves out of economic principles and superimpose this wish as if it was justified by some first principles analysis. In fact, this wishful thinking seems exactly in line with the flawed perspective that executives or director level employees will have. They don’t want to have to care about motivation and intellectual curiosity required to keep certain kinds of knowledge workers happy & productive, and spend lots of time trying to justify how their business units embody corporate platitudes about customer-driven passion. It’s quite easy to see why they would fall victim to this sort of naive wishful thinking. It’s quite similar to CTOs getting suckered by turn-key consulting solutions. It’s not even surprising that VPs & C-suite executives would be very wrong about this type of work.
opportune 8y ago
I agree and disagree with this post. I do think data scientists need to be better at data processing and do more of it. But I still think you do need a separation of labor between people setting up pipelines and people building models from the data. The real issue is that there are a lot of data science departments where they wittle away at their models in some notebook and then they're "done" once the notebook is showing the right metrics. Data scientists should be writing their models from the beginning so that they can productionize them once they are finished. There shouldn't be frequent hand off events requiring lots of communication between DS, pipelines, and data engineering teams, there should be an integration process set up so the flow of work continues to function without intervention.
thekhatribharat 8y ago
Interestingly, the article doesn’t talk about the scale of production and its effects on productivity. When you produce lots of pins, division of labor is a known way to increase productivity. A data science generalist may work fine for a small data shop but as you grow and expand data science in your organization, we know the next step to increase productivity involves specialization (AKA division of labor). It happens not just in data science, but in all business functions and with all business roles. Marketing, Sales, Finance, Engineering, Operations - every business function uses specialization to get productivity gains. So while generalists may work for you if you’re a small business or a large business spinning up a new business function, specialization is a proven economic tool for productivity gains as you grow. Interestingly, as a business function grows, the communication costs and the ensuing delays increase and this is a known side-effect of specialization within that business function. This doesn’t mean one throws away specialization and runs to the other extreme of the spectrum with their use of generalists. There’s a tradeoff organizations make here and there’s been a lot of experimentation done in this space like - Amazon's two-pizza teams (https://zurb.com/word/two-pizza-team https://zurb.com/word/two-pizza-team), Spotify’s Squads, etc - these organizational structures are not universally applicable but they’re interesting developments to look at. Shameless Plug (on current state of data science market) - https://medium.com/open-factory/state-of-the-m-art-big-data-analytics-2396c321d7b9 https://medium.com/open-factory/state-of-the-m-art-big-data-...
metakermit 8y ago
Wow, this is a cool "interactive whitepaper" website :) https://algorithms-tour.stitchfix.com/ https://algorithms-tour.stitchfix.com/
lincpa 8y ago
I'm Financial Analyst, CPA, CIA, CTA, Statistician, Expert System Developer. I independently developed a financial analysis expert system, with a strong ability to innovate and execute. All my expertise is entirely self-taught. My Project: https://github.com/linpengcheng/fa https://github.com/linpengcheng/fa My technology Blog: https://github.com/linpengcheng/PurefunctionPipelineDataflow https://github.com/linpengcheng/PurefunctionPipelineDataflow
- pirocks 8y ago
  Downvoted because I don't think this comment adds anything.
  - lincpa 8y ago
    Don't you think this is a case of data science generalist creating good products?
- lincpa 8y ago
  @pirocks, Don't you think this is a case of data science generalist creating good products? What kind of psychology makes you give a downvote? Why delete your comments again!
openloop 8y ago
I agree. This is basically siloing and it’s not good.
tomrod 8y ago
This works in environments where infrastructure can support it. It can be downright blissful!

195 points by ericcolson 8y ago

deleted 8y ago

[deleted]

A very good article, but I think that there is a missing concept - which is organisational maturity. In a fully mature data driven organisation (like... errm Google I guess - reading Jeff Deans papers anyway) there is a well developed data fabric, polished processes for providing credentials and authority, right sized resourcing pools and also substantial diversity of specialisation coupled with experience and domain insight. Specialists can flourish and deliver value out of proportion to their costs. In other, less developed, organisations there's no chance this will happen and specialists will be left floundering looking for the setting in which they can do their thang.

zwieback 8y ago
> A very good article, but I think that there is a missing concept - which is organisational maturity. Maturity and also scale - I suppose a small or even one-man shop requiring a generalist could be mature. Once you get to a certain size specialization happens automatically.
- sgt101 8y ago
  I agree that a one man shop can be "mature"; but there are many very large scale operations that have cultures that absolutely preclude speciality.

jonathankoren 8y ago

I’m not sure this is entirely true. The author is arguing for full stack scientists, and I prefer those people, but they’re hard to find, and even then you don’t necessarily want them doing everything. Worse yet, if you put someone in a full stack position, and they’re not already full stack, you need to budget a lot of mentoring, because if you don’t, you’re going to get a big pile of unmaintainable code. The author kind of builds a strawman of super specialized data scientists that constantly throw code over the wall to someone else. That doesn’t work, and you simply can’t do that unless your headcount is in the thousands. You have to have people that can productionize their work. At the same time, he’s arguing that scientists should should be maintaining their own data infrastructure, but that’s not good either. The best advice I was given was to hire people either to make you smarter, or to make you stronger/faster. You hire data scientists and ML experts to make you smarter. They should be working on problems that you can’t solve today. Infrastructure on the other hand, isn’t your product. It’s overhead. It’s a tool. Comparatively, it’s easier to hire people to build and maintain your infrastructure. Hire people to do that. All the time your scientists are dealing with infrastructure, is time they could be doing useful work. All that said, know when you should just shove the infrapeople aside and do it yourself.

gowld 8y ago
Infrastructure isn't your product. Why build an infrastructure instead of buying it?
- perpetualpatzer 8y ago
  Not OP, but given the context, it seems OP is using infrastructure to mean "all prerequisites to doing ML/data analysis work." Some of that (e.g. datawarehousing, etc.) is easier to outsource; other parts (data acquisition from your product, ETL design, etc.) are necessarily bespoke to your company an thus not readily "buyable." I understand OP to be arguing roughly "you can get a good DBA for much cheaper than you can get a good ML Engineer (much less a good ML Engineer who's ALSO a good DBA), so there's no sense in making Database management part of the Data Scientist role."
  - jonathankoren 8y ago
    You have correctly understood what I am saying.

mmsimanga 8y ago

Article's sentiments are also true for Business Intelligence. The most effective (I deliberately used work effective) BI developers have the following qualities interested in the business, able to chat to clients (emotional intelligence) and also able to code. The best BI people end up being generalists. Talkative nerds who can converse with business types and from the business end, you get the business people who are genuinely curious and willing to learn some SQL. Being able to communicate is key in BI because this enables you to focus on the right business problems.

natalyarostova 8y ago

I generally agree with this article, and I am, and continue to aspire to be, a strong generalist data scientist. However, I do still enjoy/need to have 1 or 2 really really strong quants/statistician types on my team, since they are able to solve certain problems at a level of depth I can't reach. However, if they aren't supported by generalists, they also struggle to make impact.

UncleEntity 8y ago
Yes, indeed, the main issue TFA missed out on is comparative advantage. Theoretically speaking...it's much more efficient to have the specialists doing what they do best instead of trying to learn how to optimize SQL queries or whatever.

reilly3000 8y ago

I really wish hiring managers read this. I am a data generalist, and have had no traction with obtaining even an interview for a data science job. I’ve setup a private JupyterHub where I run python ETL, interactive models, and dashboards. I deployed Metabase several times and have written hundreds of SQL queries. I’ve used Tableau with gigantic datasets. I built a front end serverless analytics pipeline from scratch with AWS that handles 30M events/mo. I've demonstrably grown revenue and margins in multiple contexts with my data products. I’m working on making a fully dynamic frontend for content recommendations. I have self-taught all of these skills in the past 3 years after a decade in sales, marketing, and entrepreneurship. What I haven’t done: a CS/math degree (mine was music), graduate work, or tech work at a household name. Lived in the Bay Area. Gotten an interview for any data job. Sigh.

walterbell 8y ago
Can you create a public demo site to showcase your multi-disciplinary skill set? Perhaps make it specific to sales/marketing, where you have domain knowledge.
5874-4b22-a4e0 8y ago
Are you assuming people with technical degrees don't have these same skills? You can also train entry level for a good amount less too.
johns 8y ago
You should put some contact info in your bio
estsauver 8y ago
If you're interested in relocating to Amsterdam, send an email to earl at apolloagriculture dot com.
deleted 8y ago
[deleted]
scarejunba 8y ago
Depends where you're applying, right? If you don't know what hyper-parameters, regularization, or cross-validation are and how they affect your work, then some jobs just won't make sense until you're able to talk about those coherently. The skills you have are useful. I know we wouldn't hire you in for a data science role on my team. I could imagine many other places they're useful, though, so perhaps it's the places you're searching.

bpyne 8y ago

This sounds suspiciously like the battle software developers have been waging with people who want to run software development in a manufacturing model. The battle itself really sucks the love of making something right out of you.

mempko 8y ago

The author points this out at the end but I want to highlight it. Adam Smith also said that division of labor makes a person "as stupid and ignorant" as a person can become. https://www.pitt.edu/~syd/ASIND.html https://www.pitt.edu/~syd/ASIND.html

mlthoughts2018 8y ago

This article is terrible. You can’t make a case by putting a bunch of unsupported assertions into section-heading fonts and then just filling in paragraphs. This reads like a desperate business person wrote it, who wishes that one full-stack set of drives made sense and coexisted in a single person to make that labor cheaper and more commidity, despite the reality that it’s simply not true. The person who spent the time to master web service frameworks, query languages and product engineering necessarily did not also master professional level knowledge of deep learning or MCMC sampling or natural language processing. The two types of people need to coexist and work symbiotically, but it’s just asinine wishful thinking to pretend like they are the same person, let alone to write a baseless essay full of assertions that if they aren’t the same person it somehow results in first principles economic inefficiency.

gk1 8y ago
You'll have to do better than an ad hominem + "the opposite is true." Author is Chief Algorithms Officer at Stitchfix and former VP Data Science & Engineering at Netflix.
- mlthoughts2018 8y ago
  No, sorry. Argument from authority doesn’t mean the original article has a cogent point. There’s no burden on anyone to refute anything from this piece, as the piece itself has not met any basic requirement of presenting facts or evidence in the first place. It’s merely a matter of fact to point out this deficiency of the article. The premises of the article could still be accurate (though I think that is fleetingly unlikely), but even if so, this article does not justify any of those claims, so nobody could know one way or the other from this article. Again, this is just a matter of observation of the justifications given. This author would personally find it more convenient if the skillset of data scientists and data platform engineers coexisted in one person who also happened to have the drive to undertake employment spanning all those skill sets, and wouldn’t become unhappy if the employer did not respect specializations. So this author has decided to read tea leaves out of economic principles and superimpose this wish as if it was justified by some first principles analysis. In fact, this wishful thinking seems exactly in line with the flawed perspective that executives or director level employees will have. They don’t want to have to care about motivation and intellectual curiosity required to keep certain kinds of knowledge workers happy & productive, and spend lots of time trying to justify how their business units embody corporate platitudes about customer-driven passion. It’s quite easy to see why they would fall victim to this sort of naive wishful thinking. It’s quite similar to CTOs getting suckered by turn-key consulting solutions. It’s not even surprising that VPs & C-suite executives would be very wrong about this type of work.

opportune 8y ago

I agree and disagree with this post. I do think data scientists need to be better at data processing and do more of it. But I still think you do need a separation of labor between people setting up pipelines and people building models from the data. The real issue is that there are a lot of data science departments where they wittle away at their models in some notebook and then they're "done" once the notebook is showing the right metrics. Data scientists should be writing their models from the beginning so that they can productionize them once they are finished. There shouldn't be frequent hand off events requiring lots of communication between DS, pipelines, and data engineering teams, there should be an integration process set up so the flow of work continues to function without intervention.

thekhatribharat 8y ago

Interestingly, the article doesn’t talk about the scale of production and its effects on productivity. When you produce lots of pins, division of labor is a known way to increase productivity. A data science generalist may work fine for a small data shop but as you grow and expand data science in your organization, we know the next step to increase productivity involves specialization (AKA division of labor). It happens not just in data science, but in all business functions and with all business roles. Marketing, Sales, Finance, Engineering, Operations - every business function uses specialization to get productivity gains. So while generalists may work for you if you’re a small business or a large business spinning up a new business function, specialization is a proven economic tool for productivity gains as you grow. Interestingly, as a business function grows, the communication costs and the ensuing delays increase and this is a known side-effect of specialization within that business function. This doesn’t mean one throws away specialization and runs to the other extreme of the spectrum with their use of generalists. There’s a tradeoff organizations make here and there’s been a lot of experimentation done in this space like - Amazon's two-pizza teams (https://zurb.com/word/two-pizza-team https://zurb.com/word/two-pizza-team), Spotify’s Squads, etc - these organizational structures are not universally applicable but they’re interesting developments to look at. Shameless Plug (on current state of data science market) - https://medium.com/open-factory/state-of-the-m-art-big-data-analytics-2396c321d7b9 https://medium.com/open-factory/state-of-the-m-art-big-data-...

metakermit 8y ago

Wow, this is a cool "interactive whitepaper" website :) https://algorithms-tour.stitchfix.com/ https://algorithms-tour.stitchfix.com/

lincpa 8y ago

I'm Financial Analyst, CPA, CIA, CTA, Statistician, Expert System Developer. I independently developed a financial analysis expert system, with a strong ability to innovate and execute. All my expertise is entirely self-taught. My Project: https://github.com/linpengcheng/fa https://github.com/linpengcheng/fa My technology Blog: https://github.com/linpengcheng/PurefunctionPipelineDataflow https://github.com/linpengcheng/PurefunctionPipelineDataflow

pirocks 8y ago
Downvoted because I don't think this comment adds anything.
- lincpa 8y ago
  Don't you think this is a case of data science generalist creating good products?
lincpa 8y ago
@pirocks, Don't you think this is a case of data science generalist creating good products? What kind of psychology makes you give a downvote? Why delete your comments again!

openloop 8y ago

I agree. This is basically siloing and it’s not good.

tomrod 8y ago

This works in environments where infrastructure can support it. It can be downright blissful!