18 ms·

Good data scientist, bad data scientist

159 points by ian-whitestone 5y ago

joncp 5y ago
Great list. The rules apply to knowledge work in general.
gyulai 5y ago
I agree with most of what he's saying but reading the first sentence almost stopped me in my tracks when I got to "obsessed". I wonder when exactly it was that "obsessed about this" and "obsessed about that" became a good thing. ...it's thrown around way too much these days, and I for one think that being obsessed with anything, regardless of how positive a thing it is, always speaks to a psychology that is defective in some way or another.
- ska 5y ago
  "focused on" is probably better terminology.
- ian-whitestone 5y ago
  Obsessed may have been overkill :)
- deleted 5y ago
  [deleted]
- concreteblock 5y ago
  Doesn't it just mean that the meaning of the word has changed?
  - xapata 5y ago
    Is changing, not has changed. If it already had, no one would remark on it.
  - gyulai 5y ago
    > Doesn't it just mean that the meaning of the word has changed? ...I do feel a bit bad amount mentioning it, because it's pretty tangential to what the article is actually about. That said: Changes in meanings of words often go hand-in-hand with broad-based changes in the way people think about something, and it's useful to reflect on whether or not one wants to go along with that thinking. There is even a bit of a clichee anyway around sciency-engineeringy folk falling within the "obsessive" range of the personality spectrum in the very original sense of the word where it might be something that a psychotherapist might work on to try and rectify. So when I see it in this particular sphere being attached to a positive value judgment and even with slightly prescriptivist overtones, then it's something that to me really "pops" and it's been happening to me more and more lately.
  - nerdponx 5y ago
    No, it's just hyperbole.
- lhnz 5y ago
  "excited by"
- SuoDuanDao 5y ago
  An interesting description of obsession I've come across is that it's what happens when the will is frustrated. So maybe temporary obsession can be a good thing, if it's a sign someone's chosen a task so difficult that they need to expand effort to overcome a significant hurdle.
- autokad 5y ago
  I guess you can't work for Amazon then. You'll never get passed the Customer Obsession LP
- salemh 5y ago
  Similar to how some companies are intent on speaking about themselves and employees as "family," you may be having the same reaction from the word usage of "obsessed." Of course, most companies say you must now be "obsessed" with customers, or quality, or some such things. Language control in this respect seems a bit easier to understand with the phrase that became very popular for companies to try and model Apples consumers during the iPod revolution (at least how I remember it while studying a bit of Industrial psychology and Marketing / Cognitive Neuromarketing (that fad died out thankfully)) I'm sure its been around for a while, but its been more "we need brand evangelist fanatic zealots" for the last 10-20 years. "Obsessed, evangelist, brand fanatic, turn your customers into zealots," etc is also used heavily in internal company "values." *Guy Kawasaki at least helped promote the idea for other companies after he left Apple: https://hbr.org/2015/05/the-art-of-evangelism https://hbr.org/2015/05/the-art-of-evangelism
didibus 5y ago
What would be the difference in role between a data scientist and a product manager in this case?
- minimaxir 5y ago
  Data Scientists can provide PMs with data and analysis to make better-informed product decisions. Then you can get into more detail, such as DS building tooling/dashboards/models for PMs/stakeholders to self-serve and save time for everyone. Yes, there's some overlap with a Data Analyst position, but there's enough day-to-day work to differentiate.
SilurianWenlock 5y ago
Is data science for most businesses just bs?
- ska 5y ago
  Not bs. But there is both a real GIGO problem, and a problem with under specification. It's certainly easy to propose DS analysis that are unlikely to have much return. Thinking "data science is hot, we should do that" is different than "we have all this data and don't understand what it means". The latter is more likely to lead somewhere interesting.
- mywittyname 5y ago
  No. But (and this is a Big But), the value of data science comes at the end of the data journey. Businesses need to be capturing data that is relevant and accurate before they can start analyzing it and deriving any value. My experience with clients is that they get a ton of value out of that first step of thinking about what information they want to collect about their customers, then actually collecting it (or, conversely, surfacing what they already collect in a meaningful way). So while they come in wanting some kind of neural network powered prediction engine or whatever, they are often really impressed by pretty basic dashboards about their customer behavior.
- graphtrader 5y ago
  10% alpha 90% useless data visualizations so management can pretend to be data driven.
vinay_ys 5y ago
Good data scientist described here seems to have unrealistic expectations at super human level of know-it-all/do-it-all. I think there are more well-established job architectures like business intelligence analyst, data engineering, user experience designers, product manager, software engineer etc - these roles in combination serve to do a lot of what is described here as data scientist. These roles are easier to hire, have well defined career paths and good ways to get job satisfaction and can scale well as the business-problem-space/orgs grows. I think the scientist label should be reserved for those who actually do the scientific mathematical research – specialists who have done deep research in specific areas. For applying pre-existing sciences to solve practical business domain problems, we need lots of engineers, analysts and managers etc who are all trained with AI-first software development practices and just a few specialist data scientists.
- commandlinefan 5y ago
  > seems to have unrealistic expectations Well, the expectations aren't unrealistic - if you were to grant the "good data scientist" a reasonable amount of time rather than demand that everything be done by this afternoon, which is what most "real data scientists" are up against.
- deleted 5y ago
  [deleted]
- monkeybutton 5y ago
  Agreed. The second point about pipelines stuck out to me: > [Good DS] will often build these pipelines themselves. Bad DS thinks it is someone else’s job. In a small environment, sure, do the job so it gets done! But in larger more corporate settings the 'cowboy' approach to pipeline building is not sustainable or even feasible. Am I a bad DS because I can't provision VMs, open firewalls, replicate production DBs and build hooks in other teams' services to expose data? No, its not my job. A good DS collaborates with other teams and sysadmins to build a pipeline that is maintainable and monitorable, and doesn't do it all themselves.
  - ian-whitestone 5y ago
sgt101 5y ago
>Good DS thinks from first principles. Bad DS accepts everything they have heard or seen as the ground truth, or the best way to do something. Domain knowledge - and the humble attitude that can get stakeholders to give it to you is fundamental to understanding data and how models will be interpreted and used. There is not enough "listen to others" in this list (although I read the "listen to customers" at the end). Listening... listening listen!
- waserwill 5y ago
  This reminds me about a time when some geneticists tried to find genes associated with a particular disease, to try to unravel why it occurs. Complex trait, no single answer, so they genotyped thousands of people with and without the disease, and ran the stats. And... nothing. What has one common name is actually several similar diseases, and the geneticists would have known that if they paid attention to the clinicians. Listening and incorporating knowledge is key. [I'm thinking of an early glaucoma GWAS, IIRC, though there are similar cases.]
  - evandijk70 5y ago
    I think this story is very, very common. Still, some complex diseases (eg. Cystic fibrosis, Down syndrome) do turn out to be simple on a genetic level, so there is some merit to this approach. Moreover, there is currently no better way to understand diseases genotyping thousands of people with and without the disease and 'running the stats', so it's worth the try
- noodlenotes 5y ago
  I would say that a good data scientist can quickly estimate where their time is best spent, either accepting what someone else has told them as-is or investigating themselves from the ground up. There's always more to investigate so using your time efficiently is one of the most important DS skills. Like solving a multi-armed bandit problem.
  - dudeman13 5y ago
    Sounds like something that is a function of your domain knowledge and your data science skills will have very little to do with it
sgt101 5y ago
Data scientists take data assets that were not designed to be used for a particular task and set them to be used systematically and with integrity for that task. It's something that comes from having lots of data in enterprises which can be exploited to create value, but can also be used to make very bad decisions and confuse the hell out of everyone. Using data and using data well are two very different things.
beforeolives 5y ago
This is a good list... for one type of data scientist - the type that has heavy involvement in product and business decisions. Other data scientists are basically software developers with a very specific domain, a third kind focus a lot more on research and many data science jobs are some blend of all of these things. My point is that the author mentions in the intro how data science is very broad and then continues to focus on what's only a subset of all data science jobs. With that in mind, the list is actually spot on - it's just good to know that it isn't relevant to many data science jobs.
- ian-whitestone 5y ago
  Agree with you that not all of these things will apply to every DS role - particularly research heavy ones. But my hope is the vast majority will.
  - mturmon 5y ago
    Yep, some research-oriented DS people are (rightly) obsessed (correct word) with a particular family of techniques (variational inference! random forests! adversarial networks!) and work to find problems to apply that family to. They literally do pattern-match on their techniques with every new problem they encounter, and move on if it doesn't fit. A lot of the other of your distinctions do still apply to such people, like knowing where the data comes from, knowing when to stop, and adjusting the message to the audience. So, still a good list. Also, even the research DS people need to evolve their techniques over time.
klmadfejno 5y ago
> Good DS starts simple, ships, and then iterates. Bad DS starts with the most advanced technique they know. > Good DS is constantly learning & evolving their toolbox. Bad DS stagnates and sticks with what they know. These are the big ones imo. But not super obvious. As a junior data scientist I never needed to use anything but regularized linear models and decision trees. Maybe a random forest but the explainability usually wasn't worth it. Recent explainability tools like SHAP have changed this somewhat. But for the most part I think its still ok for the average data scientist to be regularized linear models, decision trees, and then occasionally, idk, a LightGBM or Catboost + SHAP for explainability. A lot of people still don't know about these, and it's now a decent test for whether people are really trying to stay up to date. But if they're not, I don't really care.
- ska 5y ago
  You can't model your way out of poor data. It's a near certainty that good data + basic modelling delivers the overwhelming majority of real value, globally.
  - beckingz 5y ago
    Turns out the right data makes logistic regression go a long long way.
tmule 5y ago
I liked the article, but realize that in a decade of work in Tech, I haven’t meet a good data scientist! I’ll also add: a good data scientist knows his/her strengths, and doesn’t try to become a unicorn.
antipaul 5y ago
If there is a lot to build, like data pipelines or software apps, as opposed to just “analyze”, I think it helps to add a word for the discipline of “engineering”, eg software, data, backend engineering. The role mismatch between data and other engineers, vs actual (data) scientists, makes it difficult for decision makers to figure out which one they need References https://www.oreilly.com/content/why-a-data-scientist-is-not-a-data-engineer/ https://www.oreilly.com/content/why-a-data-scientist-is-not-... https://medium.com/airbnb-engineering https://medium.com/airbnb-engineering
ubitaco 5y ago
> Good DS understands the basics of web technology I'm not a data scientist but a portion of my job is creating pipelines, data analytics and such. I also only have a bare minimum knowledge of web technology. Why is knowledge of web technology part of being a good Data Scientist? Or is this point oriented specifically for data scientists working in web based companies? Genuinely curious. I could imagine myself working as a DS in the future and that's why I found this article interesting.
- antipaul 5y ago
  Why web technologies? You may have to build a web app to display some data or results. But like some top comments say, data science is super broad and it just depends on your team. Mature orgs and teams have a clear idea what their focus area is, while others don’t have a cogent conception of what constitutes “data science”
- jefb 5y ago
  I don't think there is a single correct answer here, but I'll offer a few insights from personal experience. Firstly, valuable data tends to live in places accessible via web technology. Maybe you need to fetch a bunch of XML files from an FTP site? Having a clear understanding of all the nuances you're about to encounter will set you up for success. Secondly, valuable data tends to be generated by web technology itself. Understanding that lifecycle can inform analytical strategy. Finally, some data scientists add value by informing decision makers. One of the most powerful things you can do for them is give them a mobile friendly secure web experience that puts the data they need directly at their finger tips. While yes, Tableau et al. are an option here, you'll be ahead of your peers by knowing how to DIY it when it counts.
screye 5y ago
This highlights one of my main complaints about the DS role. You are expected to have strong business intuition, sufficient coding skills to hold down a SWE role, a strong background in stats/math, know all the ML/DS specific skills and lastly, have technical depth in the subdomain you are looking to solve. All of this, while being paid the exact same as someone on the SWE or PM track. No one can do it all. DSs that do 70% of these are the best of the best. Mature DS groups have figured out that you have to pick your poison, and focus on archetypes rather than a 'well rounded' DS. Here are a few DS archetypes that I've seen. 1. The NLP/Vision/RL domain expert: High depth, low breadth people. Not very concerned with business intuition. Strong grasp of math for their domain. Moderate coding abilities, but pipelining for their field is fairly well defined. What is SQL? 2. The Generalist : Comes close to the 'good data scientist' outlined here. Never publishes, solves DS problems, will probably struggle to reach principal IC level in any specific product group because they lack the prerequisite depth. Will often become a manager down the line though and can also become an excellent PM at some point. SQL is their life blood. The less business savvy people see them as MBA-adjacent. But, they are super important. 3. Mr Maths or the Statistician : Pairs excellently with #4 4. The MLE who doesn't want to be an MLE - Excellent coding skills. Sufficient ML/DS skills. Just hasn't found a way to get their foot in the door to transition to a DS role without taking a pay cut. 5. The Researcher : Hiring a researcher in the wrong team can lead to a completely ineffective team. Also, not having a researcher in a team that needs it can lead to everyone going around in circles. Top DSs will manage to host a max of 2 archetypes in them. Trying to get your DS to host >2 archetypes, is a losing battle. This is as good as it is going get. Also, most teams don't need all archetypes. Identify the archetypes you need. Get some coverage over them through your hired DSs and let them continue growing along their selected archetypes.
- whatshisface 5y ago
  >All of this, while being paid the exact same as someone on the SWE or PM track. Why not pay top quality DS roles more than SWEs?
  - huac 5y ago
    often (usually?) DS are paid less than SWEs of the same level! I have plenty of cynical thoughts as to what drives that compensation gap. Maybe the simplest is just that there is high supply of people with these baseline skills and it isn't easy to distinguish if somebody is good or not.
linspace 5y ago
I think there is this false stereotype of the DS obsessed with cool techniques and detached from the business. Most DS want their work to have impact, actually like most people. But successfully applying data science is hard. We have incredibly mature tech for other problems, like for example databases, a marvel of engineering, and in comparison DS is a kludge. The value DS provides per $ is much lower although is considered a competitive advantage (DBs are a commodity) and I think this is one of the reasons it feeds this stereotype.
tpoacher 5y ago
I was hoping this would be a variant of Good Cop Bad Cop as a technique applied to datascience. It's not.
- nerdponx 5y ago
  Now that is an article I'd want to read.
albertTJames 5y ago
I feel this extends to other field. Its basically describing two of the big five personality traits conscientiousness and openness.
willdearden 5y ago
https://www.uptake.com/blog/good-data-scientist-bad-data-scientist https://www.uptake.com/blog/good-data-scientist-bad-data-sci... done here too
jll29 5y ago
A data scientist is someone that people wish was a unicorn but that is neither that nor a scientist, despite the name. People who are _actual_ scientists usually in industry go by the name "scientist" or "research scientist", although they just data just as much. You can recognize them by the peer reviewed scientific papers they publish, often preceded by filed patent applications, as their work is novel. A real scientist wonders why some people call themselves "data" scientists, because science has always been about data, modeling and measurement. But back to our "data scientist": On a good day, she is generating value from the company's data to increase customer retention. On a bad day, she is just doing the ETL prep work so the boss' other assistant can make that spreadsheet that aggregates the data that the boss' PPT slides will show.
- borroka 5y ago
  This sentiment is quite popular among those who would like to have the same popularity that data scientists currently (well, more a few years ago, since there are many more critical voices now) have, but they don't. Data science is a generic name. There are DS like me who have been "actual scientists" and others who until yesterday were working on dashboards and Excels files with 100 tabs open and pivot tables as far as the eye can see. Whatever, it is a name. What about "engineers"? It is a title with no legal value, people in the US can call themselves software engineers, but in many other countries, they could not. And who is a writer? Somebody making a living out of writing, somebody who has been published even if they got zero money for it and the magazine editor was their cousin, or else? People in my team do causal modeling, use reinforcement learning for network configuration, NLP for chatboxes, computer vision for face ID, and (again) network configuration. They are all called data scientists. Thinking that what people who have the title "Data Scientist" do is "generating value via increased consumer retention" or "ETL for Excel files for the boss" is between misinformed and laughable, but mostly laughable. The world is much bigger than that. Then, I agree that "learning from data" as a specialty has been over-hyped, and most companies do not have the maturity to take advantage of ML prediction, causal and statistical modeling, etc., but that's the nature of the world: one can take advantage of it or being bitter about it. I took advantage of the hype and I am fine, happy, and with no regrets. If tomorrow someone would propose to use for the same job the title "Data Monk" and it paid more, were more visible, and led to more career opportunities, I would grab it as quickly as I would grab 100 dollars floating in and out of the sidewalk.
analog31 5y ago
“A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly. Specialization is for insects.” -- Robert Heinlein Edit: I noticed "science" doesn't appear in the description of a good data scientist. That's ominous.
t8e56vd4ih 5y ago
most data scientist are just jupyter notebook and sklearn cowboys who know a lot of the buzzwords but lack even basic statistical understanding. and I've met a lot of data scientists.
- laichzeit0 5y ago
  Most software developers are just VSCode and JavaScript cowboys who know a lot of the buzzwords but lack even basic computer science fundamentals understanding. And I’ve met a lot of software developers.
  - t8e56vd4ih 5y ago
    sure. but most sds don't make as much of a fuzz about their work as data scientists. and from what I can tell, the average work of an sd is more challenging than that of a ds.
lxe 5y ago
For many of the points it's more like "generalist data scientist, specialist data scientist," Nothing wrong with specializing.
nnm 5y ago
"At their core, data scientists exist to create business value with data." For some DS, this is true. For other DS, they are there to create or maintain a rigorous process so that we can reliable causal inference. Example: how effective / safe is the covid-19 vaccine on trial. Thinking most of the statisticians in FDA and pharmaceutical companies.
zmmmmm 5y ago
What is strange is that people have attached a "scientist" label to a role that they say its primary purpose is: > exist to create business value with data If you want someone to essentially not be a scientist, don't call them a scientist. Calling them a scientist and then complaining that they are not relentlessly obsessed with creating "business value" is just a broken idea in the first place. The reality is, people have jumped on this label as part of the hype cycle : employers want to pretend they have real data scientist positions so they can look good to their board / investors / PR, and so they can attract top talent, while employees want to put it on their resume because they think these skills will be valuable. Neither is going to get what they want when the hype fades. It reminds me of how a long (long) time ago programmers used to be called computer scientists. It has taken a few decades but we finally now got to a position where we hire people for roles that actually correspond to what they do ... but I feel like this is now the pathway that ML / data science is on.
- Xcelerate 5y ago
  I’ve always found it funny how I have a degree in engineering and my title is “data scientist” whereas my coworkers have a degree in computer science, yet are called “software engineers”. Business titles are peculiar.
- analog31 5y ago
  Interestingly, I'm a scientist, and I definitely create business value. I'm not doing basic academic research, with is fine by me, but I do R&D on technologies that will be in our products X years from now. I'm also the keeper of "how the product works," which combines knowledge from multiple science and engineering disciplines. My job involves a considerable amount of data analysis.
fungiblecog 5y ago
Lost me at "At their core, data scientists exist to create business value with data" Pretty sure no scientist exists to create business value.

Great list. The rules apply to knowledge work in general.

I agree with most of what he's saying but reading the first sentence almost stopped me in my tracks when I got to "obsessed". I wonder when exactly it was that "obsessed about this" and "obsessed about that" became a good thing. ...it's thrown around way too much these days, and I for one think that being obsessed with anything, regardless of how positive a thing it is, always speaks to a psychology that is defective in some way or another.

ska 5y ago
"focused on" is probably better terminology.
ian-whitestone 5y ago
Obsessed may have been overkill :)
deleted 5y ago
[deleted]
concreteblock 5y ago
Doesn't it just mean that the meaning of the word has changed?
- xapata 5y ago
  Is changing, not has changed. If it already had, no one would remark on it.
- gyulai 5y ago
  > Doesn't it just mean that the meaning of the word has changed? ...I do feel a bit bad amount mentioning it, because it's pretty tangential to what the article is actually about. That said: Changes in meanings of words often go hand-in-hand with broad-based changes in the way people think about something, and it's useful to reflect on whether or not one wants to go along with that thinking. There is even a bit of a clichee anyway around sciency-engineeringy folk falling within the "obsessive" range of the personality spectrum in the very original sense of the word where it might be something that a psychotherapist might work on to try and rectify. So when I see it in this particular sphere being attached to a positive value judgment and even with slightly prescriptivist overtones, then it's something that to me really "pops" and it's been happening to me more and more lately.
- nerdponx 5y ago
  No, it's just hyperbole.
lhnz 5y ago
"excited by"
SuoDuanDao 5y ago
An interesting description of obsession I've come across is that it's what happens when the will is frustrated. So maybe temporary obsession can be a good thing, if it's a sign someone's chosen a task so difficult that they need to expand effort to overcome a significant hurdle.
autokad 5y ago
I guess you can't work for Amazon then. You'll never get passed the Customer Obsession LP
salemh 5y ago
Similar to how some companies are intent on speaking about themselves and employees as "family," you may be having the same reaction from the word usage of "obsessed." Of course, most companies say you must now be "obsessed" with customers, or quality, or some such things. Language control in this respect seems a bit easier to understand with the phrase that became very popular for companies to try and model Apples consumers during the iPod revolution (at least how I remember it while studying a bit of Industrial psychology and Marketing / Cognitive Neuromarketing (that fad died out thankfully)) I'm sure its been around for a while, but its been more "we need brand evangelist fanatic zealots" for the last 10-20 years. "Obsessed, evangelist, brand fanatic, turn your customers into zealots," etc is also used heavily in internal company "values." *Guy Kawasaki at least helped promote the idea for other companies after he left Apple: https://hbr.org/2015/05/the-art-of-evangelism https://hbr.org/2015/05/the-art-of-evangelism

didibus 5y ago

What would be the difference in role between a data scientist and a product manager in this case?

minimaxir 5y ago
Data Scientists can provide PMs with data and analysis to make better-informed product decisions. Then you can get into more detail, such as DS building tooling/dashboards/models for PMs/stakeholders to self-serve and save time for everyone. Yes, there's some overlap with a Data Analyst position, but there's enough day-to-day work to differentiate.

SilurianWenlock 5y ago

Is data science for most businesses just bs?

ska 5y ago
Not bs. But there is both a real GIGO problem, and a problem with under specification. It's certainly easy to propose DS analysis that are unlikely to have much return. Thinking "data science is hot, we should do that" is different than "we have all this data and don't understand what it means". The latter is more likely to lead somewhere interesting.
mywittyname 5y ago
No. But (and this is a Big But), the value of data science comes at the end of the data journey. Businesses need to be capturing data that is relevant and accurate before they can start analyzing it and deriving any value. My experience with clients is that they get a ton of value out of that first step of thinking about what information they want to collect about their customers, then actually collecting it (or, conversely, surfacing what they already collect in a meaningful way). So while they come in wanting some kind of neural network powered prediction engine or whatever, they are often really impressed by pretty basic dashboards about their customer behavior.
graphtrader 5y ago
10% alpha 90% useless data visualizations so management can pretend to be data driven.

vinay_ys 5y ago

Good data scientist described here seems to have unrealistic expectations at super human level of know-it-all/do-it-all. I think there are more well-established job architectures like business intelligence analyst, data engineering, user experience designers, product manager, software engineer etc - these roles in combination serve to do a lot of what is described here as data scientist. These roles are easier to hire, have well defined career paths and good ways to get job satisfaction and can scale well as the business-problem-space/orgs grows. I think the scientist label should be reserved for those who actually do the scientific mathematical research – specialists who have done deep research in specific areas. For applying pre-existing sciences to solve practical business domain problems, we need lots of engineers, analysts and managers etc who are all trained with AI-first software development practices and just a few specialist data scientists.

commandlinefan 5y ago
> seems to have unrealistic expectations Well, the expectations aren't unrealistic - if you were to grant the "good data scientist" a reasonable amount of time rather than demand that everything be done by this afternoon, which is what most "real data scientists" are up against.
deleted 5y ago
[deleted]
monkeybutton 5y ago
Agreed. The second point about pipelines stuck out to me: > [Good DS] will often build these pipelines themselves. Bad DS thinks it is someone else’s job. In a small environment, sure, do the job so it gets done! But in larger more corporate settings the 'cowboy' approach to pipeline building is not sustainable or even feasible. Am I a bad DS because I can't provision VMs, open firewalls, replicate production DBs and build hooks in other teams' services to expose data? No, its not my job. A good DS collaborates with other teams and sysadmins to build a pipeline that is maintainable and monitorable, and doesn't do it all themselves.
- ian-whitestone 5y ago

sgt101 5y ago

>Good DS thinks from first principles. Bad DS accepts everything they have heard or seen as the ground truth, or the best way to do something. Domain knowledge - and the humble attitude that can get stakeholders to give it to you is fundamental to understanding data and how models will be interpreted and used. There is not enough "listen to others" in this list (although I read the "listen to customers" at the end). Listening... listening listen!

waserwill 5y ago
This reminds me about a time when some geneticists tried to find genes associated with a particular disease, to try to unravel why it occurs. Complex trait, no single answer, so they genotyped thousands of people with and without the disease, and ran the stats. And... nothing. What has one common name is actually several similar diseases, and the geneticists would have known that if they paid attention to the clinicians. Listening and incorporating knowledge is key. [I'm thinking of an early glaucoma GWAS, IIRC, though there are similar cases.]
- evandijk70 5y ago
  I think this story is very, very common. Still, some complex diseases (eg. Cystic fibrosis, Down syndrome) do turn out to be simple on a genetic level, so there is some merit to this approach. Moreover, there is currently no better way to understand diseases genotyping thousands of people with and without the disease and 'running the stats', so it's worth the try
noodlenotes 5y ago
I would say that a good data scientist can quickly estimate where their time is best spent, either accepting what someone else has told them as-is or investigating themselves from the ground up. There's always more to investigate so using your time efficiently is one of the most important DS skills. Like solving a multi-armed bandit problem.
- dudeman13 5y ago
  Sounds like something that is a function of your domain knowledge and your data science skills will have very little to do with it

sgt101 5y ago

Data scientists take data assets that were not designed to be used for a particular task and set them to be used systematically and with integrity for that task. It's something that comes from having lots of data in enterprises which can be exploited to create value, but can also be used to make very bad decisions and confuse the hell out of everyone. Using data and using data well are two very different things.

beforeolives 5y ago

This is a good list... for one type of data scientist - the type that has heavy involvement in product and business decisions. Other data scientists are basically software developers with a very specific domain, a third kind focus a lot more on research and many data science jobs are some blend of all of these things. My point is that the author mentions in the intro how data science is very broad and then continues to focus on what's only a subset of all data science jobs. With that in mind, the list is actually spot on - it's just good to know that it isn't relevant to many data science jobs.

ian-whitestone 5y ago
Agree with you that not all of these things will apply to every DS role - particularly research heavy ones. But my hope is the vast majority will.
- mturmon 5y ago
  Yep, some research-oriented DS people are (rightly) obsessed (correct word) with a particular family of techniques (variational inference! random forests! adversarial networks!) and work to find problems to apply that family to. They literally do pattern-match on their techniques with every new problem they encounter, and move on if it doesn't fit. A lot of the other of your distinctions do still apply to such people, like knowing where the data comes from, knowing when to stop, and adjusting the message to the audience. So, still a good list. Also, even the research DS people need to evolve their techniques over time.

klmadfejno 5y ago

> Good DS starts simple, ships, and then iterates. Bad DS starts with the most advanced technique they know. > Good DS is constantly learning & evolving their toolbox. Bad DS stagnates and sticks with what they know. These are the big ones imo. But not super obvious. As a junior data scientist I never needed to use anything but regularized linear models and decision trees. Maybe a random forest but the explainability usually wasn't worth it. Recent explainability tools like SHAP have changed this somewhat. But for the most part I think its still ok for the average data scientist to be regularized linear models, decision trees, and then occasionally, idk, a LightGBM or Catboost + SHAP for explainability. A lot of people still don't know about these, and it's now a decent test for whether people are really trying to stay up to date. But if they're not, I don't really care.

ska 5y ago
You can't model your way out of poor data. It's a near certainty that good data + basic modelling delivers the overwhelming majority of real value, globally.
- beckingz 5y ago
  Turns out the right data makes logistic regression go a long long way.

tmule 5y ago

I liked the article, but realize that in a decade of work in Tech, I haven’t meet a good data scientist! I’ll also add: a good data scientist knows his/her strengths, and doesn’t try to become a unicorn.

antipaul 5y ago

If there is a lot to build, like data pipelines or software apps, as opposed to just “analyze”, I think it helps to add a word for the discipline of “engineering”, eg software, data, backend engineering. The role mismatch between data and other engineers, vs actual (data) scientists, makes it difficult for decision makers to figure out which one they need References https://www.oreilly.com/content/why-a-data-scientist-is-not-a-data-engineer/ https://www.oreilly.com/content/why-a-data-scientist-is-not-... https://medium.com/airbnb-engineering https://medium.com/airbnb-engineering

ubitaco 5y ago

> Good DS understands the basics of web technology I'm not a data scientist but a portion of my job is creating pipelines, data analytics and such. I also only have a bare minimum knowledge of web technology. Why is knowledge of web technology part of being a good Data Scientist? Or is this point oriented specifically for data scientists working in web based companies? Genuinely curious. I could imagine myself working as a DS in the future and that's why I found this article interesting.

antipaul 5y ago
Why web technologies? You may have to build a web app to display some data or results. But like some top comments say, data science is super broad and it just depends on your team. Mature orgs and teams have a clear idea what their focus area is, while others don’t have a cogent conception of what constitutes “data science”
jefb 5y ago
I don't think there is a single correct answer here, but I'll offer a few insights from personal experience. Firstly, valuable data tends to live in places accessible via web technology. Maybe you need to fetch a bunch of XML files from an FTP site? Having a clear understanding of all the nuances you're about to encounter will set you up for success. Secondly, valuable data tends to be generated by web technology itself. Understanding that lifecycle can inform analytical strategy. Finally, some data scientists add value by informing decision makers. One of the most powerful things you can do for them is give them a mobile friendly secure web experience that puts the data they need directly at their finger tips. While yes, Tableau et al. are an option here, you'll be ahead of your peers by knowing how to DIY it when it counts.

screye 5y ago

This highlights one of my main complaints about the DS role. You are expected to have strong business intuition, sufficient coding skills to hold down a SWE role, a strong background in stats/math, know all the ML/DS specific skills and lastly, have technical depth in the subdomain you are looking to solve. All of this, while being paid the exact same as someone on the SWE or PM track. No one can do it all. DSs that do 70% of these are the best of the best. Mature DS groups have figured out that you have to pick your poison, and focus on archetypes rather than a 'well rounded' DS. Here are a few DS archetypes that I've seen. 1. The NLP/Vision/RL domain expert: High depth, low breadth people. Not very concerned with business intuition. Strong grasp of math for their domain. Moderate coding abilities, but pipelining for their field is fairly well defined. What is SQL? 2. The Generalist : Comes close to the 'good data scientist' outlined here. Never publishes, solves DS problems, will probably struggle to reach principal IC level in any specific product group because they lack the prerequisite depth. Will often become a manager down the line though and can also become an excellent PM at some point. SQL is their life blood. The less business savvy people see them as MBA-adjacent. But, they are super important. 3. Mr Maths or the Statistician : Pairs excellently with #4 4. The MLE who doesn't want to be an MLE - Excellent coding skills. Sufficient ML/DS skills. Just hasn't found a way to get their foot in the door to transition to a DS role without taking a pay cut. 5. The Researcher : Hiring a researcher in the wrong team can lead to a completely ineffective team. Also, not having a researcher in a team that needs it can lead to everyone going around in circles. Top DSs will manage to host a max of 2 archetypes in them. Trying to get your DS to host >2 archetypes, is a losing battle. This is as good as it is going get. Also, most teams don't need all archetypes. Identify the archetypes you need. Get some coverage over them through your hired DSs and let them continue growing along their selected archetypes.

whatshisface 5y ago
>All of this, while being paid the exact same as someone on the SWE or PM track. Why not pay top quality DS roles more than SWEs?
- huac 5y ago
  often (usually?) DS are paid less than SWEs of the same level! I have plenty of cynical thoughts as to what drives that compensation gap. Maybe the simplest is just that there is high supply of people with these baseline skills and it isn't easy to distinguish if somebody is good or not.

linspace 5y ago

I think there is this false stereotype of the DS obsessed with cool techniques and detached from the business. Most DS want their work to have impact, actually like most people. But successfully applying data science is hard. We have incredibly mature tech for other problems, like for example databases, a marvel of engineering, and in comparison DS is a kludge. The value DS provides per $ is much lower although is considered a competitive advantage (DBs are a commodity) and I think this is one of the reasons it feeds this stereotype.

tpoacher 5y ago

I was hoping this would be a variant of Good Cop Bad Cop as a technique applied to datascience. It's not.

nerdponx 5y ago
Now that is an article I'd want to read.

albertTJames 5y ago

I feel this extends to other field. Its basically describing two of the big five personality traits conscientiousness and openness.

willdearden 5y ago

https://www.uptake.com/blog/good-data-scientist-bad-data-scientist https://www.uptake.com/blog/good-data-scientist-bad-data-sci... done here too

jll29 5y ago

A data scientist is someone that people wish was a unicorn but that is neither that nor a scientist, despite the name. People who are _actual_ scientists usually in industry go by the name "scientist" or "research scientist", although they just data just as much. You can recognize them by the peer reviewed scientific papers they publish, often preceded by filed patent applications, as their work is novel. A real scientist wonders why some people call themselves "data" scientists, because science has always been about data, modeling and measurement. But back to our "data scientist": On a good day, she is generating value from the company's data to increase customer retention. On a bad day, she is just doing the ETL prep work so the boss' other assistant can make that spreadsheet that aggregates the data that the boss' PPT slides will show.

borroka 5y ago
This sentiment is quite popular among those who would like to have the same popularity that data scientists currently (well, more a few years ago, since there are many more critical voices now) have, but they don't. Data science is a generic name. There are DS like me who have been "actual scientists" and others who until yesterday were working on dashboards and Excels files with 100 tabs open and pivot tables as far as the eye can see. Whatever, it is a name. What about "engineers"? It is a title with no legal value, people in the US can call themselves software engineers, but in many other countries, they could not. And who is a writer? Somebody making a living out of writing, somebody who has been published even if they got zero money for it and the magazine editor was their cousin, or else? People in my team do causal modeling, use reinforcement learning for network configuration, NLP for chatboxes, computer vision for face ID, and (again) network configuration. They are all called data scientists. Thinking that what people who have the title "Data Scientist" do is "generating value via increased consumer retention" or "ETL for Excel files for the boss" is between misinformed and laughable, but mostly laughable. The world is much bigger than that. Then, I agree that "learning from data" as a specialty has been over-hyped, and most companies do not have the maturity to take advantage of ML prediction, causal and statistical modeling, etc., but that's the nature of the world: one can take advantage of it or being bitter about it. I took advantage of the hype and I am fine, happy, and with no regrets. If tomorrow someone would propose to use for the same job the title "Data Monk" and it paid more, were more visible, and led to more career opportunities, I would grab it as quickly as I would grab 100 dollars floating in and out of the sidewalk.

analog31 5y ago

“A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly. Specialization is for insects.” -- Robert Heinlein Edit: I noticed "science" doesn't appear in the description of a good data scientist. That's ominous.

t8e56vd4ih 5y ago

most data scientist are just jupyter notebook and sklearn cowboys who know a lot of the buzzwords but lack even basic statistical understanding. and I've met a lot of data scientists.

laichzeit0 5y ago
Most software developers are just VSCode and JavaScript cowboys who know a lot of the buzzwords but lack even basic computer science fundamentals understanding. And I’ve met a lot of software developers.
- t8e56vd4ih 5y ago
  sure. but most sds don't make as much of a fuzz about their work as data scientists. and from what I can tell, the average work of an sd is more challenging than that of a ds.

lxe 5y ago

For many of the points it's more like "generalist data scientist, specialist data scientist," Nothing wrong with specializing.

nnm 5y ago

"At their core, data scientists exist to create business value with data." For some DS, this is true. For other DS, they are there to create or maintain a rigorous process so that we can reliable causal inference. Example: how effective / safe is the covid-19 vaccine on trial. Thinking most of the statisticians in FDA and pharmaceutical companies.

zmmmmm 5y ago

What is strange is that people have attached a "scientist" label to a role that they say its primary purpose is: > exist to create business value with data If you want someone to essentially not be a scientist, don't call them a scientist. Calling them a scientist and then complaining that they are not relentlessly obsessed with creating "business value" is just a broken idea in the first place. The reality is, people have jumped on this label as part of the hype cycle : employers want to pretend they have real data scientist positions so they can look good to their board / investors / PR, and so they can attract top talent, while employees want to put it on their resume because they think these skills will be valuable. Neither is going to get what they want when the hype fades. It reminds me of how a long (long) time ago programmers used to be called computer scientists. It has taken a few decades but we finally now got to a position where we hire people for roles that actually correspond to what they do ... but I feel like this is now the pathway that ML / data science is on.

Xcelerate 5y ago
I’ve always found it funny how I have a degree in engineering and my title is “data scientist” whereas my coworkers have a degree in computer science, yet are called “software engineers”. Business titles are peculiar.
analog31 5y ago
Interestingly, I'm a scientist, and I definitely create business value. I'm not doing basic academic research, with is fine by me, but I do R&D on technologies that will be in our products X years from now. I'm also the keeper of "how the product works," which combines knowledge from multiple science and engineering disciplines. My job involves a considerable amount of data analysis.

fungiblecog 5y ago

Lost me at "At their core, data scientists exist to create business value with data" Pretty sure no scientist exists to create business value.