5 ms·

In my experience working as a data scientist in the health tech field for the past decade (as an "epidemiologist" and "health informaticist" before "data scient

by reureu 8y ago

In my experience working as a data scientist in the health tech field for the past decade (as an "epidemiologist" and "health informaticist" before "data scientist" was a title), I've found folks who just pick up data science from an engineering background often have a simplistic view of working with data generated from human subjects. My training heavily stressed bias and confounding, study design, problem specification, validation, and interpretation of results. I've seen a lot of software engineers dabbling in machine learning jump to training a deep learning model for a problem where regex would suffice. I've also seen multiple people build models that reflect the data collection instead of the biologic/medical process, and present it without even realizing how wrong their results are. The problems in these cases is often their "objective" measures of performance (e.g., precision, recall, accuracy, AUC, whatever) look pretty good, but they don't see that it's because the model AND data are both heavily influenced by this larger problem. For example, is an increase in complexity of a particular disease due to people actually being sicker or because some payer changed a reimbursement program so now billing departments are using higher acuity diagnosis codes for their patients? That said, the best engineer I've ever met didn't have a college education. I know a bunch of awesome data scientists who have taken pretty circuitous journeys to their current career. So, "PhD required" seems like it'll lead to a lot of false negatives. So, acknowledging that, my main point is that doing data science in a meaningful and ethical way—particularly when it involves human subjects—requires a lot more thought than just being able to implement some machine learning algorithm.

kitanata 8y ago
Thank you for this. I couldn’t agree with you more. I see the exact same kind of risk aversion hiring with data scientists that I’ve seen in the past with engineers hiring other people without a specific engineering background because of the assumption that those people don’t have the right mindset or skills. My entire point is that this kind of hiring process weeds out potentially great candidates based on the false assumption that not having a PhD or some other hard to get qualification somehow makes you unqualified. I think this is the result of this “field” being relatively young in industry and it will change as it gets demystified over the next few years.
sidlls 8y ago
I've also seen multiple people build models that reflect the data collection instead of the biologic/medical process, and present it without even realizing how wrong their results are. The problems in these cases is often their "objective" measures of performance (e.g., precision, recall, accuracy, AUC, whatever) look pretty good, but they don't see that it's because the model AND data are both heavily influenced by this larger problem. This is great. It's something I notice I have to mentor my junior data scientists and ML practitioners on with some regularity. Data science and machine learning aren't necessarily useful without some additional domain knowledge and discipline to recognize the need to view the data from many different perspectives and with many different relationships highlighted. Too often they see high P/R numbers and call the job done, or something similar.
natalyarostova 8y ago
I work as a Data Scientist, but my grad degree is in Political Economy. On the math/stats side I'm definitely weaker than some of my colleagues (and I appreciate their skill, and learning from them). But I often see people with much stronger math skills than myself make very critical errors when trying to make inference on data generated by humans. I think a nice team can have some balance, with enough overlap that we can find common ground. I'd be uncomfortable without knowing I can rely on some of my peers on some of the heavier algorithmic or SDE stuff.

by reureu 8y ago

kitanata 8y ago

Thank you for this. I couldn’t agree with you more. I see the exact same kind of risk aversion hiring with data scientists that I’ve seen in the past with engineers hiring other people without a specific engineering background because of the assumption that those people don’t have the right mindset or skills. My entire point is that this kind of hiring process weeds out potentially great candidates based on the false assumption that not having a PhD or some other hard to get qualification somehow makes you unqualified. I think this is the result of this “field” being relatively young in industry and it will change as it gets demystified over the next few years.

sidlls 8y ago

I've also seen multiple people build models that reflect the data collection instead of the biologic/medical process, and present it without even realizing how wrong their results are. The problems in these cases is often their "objective" measures of performance (e.g., precision, recall, accuracy, AUC, whatever) look pretty good, but they don't see that it's because the model AND data are both heavily influenced by this larger problem. This is great. It's something I notice I have to mentor my junior data scientists and ML practitioners on with some regularity. Data science and machine learning aren't necessarily useful without some additional domain knowledge and discipline to recognize the need to view the data from many different perspectives and with many different relationships highlighted. Too often they see high P/R numbers and call the job done, or something similar.

natalyarostova 8y ago

I work as a Data Scientist, but my grad degree is in Political Economy. On the math/stats side I'm definitely weaker than some of my colleagues (and I appreciate their skill, and learning from them). But I often see people with much stronger math skills than myself make very critical errors when trying to make inference on data generated by humans. I think a nice team can have some balance, with enough overlap that we can find common ground. I'd be uncomfortable without knowing I can rely on some of my peers on some of the heavier algorithmic or SDE stuff.