10 ms·

Of the skills necessary to succeed in data science, the ability to program is actually the least important. Perhaps you were over-emphasising your software skil

Of the skills necessary to succeed in data science, the ability to program is actually the least important. Perhaps you were over-emphasising your software skills at the expense of really demonstrating proper understanding of the science of data analysis. A data science position is not going to require your experience in low-level game programming, for example. In fact, being able to program at all is secondary, since many can pick up some basic skills in languages like Octave, R or even Python, to support their mastery of the math. And it's also not so much math implementation but also a genuine scientist's eye on how to approach a problem. There are no cookie-cutter formulas you can throw at a non-trivial problem: you have to really understand the field and the art of analysis to do it well.

kitanata 8y ago
This right here is a perfect example of why this position is gatekeeped so much. Having a PhD doesn’t automatically mean you think like a scientist of a mathematician, and having a bachelor’s degree and 15 years of experience writing code and then putting in the effort to read Elements of Statistical Learning, compeleting several online courses in machine learning and data science, studying probability, combinatorics, graph theory, does not mean you don’t have a scientific mindset. A scientific mindset can be learned. You are not special because you have a PhD and I don’t. The only difference between you and me is that I had the ability to learn for free what you paid for. But when you go “oh hah, he’s just a coder and only has a bachelors degree” and you won’t even call me to talk to me that means you are missing out, and you are gatekeeping. You are not special and you are not smarter than me. You’ve just read a book I haven’t and wrote a white paper. I can read that book too. I can write a paper too. I can do everything you can do.
- haskellandchill 8y ago
  If you're in NYC, Chicago, or Boston we'll give you a shot! Email me sandy.vanderbleek@publicismedia.com. Thanks.
- erw1 8y ago
  What value does an advanced degree, in your opinion, bring to a data science position then? To hear you say it, it brings no value.
  - kitanata 8y ago
    What value does a bachelor’s degree have if some kid can just learn ruby and JavaScript on his own and go make $100k/yr at some startup? Of course a degree has value but it is possible to provide value without one. This is especially true in 2018 when anyone can learn pretty much anything online, and download whatever papers they want online and read them. Your mistake is in thinking that somehow a degree makes you special. It doesn’t. It just means you paid for a head start. Anyone can learn anything you already know and they can surpass your skill, whether or not you have a piece of paper certifying your knowledge.
    - hellofunk 8y ago
      You assume that everyone can teach themselves with the same efficiency and effectiveness as quality guided instruction from real teachers they can directly interact with. Some can, but most cannot. Education is not a waste.
    - kayoone 8y ago
      Thats a pretty ignorant statement. I'd wager that there aren't many people who could/would put the same effort into self studying that would be required to pass a CS degree with decent grades. They also would probably not learn a lot of stuff that is not interesting to them, while CS students have no choice than to go through the materials. As an employer this also tells a story about who is taking the easy route vs working through a complete program over the span of years. Obviously there are exceptions, but when there are a lot of applicants for a position, it's just an easy filter for employers.
  - kei929 8y ago
    Prestige and artificially pumping salaries. It’s all about brand building. I don’t have an advanced degree, just a bachelors of math from 1995, and have been breadboarding (and more), and coding since the 80s I can follow along with ML and have implemented toys with the ML algorithms in a couple days. It’s bourgeois intellectualism. Like a law firm only hiring from Harvard ML is automated schema design. And the current methodology has known limits of applicability This is “Mongo DB”, “devops” like hype all over again.
- resolaibohp 8y ago
  I don't think anyone doubts that the skills can be learned outside of school or fancy degrees. The problem is that there are hundreds of applicants in your situation WITHOUT experience. There are usually a couple of PHD or MS applications WITH experience for every job. Who do you think the company would give preference to?
- nycthbris 8y ago
  I think you are seriously underestimating the value of first hand experience conducting scientific research. It beats into you a mindset you can't get anywhere else. There are no books or manuals or docs to read and master it. There's only the scientific method. You start with a question, conduct experiments, analyze results, adjust your hypothesis, and repeat the process. You get very comfortable with saying "I don't know". You right at the border of the unknown and trying to navigate further. Companies are using a PhD as a proxy for having research experience because it's the the only qualification like it out there. It's a poor proxy because not all PhDs are created equal.
  - nilkn 8y ago
    What sorts of positions are we actually talking about here? How many companies actually need a research lab? I don't think anyone contests that a PhD is very useful if your job is to actually write and publish papers, attend conferences, present at conferences, etc. But many data scientists and machine learning engineers working in industry don't do any of that, and many companies have no need to have anyone on staff doing any of that either.
  - peatmoss 8y ago
    > You start with a question, conduct experiments, analyze results, adjust your hypothesis, and repeat the process. This is missing my pet step: doing the literature review. I’m pretty ambidextrous when it comes to Python and R, so I’m not typically a combatant in the data science language flamewars. But... for as much as the Python community likes to assert their superior coding chops, I’ve observed that the R community does a much better job of reading about prior art.
    - notlob 8y ago
      > This is missing my pet step: doing the literature review. One of the earliest, most important, and most useful lessons I learned from a senior grad student: "a day in the library can be worth a week at the bench."
      - 8y ago
- gervase 8y ago
  Of course it's possible that there are competent data scientists without graduate degrees; the parent was simply suggesting possible reasons why you might have been passed over for those positions, such as positioning your previous positions as more engineering-focused and less research-focused. I doubt the intent was to "gatekeep" you, but to give you suggestions on how to improve your search in the future. This is a grossly defensive overreaction to the parent reply.
- bufordtwain 8y ago
  Sure. Unjust as it may seem though, requiring a PhD in science is a free and simple way for employers to narrow down the field of candidates to those who are good at data science. It's the data science equivalent of requiring a CS degree when looking for a programmer. It's natural for employers to use that tool.
  - ralphc 8y ago
    But if data scientist is the sexiest job of the 21st century, and everyone's going to need one or more, there won't be enough PhD's to go around. Someone's going to have to "take a chance" on the lesser degreed or risk falling behind their competitors that have data scientists of some effectiveness.
    - resolaibohp 8y ago
      There are not as many data scientist jobs as the hype makes it seem. Maybe this will change in the future but right now there is not such a lack of qualified candidates to justify taking chances on lesser degree candidates. All this hype makes it so everyone wants to be a data scientist. You get people who change careers to go into this new hot career. You also have a pool of people who have been working with data well before the hype with experience. The people trying to break into data science will have a very hard time competing with the people with experience over the pool of jobs out there.
- deleted 8y ago
  [deleted]
- patientplatypus 8y ago
  I feel you bro. I've been there and done that. The thing is is that they are completely happy losing out on seeing the few candidates that are mixed in among the whole pool of non-PhDs (who are 85%+ posers) when they can select among the PhDs (who are 85%+ legit). It's a simple risk aversion tactic among companies and has nada to do with you personally. Like the man says, "nothin' personal, it's just business".
- ska 8y ago
  You are input to a classification routine. The job of those designing the routine is to reduce the false positive rate to as close to 0 as they can without incurring too high a false negative rate. So this sucks if you don't fit the model well in a way that has you often end up as a false negative - but that doesn't' mean the model is broken.
  - kitanata 8y ago
    No. The model has a problem with overfitting. That’s my point.
    - ska 8y ago
      Saying it doesn't make it true. You are claiming there is a generalization problem that causes extra error in practice. Another perfectly viable hypothesis is that the classifier is working fine, it's just tuned for true positive rate and accepts a higher false negative rate to get it. Specificity vs. sensitivity is a fundamental trade off, not a training issue (though that can make both worse)
- ared38 8y ago
  I'm sure you're extremely smart and dedicated -- I've only dreamed of putting in that much work -- but have you considered your mindset might be holding you back? Would you hire someone who had never worked in low-level game programming and didn't have a bachelors but told you "I can do everything you can do"? If you're serious about getting a PhD-level job without a PhD, getting someone to recommend and vouch for you is even more important than usual. Since you're up to date with papers, why not email researchers you admire with questions that demonstrate you deeply understand their work? Many will be too busy, but some will probably be impressed by your determination. Once you have a relationship, see if you can assist with their research, even if initially it's just grunt work. It will take time, but integrating yourself into the academic "web of trust" and maybe getting your name on some papers is the only plausible way you can expect a company that doesn't know you to take you seriously.
  - kthejoker2 8y ago
    Data science is not a PhD level job. PhDs may be well suited for it, certainly, but 80% of data science can be done by peoplenwth bachelor's degree in stats / math-heavy science. There's always a domain specificity that sometimes comes from grad school, but data is data and industries are filled with SMEs who understand the domain. Source: I hire data scientists for Fortune 500 companies.
- aje403 8y ago
  Actually no, a lot of PhD's are a bit smarter than you, some PhD's are a lot smarter than you
- ChrisLomont 8y ago
  >You are not special because you have a PhD and I don’t. >I can do everything you can do. To demonstrate that to an employer wanting PhD workers, go get a PhD like the other PhDs did. Claiming you can do what they do when you haven't done what they have done is not going to cut it. >you won’t even call me to talk to me that means you are missing out, and you are gatekeeping Gatekeeping = not spending unnecessary money and wasting unnecessary time. An employer saves significant money and time by not having to interview everyone claiming they can do what PhD can do but didn't bother to get one. Your skilled workers don't have to stop producing and do interviews, your HR people don't need to spend time and money booking flights, hotels, and such for candidates. You don't have to work through 500 resumes with 30 PhDs in the pool - you sift through 30 resumes.
- HelloMcFly 8y ago
  This comment reads as full of hubris. It is easy to say you can do what someone else does, but the fact is you haven't done it, don't have the evidence to show that you can, and are exasperated that others don't see in you what you see in yourself. Having a PhD in a relevant field is an objective signal - not proof - of capability. When these jobs are hot and candidates are plentiful, using signals to narrow the field to a group you can more rigorously interview is typically a more effective use of time than buying into everyone's self-belief. Candidly, I find most individuals from a programming background vastly underestimate the skillset required in this space. I know that is not an uncommon perception and you are likely being penalized for it, fairly or unfairly. I'll say this: anyone who refers to data science as "just linear algebra and calculus" would be immediately removed from any candidate pool I was managing. Others have evidence of capability, you do not. Programming experience is not evidence enough to elevate you above candidates with more reliable and relevant credentials. A shelf full of books is not evidence either. You either need to find a version of this job created by people that don't really know what it is they want (hint: if the Data Science JD says "Excel" that's an indicator, it's not too uncommon) to create a work history, find a way create a portfolio that you can use as evidence (e.g., Kaggle competitions, hobbyist projects with available datasets), or network with others in the industry and academia such that they will vouch for you.
  - sanderjd 8y ago
    This is all very sensible as long as it's a buyers' market. What I get frustrated by is a sellers' market where the buyers gripe about how there are not enough qualified candidates. At that point, employers should be figuring out how to identify and cultivate inexperienced potential. I thought I had sensed some of this "there aren't enough qualified candidates!" hype in data science (for instance, that's the vibe I got from the article we're discussing), but the first-hand reports from people on this thread are making me think that probably isn't the case; that is, that people in the field feel that they can be very choosey and still fill their roles. If so, it seems to me that this is all working as intended.
    - HelloMcFly 8y ago
      I don't think most organizations possess the ability to cultivate these skills. I know mine doesn't, and from the outside-looking-in we should be more likely than most.
- rifung 8y ago
  > I can do everything you can do. Sorry but how would you even know? I understand your frustrations well since I don't have a BS. Still, as much as you seem to want to talk about how capable you are, you can't seem to understand the perspective of employers. Given what you've said about your history I suspect this is not due to a lack of intelligence but empathy. Employers have to go through many candidates, each of which has some true capability but of which the employer can only see some signals. Signals have varying degrees of quality, and interviewing candidates costs time and money. That being the case, it is only natural that they try to use the strongest signals they have. Nobody believes that there are not capable people who do not have an MS or PhD as you seem to be suggesting. The reality is that the proportion of people who have a BS and can do the job is much less than the number with MS or PhDs, and so it's one of the more effective filters they have at their limited disposal. I'm sure companies are not happy about skipping great candidates like yourself, but they have not figured out a way to do so that is scalable and cost efficient. It's a difficult problem but maybe you can figure it out. > You are not special because you have a PhD and I don’t. The only difference between you and me is that I had the ability to learn for free what you paid for. How much do you even know about PhD programs? It seems like not much because PhD candidates, at least in the US, get paid. It's not much but they certainly are not paying for their education. I apologize for this unsolicited advice but your lack of humility is frankly very off putting. You sound like a very hard working person. PhD programs are extremely difficult to both be admitted into and to finish. I'd expect that you would respect others like yourself who are very hard working.
  - sonofaragorn 8y ago
    I commend you for writing this levelheaded reply. I was about to write one with essentially the same arguments but a much more aggressive tone.
- stilley2 8y ago
  Just FYI many (most?) people with STEM PhDs don't pay for it, but in fact get a stipend.
  - _raoulcousins 8y ago
    To be fair, I didn't pay for my PhD, but I gave up the opportunity to earn more than a grad student's stipend for six years.
- jriot 8y ago
  Every job is gatekeeping. You chose a path in life and it wasn't data analysis. Why would anyone interview you when there are 100's of other people that have the actual skill-set they are looking for?
- skinnymuch 8y ago
  You know PhD candidates have their tuition paid for and get a stipend. They don’t pay for it like you made it seem or seem to think.
gaius 8y ago
the ability to program is actually the least important The days of a researcher producing a model to be re-implemented for production by a programmer are over, or very nearly so. A working data scientist now is expected to produce something that can run in production. That’s something a PhD doesn’t teach and that many PhDs find an uphill struggle.
- pc2g4d 8y ago
  What would be examples of such production-ready models? Things written using TensorFlow et al?
  - gaius 8y ago
    If you being paid to research a predictor or a classifier then as soon as you have a model demonstrably better than what’s running in prod, you should be able to just drop it straight into the CICD pipeline and off it goes to make money for your employer - or for you. No more chucking a prototype over the fence to a programming team to rewrite in C++ or Java to make it “prod grade”. Right now doing this is somewhere between “state of the art” and “new normal” depending on where you sit.
- deleted 8y ago
  [deleted]
yayana 8y ago
This is all true, yet I think I would say the same thing about implementing most software projects in the modern world. At the top of most projects there is a head architect whose time coding is bordering on counterproductive. Then there are layers where only the middle is primarily concerned with programming well.. followed by ones who think more about oddities and integration with the environment it will run in. Is Data Science really fundamentally different? Or is this PHD who barely programs going to either do tasks like cleanup terrabytes of data or risk that a coder with no idea will introduce a bias in the data during that process? I find the whole emergence of the field fascinating, but I kind of feel like it is just techs recreating actuaries with what is actually a less specific education. + (And a worse academic style career path of going through an education you may never get to use instead of going up from apprentice to master)
JamesBarney 8y ago
Judging from the replication crisis in many fields, maybe getting a PhD doesn't teach the scientific mindset as much as we'd hoped.
- kei929 8y ago
  It seems to me since we don’t yet know the truth around large swaths of reality (dark matter and such topics), a replication crisis is exactly what we should expect You have to eliminate a lot of possibilities in a universe with this much detail The real issue, again IMO, is more of an “expectation crisis”. We expected to repeat this and failed to. Because we’re still a ridiculously ignorant species lacking conscious awareness of many aspects of reality
- vkou 8y ago
  When a man's salary depends on him doing his job poorly, is it really all that surprising that he will cut corners? If you don't reward (By issuing grants for) negative results, or null results, or replicating prior studies, why do you expect that scientists will aim for any of those outcomes?
sp527 8y ago
This answer belies the reality that the knowledge needed to conduct actual data science, short of genuine R&D (which is itself at best 10% of data science positions, if that), can be acquired by a sufficiently intelligent person within 6-12 months of applied training. We have developed tools to deliberately abstract away the complexity of the underlying math, and they work well. They work exceedingly well. I once took a semester-long class in data science where untrained, mathphobe business students were running various kinds of regression models on cleaned up data in WEKA (poor choice of software, yes) by the end of it. Most data scientists can and should treat the algorithms themselves as black boxes the same way software engineers treat R-B BSTs as black boxes, without necessarily having to know how they work off the top of their heads. I once worked with a Stanford "data scientist" at a top tech company who couldn't immediately recall Bayes' Rule. He wasn't stupid. His knowledge was just structured in a way that reflected the reality of his day-to-day; this is how it works for all of us. The primary skills needed are: munging, featurization, analysis (basic stats and then a few other things like ROC, etc), and perhaps most importantly (and the thing I see PhDs in particular chronically fail at) operationalization. You do not need to know heavy math to run a model over data, which is the maximum level of sophistication required for most applications of data science that generate real business value. I see similar foolishness in data science as I do blockchain, to be quite honest: people hype up and gravitate towards the cutting edge, while forgetting, ignoring, or blatantly obscuring the power of simple math on big data. I guess it's important that people have an inflated view of the complexity of what most data scientists are doing because having the role at least somewhat cloaked in mystique boosts salaries in the long run. Any argument that a reasonably intelligent person can't be trained to be a legitimately effective data scientist is a counterproductive lie.

kitanata 8y ago

This right here is a perfect example of why this position is gatekeeped so much. Having a PhD doesn’t automatically mean you think like a scientist of a mathematician, and having a bachelor’s degree and 15 years of experience writing code and then putting in the effort to read Elements of Statistical Learning, compeleting several online courses in machine learning and data science, studying probability, combinatorics, graph theory, does not mean you don’t have a scientific mindset. A scientific mindset can be learned. You are not special because you have a PhD and I don’t. The only difference between you and me is that I had the ability to learn for free what you paid for. But when you go “oh hah, he’s just a coder and only has a bachelors degree” and you won’t even call me to talk to me that means you are missing out, and you are gatekeeping. You are not special and you are not smarter than me. You’ve just read a book I haven’t and wrote a white paper. I can read that book too. I can write a paper too. I can do everything you can do.

haskellandchill 8y ago
If you're in NYC, Chicago, or Boston we'll give you a shot! Email me sandy.vanderbleek@publicismedia.com. Thanks.
erw1 8y ago
What value does an advanced degree, in your opinion, bring to a data science position then? To hear you say it, it brings no value.
- kitanata 8y ago
  What value does a bachelor’s degree have if some kid can just learn ruby and JavaScript on his own and go make $100k/yr at some startup? Of course a degree has value but it is possible to provide value without one. This is especially true in 2018 when anyone can learn pretty much anything online, and download whatever papers they want online and read them. Your mistake is in thinking that somehow a degree makes you special. It doesn’t. It just means you paid for a head start. Anyone can learn anything you already know and they can surpass your skill, whether or not you have a piece of paper certifying your knowledge.
  - hellofunk 8y ago
    You assume that everyone can teach themselves with the same efficiency and effectiveness as quality guided instruction from real teachers they can directly interact with. Some can, but most cannot. Education is not a waste.
  - kayoone 8y ago
    Thats a pretty ignorant statement. I'd wager that there aren't many people who could/would put the same effort into self studying that would be required to pass a CS degree with decent grades. They also would probably not learn a lot of stuff that is not interesting to them, while CS students have no choice than to go through the materials. As an employer this also tells a story about who is taking the easy route vs working through a complete program over the span of years. Obviously there are exceptions, but when there are a lot of applicants for a position, it's just an easy filter for employers.
- kei929 8y ago
  Prestige and artificially pumping salaries. It’s all about brand building. I don’t have an advanced degree, just a bachelors of math from 1995, and have been breadboarding (and more), and coding since the 80s I can follow along with ML and have implemented toys with the ML algorithms in a couple days. It’s bourgeois intellectualism. Like a law firm only hiring from Harvard ML is automated schema design. And the current methodology has known limits of applicability This is “Mongo DB”, “devops” like hype all over again.
resolaibohp 8y ago
I don't think anyone doubts that the skills can be learned outside of school or fancy degrees. The problem is that there are hundreds of applicants in your situation WITHOUT experience. There are usually a couple of PHD or MS applications WITH experience for every job. Who do you think the company would give preference to?
nycthbris 8y ago
I think you are seriously underestimating the value of first hand experience conducting scientific research. It beats into you a mindset you can't get anywhere else. There are no books or manuals or docs to read and master it. There's only the scientific method. You start with a question, conduct experiments, analyze results, adjust your hypothesis, and repeat the process. You get very comfortable with saying "I don't know". You right at the border of the unknown and trying to navigate further. Companies are using a PhD as a proxy for having research experience because it's the the only qualification like it out there. It's a poor proxy because not all PhDs are created equal.
- nilkn 8y ago
  What sorts of positions are we actually talking about here? How many companies actually need a research lab? I don't think anyone contests that a PhD is very useful if your job is to actually write and publish papers, attend conferences, present at conferences, etc. But many data scientists and machine learning engineers working in industry don't do any of that, and many companies have no need to have anyone on staff doing any of that either.
- peatmoss 8y ago
  > You start with a question, conduct experiments, analyze results, adjust your hypothesis, and repeat the process. This is missing my pet step: doing the literature review. I’m pretty ambidextrous when it comes to Python and R, so I’m not typically a combatant in the data science language flamewars. But... for as much as the Python community likes to assert their superior coding chops, I’ve observed that the R community does a much better job of reading about prior art.
  - notlob 8y ago
    > This is missing my pet step: doing the literature review. One of the earliest, most important, and most useful lessons I learned from a senior grad student: "a day in the library can be worth a week at the bench."
    - 8y ago
gervase 8y ago
Of course it's possible that there are competent data scientists without graduate degrees; the parent was simply suggesting possible reasons why you might have been passed over for those positions, such as positioning your previous positions as more engineering-focused and less research-focused. I doubt the intent was to "gatekeep" you, but to give you suggestions on how to improve your search in the future. This is a grossly defensive overreaction to the parent reply.
bufordtwain 8y ago
Sure. Unjust as it may seem though, requiring a PhD in science is a free and simple way for employers to narrow down the field of candidates to those who are good at data science. It's the data science equivalent of requiring a CS degree when looking for a programmer. It's natural for employers to use that tool.
- ralphc 8y ago
  But if data scientist is the sexiest job of the 21st century, and everyone's going to need one or more, there won't be enough PhD's to go around. Someone's going to have to "take a chance" on the lesser degreed or risk falling behind their competitors that have data scientists of some effectiveness.
  - resolaibohp 8y ago
    There are not as many data scientist jobs as the hype makes it seem. Maybe this will change in the future but right now there is not such a lack of qualified candidates to justify taking chances on lesser degree candidates. All this hype makes it so everyone wants to be a data scientist. You get people who change careers to go into this new hot career. You also have a pool of people who have been working with data well before the hype with experience. The people trying to break into data science will have a very hard time competing with the people with experience over the pool of jobs out there.
deleted 8y ago
[deleted]
patientplatypus 8y ago
I feel you bro. I've been there and done that. The thing is is that they are completely happy losing out on seeing the few candidates that are mixed in among the whole pool of non-PhDs (who are 85%+ posers) when they can select among the PhDs (who are 85%+ legit). It's a simple risk aversion tactic among companies and has nada to do with you personally. Like the man says, "nothin' personal, it's just business".
ska 8y ago
You are input to a classification routine. The job of those designing the routine is to reduce the false positive rate to as close to 0 as they can without incurring too high a false negative rate. So this sucks if you don't fit the model well in a way that has you often end up as a false negative - but that doesn't' mean the model is broken.
- kitanata 8y ago
  No. The model has a problem with overfitting. That’s my point.
  - ska 8y ago
    Saying it doesn't make it true. You are claiming there is a generalization problem that causes extra error in practice. Another perfectly viable hypothesis is that the classifier is working fine, it's just tuned for true positive rate and accepts a higher false negative rate to get it. Specificity vs. sensitivity is a fundamental trade off, not a training issue (though that can make both worse)
ared38 8y ago
I'm sure you're extremely smart and dedicated -- I've only dreamed of putting in that much work -- but have you considered your mindset might be holding you back? Would you hire someone who had never worked in low-level game programming and didn't have a bachelors but told you "I can do everything you can do"? If you're serious about getting a PhD-level job without a PhD, getting someone to recommend and vouch for you is even more important than usual. Since you're up to date with papers, why not email researchers you admire with questions that demonstrate you deeply understand their work? Many will be too busy, but some will probably be impressed by your determination. Once you have a relationship, see if you can assist with their research, even if initially it's just grunt work. It will take time, but integrating yourself into the academic "web of trust" and maybe getting your name on some papers is the only plausible way you can expect a company that doesn't know you to take you seriously.
- kthejoker2 8y ago
  Data science is not a PhD level job. PhDs may be well suited for it, certainly, but 80% of data science can be done by peoplenwth bachelor's degree in stats / math-heavy science. There's always a domain specificity that sometimes comes from grad school, but data is data and industries are filled with SMEs who understand the domain. Source: I hire data scientists for Fortune 500 companies.
aje403 8y ago
Actually no, a lot of PhD's are a bit smarter than you, some PhD's are a lot smarter than you
ChrisLomont 8y ago
>You are not special because you have a PhD and I don’t. >I can do everything you can do. To demonstrate that to an employer wanting PhD workers, go get a PhD like the other PhDs did. Claiming you can do what they do when you haven't done what they have done is not going to cut it. >you won’t even call me to talk to me that means you are missing out, and you are gatekeeping Gatekeeping = not spending unnecessary money and wasting unnecessary time. An employer saves significant money and time by not having to interview everyone claiming they can do what PhD can do but didn't bother to get one. Your skilled workers don't have to stop producing and do interviews, your HR people don't need to spend time and money booking flights, hotels, and such for candidates. You don't have to work through 500 resumes with 30 PhDs in the pool - you sift through 30 resumes.
HelloMcFly 8y ago
This comment reads as full of hubris. It is easy to say you can do what someone else does, but the fact is you haven't done it, don't have the evidence to show that you can, and are exasperated that others don't see in you what you see in yourself. Having a PhD in a relevant field is an objective signal - not proof - of capability. When these jobs are hot and candidates are plentiful, using signals to narrow the field to a group you can more rigorously interview is typically a more effective use of time than buying into everyone's self-belief. Candidly, I find most individuals from a programming background vastly underestimate the skillset required in this space. I know that is not an uncommon perception and you are likely being penalized for it, fairly or unfairly. I'll say this: anyone who refers to data science as "just linear algebra and calculus" would be immediately removed from any candidate pool I was managing. Others have evidence of capability, you do not. Programming experience is not evidence enough to elevate you above candidates with more reliable and relevant credentials. A shelf full of books is not evidence either. You either need to find a version of this job created by people that don't really know what it is they want (hint: if the Data Science JD says "Excel" that's an indicator, it's not too uncommon) to create a work history, find a way create a portfolio that you can use as evidence (e.g., Kaggle competitions, hobbyist projects with available datasets), or network with others in the industry and academia such that they will vouch for you.
- sanderjd 8y ago
  This is all very sensible as long as it's a buyers' market. What I get frustrated by is a sellers' market where the buyers gripe about how there are not enough qualified candidates. At that point, employers should be figuring out how to identify and cultivate inexperienced potential. I thought I had sensed some of this "there aren't enough qualified candidates!" hype in data science (for instance, that's the vibe I got from the article we're discussing), but the first-hand reports from people on this thread are making me think that probably isn't the case; that is, that people in the field feel that they can be very choosey and still fill their roles. If so, it seems to me that this is all working as intended.
  - HelloMcFly 8y ago
    I don't think most organizations possess the ability to cultivate these skills. I know mine doesn't, and from the outside-looking-in we should be more likely than most.
rifung 8y ago
> I can do everything you can do. Sorry but how would you even know? I understand your frustrations well since I don't have a BS. Still, as much as you seem to want to talk about how capable you are, you can't seem to understand the perspective of employers. Given what you've said about your history I suspect this is not due to a lack of intelligence but empathy. Employers have to go through many candidates, each of which has some true capability but of which the employer can only see some signals. Signals have varying degrees of quality, and interviewing candidates costs time and money. That being the case, it is only natural that they try to use the strongest signals they have. Nobody believes that there are not capable people who do not have an MS or PhD as you seem to be suggesting. The reality is that the proportion of people who have a BS and can do the job is much less than the number with MS or PhDs, and so it's one of the more effective filters they have at their limited disposal. I'm sure companies are not happy about skipping great candidates like yourself, but they have not figured out a way to do so that is scalable and cost efficient. It's a difficult problem but maybe you can figure it out. > You are not special because you have a PhD and I don’t. The only difference between you and me is that I had the ability to learn for free what you paid for. How much do you even know about PhD programs? It seems like not much because PhD candidates, at least in the US, get paid. It's not much but they certainly are not paying for their education. I apologize for this unsolicited advice but your lack of humility is frankly very off putting. You sound like a very hard working person. PhD programs are extremely difficult to both be admitted into and to finish. I'd expect that you would respect others like yourself who are very hard working.
- sonofaragorn 8y ago
  I commend you for writing this levelheaded reply. I was about to write one with essentially the same arguments but a much more aggressive tone.
stilley2 8y ago
Just FYI many (most?) people with STEM PhDs don't pay for it, but in fact get a stipend.
- _raoulcousins 8y ago
  To be fair, I didn't pay for my PhD, but I gave up the opportunity to earn more than a grad student's stipend for six years.
jriot 8y ago
Every job is gatekeeping. You chose a path in life and it wasn't data analysis. Why would anyone interview you when there are 100's of other people that have the actual skill-set they are looking for?
skinnymuch 8y ago
You know PhD candidates have their tuition paid for and get a stipend. They don’t pay for it like you made it seem or seem to think.

gaius 8y ago

the ability to program is actually the least important The days of a researcher producing a model to be re-implemented for production by a programmer are over, or very nearly so. A working data scientist now is expected to produce something that can run in production. That’s something a PhD doesn’t teach and that many PhDs find an uphill struggle.

pc2g4d 8y ago
What would be examples of such production-ready models? Things written using TensorFlow et al?
- gaius 8y ago
  If you being paid to research a predictor or a classifier then as soon as you have a model demonstrably better than what’s running in prod, you should be able to just drop it straight into the CICD pipeline and off it goes to make money for your employer - or for you. No more chucking a prototype over the fence to a programming team to rewrite in C++ or Java to make it “prod grade”. Right now doing this is somewhere between “state of the art” and “new normal” depending on where you sit.
deleted 8y ago
[deleted]

yayana 8y ago

This is all true, yet I think I would say the same thing about implementing most software projects in the modern world. At the top of most projects there is a head architect whose time coding is bordering on counterproductive. Then there are layers where only the middle is primarily concerned with programming well.. followed by ones who think more about oddities and integration with the environment it will run in. Is Data Science really fundamentally different? Or is this PHD who barely programs going to either do tasks like cleanup terrabytes of data or risk that a coder with no idea will introduce a bias in the data during that process? I find the whole emergence of the field fascinating, but I kind of feel like it is just techs recreating actuaries with what is actually a less specific education. + (And a worse academic style career path of going through an education you may never get to use instead of going up from apprentice to master)

JamesBarney 8y ago

Judging from the replication crisis in many fields, maybe getting a PhD doesn't teach the scientific mindset as much as we'd hoped.

kei929 8y ago
It seems to me since we don’t yet know the truth around large swaths of reality (dark matter and such topics), a replication crisis is exactly what we should expect You have to eliminate a lot of possibilities in a universe with this much detail The real issue, again IMO, is more of an “expectation crisis”. We expected to repeat this and failed to. Because we’re still a ridiculously ignorant species lacking conscious awareness of many aspects of reality
vkou 8y ago
When a man's salary depends on him doing his job poorly, is it really all that surprising that he will cut corners? If you don't reward (By issuing grants for) negative results, or null results, or replicating prior studies, why do you expect that scientists will aim for any of those outcomes?

sp527 8y ago

This answer belies the reality that the knowledge needed to conduct actual data science, short of genuine R&D (which is itself at best 10% of data science positions, if that), can be acquired by a sufficiently intelligent person within 6-12 months of applied training. We have developed tools to deliberately abstract away the complexity of the underlying math, and they work well. They work exceedingly well. I once took a semester-long class in data science where untrained, mathphobe business students were running various kinds of regression models on cleaned up data in WEKA (poor choice of software, yes) by the end of it. Most data scientists can and should treat the algorithms themselves as black boxes the same way software engineers treat R-B BSTs as black boxes, without necessarily having to know how they work off the top of their heads. I once worked with a Stanford "data scientist" at a top tech company who couldn't immediately recall Bayes' Rule. He wasn't stupid. His knowledge was just structured in a way that reflected the reality of his day-to-day; this is how it works for all of us. The primary skills needed are: munging, featurization, analysis (basic stats and then a few other things like ROC, etc), and perhaps most importantly (and the thing I see PhDs in particular chronically fail at) operationalization. You do not need to know heavy math to run a model over data, which is the maximum level of sophistication required for most applications of data science that generate real business value. I see similar foolishness in data science as I do blockchain, to be quite honest: people hype up and gravitate towards the cutting edge, while forgetting, ignoring, or blatantly obscuring the power of simple math on big data. I guess it's important that people have an inflated view of the complexity of what most data scientists are doing because having the role at least somewhat cloaked in mystique boosts salaries in the long run. Any argument that a reasonably intelligent person can't be trained to be a legitimately effective data scientist is a counterproductive lie.