10 ms·

I recently quit a gig and spent a few months looking for a data science/machine learning gig. I was surprised just how gatekeeped these positions were. Everyone

• boxy310 8y ago
  Well - there _is_ a bit of Student's Paradox involved in requirements-gathering for machine learning or analytics, and it's not always apparent that you're using the right algorithm for the problem or what specific metrics you actually need to be optimizing. Most of the time when I've gone through User Story collection processes, the actual end-state users describe is vague, not based on hypothesis testing, and not supportable by the data available onhand. A big part of this process is discovering what the data _can_ tell you, and if necessary to rewrite the technical requirements entirely to align with the actual business need. Even when a customer's given me a "clean" dataset, I've had to write 400+ lines of code to do the feature engineering on a relatively straightforward logistic regression. Then there's all the other times when a customer asks me to deploy one type of algorithm, and their business problem is actually solved by an entirely different class of algorithm entirely. Zayd over at Stanford has a nice blog post [1] describing why machine learning is several more dimensions of complexity compared to traditional software development. There _is_ a specific set of Data-first skills that is complemented by dev and CS experience, but a fundamental reason why ML projects fail is due to lack of appreciation for the many different skillsets needed to succeed. [1] http://ai.stanford.edu/~zayd/why-is-machine-learning-hard.html http://ai.stanford.edu/~zayd/why-is-machine-learning-hard.ht...
  • deleted 8y ago
    [deleted]
  • rectangletangle 8y ago
    Your article eloquently sums up my thoughts on the matter. Data science is like software engineering, but with bugs that tend to have particularly far error localities. You don't get a clean orderly traceback, instead you get vague symptoms, where the model behaves "weird" for no obvious reason. I've found encapsulating the data preprocessing steps in pure functions helps ensure that the data cleaning can easily/quickly be debugged. When it comes to the actual model, there is no substitute for thoroughly understanding the characteristics of your dataset. Finally when it comes to model selection, a good scoring metric is absolutely necessary; this is entirely dependent on what you're actually trying to accomplish with the model. So there is little universal advice. When it comes to the long iteration cycle, the only bandaid I've been able to find is a solid test suite, and thorough code review. This makes it less likely you'll introduce unintended problems. Basically you have to move slow and deliberately, instead of "move fast and break things." Far from impossible, but certainly more difficult than "traditional" software engineering. It's like transitioning from debugging interpreter tracebacks generated by a simple toy script, to debugging a >40k LOC application written in a dynamic language, which happens to be intermittently segfaulting in production.
• make3 8y ago
  In real life, problems are hard and require theoretical investigation to do well. That's the actual job! The theoretical investigation. Not the programming. The programming is really easy. which is part of why having 15 years in software doesn't have so much weight.
  • PunchTornado 8y ago
    I would say the non-programming part, the theoretical investigation is really easy. Plus the majority of data science do data cleaning 80% of their time.
    • make3 8y ago
      If you're a programmer by formation, then indeed you will be given a programming intensive, theoretically non-intensive task (or be found to be qualified for such a job). If you are more theoretically inclined, unsurprisingly, you end up with a more theoretically heavy job.
  • maxxxxx 8y ago
    How many data scientists really do theoretical investigation? From what I see not too many. It feels a little like looking only for devs who can write complete operating systems and then having them write CRUD apps.
    • gaius 8y ago
      90% of data science time is spent sourcing and then cleaning the data. 10% is spent preparing and delivering reports to stakeholders. The ML time is a rounding error. The job in which you are magically given a pristine dataset and can just tweak ML models all day doesn’t exist.
      • Puer 8y ago
        The running assumption on HN seems to be that ML/data science jobs are easy because you can just "plug in" already implemented algorithms and just expect things to work. I think the barrier for most of these jobs is a proper statistics education--one that involves active exploration of real world data that's imperfect and has to be cleaned. Very rarely is there an obvious solution to big data problems and much of the work is tedious parameter tuning that absolutely does require knowledge of the math and domain and not just how to write k-Means.
• CuriouslyC 8y ago
  I had the same experience in my last job search. I only did a bachelors. Even with many projects on my resume that were completely data science focused, my response rate was almost zero for those types of positions. My response rate was very good otherwise. It's frustrating. Based on my conversations at data science/analytics meetups, my knowledge is greater than that of the average employed data scientist. My suggestion is to get a job somewhere with a data science team, do a data science side project with company data, and demo it to the data scientists. That is probably your best bet to get a foot in the door.
  • confounded 8y ago
    > Based on my conversations at data science/analytics meetups, my knowledge is greater than that of the average employed data scientist. Sounds like you may already be too good at data science to ever get any actual experience of it.
    • CuriouslyC 8y ago
      Not sure if you're intending to be snarky or not, but I mentioned the fact that I already had plenty of professional experience doing data science.
  • haskellandchill 8y ago
    Hi, if you're still looking for a data science role and are based in NYC get in touch. sandy.vanderbleek@publicismedia.com Thanks.
• mikert5671 8y ago
  I actually think that machine learning can be much harder than you are claiming here, and that your inexperience is showing when you make such claims. Sure a toy problem from some dataset in a book or pulled off the internet is easy. So is implementing a k-means algorithm. But go to a large corporation with 100 datasets, all with 100 fields each, each having significant seasonal bias (among other biases), and build something that lasts, works, is clean, and is better than someone else can build. You need to convince them to trust you, maybe you can write an algorithm that works, but the business people cant understand that algorithm, and so they have to trust your word.
  • nightski 8y ago
    Often this comes down to the ability to communicate, work with people, and understanding real world business. In addition, often the more simple a learning method used the better. I actually find most businesses needs can be solved by straightforward analytical methods such as traditional statistics. Very few situations require actual machine learning and it is often misapplied. In fact, having a PhD does not tell you a whole lot about these skills. More advanced modern techniques such as deep neural networks, reinforcement learning, etc.. are all extremely proficient at certain niche problems but these do not come up nearly as often in a business context. This is why I don't advertise myself as a machine learning engineer. Rather, I am a business consultant that knows when, and when not to utilize machine learning methods.
  • randcraw 8y ago
    So true. Missing from university textbooks is the fact that real world data is often dirty as hell, inconsistent, distorted, biased, incomplete, and/or just plain invalid (measured the wrong source or too imprecisely to be useful). Unless your DS group is big enough to warrant hiring data engineers / cleaners, as a data scientist it'll be your job not only to eventually choose the algorithm, but foremost, to confirm that the data is sufficient to serve the intended purpose of mining it, ideally before you waste a lot of time curating it or paying for a raw data dump you can't use.
• aje403 8y ago
  The average CS student didn't do that great in LA/Calc. Most CS programs don't require vector calc. At an average uni, the vector calc class/linear algebra will be extremely forgiving and you really don't need to understand the material. I don't think any undergrad CS program in the country teaches enough statistics and probability to understand some of the harder latent models or generative modeling. The math can get arbitrarily hard enough where a lot of decently ranked universities don't even teach the classes required to understand some papers, but that's overkill. An Udacity gold star isn't really a replacement for that either. So basically, people like you are a huge risk for a company. You really can't prove you have anything more than a practical ability to use the algorithms. Most algorithms are hard to implement from scratch (robustly, let alone correctly), and you would be doing so by reading papers you have a scant understanding of.. I'd bet a large sum of money 2/3 of the candidates applying for a machine learning role without a PhD could not even provide any derivation of OLS or something like that if it came up in an interview. Incoming downvote train, choo choo
  • jlelonm 8y ago
    What're some examples of a "harder latent model"
    • aje403 8y ago
      Here’s a really hyperbolic example: https://arxiv.org/pdf/0908.4425.pdf https://arxiv.org/pdf/0908.4425.pdf
• chestervonwinch 8y ago
  For what it's worth, I do have a PhD in math with research in machine learning, and I still had a hard time getting calls back. To be fair to myself, I'm speaking from my experience of grad-school-to-industry transition, and the lack of industry experience is a likely big reason for not getting as many calls back as I had anticipated.
• cup-of-tea 8y ago
  You're right, it's not that hard. A lot of the gatekeeping is because the departments are run by nontechnical people who just don't understand your skill set. They also need to guard the bullshit and not let anyone realise it's just really shoddy programming at the end of the day.
  • mikert5671 8y ago
    this is 100% wrong
    • deleted 8y ago
      [deleted]
    • whataretensors 8y ago
      That's not a rebuttal. It's absolutely non-technical gatekeepers looking for certificates. Graduates get mad because they realize they've wasted a bunch of money on something they can learn for free, so they reject the notion and make others go through the same abusive system.
      • aje403 8y ago
        Do you have any proof that non technical managers are setting the hiring bar and writing the requirements for data science positions at most companies, which is what it seriously sounds like you’re implying. The guy above you was joking
  • drharby 8y ago
    I love the sarcasm, but id advise you to mark it as such. Hn readers are very literal on the whole.
• prions 8y ago
  Saying ML is "just linear algebra and calculus" is a gross simplification. As a dev, you can pick up some popular ML frameworks and learn the basics relatively quickly. The difficulty in this field comes from the amount of theoretical knowledge you need to interpret your results. All these data science bootcamps/learn quick schemes are like teaching a blind person to drive a racecar. He can work the pedals and the steering wheel - but has no idea where he's going.
  • Puer 8y ago
    Exactly this. Implementing algorithms that work on well-defined clean data isn't difficult. Working with naive problems where the right algorithm isn't clear or can't be directly implemented is difficult. In many cases cost of implementation is significant. You can't just apply a model to millions of points of data and blindly expect to get anywhere without fundamentally understanding the data you're working with first. That's what separates data scientists from data engineers, and the ability to take a naive problem and work with it to a conclusion in a manner that's understandable both from a technical and business point of view is exactly why having a PhD is typically a barrier of entry because that's all academics in data science do all day!
• tarunkotia 8y ago
  There was a popular quote in 70's & 80's in corporate America "nobody ever got fired for buying an IBM machine" and that had more to do with branding than anything else. It's the same reason certain law firms hire only from Harvard, you are not second guessing the candidate, as far as the qualifications are concerned. It certainly does not guarantee that the candidate will be good but does fit the requirements. I think this is a problem for candidates who don't meet the set requirements, but this can also be an opportunity to differentiate from rest of the pack.
• joker3 8y ago
  Data science is like security: it's very easy to come up with something that looks really good but is horribly broken in subtle ways that you won't find until it's too late. I think there's a lot of value in having lots of people be literate at it, but part of that literacy is understanding that it's a hard problem and that it does need attention from dedicated specialists.
  • fnbr 8y ago
    I would certainly agree with you, but I don't think someone with a master's degree is necessarily an expert. What's important, IMO, is having practical experience in data science, which may or may not come with a graduate degree.
    • joker3 8y ago
      No, it takes a lot more than a master's degree. Even a PhD is only really entry-level. There's no substitute for practical experience and talking to someone very experienced. Shame there aren't a lot more very experienced statisticians and data scientists out there.
      • Nomentatus 8y ago
        Shame there aren't more concert pianists out there... but given the leverage big data can give (a small percentage change in what a big organization does can still save or earn big bucks) this is a competitive position, just as playing concerts is only for the few. You want a better employee than your competitors have. You'll take all the expertise you can get, plus the ability to invent new techniques as needed. Everybody wants the very best 'cause that last 1% of skill can still be worth a lot of money. Everybody wants someone who can hit a lot of long balls for their team, as it were. There might be a floor on the skill level you could make some use of, but there's no ceiling.
  • mywittyname 8y ago
    > it's very easy to come up with something that looks really good but is horribly broken in subtle ways that you won't find until it's too late. My experience has been that this doesn't matter in a lot of cases. A product can be both, highly useful and subtly broken. One of the biggest boons to my productivity was realizing when something is good enough and to move on. You can waste so much time tuning for precision and recall.
• amrrs 8y ago
  That's very valid because Recruiters are dumb confused. I'm an undergraduate engineer with decent experience in Analytics and my resume doesn't even get into some company because I don't have Masters while my friend who did an MS in Data science from UK is also not getting a job because she just pivoted her career from Mainframe to data science with that MS and they don't want a fresher. Hence you are left with a minimal talent pool that gets to hop here and there increasing their market value while the demand doesn't reduce and at the same time with this fancy word PHD every company is ants to get them or at least an MS just for God knows what reason. In an ideal scenario, You are just exploring data and getting actionable insights ( I'm not talkign about publishing paper nor programming self.driving cars). I'm an average guy knowing R and Python with Pandas and Scikit learn. Anyone who can connect the dots can learn these tools and become a good data scientist, but the blind belief of these recruiters just kill those hope and look for the hardly available job hoppers. Such a paradox self created by them.
• galieos_ghost 8y ago
  The Data Scientist "shortage" is another scam by the big tech companies to drive down wages. My boss had hundreds of applications for the position he was looking to fill and the majority were so well qualified that it was basically a coin flip between multiple PhDs with great resumes. Tech worker's arrogance reminds me of "made in the USA" factory workers who thought their jobs were safe. If people don't start pushing back wages will be pushed down to global equilibrium. Based on how things are going I'm predicting neo-feudalism thanks to the joys of open-boarders globalism putting all power into the hands of corporations.
  • lainga 8y ago
    wouldn't a real or fake shortage of talent drive wages up, not down?
    • ItsMe000001 8y ago
      Yes, no. Yes if it's real. No if it's fake. This still ignores the question of if entry limits for jobs are set reasonably or not though, you can create a real shortage by limiting yourself needlessly. At which point those who think that it is needless will call it "fake", so which word you use also depends on opinion and interpretation of data :-)
    • usrusr 8y ago
      A fake shortage drives wages down by increasing supply. Also, you create a fake shortage by driving wages down: "we have so many positions that we cannot fill (at the price that we are willing to pay)"
• dominotw 8y ago
  I think it might be a easier path if you apply for "data engineering" jobs and convince internally to give you a chance at data science. Get data science on your resume and look for next job.
  • randcraw 8y ago
    Agreed. A DE job is based on more traditional software skills, expecting you to understand the priorities and purposes of an given ML technique, without having to choose, architect, or defend the choice of that technique over alternatives. I can see most of the OP's objections as being more appropriate to the superciliousness of hiring software PhDs into DE jobs rather than DS. The DS role is more quantitative and theoretical (befitting a PhD), while the DE is more an engineer and experimentalist (befitting a less academic mindset).
• hellofunk 8y ago
  Of the skills necessary to succeed in data science, the ability to program is actually the least important. Perhaps you were over-emphasising your software skills at the expense of really demonstrating proper understanding of the science of data analysis. A data science position is not going to require your experience in low-level game programming, for example. In fact, being able to program at all is secondary, since many can pick up some basic skills in languages like Octave, R or even Python, to support their mastery of the math. And it's also not so much math implementation but also a genuine scientist's eye on how to approach a problem. There are no cookie-cutter formulas you can throw at a non-trivial problem: you have to really understand the field and the art of analysis to do it well.
  • kitanata 8y ago
    This right here is a perfect example of why this position is gatekeeped so much. Having a PhD doesn’t automatically mean you think like a scientist of a mathematician, and having a bachelor’s degree and 15 years of experience writing code and then putting in the effort to read Elements of Statistical Learning, compeleting several online courses in machine learning and data science, studying probability, combinatorics, graph theory, does not mean you don’t have a scientific mindset. A scientific mindset can be learned. You are not special because you have a PhD and I don’t. The only difference between you and me is that I had the ability to learn for free what you paid for. But when you go “oh hah, he’s just a coder and only has a bachelors degree” and you won’t even call me to talk to me that means you are missing out, and you are gatekeeping. You are not special and you are not smarter than me. You’ve just read a book I haven’t and wrote a white paper. I can read that book too. I can write a paper too. I can do everything you can do.
    • haskellandchill 8y ago
      If you're in NYC, Chicago, or Boston we'll give you a shot! Email me sandy.vanderbleek@publicismedia.com. Thanks.
    • erw1 8y ago
      What value does an advanced degree, in your opinion, bring to a data science position then? To hear you say it, it brings no value.
• resolaibohp 8y ago
  When my company posted a data science job we received something like 300 applications on the first day. When looking at them most were people with no experience but a fresh degree or bootcamp grad plus all the other people like you who want to change careers from software with portfolios of data science projects. The field is absolutely saturated with people who want to be a data scientist but have no experience. This is where some of that gate keeping comes from. The people who have it easy are the ones with a MS or PHD and years of experience doing data science work at companies under their belt. There are very few of these people right now. There is this idea that data science is needed everywhere and there is a HUGE supply of jobs. As an example if you search for data scientist jobs at glassdoor.com in San Francisco there are ~2000 jobs. If you search for software engineer in San Francisco there are ~9000. Similar ratios can be found in any major tech city. Data science does not scale like software engineering in companies but the narrative out there is that this is the job to be in and there is this huge unmet need. It is all hype.
  • stcredzero 8y ago
    The field is absolutely saturated with people who want to be a data scientist but have no experience. This is where some of that gate keeping comes from. What are the data science "gotchas?" A lot of people can pick up basic programming in a weekend, but they wouldn't necessarily know what they don't know and might well get deeply mired in problems with concurrency or algorithmic complexity. It's such "gotchas" which justify gatekeeping. Otherwise, gatekeeping is just unproductive manipulation of the market.
    • Puer 8y ago
      I mean, in my graduate ML classes the first "homework" assignment was always just a problem set of probability and distribution questions. They were simple questions, but you'd be surprised how many people dropped the class after that first assignment. There are a lot of people out there who have memorized how to implement k-means and PCA but would absolutely struggle to understand what they're actually doing or interpret the results in a meaningful way. A HUGE part of being a data scientist is presenting information in a useful way. That's why PhDs are favored because with their experience having to write grants to get funding for their research they're exactly the type of people that can take a naive problem, work it to a result, and then sit in front of a board room of non-technical people and explain why their result was worth the money that was given to them.
• jtmcmc 8y ago
  I agree it got quite crazy. I had several years experience as a data scientist and 2 years of PhD work in machine learning and after a few years not in software (running a non profit) it was incredibly hard to get call backs. I ended up getting totally burnt out on the search and the field in the end.
• nu-world-odor 8y ago
  that's why I don't bother ML/data tracks at conferences. What's the point of these "ML for everybody" tracks?
• nu-world-odor 8y ago
  yeah every IT conference now has ML/data track to train the legions so they won't need to create positions for PHD candidates...
• auntienomen 8y ago
  If your job applications emphasize the same material as this comment, it's no wonder you aren't getting callbacks. You say you've got a wide range of programming experience. Great, that's usually evidence of intelligence. You say you've done AI in some fashion. Also good. Then you go off on a jag about how simple the math is and how easy to implement the algorithms are. Totally true, but this isn't what data scientists are paid for. Data science/machine learning positions are more about understanding the limitations and pathologies of the algorithms, the data, and their interactions. This isn't necessarily hard, but it can be -- and the pay tends to scale in proportion to the difficulty. Since theory doesn't provide much guidance -- you can learn everything that anyone knows in a year -- employers will necessarily prefer someone who can demonstrate practical experience with data. Selecting for PhDs is one way of filtering for that. If you've got experience with data, emphasize that and you should get callbacks.
  • anatari 8y ago
    This is spot on. As a hiring manager for these type of positions, the primary thing I look for is a track record of using data to achieve business results. This of course creates a barrier for entering this field and makes those with this experience even more highly sought after.
• superhuzza 8y ago
  I think you're wrong about the math requirements. The math is not very abstract, but requires a much deeper understanding of statistics than your post suggests. That's probably why you struggled to get hired. My partner has a successful career in data science, but she has multiple degrees in mathematics - significantly more than your typical engineer is exposed to.
• sjg007 8y ago
  A data scientist is a statistician who programs.. :). That’s the real issue. The gate keepers will be someone with said advanced degree who is filtering for people like them. Ironically that’s a bad way to filter but it happens.
• reureu 8y ago
  In my experience working as a data scientist in the health tech field for the past decade (as an "epidemiologist" and "health informaticist" before "data scientist" was a title), I've found folks who just pick up data science from an engineering background often have a simplistic view of working with data generated from human subjects. My training heavily stressed bias and confounding, study design, problem specification, validation, and interpretation of results. I've seen a lot of software engineers dabbling in machine learning jump to training a deep learning model for a problem where regex would suffice. I've also seen multiple people build models that reflect the data collection instead of the biologic/medical process, and present it without even realizing how wrong their results are. The problems in these cases is often their "objective" measures of performance (e.g., precision, recall, accuracy, AUC, whatever) look pretty good, but they don't see that it's because the model AND data are both heavily influenced by this larger problem. For example, is an increase in complexity of a particular disease due to people actually being sicker or because some payer changed a reimbursement program so now billing departments are using higher acuity diagnosis codes for their patients? That said, the best engineer I've ever met didn't have a college education. I know a bunch of awesome data scientists who have taken pretty circuitous journeys to their current career. So, "PhD required" seems like it'll lead to a lot of false negatives. So, acknowledging that, my main point is that doing data science in a meaningful and ethical way—particularly when it involves human subjects—requires a lot more thought than just being able to implement some machine learning algorithm.
  • kitanata 8y ago
    Thank you for this. I couldn’t agree with you more. I see the exact same kind of risk aversion hiring with data scientists that I’ve seen in the past with engineers hiring other people without a specific engineering background because of the assumption that those people don’t have the right mindset or skills. My entire point is that this kind of hiring process weeds out potentially great candidates based on the false assumption that not having a PhD or some other hard to get qualification somehow makes you unqualified. I think this is the result of this “field” being relatively young in industry and it will change as it gets demystified over the next few years.
  • sidlls 8y ago
• ellisv 8y ago
  You might be a great candidate but how would anyone know? Something on your application needs to stand out. If you don't have a degree that's fine but recruiters need some way to filter through resumes. I'm generally not a fan of cover letters but write one and make a case if you have one.
• whataretensors 8y ago
  This is commonplace. I think it's due to the dilbert effect. If you can't judge candidates based on ability, you have to judge them based on accreditation. Larger organizations tend to have more layers of non-technical people evaluating technical candidates. It's not particular to ML, though ML is worse than others. CS degrees are luckily becoming less needed for basic programming jobs.
• vowelless 8y ago
  This is a bizarre comment. You have 15 years of software development experience and you are surprised this experience doesn't carry over to data science or ML directly? Why should someone bring you in? Do you have a kaggle profile with your notebooks that people can look at? Have you replicated research papers ? Do you have any independent results that you can share, publish, or talk about?
• lordnacho 8y ago
  Well to put it in a data science terms, the employers are looking for precision, at the expense of recall. They don't care that they don't find all the good data scientists so long as the people they do find are good data scientists. I actually think that requiring an advanced degree doesn't help toward that end. In the finance world we call these people quants. And actually in my experience having a phd does very little for someone; the critical skill required is software engineering. Thinking about investment strategies is about testing hypotheses, and you can't test things properly if you don't know a few things about how to organize code. This is actually an insidious problem, because there's nobody telling you how to actually build an alpha generating strategy. And if you can't investigate properly, you fall into all the traps (you make excuses to do the following): too many features, choosing too small a sample that happens to do well, filtering the data in so may ways one of them is bound to "work", and so on. Imagine that you're a chef, but you can't chop stuff effectively. You would then work around that limitation, maybe work on dishes where it's not needed, or perhaps get a junior guy to do the chopping. You might think this solves the problem, but actually it just swaps one problem for another, because now you need to communicate and coordinate with this other person. Or you don't explore that whole area of food with chopped stuff in it. CI pipelines, version control (branching, diffs, etc), database maintenance, a bit of OS basics. All things that tended to differentiate the productive quants from those who merely thought they were useful. I've seen things done in a few weeks that others had spent years not achieving. As for the ML skills themselves, you do need a bit of math to do it, and the math is relatively easy to learn. There's loads of materials to help you as well. What's not explained so much is certain philosophical issues around what is being examined. A course in economics has examples of these things: endogeneity, Lucas critique (which is Hume rehashed), experiment design (do the observations mean what you think?).
• deleted 8y ago
  [deleted]
• sanderjd 8y ago
  I had the same experience recommending private sector jobs to a scientist friend of mine. He just finished a Ph.D. in chemistry at a very prestigious program. This (inevitably) involved doing a lot of analysis of experimental results in Python. "You're not a real data scientist" was the over-arching conclusion of his effort to get a job doing data analysis, even for comparatively lame old-school BI type analyses. This is very silly and unbelievably short-sighted.