10 ms·

Some Reflections on Being Turned Down for a Lot of Data Science Jobs

288 points by emplynx 10y ago

rubayeet 10y ago
"Networking is still king" My takeaway from it.
- wgyn 10y ago
  I think that just "networking" would be the wrong takeaway, though. The author[0] is pretty active on Twitter / the blogosphere and, even though I don't follow him personally, his writing has popped up on my feed a number of times. He's likely meeting people and having interesting conversations ("networking") because he has interesting things to say. [0] https://twitter.com/tdhopper https://twitter.com/tdhopper
  - mindcrime 10y ago
    Not just blogging, but Tim is also active in terms of attending / speaking at various industry conferences and what-not. Whether he is doing it simply to "build his brand" or because he genuinely enjoys it, or both, is something I can't speak to. But I think he's definitely "meeting people and having interesting conversations".
    - emplynx 10y ago
      Hi Phil!
      - mindcrime 10y ago
        Oh no, I've been outed!
pascalxus 10y ago
The real reason you were rejected, or anyone applying in a highly competitive field: an oversupply of qualified candidates.
- ryandrake 10y ago
  But, wait! I thought there was a "shortage of tech workers."
  - x0x0 10y ago
    Which must necessarily be urgently addressed by open immigration for anyone who can write code! The future of silicon valley demands it!
- ska 10y ago
  In this particular case (data science) it is more an oversupply of candidates (qualified and not), plus difficulties defining and measuring "qualified", plus buzz. It's a difficult enough field to hire in when you understand what it is (and isn't) - and lots of companies are trying to do it with far more vague goals.
  - throw_away_777 10y ago
    Why do you think candidates for data science jobs aren't qualified relative to other positions?
    - yummyfajitas 10y ago
      I've probably interviewed about 70-100 such people in the past year and a half. Exactly 1 such person was qualified (I hired him). The issue in my view is the following: people who know both statistics and computer science are extremely rare. People who actually understand statistics are rare. I can probably weed out 1/3 to 1/2 of candidates simply by asking what a p-value is, or what precision/recall are (this includes people who said they worked in search). Of the ones who know basic stats, most are neither good at nor interested in programming. They just want to use existing libraries to crunch numbers in a Jupyter notebook, then hand that off to the developers. Finding a person who can come up with a predictive model, understand what they did, optimize it without breaking it's statistical validity and deploy it to production is very hard. (If you can do this, I'm hiring in Pune and Delhi. Email in my profile.)
CalChris 10y ago
Companies often use interviews as a time to figure out what they're really looking for. For startups, this transcends data science. It might be the one time that week they focus on that need. Networking is still king. Exactly and this also argues against wanting to get hired to work remotely.
- 102142592235ad 10y ago
  >Exactly and this also argues against wanting to get hired to work remotely. True, though not unheard of. I worked at a startup that hired remote devs via networking of that nature. It seemed to work well for them. Eventually the entire team migrated to the same area but things operated smoothly remote for the time being.
- emplynx 10y ago
  Not sure networking is contrary to working remotely. Opportunities for networking online are far bigger than in-person.
  - CalChris 10y ago
    You may be talking about contracting and the OP is talking about hiring. Moreover while the opportunities for networking online are countably infinite they can be thin opportunities. I know Joe from HN doesn't carry the same weight as I know Roger from Google. It carries weight just not nearly as much.
motivic 10y ago
This is a bit off-topic but one thing I'm curious is how the author manages to interview with these companies while holding down a full-time job and keep up-to-date with the latest developments. Interview for a data science position is typically a drawn-out process, with multiple rounds of interviews and possibly take-home projects. I found them to be very time and energy consuming. To share my story, I also had a difficult time transitioning into a data scientist role after leaving academia (pure mathematics), and I always thought the root cause was my lack of experience and competency. So instead of keeping on applying, I spent over a year just to sharpen up my skills. It paid off in the end. How can one develop his/her skills and cultivate expertise if one is job-shopping all the time (possibly aimlessly)?
- motivic 10y ago
  A better but more difficult approach is to distinguish oneself and have the companies go after you.
- eanzenberg 10y ago
  I think being a good DS requires focused learning. For example, when I started out, I didn't know much about neural networks and their different architectures. I tend to find time outside work to go over papers, thesis, or watch lectures on youtube to keep myself up to date so that now I can describe cnn and rnns to tech and non-tech people.
eanzenberg 10y ago
My analytical thought process on DS interviews: - Signal is still quite low among noise, even with long multiple interviews, take-home homework, coding challenges, etc. Most relevant data is still hidden and takes months-years to come out. - Companies seek to minimize false-positives much more than minimizing true-negatives. - It's a numbers game from both ends because the probabilities are low, due to above 2 points.
- FireBeyond 10y ago
  Woe betide the company that asks for a take home project, especially one that isn't just 50 lines of an algorithm (I'm talking one that I recall that asked for parsing Apache logs from a stream, displaying moving averages for URLs, aggregates, having high water mark "alerts" and resetting when rolling averages dropped below that, some unit tests and docs... And doesn't bother to respond after you submit. I'm certainly not the world's best coder, but my solution met all those requirements in a reasonably elegant manner, and took several hours (10-20?) to make comprehensive. That gets you on my shit list, and people I know get to hear about it, too.
  - eanzenberg 10y ago
    Same FireBeyond. Even worse, I've gotten feedback on a take-home project where the solutions had errors in it and I had the correct response! This, btw, was from one of the huge tech-companies in SF. Since it was the recruiter who went through the solutions, I couldn't nor cared to correct their mistakes :)
minimaxir 10y ago
Many recommend setting up an online portfolio for these types of positions. But I've applied to a number of Data Analyst/Scientist jobs recently and I am immediately rejected almost every time despite highlighting my blog/portfolio (http://minimaxir.com http://minimaxir.com) and my GitHub with open-source code/Notebooks for each and every post (https://github.com/minimaxir https://github.com/minimaxir), both of which have topped HN on occasion. Internal recruiters have hinted that my Software QA Engineer background + no CS degree implies I have no technical skill.
- chronicx0 10y ago
  Because there are thousands of people like myself with blog posts, GitHub repos, HN submissions, AND software engineering and/or data science experience already who are being selected over yourself.
- EternalData 10y ago
  I've seen your posts on Spark. That's sad to hear. Are you applying online or via getting coffees with people in the data science team and just talking shop? The latter seems to have a high conversion rate if you can nail it.
  - minimaxir 10y ago
    Mostly online; I've been too busy to network during the weekdays unfortunately, which is something I'm working to fix.
- muninn_ 10y ago
  Yeah. The problem is recruiters are almost universally non-technical, so they can't properly gauge your talent nor do they understand the job requirements fully.
  - cwilkes 10y ago
    I wouldn't limit it to recruiters ... some engineers see the word "QA" or "SDET" and think the guy isn't worthy of being a "real" software engineer, or if they do then it is at the lowest rung of the ladder. Really annoying. In the OP's case I would change their title (if applicable) to "QA / engineer" as a) that probably paints a more accurate picture of what you were doing and b) gets around yokels that look down on QA/SDET.
donovanm 10y ago
Is there a typical path for getting into Data Science? Posts like this make it seem like it's the wild west
- dbecker 10y ago
  I've had a few DS jobs. And I've been a hiring manager at two of those jobs. I can confirm that it's roughly the wild west still.
- emplynx 10y ago
  It's a Wild West.
deleted 10y ago
[deleted]
CSDude 10y ago
At least you have been told the reasons, even if they are true or not. I recently had many tech/behavioral interview with team and passed, but after a review with VP of X, saying we decided not to move forward is way worse than this. I still keeep wondering 'What is wrong with me?' even after years of interviews. I have some suspicions, but never a promising answer. If I oneday found a company, first company policy would be to tell the candidates to tell why they were not hired in a polite way.
- kolinko 10y ago
  How about talking to a therapist / counsellor / coach that specialises in professionals? Even one or two sessions can help you a lot - they are trained in pinpointing "soft" issues.
- jedberg 10y ago
  > first company policy would be to tell the candidates to tell why they were not hired in a polite way. And your lawyers would strongly advocate against that. In most cases they can't/won't tell you because it could open up legal liability. If they say a reason for you, but then hire someone else to whom that reason also applies and you find out, you could sue them. Or you could find a way to twist it into something against a protected class. Too many ways for the company to get screwed. That's why they all say "we've decided to go another way" or something equally generic.
  - yuhong 10y ago
    This is part of why I dislike employment anti-discrimination laws. They probably worked better for manual labor jobs. Of course, there is also firing where the laws vary by state. I think CA is one of the most strict, right?
    - deleted 10y ago
      [deleted]
- emplynx 10y ago
  The reasons are rarely shared unsolicited. It's usually because I asked (no harm in that) or knew someone on the inside.
SatvikBeri 10y ago
I manage a data science team and revamped the hiring process pretty substantially about a year ago, to good results. Nothing in here is particularly original, but here's what we do: 1. Break down "data science" into several different roles–in our case, Analyst (business-oriented), Scientist (stats-heavy), Engineer (software-heavy). Turns out that what we mostly want are Engineers-Analysts, so our process screens heavily for those. 2. Figure out which types of people can be trained to be good at those roles, given the team's current skillset. I opted to look primarily for people with strong analysis skills and some engineering. 3. Design interview tasks/questions that screen for those abilities. In my case, the main thing I did was make sure that the interviews depended very little on pre-existing knowledge, and a lot on resourcefulness/creativity/etc. E.g. the (2-hour) takehome is explicitly designed to be heavily googleable. 4. Develop phone screens that are very good at filtering people quickly, so that we don't waste candidates' time. By the time someone gets to an onsite interview on our team there's something like a 50% chance they'll get an offer. On the candidate side, when I'm applying I try to figure out first and foremost what a company means by "data scientist", usually by networking & talking to someone who already works there. This filters out maybe 90% of jobs with that title, and then I put more serious effort into the rest.
- lifeonahilltop 10y ago
  Can I ask what are the rough percentages of the three types of data science jobs in the industry? And is there an easy way to search for only one type of jobs (such as using some particular keywords on indeed.com)? I have some advantage only in the scientist/stats-heavy type of data science jobs and would like to better focus my job-searching effort. Thanks.
- lukejduncan 10y ago
  This is a great breakdown. Data Science can mean a lot of different things and I think the three roles you defined are spot on. At LinkedIn, what used to be called Data Science is now exclusively analyst roles. We have a separate org (hundreds of people) called Relevance which is part of the engineering org and is composed of scientists and engineers. Everyone in that org falls somewhere on the scientist to engineer spectrum with ML / Stats backgrounds and a bias towards ML engineers. Most of the people who interview would consider the relevance roles "Data Science" but the term within the company is still tightly coupled with pure analyst functions.
deleted 10y ago
[deleted]
strictnein 10y ago
> "Quite honestly given your questions [about vacation policy] and the fact that you are considering other options, [we] may not be the best choice for you." I had a very similar experience. Job offer was basically on the table and then they balked because I mentioned that I had another offer (at a larger company, which they seemed shocked/annoyed with) and I had a question about parking at their new offices. The current offices had free parking, so I had simply asked if the new ones would too. The response was very cold (must have a been the source of internal strife, I guess?). An hour later I got a call saying they were no longer interested.
- draw_down 10y ago
  Wow!
- dsr_ 10y ago
  It is always the case that you are better off learning that the company is dysfunctional before your acceptance than after. It still hurts, of course.
- komali2 10y ago
  Both of your points I find extremely surprising. I've found that mentioning that I have other offers makes me a more competitive pick (this reflects my experience when I was a recruiter as well), and that's very weird that they balked at your question about parking. Honestly you probably dodged a bullet.
  - strictnein 10y ago
    That's been my takeaway too. Company was purchased by a huge private equity firm last fall (KKR), so I probably would have been out of there by now regardless.
- BickNowstrom 10y ago
  Mentioning that you had another offer can be tricky. Did they ask you if you were seeing other companies? Did you just put it out there? Did you put them on a deadline? Mentioning that you have another offer is seen as a negotiating tactic. Then they have to compete/bid against the other offer, and you are removing some of their leverage/power. A strong reaction to such a move can be to cut off the interview, so to regain back their power.
draw_down 10y ago
> "Honestly, I think the way you communicated your thought process and results was confusing for some people in the room." We got confused, and then we stopped. The end.
kafkaesq 10y ago
So you're at least occasionally getting actual feedback from some of these discussions. That's quite something, actually. BTW, as to some of that "feedback": Honestly, I think the way you communicated your thought process and results was confusing for some people in the room. "Okay, pal - let's put your people up in a room full of strangers (some of whom show through their body language and/or constant phone-checking that they pretty much don't really want to be there in the first place), make them answer some made-up questions (which may or may not be coherently presented; or even particularly relevant to the job description) -- combined with the background stress of being unemployed, and/or stuck in job they absolutely HATE, and can barely stand another day of -- and see how they do." Quite honestly given your questions [about vacation policy] and the fact that you are considering other options, [we] may not be the best choice for you. Great -- so they're basically accusing you of being a slacker (not really into the work, only interested in what's in it for you, etc). Which is quite a presumptuous attitude to take in response to a perfectly reasonable question about the value proposition they're asking you to consider (a question for which it might be in better form to wait until the later stags of the negotiation process to bring up... but that's a very minor style point that you definitely shouldn't be dinged for). "Quite honestly" that attitude sucks. And you don't need to feel bad about being "rejected" by people like that.
- FeatureRush 10y ago
  Without context the second quote sounded to me not like an accusation of slacking but like a warning that the company is bad place to work at (poor life-work balance)...
  - jlarocco 10y ago
    Completely agreed. An interaction like that during or after an interview would be a big red flag that it's not a place I want to work.
  - mulmen 10y ago
    I recently asked about the "unlimited vacation" policy and received a vague answer about "establishing expectations with your manager" and then was told they went with someone else. I thought about if I should ask or not but I honestly don't understand how such a policy could be good for employees so I wanted an idea of the culture that allows it. If that question contributed to them not giving me an offer then good riddance so I don't regret asking.
wakkaflokka 10y ago
I was coming from academia with no real competitive background in machine learning/predictive analytics/statistics when I started applying for data science positions. I was coming from a post doc in computational neuroscience, and had done ML/your run-of-the-academic-mill stats as part of my thesis, but never had formal training in it. I landed a lot of interviews (and applied for an ENORMOUS amount of positions), but it took me a while to land a really solid offer. So just keep at it. Once I got that offer and 'data scientist' was listed as a position on my resume, I've had at least one or two recruiters reach out to me each week. Not just your 'bulk tech recruiter', but individual hiring managers/team leads from companies who had no interest in me prior to the title change. Hell, I had a manager I had already interviewed with from another company (and got rejected) reach out to me TWICE to come back in. To be honest, I'm not entirely sure he remembered that I had interviewed there only a month earlier. All after I had the title change on my resume. What I'm getting at, is for me, the title change really opened up doors. I figure 'data science' is such a huge buzzword now, recruiters look more for that buzz word than they do for the actual content of what you've done on your resume. I'm sure it will die down at some point. This may be more prominent outside of silicon valley where I am, and (in my opinion without any facts to back it up), where I feel more weight is given for the title than the substance.
- aeorgnoieang 10y ago
  I still feel grateful for an otherwise-pretty-terrible employer for gifting me with a 'developer' title. Regardless of how much experience or aptitude one has or can demonstrate programming or building software, titles are crucial.
bsg75 10y ago
> Quite honestly given your questions [about vacation policy] and the fact that you are considering other options, [we] may not be the best choice for you. Dodged a bullet on that one: - PTO is a touchy subject? - They are looking at more than one candidate, why would any candidate limit themselves to one potential employer?
- hudibras 10y ago
  Yeah, that's just weird. Paid time off is part of the total compensation package; it would be strange if a candidate didn't ask about it.
  - cwilkes 10y ago
    And if they are touchy about it probably also means that they won't pay it out if you leave ... that's the law in California but not in other states like Washington. I was shocked to hear that, apparently it is somewhat common in retail jobs not to get paid out. Happened to me at a company that claimed to be more of a technology company. Maybe they don't think word gets around. Or that smarter employees will just take a lot of time off before quitting, which does a lot more harm to the company than if they just paid out the time owed to the employee.
anotheryou 10y ago
While I find it reasonable, I would still not ask about vecation policies in the interview, but rather when they decided they want me and we go through the details. Why would you ask before? (honest question)
- emplynx 10y ago
  I actually had an offer in hand, and it was rescinded after I asked about PTO.
  - anotheryou 10y ago
    Than it's good you weeded that one out :)
- maxxxxx 10y ago
  Why not?
  - anotheryou 10y ago
    Not to give them any reason to doubt my working moral.
    - maxxxxx 10y ago
      If asking about vacation policies makes them doubt your working morale I would recommend to run quickly. Is asking for money offensive to them too?
      - anotheryou 10y ago
        It just came to my mind that I'm spoiled by the german vacation standards. It probably makes a lot more sense elsewhere.
deleted 10y ago
[deleted]
alain94040 10y ago
You never really know That's actually the only correct answer. Having been on both sides of the table for many years, I can pretty much guarantee that whatever reason the candidate is given is nowhere close to the actual reasons. There may not be any specific reason why we didn't pick you, but we'll give you a tiny sample anyway. So you think that's the reason - it's not. In other cases, we have a strong reason not to pick you, but it's embarrassing so we feed you a bogus reason instead. And in more cases than I'd like, there is no problem on your side, we are having internal issues that we can't reveal anyway. Any reason you hear from us in that case is completely irrelevant. By the way, when you evaluate people in interview, you really need to figure out a vector: where they are today in terms of knowledge and experience, but also in what direction they are going (fast learner or not, high potential, etc.). Which is why sometimes you can hire someone who has less relevant experience, but you think they'll learn fast and are very smart, and sometimes you are looking for the perfect match to the current position, but don't care too much whether they can pick up new stuff or not.
- deleted 10y ago
  [deleted]
- bbarn 10y ago
  HR Departments have also groomed hiring managers not to ever give a reason why they say no. Every reason you provide is grounds for speculation and potential law suits.
megablast 10y ago
> you rarely truly know why you were turned down. Bit of a silly article. You could say this about anything. And I don't see what the problem is with never being contacted, if someone doesn't want me for whatever reason, I don't really want to hear from them again.
- emplynx 10y ago
  K
gillianlish 10y ago
ironically, the unwashed masses that data science has 'targeted' for 'optimization' go through the exact same process and feel the exact same emotions. apply for 100 jobs, get zero response, and zero reason why. be told illogical things. be told out right lies. welcome to capitalism. welcome to the workplace that is run entirely by data. (or by people who only care about data). welcome to the future of humanity.
ENGNR 10y ago
Clicked hoping he was going to do data science on his failed interview data
carterehsmith 10y ago
That sucks. That said, what is this? a) "I cannot do X" b) "But here is some advice on how to do X" I mean... why would anyone listen to the advice that is proven to lead to failure? Serious question.
segmondy 10y ago
I looked at this resume. it looks good but his job history is a major concern. Of course, some might say why even invite him to an interview, perhaps they gave him a chance only to discover in the interview process that he's really impatient and not willing to stick around for long? There is a ramp up time for new hire, which could be a couple of months. So durations of a year or less doesn't look good. I personally like to see minimum of 2 years for each job. Of course, too long can be a concern too unless they really grew in their role. I agree with his conclusion, Network is king, but I also believe in listening to the universe. If everyone is "wrong", maybe you are the problem. If lots of people don't wish to hire you after you have solid experience in the industry, something is wrong with you and you are being stubborn by refusing to recognize it and fix it.
graphememes 10y ago
I don't have much Data Science job hunting experience, however I have applied to and been hunted by over 100 startups, 24 enterprise companies, and 12 large "startups". I've had the following: - 106 Rejections - 39 Offers - 24 Refused - 15 Contemplated - 5 Taken Roles Applied for: - Software Engineer (55rj, 8o, 6r) - Product Manager (5rj, 6o, 5r) - Senior Software Engineer (31rj, 23o, 22r) - Product Lead (0rj, 1o, 1r) - Senior Product Manager (9rj, 3o, 2r) - VP of E (2rj, 1o, 1r) - CTO (0rj, 1o, 0r) It's been a journey. I don't keep an exact list so this is from looking at my calendar and doing some basic math. This certainly doesn't include all of the role changes within companies. Things to note: - People conflate Agile & Scrum a lot. - Make it easy for people in the room to know what you personally accomplished during your career. - Make an impression, don't overdo it however. - Make your descriptions easy to understand. - Don't just list tech stacks on your resume, list achievements. - Stay consistent. - Ask questions. Also Note: Regardless how impressive your Github & Stackoverflow accounts might be, you will still be asked to do a stupid code challenge. It irks me. I understand the reasoning for it when you don't have a viable pool of information on the individual, but when you do... It's just disrespectful. Few places I have been rejected by: Salesforce, Atlassian, Trello, Twitch, Segment, AirBnB, Twitter, OKCupid, LinkedIn, Shippo, and many many more. Some places I have been offered by: Amazon, VMWare, Oracle, Google (twice), Apple (twice), Venmo, Auth0, Discord, Youtube, Hitbox, Steam, Pusher, Cloudflare
- michaelscott 10y ago
  This is a really interesting set of stats you have here. Heartening in the sense that rejection is a common and overwhelming likelihood and shouldn't be a source of dejection, but also kind of depressing in the sense that it's clear many, many companies (including some big ones) have a terrible hiring process.
  - graphememes 10y ago
    Yes, for example some take months with everyone loving you the entire way, for a single person to interview you and reject you without a reason. I'm glad I had those interviews early on, so now when I see an interview start to drag on, I cut it short as it doesn't look like a priority on their end.
edblarney 10y ago
"Quite honestly given your questions [about vacation policy] and the fact that you are considering other options, [we] may not be the best choice for you." You wouldn't want to work there. " but the team has decided to keep looking for someone who might have more direct neural net experience." Fair enough - but this has to be slim pickins. How many AI jobs are there out there? Realistically, very few.
preordained 10y ago
The too many jobs knock I can understand...I'm just plain old Midwest developer guy, so maybe I don't understand the nuances of a market like Silicon Valley or the like that is inundated with software jobs. Still, when I read HN I often get the impression that there is more focus on getting that next great offer than actually doing a given job...so much so that the ability to be perceived as valuable at interview/screening time seems like the mostly highly focused on or developed skill for some people. It seems like there are many professional interviewees out there these days. I have a hard time understanding. At least for what I've seen/done, it would take about a year of experience to be of any real value. I'd say you start seeing real dividends from an employee near the three year mark. I think the primary exception would be where you have a big gaping hole in an organization...like building a data science program or something from the ground up. But if you've got software and customers long long past the 1.0 stage to support, and someone is going to (someday) understand/contribute to the core? Think about Google's monolith or the Linux kernel...sure most software isn't that mature/grand, but there are many projects out there that are closer to that than greenfield. And it's not just the code, it's the developed relationships/rhythm with coworkers and customers. Maybe I've explained it to myself, I don't know. The difference may be startup companies or projects versus mature ones. At least in the latter case, it seems to me that if a company retains an engineer for less than 2-3 years, they've almost certainly lost on that investment.
- emplynx 10y ago
  You may well be right.
autokad 10y ago
I can't even get a call back for an interview lined up. After hearing people get tons of unsolicited mails on linkedin from recruiters (while I got none), I tried adding more words in my linkedin to show up on searches, but still nothing. Does anyone have any suggestions? Here is my linkedin account: https://www.linkedin.com/in/karl-dailey-02557b65 https://www.linkedin.com/in/karl-dailey-02557b65
LeanderK 10y ago
Well, it also seems like he applied to a lot of jobs (he worked for 5 companies since 2012, for many only under a year!). He must be constantly looking for a new job. I know switching jobs is common here, but i would think that sticking at least 1 to 2 for a job would be normal (assuming it works out).
mattfrommars 10y ago
Ok, this is a little worrying. I've read through the all the conversation here but is it really that difficult to get jobs for people who already are involved in Data Science? Here I was thinking if I do a course on Data Science fundamentals of computer science online, I would on tract of getting a computer science/data science job. My future really rests upon this. I have a degree in mechanical engineering with irrelevant experience. My only regret is not doing bachelors in computer science. I wouldn't lie if all the post got me little tensed and made me thinking, "what am I doing with my time". About to finish introduction to Computer Science using Python on Edx.

288 points by emplynx 10y ago

rubayeet 10y ago

"Networking is still king" My takeaway from it.

wgyn 10y ago
I think that just "networking" would be the wrong takeaway, though. The author[0] is pretty active on Twitter / the blogosphere and, even though I don't follow him personally, his writing has popped up on my feed a number of times. He's likely meeting people and having interesting conversations ("networking") because he has interesting things to say. [0] https://twitter.com/tdhopper https://twitter.com/tdhopper
- mindcrime 10y ago
  Not just blogging, but Tim is also active in terms of attending / speaking at various industry conferences and what-not. Whether he is doing it simply to "build his brand" or because he genuinely enjoys it, or both, is something I can't speak to. But I think he's definitely "meeting people and having interesting conversations".
  - emplynx 10y ago
    Hi Phil!
    - mindcrime 10y ago
      Oh no, I've been outed!

pascalxus 10y ago

The real reason you were rejected, or anyone applying in a highly competitive field: an oversupply of qualified candidates.

ryandrake 10y ago
But, wait! I thought there was a "shortage of tech workers."
- x0x0 10y ago
  Which must necessarily be urgently addressed by open immigration for anyone who can write code! The future of silicon valley demands it!
ska 10y ago
In this particular case (data science) it is more an oversupply of candidates (qualified and not), plus difficulties defining and measuring "qualified", plus buzz. It's a difficult enough field to hire in when you understand what it is (and isn't) - and lots of companies are trying to do it with far more vague goals.
- throw_away_777 10y ago
  Why do you think candidates for data science jobs aren't qualified relative to other positions?
  - yummyfajitas 10y ago
    I've probably interviewed about 70-100 such people in the past year and a half. Exactly 1 such person was qualified (I hired him). The issue in my view is the following: people who know both statistics and computer science are extremely rare. People who actually understand statistics are rare. I can probably weed out 1/3 to 1/2 of candidates simply by asking what a p-value is, or what precision/recall are (this includes people who said they worked in search). Of the ones who know basic stats, most are neither good at nor interested in programming. They just want to use existing libraries to crunch numbers in a Jupyter notebook, then hand that off to the developers. Finding a person who can come up with a predictive model, understand what they did, optimize it without breaking it's statistical validity and deploy it to production is very hard. (If you can do this, I'm hiring in Pune and Delhi. Email in my profile.)

CalChris 10y ago

Companies often use interviews as a time to figure out what they're really looking for. For startups, this transcends data science. It might be the one time that week they focus on that need. Networking is still king. Exactly and this also argues against wanting to get hired to work remotely.

102142592235ad 10y ago
>Exactly and this also argues against wanting to get hired to work remotely. True, though not unheard of. I worked at a startup that hired remote devs via networking of that nature. It seemed to work well for them. Eventually the entire team migrated to the same area but things operated smoothly remote for the time being.
emplynx 10y ago
Not sure networking is contrary to working remotely. Opportunities for networking online are far bigger than in-person.
- CalChris 10y ago
  You may be talking about contracting and the OP is talking about hiring. Moreover while the opportunities for networking online are countably infinite they can be thin opportunities. I know Joe from HN doesn't carry the same weight as I know Roger from Google. It carries weight just not nearly as much.

motivic 10y ago

This is a bit off-topic but one thing I'm curious is how the author manages to interview with these companies while holding down a full-time job and keep up-to-date with the latest developments. Interview for a data science position is typically a drawn-out process, with multiple rounds of interviews and possibly take-home projects. I found them to be very time and energy consuming. To share my story, I also had a difficult time transitioning into a data scientist role after leaving academia (pure mathematics), and I always thought the root cause was my lack of experience and competency. So instead of keeping on applying, I spent over a year just to sharpen up my skills. It paid off in the end. How can one develop his/her skills and cultivate expertise if one is job-shopping all the time (possibly aimlessly)?

motivic 10y ago
A better but more difficult approach is to distinguish oneself and have the companies go after you.
eanzenberg 10y ago
I think being a good DS requires focused learning. For example, when I started out, I didn't know much about neural networks and their different architectures. I tend to find time outside work to go over papers, thesis, or watch lectures on youtube to keep myself up to date so that now I can describe cnn and rnns to tech and non-tech people.

eanzenberg 10y ago

My analytical thought process on DS interviews: - Signal is still quite low among noise, even with long multiple interviews, take-home homework, coding challenges, etc. Most relevant data is still hidden and takes months-years to come out. - Companies seek to minimize false-positives much more than minimizing true-negatives. - It's a numbers game from both ends because the probabilities are low, due to above 2 points.

FireBeyond 10y ago
Woe betide the company that asks for a take home project, especially one that isn't just 50 lines of an algorithm (I'm talking one that I recall that asked for parsing Apache logs from a stream, displaying moving averages for URLs, aggregates, having high water mark "alerts" and resetting when rolling averages dropped below that, some unit tests and docs... And doesn't bother to respond after you submit. I'm certainly not the world's best coder, but my solution met all those requirements in a reasonably elegant manner, and took several hours (10-20?) to make comprehensive. That gets you on my shit list, and people I know get to hear about it, too.
- eanzenberg 10y ago
  Same FireBeyond. Even worse, I've gotten feedback on a take-home project where the solutions had errors in it and I had the correct response! This, btw, was from one of the huge tech-companies in SF. Since it was the recruiter who went through the solutions, I couldn't nor cared to correct their mistakes :)

minimaxir 10y ago

Many recommend setting up an online portfolio for these types of positions. But I've applied to a number of Data Analyst/Scientist jobs recently and I am immediately rejected almost every time despite highlighting my blog/portfolio (http://minimaxir.com http://minimaxir.com) and my GitHub with open-source code/Notebooks for each and every post (https://github.com/minimaxir https://github.com/minimaxir), both of which have topped HN on occasion. Internal recruiters have hinted that my Software QA Engineer background + no CS degree implies I have no technical skill.

chronicx0 10y ago
Because there are thousands of people like myself with blog posts, GitHub repos, HN submissions, AND software engineering and/or data science experience already who are being selected over yourself.
EternalData 10y ago
I've seen your posts on Spark. That's sad to hear. Are you applying online or via getting coffees with people in the data science team and just talking shop? The latter seems to have a high conversion rate if you can nail it.
- minimaxir 10y ago
  Mostly online; I've been too busy to network during the weekdays unfortunately, which is something I'm working to fix.
muninn_ 10y ago
Yeah. The problem is recruiters are almost universally non-technical, so they can't properly gauge your talent nor do they understand the job requirements fully.
- cwilkes 10y ago
  I wouldn't limit it to recruiters ... some engineers see the word "QA" or "SDET" and think the guy isn't worthy of being a "real" software engineer, or if they do then it is at the lowest rung of the ladder. Really annoying. In the OP's case I would change their title (if applicable) to "QA / engineer" as a) that probably paints a more accurate picture of what you were doing and b) gets around yokels that look down on QA/SDET.

donovanm 10y ago

Is there a typical path for getting into Data Science? Posts like this make it seem like it's the wild west

dbecker 10y ago
I've had a few DS jobs. And I've been a hiring manager at two of those jobs. I can confirm that it's roughly the wild west still.
emplynx 10y ago
It's a Wild West.

deleted 10y ago

[deleted]

CSDude 10y ago

At least you have been told the reasons, even if they are true or not. I recently had many tech/behavioral interview with team and passed, but after a review with VP of X, saying we decided not to move forward is way worse than this. I still keeep wondering 'What is wrong with me?' even after years of interviews. I have some suspicions, but never a promising answer. If I oneday found a company, first company policy would be to tell the candidates to tell why they were not hired in a polite way.

kolinko 10y ago
How about talking to a therapist / counsellor / coach that specialises in professionals? Even one or two sessions can help you a lot - they are trained in pinpointing "soft" issues.
jedberg 10y ago
> first company policy would be to tell the candidates to tell why they were not hired in a polite way. And your lawyers would strongly advocate against that. In most cases they can't/won't tell you because it could open up legal liability. If they say a reason for you, but then hire someone else to whom that reason also applies and you find out, you could sue them. Or you could find a way to twist it into something against a protected class. Too many ways for the company to get screwed. That's why they all say "we've decided to go another way" or something equally generic.
- yuhong 10y ago
  This is part of why I dislike employment anti-discrimination laws. They probably worked better for manual labor jobs. Of course, there is also firing where the laws vary by state. I think CA is one of the most strict, right?
  - deleted 10y ago
    [deleted]
emplynx 10y ago
The reasons are rarely shared unsolicited. It's usually because I asked (no harm in that) or knew someone on the inside.

SatvikBeri 10y ago

I manage a data science team and revamped the hiring process pretty substantially about a year ago, to good results. Nothing in here is particularly original, but here's what we do: 1. Break down "data science" into several different roles–in our case, Analyst (business-oriented), Scientist (stats-heavy), Engineer (software-heavy). Turns out that what we mostly want are Engineers-Analysts, so our process screens heavily for those. 2. Figure out which types of people can be trained to be good at those roles, given the team's current skillset. I opted to look primarily for people with strong analysis skills and some engineering. 3. Design interview tasks/questions that screen for those abilities. In my case, the main thing I did was make sure that the interviews depended very little on pre-existing knowledge, and a lot on resourcefulness/creativity/etc. E.g. the (2-hour) takehome is explicitly designed to be heavily googleable. 4. Develop phone screens that are very good at filtering people quickly, so that we don't waste candidates' time. By the time someone gets to an onsite interview on our team there's something like a 50% chance they'll get an offer. On the candidate side, when I'm applying I try to figure out first and foremost what a company means by "data scientist", usually by networking & talking to someone who already works there. This filters out maybe 90% of jobs with that title, and then I put more serious effort into the rest.

lifeonahilltop 10y ago
Can I ask what are the rough percentages of the three types of data science jobs in the industry? And is there an easy way to search for only one type of jobs (such as using some particular keywords on indeed.com)? I have some advantage only in the scientist/stats-heavy type of data science jobs and would like to better focus my job-searching effort. Thanks.
lukejduncan 10y ago
This is a great breakdown. Data Science can mean a lot of different things and I think the three roles you defined are spot on. At LinkedIn, what used to be called Data Science is now exclusively analyst roles. We have a separate org (hundreds of people) called Relevance which is part of the engineering org and is composed of scientists and engineers. Everyone in that org falls somewhere on the scientist to engineer spectrum with ML / Stats backgrounds and a bias towards ML engineers. Most of the people who interview would consider the relevance roles "Data Science" but the term within the company is still tightly coupled with pure analyst functions.

deleted 10y ago

[deleted]

strictnein 10y ago

> "Quite honestly given your questions [about vacation policy] and the fact that you are considering other options, [we] may not be the best choice for you." I had a very similar experience. Job offer was basically on the table and then they balked because I mentioned that I had another offer (at a larger company, which they seemed shocked/annoyed with) and I had a question about parking at their new offices. The current offices had free parking, so I had simply asked if the new ones would too. The response was very cold (must have a been the source of internal strife, I guess?). An hour later I got a call saying they were no longer interested.

draw_down 10y ago
Wow!
dsr_ 10y ago
It is always the case that you are better off learning that the company is dysfunctional before your acceptance than after. It still hurts, of course.
komali2 10y ago
Both of your points I find extremely surprising. I've found that mentioning that I have other offers makes me a more competitive pick (this reflects my experience when I was a recruiter as well), and that's very weird that they balked at your question about parking. Honestly you probably dodged a bullet.
- strictnein 10y ago
  That's been my takeaway too. Company was purchased by a huge private equity firm last fall (KKR), so I probably would have been out of there by now regardless.
BickNowstrom 10y ago
Mentioning that you had another offer can be tricky. Did they ask you if you were seeing other companies? Did you just put it out there? Did you put them on a deadline? Mentioning that you have another offer is seen as a negotiating tactic. Then they have to compete/bid against the other offer, and you are removing some of their leverage/power. A strong reaction to such a move can be to cut off the interview, so to regain back their power.

draw_down 10y ago

> "Honestly, I think the way you communicated your thought process and results was confusing for some people in the room." We got confused, and then we stopped. The end.

kafkaesq 10y ago

So you're at least occasionally getting actual feedback from some of these discussions. That's quite something, actually. BTW, as to some of that "feedback": Honestly, I think the way you communicated your thought process and results was confusing for some people in the room. "Okay, pal - let's put your people up in a room full of strangers (some of whom show through their body language and/or constant phone-checking that they pretty much don't really want to be there in the first place), make them answer some made-up questions (which may or may not be coherently presented; or even particularly relevant to the job description) -- combined with the background stress of being unemployed, and/or stuck in job they absolutely HATE, and can barely stand another day of -- and see how they do." Quite honestly given your questions [about vacation policy] and the fact that you are considering other options, [we] may not be the best choice for you. Great -- so they're basically accusing you of being a slacker (not really into the work, only interested in what's in it for you, etc). Which is quite a presumptuous attitude to take in response to a perfectly reasonable question about the value proposition they're asking you to consider (a question for which it might be in better form to wait until the later stags of the negotiation process to bring up... but that's a very minor style point that you definitely shouldn't be dinged for). "Quite honestly" that attitude sucks. And you don't need to feel bad about being "rejected" by people like that.

FeatureRush 10y ago
Without context the second quote sounded to me not like an accusation of slacking but like a warning that the company is bad place to work at (poor life-work balance)...
- jlarocco 10y ago
  Completely agreed. An interaction like that during or after an interview would be a big red flag that it's not a place I want to work.
- mulmen 10y ago
  I recently asked about the "unlimited vacation" policy and received a vague answer about "establishing expectations with your manager" and then was told they went with someone else. I thought about if I should ask or not but I honestly don't understand how such a policy could be good for employees so I wanted an idea of the culture that allows it. If that question contributed to them not giving me an offer then good riddance so I don't regret asking.

wakkaflokka 10y ago

I was coming from academia with no real competitive background in machine learning/predictive analytics/statistics when I started applying for data science positions. I was coming from a post doc in computational neuroscience, and had done ML/your run-of-the-academic-mill stats as part of my thesis, but never had formal training in it. I landed a lot of interviews (and applied for an ENORMOUS amount of positions), but it took me a while to land a really solid offer. So just keep at it. Once I got that offer and 'data scientist' was listed as a position on my resume, I've had at least one or two recruiters reach out to me each week. Not just your 'bulk tech recruiter', but individual hiring managers/team leads from companies who had no interest in me prior to the title change. Hell, I had a manager I had already interviewed with from another company (and got rejected) reach out to me TWICE to come back in. To be honest, I'm not entirely sure he remembered that I had interviewed there only a month earlier. All after I had the title change on my resume. What I'm getting at, is for me, the title change really opened up doors. I figure 'data science' is such a huge buzzword now, recruiters look more for that buzz word than they do for the actual content of what you've done on your resume. I'm sure it will die down at some point. This may be more prominent outside of silicon valley where I am, and (in my opinion without any facts to back it up), where I feel more weight is given for the title than the substance.

aeorgnoieang 10y ago
I still feel grateful for an otherwise-pretty-terrible employer for gifting me with a 'developer' title. Regardless of how much experience or aptitude one has or can demonstrate programming or building software, titles are crucial.

bsg75 10y ago

> Quite honestly given your questions [about vacation policy] and the fact that you are considering other options, [we] may not be the best choice for you. Dodged a bullet on that one: - PTO is a touchy subject? - They are looking at more than one candidate, why would any candidate limit themselves to one potential employer?

hudibras 10y ago
Yeah, that's just weird. Paid time off is part of the total compensation package; it would be strange if a candidate didn't ask about it.
- cwilkes 10y ago
  And if they are touchy about it probably also means that they won't pay it out if you leave ... that's the law in California but not in other states like Washington. I was shocked to hear that, apparently it is somewhat common in retail jobs not to get paid out. Happened to me at a company that claimed to be more of a technology company. Maybe they don't think word gets around. Or that smarter employees will just take a lot of time off before quitting, which does a lot more harm to the company than if they just paid out the time owed to the employee.

anotheryou 10y ago

While I find it reasonable, I would still not ask about vecation policies in the interview, but rather when they decided they want me and we go through the details. Why would you ask before? (honest question)

emplynx 10y ago
I actually had an offer in hand, and it was rescinded after I asked about PTO.
- anotheryou 10y ago
  Than it's good you weeded that one out :)
maxxxxx 10y ago
Why not?
- anotheryou 10y ago
  Not to give them any reason to doubt my working moral.
  - maxxxxx 10y ago
    If asking about vacation policies makes them doubt your working morale I would recommend to run quickly. Is asking for money offensive to them too?
    - anotheryou 10y ago
      It just came to my mind that I'm spoiled by the german vacation standards. It probably makes a lot more sense elsewhere.

deleted 10y ago

[deleted]

alain94040 10y ago

You never really know That's actually the only correct answer. Having been on both sides of the table for many years, I can pretty much guarantee that whatever reason the candidate is given is nowhere close to the actual reasons. There may not be any specific reason why we didn't pick you, but we'll give you a tiny sample anyway. So you think that's the reason - it's not. In other cases, we have a strong reason not to pick you, but it's embarrassing so we feed you a bogus reason instead. And in more cases than I'd like, there is no problem on your side, we are having internal issues that we can't reveal anyway. Any reason you hear from us in that case is completely irrelevant. By the way, when you evaluate people in interview, you really need to figure out a vector: where they are today in terms of knowledge and experience, but also in what direction they are going (fast learner or not, high potential, etc.). Which is why sometimes you can hire someone who has less relevant experience, but you think they'll learn fast and are very smart, and sometimes you are looking for the perfect match to the current position, but don't care too much whether they can pick up new stuff or not.

deleted 10y ago
[deleted]
bbarn 10y ago
HR Departments have also groomed hiring managers not to ever give a reason why they say no. Every reason you provide is grounds for speculation and potential law suits.

megablast 10y ago

> you rarely truly know why you were turned down. Bit of a silly article. You could say this about anything. And I don't see what the problem is with never being contacted, if someone doesn't want me for whatever reason, I don't really want to hear from them again.

emplynx 10y ago
K

gillianlish 10y ago

ironically, the unwashed masses that data science has 'targeted' for 'optimization' go through the exact same process and feel the exact same emotions. apply for 100 jobs, get zero response, and zero reason why. be told illogical things. be told out right lies. welcome to capitalism. welcome to the workplace that is run entirely by data. (or by people who only care about data). welcome to the future of humanity.

ENGNR 10y ago

Clicked hoping he was going to do data science on his failed interview data

carterehsmith 10y ago

That sucks. That said, what is this? a) "I cannot do X" b) "But here is some advice on how to do X" I mean... why would anyone listen to the advice that is proven to lead to failure? Serious question.

segmondy 10y ago

I looked at this resume. it looks good but his job history is a major concern. Of course, some might say why even invite him to an interview, perhaps they gave him a chance only to discover in the interview process that he's really impatient and not willing to stick around for long? There is a ramp up time for new hire, which could be a couple of months. So durations of a year or less doesn't look good. I personally like to see minimum of 2 years for each job. Of course, too long can be a concern too unless they really grew in their role. I agree with his conclusion, Network is king, but I also believe in listening to the universe. If everyone is "wrong", maybe you are the problem. If lots of people don't wish to hire you after you have solid experience in the industry, something is wrong with you and you are being stubborn by refusing to recognize it and fix it.

graphememes 10y ago

I don't have much Data Science job hunting experience, however I have applied to and been hunted by over 100 startups, 24 enterprise companies, and 12 large "startups". I've had the following: - 106 Rejections - 39 Offers - 24 Refused - 15 Contemplated - 5 Taken Roles Applied for: - Software Engineer (55rj, 8o, 6r) - Product Manager (5rj, 6o, 5r) - Senior Software Engineer (31rj, 23o, 22r) - Product Lead (0rj, 1o, 1r) - Senior Product Manager (9rj, 3o, 2r) - VP of E (2rj, 1o, 1r) - CTO (0rj, 1o, 0r) It's been a journey. I don't keep an exact list so this is from looking at my calendar and doing some basic math. This certainly doesn't include all of the role changes within companies. Things to note: - People conflate Agile & Scrum a lot. - Make it easy for people in the room to know what you personally accomplished during your career. - Make an impression, don't overdo it however. - Make your descriptions easy to understand. - Don't just list tech stacks on your resume, list achievements. - Stay consistent. - Ask questions. Also Note: Regardless how impressive your Github & Stackoverflow accounts might be, you will still be asked to do a stupid code challenge. It irks me. I understand the reasoning for it when you don't have a viable pool of information on the individual, but when you do... It's just disrespectful. Few places I have been rejected by: Salesforce, Atlassian, Trello, Twitch, Segment, AirBnB, Twitter, OKCupid, LinkedIn, Shippo, and many many more. Some places I have been offered by: Amazon, VMWare, Oracle, Google (twice), Apple (twice), Venmo, Auth0, Discord, Youtube, Hitbox, Steam, Pusher, Cloudflare

michaelscott 10y ago
This is a really interesting set of stats you have here. Heartening in the sense that rejection is a common and overwhelming likelihood and shouldn't be a source of dejection, but also kind of depressing in the sense that it's clear many, many companies (including some big ones) have a terrible hiring process.
- graphememes 10y ago
  Yes, for example some take months with everyone loving you the entire way, for a single person to interview you and reject you without a reason. I'm glad I had those interviews early on, so now when I see an interview start to drag on, I cut it short as it doesn't look like a priority on their end.

edblarney 10y ago

"Quite honestly given your questions [about vacation policy] and the fact that you are considering other options, [we] may not be the best choice for you." You wouldn't want to work there. " but the team has decided to keep looking for someone who might have more direct neural net experience." Fair enough - but this has to be slim pickins. How many AI jobs are there out there? Realistically, very few.

preordained 10y ago

The too many jobs knock I can understand...I'm just plain old Midwest developer guy, so maybe I don't understand the nuances of a market like Silicon Valley or the like that is inundated with software jobs. Still, when I read HN I often get the impression that there is more focus on getting that next great offer than actually doing a given job...so much so that the ability to be perceived as valuable at interview/screening time seems like the mostly highly focused on or developed skill for some people. It seems like there are many professional interviewees out there these days. I have a hard time understanding. At least for what I've seen/done, it would take about a year of experience to be of any real value. I'd say you start seeing real dividends from an employee near the three year mark. I think the primary exception would be where you have a big gaping hole in an organization...like building a data science program or something from the ground up. But if you've got software and customers long long past the 1.0 stage to support, and someone is going to (someday) understand/contribute to the core? Think about Google's monolith or the Linux kernel...sure most software isn't that mature/grand, but there are many projects out there that are closer to that than greenfield. And it's not just the code, it's the developed relationships/rhythm with coworkers and customers. Maybe I've explained it to myself, I don't know. The difference may be startup companies or projects versus mature ones. At least in the latter case, it seems to me that if a company retains an engineer for less than 2-3 years, they've almost certainly lost on that investment.

emplynx 10y ago
You may well be right.

autokad 10y ago

I can't even get a call back for an interview lined up. After hearing people get tons of unsolicited mails on linkedin from recruiters (while I got none), I tried adding more words in my linkedin to show up on searches, but still nothing. Does anyone have any suggestions? Here is my linkedin account: https://www.linkedin.com/in/karl-dailey-02557b65 https://www.linkedin.com/in/karl-dailey-02557b65

LeanderK 10y ago

Well, it also seems like he applied to a lot of jobs (he worked for 5 companies since 2012, for many only under a year!). He must be constantly looking for a new job. I know switching jobs is common here, but i would think that sticking at least 1 to 2 for a job would be normal (assuming it works out).

mattfrommars 10y ago

Ok, this is a little worrying. I've read through the all the conversation here but is it really that difficult to get jobs for people who already are involved in Data Science? Here I was thinking if I do a course on Data Science fundamentals of computer science online, I would on tract of getting a computer science/data science job. My future really rests upon this. I have a degree in mechanical engineering with irrelevant experience. My only regret is not doing bachelors in computer science. I wouldn't lie if all the post got me little tensed and made me thinking, "what am I doing with my time". About to finish introduction to Computer Science using Python on Edx.