37 ms·
How to hire engineering talent without the BS
- lbriner 4y agoI agree with others that these posts only have limited value perhaps to people who don't have a clue. For those of us who have done this lots, it doesn't really say much but it does allude to a common fallacy, that the interview itself is the most important part. Other things that are actually useful to consider (others have mentioned these too). 1) Your company culture is important for you to know; for you to codify and for you to communicate to your candidate. 2) In some companies, you will have tonnes of unsuitable applicants because of your brand, you should not optimise for people that aren't suitable - filter them asap 3) Your whole onboarding is a lot wider than just an interview. For many of us, we should be asking where/how we would expect to find our candidates and optimise those places. Do we visit hackathons? Is our recruitment page(s) clear and does it articulate what we want? 4) Your entire process needs to be like an interative development. Did you hire a bad person? What could you change to catch that in an interview? Wrong technical questions? Not enough about comms or culture? 5) In many cases, a company will hire someone that comes across well even if they don't necessarily tick the boxes so don't assume that you didn't get the job because you failed the whiteboard test, maybe you weren't as good as you thought? 6) Candidates need to think carefully about their approach to interviews, I would say more candidates than not seem almost entirely unprepared for normal interview questions and perhaps expect their ambience will get them the job! Study, swot up, never used owasp for web apps? Don't say that in an interview, spend 10 minutes learning about the top 10 and answer confidently. 7) Recruiters are in it for large commissions. They kind of care only in-as-much as getting a good reputation might help them but everything is second to money so don't expect them to place you well, don't expect them to sell you properly and if you get rejected, don't expect them to call you!
- Eumenes 4y agoembedded tweets, comics, memes, and cartoons ... dumbing down a complex subject. Next.
- einhverfr 4y agoGenerally good tips, but I have noticed a few other things too. 1. If there are challenges, particularly if they are take home tests, it is important to make these reflect the sort of work someone will do without raising concerns that the work will be used by the company without pay. Candidates will spend time on relevant challenges and be happy. They will not be happy about irrelevant challenges. And interviews go both ways. 2. Dispense with "good questions" and go instead with "what do I want to know about a candidate. 3. Ask yourself before you start hiring, "What makes those who are successful at this company successful?" And from there, start building your interview structure. Not every company will be the same, or will be good matches for the same candidates. The key should be to figure out what you need and use the interview to determine if the candidate actually is a good fit. Unfortunately this cannot have data because it relies on a bunch of human judgment calls.
- warmcat 4y agoQuestion for the greater HN community? Would it help if software engineers demonstrate their skills on an ongoing basis via a third party which accesses them periodically without the pressure cooker environment of technical interviews and companies just do cultural fit type of interviews based on the previous assessments of their technical skills?
- JonChesterfield 4y agoIt'll be borderline impossible to persuade experienced engineers to do that.
- warmcat 4y agoI think so too. But the incentive of not doing tech interviews the traditional way will be a bit tempting too.
- natasiajackson 4y ago[flagged]
- joanne123 4y ago[dead]
- george574 4y ago[flagged]
- jesalg 4y agoAs someone who's been around the tech industry for a while, I know firsthand all the sausage-making that goes into building a great technical hiring funnel. On the flip side, as a job seeker, I also know how demoralizing it can be to go through a broken hiring process that doesn't accurately reflect your abilities. With recent layoffs and many talented professionals on the job market, I was compelled to write a blog post about how to build an inclusive hiring culture and find exceptional engineering talent. If you're involved in your organization's technical hiring process at any stage, I encourage you to give this a read. I share some best practices for conducting effective interviews and improving your own hiring process. Let me know what you think!
- realjhol 4y ago> inclusive hiring culture and find exceptional engineering talent. Thats a contradiction in terms. Building something exceptional always involves excluding mediocrity
- mikrl 4y agoNo it is not. You can exclude mediocrity while also being exclusionary on other axes. They are unrelated issues. In fact, I’ve even heard of exceptional people being abused/bullied for belonging to the wrong group to the point of being told “you couldn’t have done that” which itself is an assertion of their supposed mediocrity for exclusionary reasons. Don’t assume you need to be bigoted to exclude mediocrity. Discriminating, yes, but not discriminatory against groups that inclusive hiring policies attempt to protect.
- jbmsf 4y agoI'm sure that sounded smart when you wrote it. An inclusive interviewing process does not mean that you hire everyone. It means you reduce the weight of people's biases as part of identifying who you hire (because people turn out to be quite bad at prediction in hiring).
- rfrey 4y agoWell, felt smart, anyhow.
- jbmsf 4y agoI agree with the conclusions, though I've seen the structured part go wrong, e.g. the interviewer is so dedicated to following the structure of the process that they forget about the empathetic part. These interviews look more like scripts than exploration of a candidate. So I'd add another criteria: interviewers need to be trained!
- criteriums 4y agoSingular: Criterion Plural: Criteria SCNR
- grrdotcloud 4y agoMy biggest recommendation is that those working directly with the new hire, peers, direct reports, subordinates, counterparts, have a vote or veto power. HR and recruiting relationships are sparse at best.
- toomuchtodo 4y agoHave to also have a challenge process. I built an infosec talent pipeline for a fintech; stakeholders get a veto but the hiring mgr can go back to the veto voter and ask them to dive deeper (and possibly perform an additional candidate call) to confirm the veto. >1 veto = no hire. It’s working well, and has avoided at least two false negatives since implementation within the last six months.
- kortilla 4y agoHave the “1 veto but hire anyway” cases gone poorly?
- criteriums 4y agoA scenario in which I can imagine this backfiring is if the new hire and the vetoer end up working together. With an open mind that could be overcome, as in getting positively surprised, but let's be honest, how many people do you know who really have such an open mind?
- toomuchtodo 4y agoNo, both going great, which means (it appears) I’m balancing the org’s health and need to succeed with giving candidates an opportunity they might not otherwise have had. The purpose of the system is what it does. If desired state is not emerging, we must adjust and observe accordingly.
- zeroonetwothree 4y agoIt sounds good but do we have any evidence that this actually works? There’s so many of these speculative “how to interview” posts but it’s all just cargo culting.
- humanrebar 4y agoIf you really want to hire engineering talent, paying above "competitive salary" is very important. It's a bit orthogonal to the concerns in this article, but in some ways it's much more important. What I wonder about is given an org that is able and willing to compensate at market clearing rates, how do they get the word out well enough to get engineers interested. Because the other big BS in hiring is the whole recruiting side of things.
- Zetice 4y agoThis is focused on finding technically skilled engineers, but I think you can get a more wholistic (holistic?) view of the person by asking them to walk you through their work history, project by project, and call a subset of the people they’ve worked with. It’s more conversational, and you don’t have to live in hypotheticals. We all know that skilled engineers will learn whatever skills they need to on the job, so less and less am I interested in what they can do in the interview pressure cooker.
- sokoloff 4y ago> the best experiences were when the interviewer wanted me to succeed, was emphatic I assume you mean empathetic. Same word is spelled “Emphethatic” later. (I tried finding a way to reach you privately, but your site “about” says you have contact methods on the left but, on mobile, there is no left…so here will have to do.)
- jzombie 4y agoThe next time any recruiter asks me to do a "homework" assignment, I will ask them to write me a 15-page essay explaining how that homework assignment will actually be used, to what extent it will be reviewed, and what criteria it will be judged on. If the comeback remark is something like, "if you really want this job," I will reply, "if you really want to hire me." My current job, I told them that I was too busy to do such a thing (and I was), and got hired anyway. Nobody w/ actual responsibilities in their life should be coerced into doing something for free, for someone they do not know. Will an attorney give you free legal advice until you decide they are fit to represent you, or will you get free surgery until someone proves they won't completely butcher you? Can you drive a car for free (for 5 - 8 hours) until you decide that's the car you want to buy? Why is it any different in the software industry? Because we just clack on our keyboards all day and do nothing?
- disruptiveink 4y agoBecause take-home exercises are both the best way to evaluate a candidate (I have never gotten a nasty "surprise" while hiring in companies that did take-home exercises: I mean it, absolute 0% "bad hire" rate, the only bad hire I got was a candidate that was allowed to skip the take home exercise) and they allow you to find absolute jewels amongst the candidate pool who are great, reliable engineers but regularly bomb regular interviews. While there are people who are naturally great at interviewing in person or who don't mind grinding leetcode for free for months on end, there is a whole "base of the iceberg" population who a) can't spend months grinding leetcode on end but they can definitely spare a couple of afternoons for one job they are interviewing for – these are normal people with normal jobs and a normal family who don't interview every month just for kicks, they do this a couple of times every 2-3 years, and, b) just cannot code while under stress. You may scoff at b) if you've never experienced it, and while I found it really hard to understand for a while as I also have no issues with getting into deep focus while people are looking and/or trying to talk to me and there's a time limit, it is absolutely real. Unfortunately, there is an extremely vocal minority (you) who go absolutely ballistic when asked to do a take-home exercise, who absolutely ruin it for everyone else and make hiring managers shit their pants whenever take-home exercises are suggested. I honestly don't understand why there's such an outrage, take-home exercises are the minority already, because there's a strange huge backlash. Sure, if you're a great communicator (usually native English speaker) and grind leetcode for fun so you can shove 5 interviews in one week, you hate take-home exercises. And that's fine, really, just please apply to the leetcode grinding contests and stop poisoning the well for the rest of us who would like to cater to the large amount of extremely competent engineers that don't fit that persona. > Nobody w/ actual responsibilities in their life should be coerced into doing something for free, for someone they do not know. I take it you haven't interviewed for a SE in a while? The status quo requires you to perform months of unpaid labor, not days, just in the form of memorizing the solutions to every single Leetcode Hard problem. How is that better?
- howling 4y ago> Google: 90% of our engineers use the software you wrote (Homebrew), but you can’t invert a binary tree on a whiteboard so fuck off. — Max Howell (@mxcl) If inverting a binary tree means swapping the left and right subtrees of every node, I wouldn't want to work with someone who can't do that either and Google is definitely right to reject him.
- regularjack 4y agoNot knowing how to do something from the top of your head is not the same thing as not being able to do it.
- deathanatos 4y ago… but that's an interview. If you cannot, within the time, demonstrate any ability, why should you be hired? The question above, as clarified, is not complicated, nor does it rely on memorization or some "trick": anyone purporting to be a SWE should be able to write an essentially de novo solution to it. (And in my own technical interviews, there are multiple questions, to specifically hedge against any one being "that one question a good candidate is going to miss because it's just not their day". It doesn't happen: it's either all or nothing.)
- jmd42 4y agoI'm on the same page. Sure, your day-to-day work may not involve manipulating binary trees. But presumably it does involve working with variables, objects, references, manipulating data of some kind... And if you're comfortable with the fundamentals of those, then this is something you should be able to figure out even if you've never heard of a "binary tree" before, once somebody has sketched it or shown you the definition of their TreeNode class, right? It honestly baffles me how people consider this something which needs to be drilled or memorized. There are absolutely algorithmic questions which would fall into that category. But if somebody considers this to be one of them - or something like "find the smallest number in an array" - then I have to question whether they have an understanding of the most fundamental concepts in programming... Or if they get through each day solely using things they've memorized by rote, or looked up, and they don't really have any idea how any of the foundations they're building on actually operate.
- 5350-uiop-1130 4y agoDev for 16 years. The people who do the interviewing are themselves different now. They have different values. Different expectations of what is normal or important. "Culture", "team fit" and other bs. Everything changes. New people come who don't know the past.
- 0xB31B1B 4y agoThis misses 90% of what I, a startup CTO, find valuable in technical hires. What I want to know is: what type of projects have you worked on, how did you develop expertise in those systems, what level of ownership over your work did you display, how well were you able to plan and design the solution to a problem, and how did you handle the execution over the X months of work to make it go live. Demonstrate expertise, curiosity, and ownership. “System design” is like 5% of the work we do, and it’s important, but putting the designs in motion and driving value from them is 95% of our time and that’s something we do not screen well for. The way that I do this now is a process with a soft skills interview, a coding interview, then a “case study/system design” interview where I have candidates write a system design doc at home for a project they have worked on IRL and use that as a starting point for a 45 minute panel convo where we review the doc and ask questions about their choices and how execution went.
- gardenhedge 4y agoIs that 3 separate interviews for your startup? Or is it done in one session? If it is the former you're missing out on lots of potential candidates
- teaearlgraycold 4y agoWhy would you say no to a job because they want you to interview for a few hours? As a candidate I always respect the employers more if they can put together a coherent 4+ hour interview. Why would I want to work somewhere where my co-workers were only briefly vetted?
- gardenhedge 4y agoIt's not "no" to the job. It's "no" to the interview(s) process.
- teaearlgraycold 4y agoThey're tied together. Once you join you'll be in a place that uses that interview process.
- ffssffss 4y agoIt's not a bad post per se but we've been reading similar, anecdotal blogs like this about making the interview process kinder for decades. Yet the only companies in a position to do a rigorous statistical test - large tech cos - stick with the traditional, somewhat adversarial whiteboarding process. I would even suggest that a strictly technical whiteboarding process can be less biased than what the author describes, because you can so regularly grade everyone on the same exact rubric. That's tougher when "pair programming" or doing a take home. Also, stop giving take home projects. Bad candidates will cheat them and good candidates will not even do them. If one of the random startup names listed on the author's site sent me a 12 hour take home project I would delete the email. Do you think they pay twice as much as the bigger company that only makes you waste 6 hours doing a whiteboard? I doubt it.
- fallingknife 4y agoLarger tech cos with very high TC know that they will always have a pipeline of more qualified candidates, so a false negative has basically no cost to them, whereas a false positive has a significant cost. I think the reason that they run these processes is because they have a very low false positive rate, and so long as that is true, it doesn't matter to them how high the false negative rate is. And I think that smaller companies copy this as a part of the tendency to copy large companies without thinking about whether the thing they are copying actually makes sense at their scale. In this case it can be very damaging, because false negative for a startup with a limited pipeline can be very bad.
- ryandrake 4y agoAlso, as far as I know, nobody measures false negatives in hiring. How would you even do it? Keep track of everyone you rejected, and then 5, 10, and 20 years later check on their career? I'd be fascinated to see the results of such a study. I'm sure it would be super valuable if you could find some kind of pattern where some filter is falsely excluding candidates that are actually great.
- StevenWaterman 4y ago
- sophonX 4y agoYou should mention, what kind of teams you've worked with and what kind of stuff you've built. Every team has different requirements and hiring bar. In my previous company the low bar caused not so good (able to understand stuff, knowledge and connect dots) people be a burden to rest of team. Heck in 2 years, 4 important people have left the team due to hiring a bad manager (has neither tech nor soft skill(s)). There wasn't growth in that team due to mediocre hiring and eventually all the good ones - left to other companies. My current team is an infra platform and has lot of growth as IC. Everyone is learning something in-depth and are explorers - rather than blind sheep. The bar here is higher than the one for my previous team. Our team requires you to know about whatever you talk on, not just usage but it's internals - why ? That's what we do daily. It can be about scheduler, checkpointing, auto scaling, concurrency, different data structures & algos, integrating with ecosystem, etc. Even soft skills - like helping others, taking feedback, communicating clearly, etc. Yeah so, mediocre will always be a burden to team.
- snozolli 4y agoSince it mentions the infamous interview challenge, I'll ask: has anyone ever "inverted" (i.e. swapped left and right recursively) a binary tree in production code? I can't think of any reason why anyone would ever do this. Just navigate the tree in the reverse of your normal direction instead. Why not ask the much more interesting and potentially useful question of balancing a binary tree? Or do something else recursive, if that's what you're after.
- sophonX 4y agoYou are just ignorant. You should read about distributed systems. Google around for Zookeeper and it's zab paper. Every (recently with raft protocol) multi master distributed system out there interacts with Zookeeper for assigning leader and maintain configuration. Do you know how the syntax or api calls look like ? They are node path in a tree. You want to store something ? That's a tree path again. You want to listen to some change ? It's tree path again. As I said, most people are ignorant here and don't do "true" computer science engineering in daily life. Most devs simply convert business logic into bunch of apis + adhoc implementation. Did you ever work at a banking firm ? I've read their codebase - they are structured as trees, every damn thing is tree. It's a headache to navigate, code, heck even the objects are literal trees. As I said, people are ignorant and think world revolves around them.
- jmd42 4y ago> Why not ask the much more interesting and potentially useful question of balancing a binary tree? Honestly? Because that's harder. The swapping question is basically a softball / FizzBuzz-style question to test the most basic familiarity with data structures, pointers/references, and recursion.
- regularjack 4y ago> Why not ask the much more interesting and potentially useful question of balancing a binary tree? Or even just when would you use a binary tree? Figuring out which data structure is appropriate for the problem at hand is the hard part, how to implement operations on the data structure is easy in comparison, you can just Google it.
- deleted 4y ago[deleted]
- Keyframe 4y agoOn a smaller size (company) it's relatively easy: pay well, don't oversell the position, send a small assignment representative of work and give them ample time to solve on their own OR ask for references you can talk to from previous workplaces. Game changes if you actively contacted someone.. if you're no BS, assumption is you know who you contacted and why, hence only thing to do, once contact established is not to oversell and pay well. Pay-well can constitue compensation as well as time.
- 908B64B197 4y agoI think there's a fundamental misunderstanding about the purpose of the whiteboard interview. The point is to eliminate, as fast as possible, candidates who simply cannot code [0] [1]. You can't do that with a take-home (and I'm against take home as the signal to noise ratio is too low) because people will cheat and have them done by someone else. I've heard horror story of a "senior" engineer from "his country's top school" being interviewed for a technical position by several non-technical managers and HR reps. They only included an engineer in the final round, which was basically supposed to be rubberstamped anyways. He was then asked to implement something trivial like fizzbuzz or wordcount on the whiteboard. The candidate then became extremely defensive and tried to argue that such task was "beneath him", arguing for a good 15 minutes why he shouldn't have to do it. Then the dev just left the room and said that he used this question as a warmup with new hires and it typically takes them less than 10 minutes. Now, a lot of folks do whiteboard interviews wrong. They often expect to get the exact implementation of an algorithm they found in a textbook and for code on the board to compile. This isn't the point of whiteboarding. Doing this only promotes rote memorization. A good whiteboard interview should be a toy problem that can be solved in several different ways by using different strategies or data-structures. The idea is to see how the candidate will break down the problem. Is the candidate able to formulate test cases, write a simple implementation, verify his code and correct the implementation should it fail a test? On the more meta side of things is the candidate able to take feedback and explain why a certain strategy was chosen? Of course it's not representative of real world engineering but it's a good way to peek at someone's ability to debug and reason about programs; these abilities translate well into debugging and design. Especially at the college level, I really can't make any assumptions on what the candidates know. I'm not judging their knowledge of the standard library of X programming language or the framework-du-jour but their ability to learn it fast. Now the hard part isn't so much to create an interview process that works well, but to create a pipeline that feeds into this interview process that has a high signal to noise ratio. In my experience, the best predictors of a good signal to noise ratio was to select for CS fundamentals, good references and offer above market comp. The latter is especially crucial now since there's no more "local market" to speak of now that remote work is a lot prevalent. The "local market's" best devs are working for SV firms at SV salaries mentoring SV employees. [0] https://blog.codinghorror.com/why-cant-programmers-program/ https://blog.codinghorror.com/why-cant-programmers-program/ [1] https://economictimes.indiatimes.com/tech/ites/95-engineers-in-india-unfit-for-software-development-jobs-report/articleshow/58278004.cms https://economictimes.indiatimes.com/tech/ites/95-engineers-...
- bobleeswagger 4y ago"If you wanna hire great people and have them stay working for you, you have to be run by ideas, not hierarchy. The best ideas have to win, otherwise good people don't stay." - Steve Jobs I think the real disconnect with the 'inclusive culture' boom comes because humans are involved so heavily in the process. The _idea_ is great, we want to be fully aware of our internal biases and avoid having them color our perception as much as possible so we do not shoot ourselves in the foot. In practice, I have yet to see inclusivity programs at corporations be anything more than virtue signaling, and an opportunity to exclude others under the guise of "inclusivity" wink wink. Remember 'affirmative action'? It's palpably Orwellian that inclusivity is newspeak; what we call it now.
- t8sr 4y agoWe all want "inclusive hiring culture", "exceptional engineering talent" and a frictionless hiring process, but IMO you can't have all three. Like them or not, leetcode* interviews actually give a chance to people who can't do a home assignment, or come from a background that didn't let them have a bunch of code on Github. In that sense, it's the more fair way to test people's aptitude, and will find exceptional talent from all kinds of different backgrounds. If you still want "exceptional talent", but not algorithmic interviews, then you end up biasing towards white guys who have a ton of projects to show you. Actually, I think this should be verifiable. Select some companies that we think have exceptionally high bar (you could use compensation as a proxy, acknowledging it's imperfect). Then classify them based on whether they do "leetcode" interviews or not, and check their diversity reports. My bet would be that the "leetcode" companies do significantly better. * Caveat is that companies people think do "leetcode" actually usually ban questions that appear on leetcode.
- lolinder 4y ago> Like them or not, "leetcode interviews" actually give a chance to people who can't do a home assignment Not really—if I don't have time to do an hour-long take-home assignment, what makes you think I have time to practice leetcode-style questions? The take-home assignment is usually testing skills that I actually use in my job on a regular basis, so I don't need extra preparation, I just need a block of time to sit down and do it. I agree that expecting people to be able to show side projects is a mistake, though I'm not sure why you think that the bias there would be racial—I would imagine it would be much more a filter that excludes people with families and/or non-computer hobbies.
- t8sr 4y agoI guess my assumption is that measuring cognitive ability is better than specific skills. The algo interviews, when done well, do that better than interviews focused on experience. There are of course a lot of caveats - experienced candidates should be treated differently from entry level. It’s also true that, e.g. Google has a weird fetish about dynamic programming questions and other problems that people are unlikely to figure out without having taken a class in them. On the other hand, take home assignments take up time and room you might not have. It’s easier to do those when you’re a man in a developed country than, e.g. a single mother in Alabama. So its basic, I think algo interviews are a good way to hire junior level engineers.
- Scubabear68 4y agoWhile this seemed to start strong, I don’t buy into the “structured” portion of this blog post. The referenced research does not seem relevant to hiring engineers. In fact, the opener for the first reference says they researched “ 19 male applicants for life insurance sales”positions”. This is a “mountain” of evidence in hiring engineers? My own interviews have a list of topics I want to cover (non functional requirements, data experience, app design, infrastructure, etc), so I guess there is some structure. But I mostly run the interview based on their own experiences and projects they have worked on. So we will focus on applications and systems they have worked with in the past. And then I see how deep down those rabbit holes of their own system they can go.
- throwway234321 4y agoMy primary programming languages are not allowed in leetcode interview sessions so a lot of the challenge is remembering how to use Ruby or Python on the spot, and also to think imperatively. Lot of my thinking is based on visuals and emotions -- It's challenging for me to transcribe to English on demand and it interrupts my process -- it's somewhat like painting. I always shine on take-homes since I'm allowed to be my authentic self. I'm enabled and have the full capacity to do my rituals, routines, and quirks. Admittedly, this means I won't succeed in cooperative environments like pair programming. I'm better off left to my own devices.
- lopkeny12ko 4y agoA lot of the commenters in this thread (and elsewhere on HN) flat out refuse to do take-home assignments, live algorithms coding, 4+ hours of onsite rounds, etc. Yet every interview I've ever done in the last decade+ with FAANG and FAANG-adjacent companies have always been like this. So where are all you interviewing that pays competetively without this "traditional" interview loop?
- lolinder 4y ago> that pays competetively I think this is the broken assumption—there are a lot of us who simply are willing to accept a sub-FAANG wage in exchange for a work environment/interview process that we feel respects us.
- orangesite 4y agoGood jobs have friendly practices, bad jobs have awful practices. I quite like how things work currently. It's easy to tell the difference.
- lapcat 4y agoWhat bothers me about tech hiring is that tech companies overthink it. To use a housing analogy, they act like they're signing a 30 year mortgage when they're only signing a 1 year lease. Engineers come and go all the time. At present, tech companies are laying off engineers by the thousands. Think of how much time, effort, and money was spent hiring those thousands of engineers! It's a giant waste. Premature optimization is the root of all evil, and that applies not just go writing programs but also to hiring programmers. It's funny how they claim that a bad hire is devastating, and they can't rid of them easily, but somehow they can do mass layoffs and get rid of a bunch of engineers easily.
- jarjoura 4y agoI wholeheartedly disagree! It takes months, sometimes even half that year for engineers to fully ramp-up on teams and integrate into the culture of a company. Yes, you are expected to hit-the-ground running on day one, but no one will immediately operate at their full potential. Even with all the shared best practices in the world, the secret sauce is the part you have to learn. As an employer it's very hard to know if the reason for someone's uneven performance is due to ramp-up or if they are just not a good fit. Without a rigorous interview process, so many months would be wasted waiting to get a clear signal on that person. That also doesn't account for complete cultural mismatches that cause instability in teams and hurt the impact of your other employees. Another implied reason, good engineers want to surround themselves with other good engineers. So knowing its hard to get into a company signals to each applicant that the other employees there made it through that process.
- lapcat 4y ago> It takes months, sometimes even half that year for engineers to fully ramp-up on teams and integrate into the culture of a company. Maybe that's because companies tend to hire whiteboard-master generalists rather than subject-matter specialists who may not be great at standardized technical interviews. ;-) Also, if the company culture is ultra-bureaucratic, maybe the company should fix that instead of wasting months on every new hire. Seriously, if a new engineer can't commit code within the first week, that's a company problem, not an engineer problem. Of course their code shouldn't go directly into production, but that's true of any new code. Give them something small to start, like some bugs to fix. > That also doesn't account for complete cultural mismatches that cause instability in teams and hurt the impact of your other employees. Technical interviews can't determine this. > knowing its hard to get into a company signals to each applicant that the other employees there made it through that process. I realize that's a signal, but it's not necessarily a good or accurate signal. I think it's mostly PR and hype. Reminds me a lot of fraternity hazing. Google engineers believe they're the best, and some of them may be, but some of them don't impress me at all. And as I mentioned, engineers tend to move from company to company anyway, so if Google engineers are "the best", they're constantly losing the best too.
- siliconc0w 4y agoI think something like this works well: 0. 45 minute homework/prescreen. Provide an (optional) pre-setup environment so it's mostly about coding and not about building/installing deps. 1. on-site where you chat about your solution, mostly an ice breaker/introduction to the team. 2. pair-programming to extend the homework or work on a simplified but real problem encountered day to day, open book 3. design review 4. code review 5. behavioral / case study All of these can be pretty objective and don't rely on any memorization. All this should be pre-canned so individual proctors don't come up with their own questions and you're comparing candidates around the same prompts. It's amazing how few companies even manage these basic steps. I think most importantly the hiring should be done by a committee of actual practicing engineers - that means if you have checked in code in six months you aren't a vote on the committee.
- stickyricky 4y agoImagine the other side. You're more than likely doing this for multiple companies. Plus working your current job. Plus taking care of your kids! You have to devote minimum 5 hours per interested company. IMO that's ridiculous. I also think its why startups skew so young. I don't have the energy to put up with it anymore.
- higeorge13 4y agoSorry, but take home test as the first step to filter them is a big no for me. Why should i waste time on a test before even meeting anyone? 1, 2 and 5 should be more than enough. You get to talk to them about their past expertise and even combine it with some design discussion, you get a pair programming session and a final casual discussion. Why do you need everything else?
- siliconc0w 4y agoThey call it a funnel for a reason - most candidates are not worth bringing onsite - which is expensive for both sides. So you need a screen anyway and homework should, if it's well designed, take similar amount of time to a screen. In my experience it takes most teams at least six months to get an engineer productive and a year before they're really hitting on all cylinders. If you're asking a company to spend six months to a year training you, giving up a day isn't (IMO) a huge ask. As a candidate you really shouldn't need to go onsite at more than a handful of companies or you're really wasting everyone's time.
- andrewstuart 4y agodeleted too negative, upon refection.
- vore 4y agoWell, what did you expect? When interviewing someone, I would still want someone to work through a problem live with me to see how they solve some problem. I don't care about the end result, I want to see how you're getting there.
- andrewstuart 4y agodeleted too negative.
- vore 4y agoIf someone gave me a ChatGPT answer and nothing else, I don't have any way of telling if they're using it to boost their productivity or if they're just a complete charlatan copy-pasting off of ChatGPT who can't solve anything as soon as ChatGPT gets a little wonky. My bet is on the latter -- maybe it's my loss, but I would rather they have just worked through the problem without ChatGPT in the first place so I didn't have to guess which side of the spectrum they fell on.
- hello_moto 4y agoAnd the adventure to fix the issue of ENG interview saga continues... "Process is broken" continues to be the theme where no parties agree whether a basic algo or leetcode or takehome is sufficient yet continuously reject professional designation. Folks, keep in mind that at the end of the day, you are hiring a person, not an object with a bunch of methods.
- punkbit 4y agoHiring...my partner had an interview booked for a Friday at 5pm. The interviewer didn't show up. My partner end up emailing the person, whom apologised and then jumped on a 10m call, 10m before 6pm. My partner, as I do, spend a lot of time researching the company, preparing for the interview, etc. How many countless stories like that exist? For example, people messaging for a chat "found your work on project X and saw your github account and found about your past projects, we are looking for some one like you", then on the day "oh sorry to let you know last minute but X and Y happened". Bunch of time wasters! This is the reality! Once in the job, it's funny to see who actually does the work. Zero contributions for days, etc. There are a lot of people out there handling these processes and they are bad, really bad!
- FeistySkink 4y agoAnd then interviewers and recruiters go on LinkedIn lamenting candidates ghosting them on a regular basis. I constantly have to fight the urge to post screenshots under their posts where they fail to get back to me after reaching out first.
- notmars 4y agoAs a startup CTO that has done 15+ years of this, I believe very strongly that your hiring process relfects deeply on the core software belief system you hold on. That's why bureaucratic/non-engineering organisation will tend to over-emphasize references and tests, big tech will over-emphasize CS40021 style exercises and whiteboarding "shame on you", and the rest of us, other stuff. My advice for job-seeker: look very deeply why they ask things during the process and you will be able to fairly predict your future there. Make sure it matches you needs and wants. For the process-builder: are you sure those deeply-held beliefs are filtering what you need or is it filtering what you want...?
- mgl 4y agoAbsolutely agree, and smaller companies can even afford to fully disclose their core values and non-technical expectations towards the candidates to make sure the process is a win-win for both parties. We update and share these points within the team and with the applicants even before they make the first contact with us: https://stratoflow.com/our-recruitment-and-onboarding-process-for-software-developers/ https://stratoflow.com/our-recruitment-and-onboarding-proces...
- ChaitanyaSai 4y agoNice list. What does "Distanced from themselves and their source code." mean?
- sophonX 4y agoThe code has bugs and might cause a new pandemic. So they follow social distancing.
- scarface74 4y agoThat’s all nice in theory, but new college grads want to make as much as possible. They don’t do that by “arguing that gravity shouldn’t exist”. They do that by playing by the rules as they exist. That means “grinding leetcode and working for a FAANG” (c) r/cscareerquestions. I am 25+ years in the industry. But if any new college grad asks me for my advice. That’s what I tell them. I definitely wouldn’t tell them to take a chance of working for any non public company hoping their “equity” may be worth something because they heard that early engineers at Uber struck it rich.
- jedberg 4y agoWe need an industry agreed-upon certification exam. When you apply for a job as a doctor, they don't make you demonstrate surgical techniques. They take your board certification and ask you questions about what you've done in the past, things that went wrong and what you learned from that experience, and so on. They just assume you have the technical skills if you have the license. Now, I'll admit that with doctors you are legally required to have the license, and we certainly don't need to go that far. If a small startup wants to take a chance on "unlicensed software engineers" or even offer to pay for the exam as a job perk (like a lot of law firms do for their interns), then great! But I can see a lot of time and effort saved if all the big enterprises would get together and come up with a national certification exam that you take once. Or even better, a series of exams for junior/senior/staff/principle, so neither candidates nor hiring managers have to waste time on tech assessments. One of the keys would be making the exam inclusive for neurodivergent candidates, people with disabilities, etc. But this can be solved.
- ShredKazoo 4y agoHow about LinkedIn certificates? Last I checked Triplebyte has a certification you can put on your LinkedIn if you do well enough on their process.
- VirusNewbie 4y ago>We need an industry agreed-upon certification exam I'd be OK with this if all the answers were just shown to various companies so they could see strengths and weaknesses. If some company doesn't care about DP or low level OO design they can throw out those scores, etc. I certainly got annoyed in my last job hunt where I had to answer some basic whiteboard easy level LC questions over and over and over. Thank god I got to skip the Google screen. If one more person made me do basic BFS or something I was going to freak out. I understand why they asked it, but I had to keep coding it over and over and over...
- jedberg 4y agoLaw and medical licensing both require showing basic proficiency in all areas. I don't see why we wouldn't do the same, and then let you get specialization afterwards.
- jsdeveloper 4y ago[flagged]
- dennis_jeeves1 4y agoLet me mention the elephant in the room: You the candidate, did not get the job because they did not like you ( e.g. you had a voice similar to the kid in school who was bully to your interviewer etc.). For most software positions out there, a relatively mediocre level of skills is sufficient. No fucking need to hair split on a person's technical skill. If you truly care about the candidate first judge him on the 'cultural' fit. If he has crossed that barrier then the following advice from the article is a great one: >Give candidates a heads-up about the attributes or topics the interview will cover and any other information you can reasonably share upfront. All other advise in the article like paid assignment etc. are also great.
- zeroonetwothree 4y agoUsually you have 3-6 interviewers. It seems unlikely they would all "not like you" as the reason for not getting hired. Now sometimes a single very strong no is enough, but IME a strong yes can easily override that. I myself have interviewed 400+ people and have definitely had cases where I got people hired over objections because I thought they were really good. I would say it's more common to not get hired because you didn't have a strong yes. Maybe everyone thought you were ok but none of the interviews really wanted to argue for your case. There are also cases of people that are just obviously not qualified, like every single interviewer says 'no hire'.
- dennis_jeeves1 4y ago>I myself have interviewed 400+ people and have definitely had cases where I got people hired over objections because I thought they were really good. You brought up (perhaps inadvertently) a good point here , most interviewers in the field are young, and their callowness is obvious, often you need senior people with the vetoing power to override their shallow analysis of a candidate.
- TrackerFF 4y agoFWIW, we’ve passed on (what turned out to be) the same excellent candidate(s) multiple times because the other candidates were seemingly stronger. If you have two almost identical candidates, but the other has better credentials, it’s difficult to turn them down - if you have some HR guidelines to follow
- MPSimmons 4y agoI have found success by doing the following (when hiring for an engineering role): 1) Doing a relatively shallow but wide survey of the technology I'll expect them to be responsible for. Because we use k8s, I steal the old "type google.com into your browser" question and make it, "I type 'kubectl get pods' and hit enter. What happens to make the list of pods show up on my screen?". From there, you can dive into basically any part of the stack you want. I'll often ask them to explain to me what the difference between a container and a VM is, as though I were an intern, and then I'll ask probing questions about things they get wrong or things they leave out that I think are relevant. This isn't to pass/fail them, necessarily (though some people have done so badly that they essentially failed themselves), but it's to see where their familiarity and comfort level is with the tech at hand. 2) talking through their resume with them, and doing a deep-dive into a couple aspects of their recent history - why did they do a thing? What alternatives did they explore? What was the reason they went with what they did? How did they implement it? What problems happened? Who did they collaborate with and what was the precise scope of their involvement? How did they measure success, and what was the follow-up? I don't expect anyone coming in to have a deep knowledge of the tech stack we have. I do expect someone to have deep knowledge of the technology that they put on their resume, though. My hit / miss rate is pretty decent. There have been a couple of times that I said no when I should have said yes, but I'm okay with that ratio.
- logicalmonster 4y agoMostly good article: and totally agree that leetcode style interviews under intense time pressure are a wasteful plague on the industry. A good web developer could go through a bunch of interviews without ever once discussing designing APIs, HTTP requests, web security, or any one of a 100 practical topics which they deal with nearly daily, but might be asked to solve a pile of different leetcode questions (99% of which are close to irrelevant for the vast majority of typical day to day web-development). If I could give one minor persuasive writing critique to the author of this article though, I'd suggest not emphasizing inclusivity (which is basically bog standard, meaningless corporate drivel by now), but emphasizing that changing the typical interview pattern ensures that you're casting the widest possible net for talent. There's business people out there that couldn't give half an ounce of care for doing a solid for whatever the heck they might think a neurodivergent is, but if you emphatically frame this a bit different (solely as the company missing out on talent) I think the argument instantly becomes a lot more appealing to a pretty large group in the business world.
- Darrenbarret 4y ago[dead]
- blindriver 4y agoAll of these "this is how you hire" posts are utterly worthless without data afterwards to prove that their selection process results in high performance. Does any of these posters actually correlate interview scores with performance reviews? No. It's just people very confidently announcing their own biases beliefs without any reason except their overconfidence. At least the reason why we have Leetcode questions is because Google did the research and came to the conclusion that those are were good at algorithms ended up being more successful at Google, and THAT'S why we are all suffering through LC. But now that LC has been gamed, I would love to see what the results are as to what makes a successful interview.
- pjmlp 4y agoAndroid code is such a good example of that hiring process works in practice, it is just brilliant piece of art. Where would we be without such hiring processes that managed to have such set of genius work on Android. /s
- TheBigSalad 4y agoIt's a pretty successful software project.
- cloogshicer 4y agoThis assumes that performance can be meaningfully measured. Which is incorrect, in my experience.
- blitzar 4y agoAlso - does it really really matter? 101% worker vs 95% worker but one is an insufferable pr*k who makes everyone else 75% workers ... Everyones approach looks like it works because if everyone is between a 90-110% worker and the team gets along and works together you will get solid results.
- TexanFeller 4y ago> Does any of these posters actually correlate interview scores with performance reviews? Big assumption that performance can be reviewed and scored in some objective way, or if it is that this actually happens. I've seen lots of review processes that pretend to be objective, but absolutely none of them actually were.
- mountainriver 4y agoThis is great, the approach I’ve seen work best in a couple orgs now is giving the candidate options. You can either walk us through and open source project you wrote, do a take home test, or do a live coding challenge. There is a large diversity in how developers are effective. When you force people into one funnel you lose the rest of the ecosystem. Meet people where they are, the only metric that should matter is effectiveness
- ShredKazoo 4y agoDoesn't that make it harder to do an apples-to-apples comparison between candidates though?
- mountainriver 4y agoThis is the criticism I hear but I think it depends on how you look at engineering and bias in general. I just see different people with different skillsets and pick the best one for the team. I have yet to run into a situation where the ability to code was the determining factor between two candidates. I imagine it could happen but so far so good
- ShredKazoo 4y agoInteresting take.
- pwpw 4y ago> In fact, you may be doing a disservice to yourself by filtering out slow thinkers or neurodivergent candidates that are likely to not shine thinking on their feet. Yes, yes, 1000x yes. I was recently rejected after two technical interviews from a company that my former principal engineer that I worked directly under had referred me to. The position was to work under them again, which is why they referred me. The feedback I got from the recruiter was that it wasn’t the result they had expected, and I hadn’t achieved a specific number on their technical assessment. My understanding of this after some discussion was that the lower score was due to my speed in answering the leetcode style questions in the live interview with another engineer. Here’s the secret I never brought up with the company while interviewing: In high, school, college, and for the CPA exam, I received accommodations for extended time and testing in isolation to reduce distractions from my ADHD. With those accommodations bringing me up to an even playing field with a neurotypical test taker, I was able to get into a good university, graduate with a bachelors and masters at the top of my class, and pass all four sections of the CPA exam on my first attempt. In the real working world, I have never needed extended time. I always deliver what is asked of me on time while I have witnessed neurotypicals show up to meetings with their work majorly behind. I have always hesitated to bring this up with companies because I fear they will make the incorrect assumption that extended time on testing implicates that I will be a slower worker, which I have not found to be the case. I don’t want to introduce any biases for the interviewer to pick up. For whatever reason, testing with pressures absolutely slows down my thinking. In the real world, I have found when I face particularly tough problems, I find solutions after going on a 15 minute walk outside or while taking a shower in the morning. You cannot test for that style of problem solving in these high intensity algorithm technical interviews. I certainly miss having a CPA license as evidence that I was a competent individual from my previous accounting career, which allowed all parties to skip technical questions in the interview and instead focus on fit for both sides. The software engineering industry suffers from too great of an emphasis on absolute performance levels in my opinion. To pass a section of the CPA exam, one needs to score a minimum of 75. What do you call an accountant that passed every section of the CPA exam with 75s? A CPA.
- ShredKazoo 4y agoWhy not just apply at companies that focus on take-home problems?
- matt3210 4y agoI interviewed at a startup with ex Amazon employees as founders, and they had some crazy systems design and a leet code style challenge. I ended up doing entry level, basically data entry, work for 6 months before leaving.
- polalavik 4y agoI wish a company would run an experiment where they just higher at random from a stack of team vetted resumes. I think the results would almost be the same as being super picky. Why can’t anybody trust verifiable info these days - that you worked for $COMPANY doing some $JOB for $YEARS. Why are resumes completely thrown out the window and you start from ground zero on a whiteboard when you have over a decade of experience. I wasn’t practicing leet for the last decade, I was doing actual engineering. I work in aerospace which often does a STAR behavioral interview (very light to zero technical interview) and I can honestly say that I work with some of the brightest people I’ve ever met.
- zamnos 4y agoUnfortunately the STAR behavioral questions are even easier than Leetcode to study up for, and they're a pretty common interview set for managers or to set leveling, so a prepared candidate will ace those questions.
- polalavik 4y agoMy point was more that you can get capable engineers without any technical interview process.
- zeroonetwothree 4y agoI've interviewed people from $BigTechCompany (with 5+ years experience) that couldn't do some basic whiteboard coding question. I wasn't asking any dynamic programming-type obscure thing that no one actually uses, but rather a realistic question that they might actually have to code (indeed it was based on a problem I had to solve myself at my job). So it's not that I don't trust that they didn't work there, but that doesn't mean that they can do a good job here. Every company has low performers, perhaps their previous employer was just bad at detecting them.
- neon_electro 4y agoCare to share the example problem?
- claytongulick 4y agoI have an interview process that's both easy and hard. It's kind of a "choose your own adventure" style interview. I explain upfront what the process will be, that during the interview "I don't know" is a preferred answer than BS. Then I start with this question: "Your task is to take some data in from a user, store it, and then present it back to the user. How do you do it?" Based on their answer and follow up questions, I follow them down the path of their preferred stack. This is rare, but I love it when prior to answering, a candidate asks me clarifing questions, like "how many concurrent users will the system need to support? What sort of performance is necessary?" Etc... Many just assume that I'm asking about web development, so if they go down that path I challenge their assumptions, and follow up asking about their reasoning. If they pick a framework, I ask what the advantages and disadvantages there are to that framework and how it compares to others. Same with databases, etc.. It becomes pretty clear quickly what sort of level they're at. Many times the answer on a framework question is "that's what I used at my last job". That's not necessarily a poor answer, but it is informative. I try hard to make it a casual conversation, like the sort you'd have with a technical stranger at a bar or something, though I understand that's impossible given the power dynamics and stakes involved. Still, I try. So far, it seems to work pretty well for me. I have the luxury of keeping my team small, so I don't have to come up with a scalable "one size fits all" process, I'm able to keep it personal and relevant to the candidate and their experience.
- pugworthy 4y agoIt's too bad it's not a culture of interview questions to learn how well someone will work with you and your team, how much of a creative they are, a coach, a mentor, etc. I'd pick someone who really clicks with the team and is smart any day over someone who's brilliant but hard to work with.
- dmundhra 4y agoNice article talking about the common pitfalls popularized by likes of Google and Palantir a decade or so ago. Some of these methods work if you are hiring enmasse and are ok to tradeoff identifying some outliers (like in homebrew creator's example) to efficiency. This is one of the reasons I built nitrohire.co to be able to get more relevant information about a candidate in a different way! The basic question is why try to guess something about a candidate that their peers might already know and help you make more informed decisions
- sam0x17 4y agoA lot of people are in the "if I could just hire right I'd be fine" camp when they should be in the "why is the way I'm trying to build this stupid" camp. Address the root problem, go in with the assumption that it will be easier to start over from scratch and rebuild with 3x the productivity based on what you've learned about what _doesn't_ work, and it will be easy to find people who want to work on your app/platform/etc. I've advised far too many mid to late stage startups where what they really need is to just take what they've learned, rm -rf, and quickly build something that works now that they've gone through the growing pains, and far too many have taken this option later than they should have.
- ShredKazoo 4y agoWhat does rewriting your codebase have to do with hiring?
- sam0x17 4y agoCompanies that have trouble hiring and retaining right now have at least one of three things: a stinky codebase, a stinky culture/benefits/work-life-balance (i.e. not WFH, micro-managing, etc), or uncompetitive salaries. The ones that have none of the latter problems sit around scratching their heads and writing posts like OP
- rqtwteye 4y agoFor a while my company did aptitude tests which were basically a kind of IQ test. Between good results on this kind of test and having a good conversation in an interview I felt there was a very good correlation for doing well on the job. I value this higher than specific knowledge. I feel smart people that can communicate and get along with others will usually deliver high performance.
- YZF 4y agoBoth are needed in a healthy team. Unless you're building trivial things then just being smart and communicative doesn't cut it. I would say you need a mix of 50% really smart people, that get along with others, and have the specific knowledge required, and 50% smart people that get along with others, and have some basic skill. That mix +/- is a good team. A team of smart people that have good people skills (EDIT: but lack specific/domain knowledge/experience) will consistently produce terrible software.
- Nginx487 4y agoFor the small startups, who don't have time for BS like leetcode, I advise asking the candidate's code first of all. The senior-level developer almost definitely should have GitHub with his portfolio- or pet-projects. After carefully looking through his code, it becomes 100% clear do we want to talk to him or not. After that we usually had one interview, something like system design + soft-skills. Subjectively, I recall the hiring experience as very successful compared to what I experienced interviewing people for major tech companies according to their guidelines.
- devnullbrain 4y ago>his There's the rub. Women are disproportionately likely to spend time caring for children and have less free time available for hobbies. Personal projects might be a positive indicator that someone really cares about computing but the projects I've been praised for in interviews make up ~1/500th of the time I've spent developing my skills. It can objectively display the quality of code that a candidate can write but its absence isn't a guarantee that the skills don't exist.
- notShabu 4y agoIMO interviews are designed solely to maximize team size. This requires filtering for individuals for fit well into a hierarchy without disrupting the structures above and below them. Being "good at the job" is actually bad because it reduces the team's required size. Almost all the incentives revolve around this. Not only compensation, but entire hierarchies that revolve around EB1C dangling. (managers and executives at multinationals can get a green card faster)
- cubano 4y agoWith all the overt and hidden biases that us flawed humans live with daily, the idea that you will be able to pick really skilled devs consistently is laughable unless your doing some sort of blind interviews like symphony orchestras do nowadays. It's so blatantly obvious that interviewers are basically trying to hire themselves, and will almost always select candidates whom they share the most personality traits with. Also, I see the hiring process as similar to wanting to be a politician ie anyone who really wants to be one and is just really good at it should never be given the job. The people who impress you the most almost surely have simply put a ton more effort into gaming the process with long leetcode sessions, live interview practices, and other bullshit tricks to convince you that they are the best person to hire. With so much as stake, why wouldn't young devs spend tons of time working on interviewing skills and not really giving a damn about developing the real skills needed to be a goto resource at Big Software? It's very much like taking steroids in professional sports...well no shit your taking PEDs when your career paths are either making generational wealth fucking with a ball or working selling Jordans at the local shoe store.
- jl2718 4y ago"invert a binary tree" This didn't make any sense to me, so I looked it up, and it seems exactly as senseless as I had thought. Is the 'inversion' not the same topology as the input? my solution: "auto invert(regular_tree x){return static-cast<inverted_tree>(x);}".
- devnullbrain 4y agoIt means swap the left and right pointers for each node.
- compressedgas 4y agoWhen I tried to find what was meant by "invert a binary tree", the only thing that made sense was the operation of reversing the tree: ((1 2) (3 4)) becomes ((4 3) (2 1))
- steve76 4y ago[dead]
- Uptrenda 4y agoTbh the way that hiring is done in tech is just lazy and reeks of mediocrity. Memorizing 'take home' assignments, banks of algorithms, pair coding, white board interviews, what a joke. Companies want to place all the burden on candidates and spend as little money as possible. It doesn't matter if they waste people's time. Whatever happened to getting to know a candidates work? How about look into the work that a person has done and take the time to understand where a person's skills are. The problem with tech hiring is we have people trying to cut corners. So-called 'non-technical' recruiters doing interviews with a checklist, companies that treat people like hoop-jumping monkeys, and generally f*king idiots that won't do their job (they get paid for it, why again? They're not actually doing their job.) Hiring is not a complex problem. The problem is literally incompetent people doing hiring.
- Joel_Mckay 4y agoMake a sign that reads "Free Starrett and Mitutoyo gifts if you like Maxwell's equations"... Yes they know its a 100% a trap, but good engineers won't be able to resist responding on the off chance it is real. Don't click this... it is clearly a trick... you already knew they never have overstock sales. yet had to click this anyways... =) https://tinyurl.com/mitutoyocoupon https://tinyurl.com/mitutoyocoupon
- reverseblade2 4y agoThe sad fact is usually your assumed nationality, your assumed gender, your skin color (not necessarily being white is always good), your assumed origin and assumed religion matters more. Depending on these, you end up with different attitude and questions.