21 ms·

Green Lumber Fallacy in Software Engineering

• bsedlm 5y ago
  > solving DSA-style problems and building software are two different things disagree. I'd say that both are needed skills for software. Software is made layers, like a cake. Maybe DSA style problems are like grounding the flour (or making the oven itself?) and the other building skills (which I agree, seem like different skillsets) are about decorating the cake and mixing the dough or something. So in practice nobody is grinding their own flour; the flour is being bough already processed (in programming this'd correspond to importing the algorithms from a lib)
  • buscoquadnary 5y ago
    I agree to a certain extent, it has been in my mind the big difference between impactful engineers and engineers that do a lot of stuff, it seems that many people can write code, but being able to conceptualize the software at the layer of DSA's allows one to flow fluidly from multiple levels of abstractions and aspects of the problem more seamlessly. That being said I also have serious beef with "architects" whose primary capabilities are programming in powerpoint and being professional XML developers.
• zwieback 5y ago
  sure, but if performance in those types of questions are a good predictor of success in the job then I'd go on asking them. Probably they aren't but just saying "I won't need this in my job so I won't ask in an interview" is short sighted.
• invalidname 5y ago
  I very much agree. But this is written in a somewhat "this is bad" style and misses the "you should do this, instead" part. This article contains both: https://talktotheduck.dev/debugging-the-technical-interview-methods-and-cheating https://talktotheduck.dev/debugging-the-technical-interview-...
• mytailorisrich 5y ago
  Data structures and algorithms are important to very important in software engineering and I think it's quite right to include questions on them in interviews. But that's not the same as knowing by heart how to implement non-trivial algorithms. There's the green lumber fallacy. To have a feel for the 'right' data structures and algorithms to use for specific problems is a key skill. But then the detailed algorithm is only one Google away these days, although I would expect someone to be able to derive the simple ones themselves.
  • corecursion 5y ago
    This! Data structures obviously do matter for software engineering. But memorizing the solutions to a handful of random, difficult, data structure problems isn't very useful for a real job and shouldn't be part of the hiring process.
  • deleted 5y ago
    [deleted]
  • 0xbadcafebee 5y ago
    Data structures and algorithms are just parts, like screws and nails. When you're building a house, you don't need to remember if you should use 2 18-penny nails or 3 16-penny nails, or if you can use screws, and what kinds. What matters is that you know how to go look up the building code, find out what's required, use those parts, know how to verify they're installed correctly, and know the consequences to doing it wrong. Once you've built a dozen houses you may remember these things implicitly, but you never need to remember them. You do need to know why they matter, how to look them up, and how to adapt them to your particular job.
    • mytailorisrich 5y ago
      This is not a good analogy as it seems to go very much into details. If you're a builder I might expect you to know when to use a steel beam rather than timber even if you don't know by heart what grade and size for any random load (I don't know anything about building but that feels like the right level). Likewise, if you're a software engineer I expect you to know what's a hash table, a binary tree, a linked list, etc., what are their pros and cons, and when you might want to use each of them. But in general I don't expect you to be able code a tree inversion off the top of your head. Obviously expectation of detailed and specific knowledge has to depend on previous experience and role.
• compiler-guy 5y ago
  The test isn't so much about the ability to invert a binary tree (for example), but the ability to problem solve complicated problems where the obvious solution is probably not the best one. In interviewing, it is hard to produce real world problems that don't require a huge amount of context and details of some obscure software stack, so we go with abstract problems which are kind of irrelevant to real work, but the problem solving approaches you need to take are very relevant. Thinking about the requirements, thinking at the right level of abstraction, defensive coding, to name just a few. Communication and whatnot are all on top of that, and yes, absolutely do need to be evaluated. I'll also note that MAANG companies have extremely sophisticated software stacks (and multiple ones for each company!) that were built by people who were hired using that method. If it is wrong, it's working pretty darn well. There are many things I would change if I could, but we do have an existence proof that, in the aggregate at least, the people this interviewing method selects for can build amazing software stacks.
  • disgruntledphd2 5y ago
    If we're going to use Meta for the acronym, then for God's sake can we call them MANGA? More generally, the absurd success of most of Big Tech has not come from better technology but extremely high margins facilitated by software. Apart from the core ranking/ads problem that M and G solve, most of it doesn't need to be that complicated. I would venture that lots of the cool technology stuff is designed to keep engineers happy so that the money train continues. Certainly the one I was at built a lot of cool stuff, but there was so much reinventing of the wheel that I suspect that the core business and products could have been served by a much much smaller engineering org.
    • osigurdson 5y ago
      According to Jim Cramer (who came up with the FAANG acronym), we should now be using MAMAA (Meta, Apple, Microsoft, Amazon and Alphabet). https://finance.yahoo.com/news/not-faang-mamaa-jim-cramer-163000145.html https://finance.yahoo.com/news/not-faang-mamaa-jim-cramer-16...
      • 5y ago
• floober 5y ago
  In the physical world there are folks who design bridges, dams, buildings, telescopes, furniture, and a million other things. All of those have unique requirements and skill sets. In software, we also make a huge number of categorically different systems each with its own skill set, but we call all of it 'software engineering.' There are folks who are great at their job (called "software engineering") and their work never invokes their knowledge of algorithms. There are also folks who spend day in and day out tuning and designing new systems and deep knowledge of algorithms is essential (we also call this "software engineering").
  • 0xbadcafebee 5y ago
    I completely agree, we need better compartmentalization/specialization and a better way to target specific skillsets and experiences. Not knowing how to hire the right people is a source of so many problems in the industry. 20 years ago, all you had to do was get a "certification" in some technology and you'd immediately be hired to work on it. That led to a lot of really bad people getting hired because all they did was learn how to pass a test. Today, hiring in tech doesn't require any piece of paper at all. But that means that instead of relying on some standard format to prove your baseline body of knowledge, the interview process now has to do it ad-hoc, and it never seems to fit the role's actual requirements. And nowhere in the industry have we ever required training on how to actually do a job. Does the candidate know what an SDLC is? Do they write ADRs? Have they juggled multiple changes in flight on a team with large codebases? Do they have a solid grasp of the strange subtle quirks of their tech? Have they learned to be judicious in their decisions and weigh the many long-term pros and cons? Have they ever developed any project with a team? Trades typically require an industry board to certify them, and then often require years of apprenticeship under journeymen or masters. I think these two would go a long way towards leveling the incredible amounts of variation in candidates and eliminate these ridiculously ill-suited interviews.
  • vajrabum 5y ago
    I've spent a lot of time fixing or working around messes caused by people who don't know algorithms and data structures. If you don't know about those problems then you're likely to solve one without recognizing it and do a poor job of it. Worse if you don't have data structures and algorithms training you're unlikey to be able to reason about time and space complexity. Unlike at startup or enterprise dev shops, at the scale of FAANG, those sorts of problems (e.g. turning n into n*2 and the like) at a backend are very likely going to cause crashes or something essential to time out. If you haven't worked on a service with 5-10K or more servers you aren't likely to really get how quickly and how frequently things are going to go bad if you aren't very careful. There aren't many if any engineers designing bridges who haven't had the entire curriculum. If you want to practice independently as an engineer then you have to get a license which involves taking a hard test which recaps your entire undergrad curriculum and maybe more. I peeked over my brother's shoulder when he was studying for the EIT. Not easy to pass. And like doctors and lawyers they have to keep up with their professions to renew their licenses. So, no.
• strict9 5y ago
  I think this phenomena is at least partly responsible for why so many devs stay in the same position and why they are difficult to find/hire. Before a tech interview, good candidates will spend a large amount of time doing 'homework' which is just practicing answering the asinine questions that tech interviewers give that is usually barely relevant to the position, if at all. Equally, if not more important, is the candidate's experience or ability to navigate legacy code, testing philosophy, handling incomplete/inconsistent requirements, and attention to detail, among other traits. Of course these won't merit the same attention in an interview. On the one hand firms have difficulty finding good candidates, on the other hand they are giving ridiculous tech interviews that aren't a great judge of what kind of employee they will be.
  • ejb999 5y ago
    >>partly responsible for why so many devs stay in the same position That is why I am staying put for the time being; hate the interview process, hate being asked stupid questions about solving problems that I will never have to solve in real life. Ask me about projects I have worked on, ask me about code I have developed ask about my experience etc - but I am still looking for the development job where I will be required to solve Hanoi's tower problem.... In fact, for the current job I am in got thru the initial interviews and then was told I needed to take online 'skills assessment' test to move forward - I told the interviewer (nicely) 'forget it, I don't want the job if I have to take the test; if my resume and track record isn't enough then maybe this isn't right for me.' (keeping in mind that I have a 30 year track record, not a recent college grad) Guess the hiring manager agreed, because they hired me anyway (still haven't taken the test).
    • bredren 5y ago
      I have a very talented SWE friend who only left a now-public unicorn because they were recruited by a former colleague at a startup they knew wouldn't involve leetcode in the interview process. I have another very talented SWE friend who has been doing leetcode exercises off and on for months and is struggling to start doing applications even though they do not like their current work and are undercompensated. They fear they will fail algo / data structures and yet this person has an excellent public track record of quality contributions and comms in FOSS. From what I've observed leetcode screens are problematic enough that they are causing market inefficiency.
• dottedmag 5y ago
  > I’ve never heard of an ex ICPC competitor go on to build a great piece of software Telegram.
  • dadkins 5y ago
    Also, Craig Silverstein, 1993 ICPC winner and Google employee number one? https://en.wikipedia.org/wiki/Craig_Silverstein https://en.wikipedia.org/wiki/Craig_Silverstein
  • yongjik 5y ago
    I know a guy who had won gold medal in International Olympiad in Informatics (for high schoolers) - he's now the CEO of Korea's largest food delivery company.
  • TheRealNGenius 5y ago
    Vitalik Buterin did IOI I think and made Ethereum Guy who made Quora did ICPC or IOI GP probably deaf
• tanseydavid 5y ago
  Author wrote: "These test-like interviews mistake the trees (data structures and algorithms) for the forest (building software)." Should say: "interviews mistake the b-Trees for the forest"
• kokanator 5y ago
  The post arrives at the wrong conclusion. DSA style problems are good if used correctly. Give a candidate a good challenge, preferably with some requirements that can have different interpretations, ask them questions, help them and collaborate with them. The analysis of the whole exercise should be your hiring determination not whether the problem was actually solved. Did the candidate: - stick with the problem? - ask meaningful questions about requirements? - ask for help from a senior when stuck? - explore different avenues to solve the problem? - handle criticism? Of course it is great if they solve the problem but often that often only shows you that they have memorized algorithms. ( we have the internet these days )
  • d4nyll 5y ago
    I agree with everything you said. The best interviews ive had are those where I felt like I am working together with the interviewer. I think being able to handle criticism is an important trait to look out for. Many times interviewees will defend their method too aggressively. But if you have to pick between two people who are equally pleasant to work with, but one was able to solve the problem, you'll probably pick the one who solved the problem. When the competition is fierce, the candidate would probably still have to spend some time practicing on these problems.
    • lhorie 5y ago
      I interview candidates at a company with a relatively high volume of interviews and I use DSA criteria in my interviews. But here's what I do: - I phrase the question in a way that has some semblance of day-to-day relevance. That is to say that at some point in the process of coming up with a solution, the ability to apply a relevant data structure will come up, but it will be in service of an end goal that looks like the deliverable of a sprint task. - I come into the interview aware of multiple solutions and I am open to any of them. - I pace feedback so that the candidate actually solves the problem by the end of the interview, no matter their level (which does mean, in some cases, literally spelling out the step to unblock themselves). The rationale is that solving a hard DSA question doesn't give me all that much signal in and of itself. Watching a candidate bang out something with a level of complexity a little higher than fizz buzz is usually sufficient to evaluate whether the candidate has familiarity with the language. The choice of idioms and APIs can tell me things about their relative level of expertise with the stack (i.e. it can generally be safely assumed that an already employed candidate can hold their candle, and the question for me is more along the lines of "to which extent"). During the course of an interview, I can usually pick up a distinct and noticeable difference in focus between candidates, especially surrounding topics related to proactiveness/curiosity (e.g. does the candidate have understanding of aspects one abstraction level lower than the API they usually use, are they aware of well known pros and cons of some specific idiom, does their argumentation seem derived from personal experience vs parroted from a hivemind, etc). This tends to correlate surprisingly accurately to how much autonomy and growth they demonstrate on the job. "Hardcore" DSA evaluation only really comes in as a criteria to determine whether the candidate is of very high quality when most other criteria have already been evaluated as acceptable/desirable. These nice-to-have criteria come into play in some cases where I want to advocate for the candidate when the evaluation panel is split due to one seemingly bad session (possibly due to factors such as nervousness or mixed signals), or inversely when the role logically demands a higher bar but the panel is situationally incentivized to hire down to meet a quota. I've been told by several candidates that they appreciate my interviewing style, and conversely, I feel like I get a much better feel for the candidate than strictly evaluating DSA skills and nothing else.
• d4nyll 5y ago
  Personally, I dislike competitive programming interview problems. Competitive programming, when viewed as a sport, requires training and practice. It's a muscle that you must constantly train. If you stop for a few months, it may take you a few days or weeks to get back to the previous level. So it's always frustrating to find a new job because I know I'll have to spend weeks practicing on problems that will have no relevance to the posts I'm applying to. To me, it's a waste of time. I think companies that uses competitive programming problems (that are irrelevant to their engineers' day-to-day jobs) end up hiring people who are very good at interviews and perhaps not as good at their actual jobs. Of course, there are jobs which deals heavily with algorithms and optimizations, for which these types of interviews are relevant. But the article is not talking about these relevant cases.
• daotoad 5y ago
  Excessively complex DSA questions are not very useful for me as an interviewer. I prefer to offer a fairly simple test problem that has room for elaboration. For example, when evaluating SQL skills, looking how the candidate would model mapping between companies and stock ticker symbols is instructive. You can start with a simple flat table, extend to one to many relationships, discuss when normalization and denormalization make sense. You can keep elaborating this, too--add in multiple exchanges to get many to many, and so forth. This lets you see how the candidate evolves a design in the face of changing requirements, which is critical. These kinds of questions are great for leveling an applicant. How a junior dev answers these questions will be very different from what a senior dev has to say.
  • dnissley 5y ago
    This sounds like the system design round at many companies.
• unwind 5y ago
  While trying to read more about the actual case of the trader who didn't know what he was trading, I found [1] which contains this quote: > The “green” in green Douglas fir refers to the fact that it has been newly cut (it has not been dried), just like someone who is new at something is referred to as green. This was interesting/funny to me, since I was sure that the name "green lumber" comes from the fact that it is green if you look at it (being plant material which was recently alive). I'm not saying a full-size tree will be green like your kitchen garden basil stalks inside, but e.g. under the bark there are green hues. I would never have thought it came from the "green = n00b" connection. Am I the mistaken one, now? [1] https://with.thegra.in/green-lumber https://with.thegra.in/green-lumber
  • djur 5y ago
    See also "green cheese". "Green" has metaphorically meant "fresh, new, immature" in English for a long time. I don't think that quote is claiming that it _came from_ "green = n00b" but that it shares the same source.
  • caethan 5y ago
    It's the other way around: the "green = new" is derived from the usage of green timber, where newly cut timber is literally green.
• deleted 5y ago
  [deleted]
• JavaBatman 5y ago
  > All the MAANG companies use algorithmic and system design style questions as their main metric for hiring candidates. The internal justification for doing so is likely a combination of “this is what everyone else does” and “it’s a quick way to evaluate someone’s skill”. They argue that this is the only way to weed out all these candidates. It is MAANG's version of the SAT. > the best solution is to have trial work periods. There’s no better way to see how someone performs at the job than having them actually do the job. Agreed. But how do they implement this?
• tetsuhamu 5y ago
  Reasons we test for data structures and algorithms: - The work sucks because existing employees don't know DSA's - We want people to re-implement and refactor with better DSA's - We want to know the candidate studied for the interview (took it seriously) - We actually do want them to know DSA's when joining the team - It's the least you can do when applying for a CS job Most recruiters provide links and resources, the material isn't a mystery.
  • AnimalMuppet 5y ago
    The material is irrelevant, at least for most software engineers. (For a FAANG, not so much. But for most places? Irrelevant.) I understand the time-space tradeoffs of the various STL collections, and the Java collections. In 35 years, that's been all I have needed. (And, if it does come up, why spend months memorizing what I can spend minutes googleing?) I am not a huge outlier. Interview for what you need. Anything else is wasteful. Do your people spend most of their time trying to squeeze the absolute most efficiency out of their data structures and algorithms? If so, yes, interview for that. If not, though, then don't interview for that. Leetcode interviews when that doesn't match the work are just abuse. "Here, take months of your spare time learning to jump through this hoop that's actually irrelevant to the job." That's abuse. The only way that makes any sense is if you need employees that you can continue to abuse after you hire them. And if so, then I don't want to work for your company. As I said, FAANGs are an exception. They need people who can go from n (log n)^2 to n log n. It makes a huge difference to them. If that's your company, then I'm not talking about you.
    • wiseowise 5y ago
      You're making it sound like only FAANGs operate on such a scale.
      • AnimalMuppet 5y ago
        I didn't mean to do so. But I would say that the large majority of software engineers work on stuff that doesn't operate at that scale. [Edit: Or perhaps I should say that far more companies interview as if they operated at that scale than actually operate there.]
• rojobuffalo 5y ago
  > I’d take working with someone who’s great at naming functions and variables over someone who can code a solution to the knapsack problem. totally agree
• mesozoic 5y ago
  The real question is why do we keep doing this type of interview even though it seems everyone knows this is a problem?
  • wiseowise 5y ago
    Because they're not a problem?
• MattGaiser 5y ago
  A lot of it is that building successful software requires product knowledge. I have never used that in my career as a dev (despite being hired for it once). Others hand that to me. I’ve also never gotten any credit whether customers like what was built. Homebrew is a great product. But in a tech firm, the product manager gets the credit for its greatness.
• mrkentutbabi 5y ago
  If I have money for every article that rail against DS&A algorithm.... I don't understand why people hate DS&A so much. Just do it. Get money, jump ship, get more money, it also makes you a better engineer. It gives you better compensation, it helps you, helps everyone, helps your team, your company. It helps avoid wasting time as well on both sides. There is no downside of studying DS&A and Leetcode, if there is, the downside is minimal. And yes, you can have DS&A knowledge without handicapping your other software engineering knowledge. It is a fallacy to think that a Leetcode monkey wouldn't be able to code resilient, robust software, with good variables and good readable, maintainable code, and vice versa. Like, seriously. Why this is even an argument? And after DS&A interview, there is behavioral questions interview. If someone is a crazy person, ideally behavioral questions would weed that out. I don't understand why people prefer "here take home questions/work for 5 hours for free" over DS&A interview. Also, it is "Green Lumber fallacy" to think that a knowledge of "how webpack 1, 2, 3, 4, and nth build system in JavaScript here works" will make you a better engineer. Those kinds of knowledge aren't long lasting, and it is also quite easy to learn relative to DS&A. Here is the problem with these kinds of articles. Majority of them are written by people who hate these kinds of interview questions, and by extension, they probably aren't that good at it. There are articles that sing praise of DS&A interview but of course it doesn't get here on HN because it is not controversial and the number of bitter SWE who don't have DS&A knowledge overwhelms those who do. Not saying the author doesn't have DS&A knowlege, just my generalization. And yes of course there are people who excel at software engineering without DS&A, but that's not the point of DS&A interview. The point of DS&A interview is as fast as possible system to vet engineers that aren't time wasting on both sides. It accomplish its purpose nicely. Engineers aren't as non-fungible as they think. Deal with it.
  • MattGaiser 5y ago
    My objection is maintaining an entirely unrelated skillset that I have never encounter in my day to day work. It is a skill that you will lose if you spend a year without interviewing/leetcode practicing. Sure I’ll do it, but you may as well test me on painting famous oil artwork from memory too. > I don't understand why people prefer "here take home questions/work for 5 hours for free" over DS&A interview. The alternative is hundreds of hours of speculative investment memorizing books/websites of algos. People will take full time courses on passing these interviews.
• jcrites 5y ago
  One employer that I respect gave me what I thought was one of the best technical interview questions that I have seen at any recent company where I have interviewed: Design a binary tree containing integers. Design a function to serialize this to a byte array (or byte stream); and design a function to deserialize that same output back into its original tree form. I thought this was completely reasonable and also very practical, as it tests data structure knowledge to the extent that is likely to come up during real programming tasks, in a context that is entirely plausible as well (needing to serialize data in order to store it or pass it between systems, etc.) I had to look up what it means to invert a binary tree since I hadn’t heard that term in a while. It seems like something you’d be more likely to do with a binary search tree than an arbitrary binary tree, but the operation makes sense on both. (Given that a binary tree represents a sequence of elements, “inverting” the tree means constructing a tree representing the same sequence of elements in reverse.) If you realize that inversion is as simple as swapping the left and right edges for every node — either in place or by constructing a new tree - then the problem is actually fairly simple. … as long as the candidate has a clear understanding of what “inverse” means — I might clarify and ask them to “reverse” the tree - it doesn’t seem like a particularly difficult interview question. However I’m not a fan of interview questions that require a “flash of insight” - even one such as “oh this question has a simple solution: swap left and right of each node” - since candidates might get tripped up looking for traps that require algorithms something more complex than the obvious. Also I think that kind of task is rather removed from the kind of problem that we typically work on as software engineers on a daily basis. Serializing and deserializing data structures is something that I do in one fashion or another not infrequently - usually not with custom code but I think a competent programmer should be able to write that code. Binary trees and algorithms on them are not something that come up very often in practice in my experience. They might come up if you were building a collections library or a particularly optimal solution to a large scale problem. Otherwise, I don’t think I’ve seen a binary search tree in userland business logic the entirety of my professional career. On the other hand I’ve definitely had to write serialization/deserialization functions for object graphs. Although this is less common now that there are a variety of good serialization libraries and RPC tool kits, I find it’s often still necessary to convert between their generated structures and the native ones used by business logic. In conclusion: the question seems like one that I would expect a competent developer to be able to solve, as long as they’re given a clear understanding of what the problem actually means (i.e. explain what it means to invert a tree and not takeoff points for not knowing that) - but it’s not a good problem IMO because it’s not the kind of code one would typically need to write while solving routine business problems. All that being said, Google also rejected me during my last round of interviews; I believe this was because I was transparent with the recruiter about my interviewing at other companies and offers that I had, and Google’s offer (per the recruiter) would have been for considerably lower compensation. They said they did not want to compete on compensation because it would be unfair to their existing employee population, and so - per my best read of the situation (there was no discussion about my interview performance, and rather about this) - they decided not to make an offer that was lower on compensation, and potentially also on comparative level. So I might not be the best person to comment on Google‘s hiring practices. During a previous interview they did give me an offer though so shrug. (I interview periodically to benchmark comp and stay sharp)
• captainbland 5y ago
  I can't help but feel that this makes a better point about the lack of intrinsic merit in trading and more generally other profit making pursuits.
• caethan 5y ago
  I've heard bitching about some of the interview questions our team asks before, but here's the thing: each of those questions is about a problem our team actually had to solve before, reformulated into an interview-style question. Yes, we've had to use Hamming distances, worry about the scaling (N log N vs N^2) of particular solutions, use error-correcting codes, interesting data structures and all of that. Is that most of the job? No, we do a lot of more boring stuff too, but the algorithms and data structures are definitely a part of it. I don't want someone who can only glue pieces together, developing novel tools to solve the problem is important too.
  • riddleronroof 5y ago
    I bet you have. But how long did you have to solve it? Did you have access to the internet, talking out ideas with your colleagues, coffees? I agree that you can’t simulate real scenario in an interview, but you can also acknowledge that the process is a little cartoonish.
    • caethan 5y ago
      Interviewees are encouraged to use Google, code on a laptop rather than on a whiteboard, etc. Sadly, I only get an hour to interview people rather than a week. But I have had people complain about "completely unrealistic problems unrelated to the real work" when the problem is something I literally had to solve 3 months ago.
      • EdwardDiego 5y ago
        That sounds far more reasonable.
      • jzoch 5y ago
        To be fair, you could just give them the question ahead of time. Email them 3 days before their interview with the question and say see you soon. That is much closer to actual conditions
  • ZephyrBlu 5y ago
• erehweb 5y ago
  Article says "There’s no better way to see how someone performs at the job than having them actually do the job." and argues for a trial work period. Well sure, except the first part of any job is coming up to speed on the domain, the codebase, who the key people are, etc. So if you want to see how someone really performs, you're going to have to wait a while past that initial period. And that means you're going to have a lot of people who won't make it past the trial needing support.
  • CobrastanJorji 5y ago
    For very small companies, hiring someone with the right skills who will work well with the team is a crucial, "this might make or break our company" decision. I've seen a couple of companies who solve this by having the candidate spend a day pairing with someone on a real problem. When this works (whatever's being worked on is small enough to be understandable, the language/platform/domain/tool isn't completely foreign to the candidate, etc), it's a really great signal. But for very large companies, it's still really bad to hire bad candidates, but it's also important to be efficient. You need multiple opinions on candidates, but the interviewers can't spend a whole day each; it's too expensive. But on the other hand, you have basically an unlimited pool of candidates. So you do whatever's both fast and is unlikely to produce bad "hire" decisions, and supposedly a series of algorithm puzzlers do a good job of being fast and producing a low false positive rate. I'm not sure if that's actually true, but that's the argument. I am sure that the process rejects a huge number of perfectly good programmers, though.
  • lhorie 5y ago
    Yeah, that is one of those claims that sound nice in theory but kinda come crashing down when rubber meets the road. Companies do in fact engage in a process of getting someone in and doing an actual job, they're called internships. The problem is that these arrangements typically involve someone dedicating some non-trivial amount of their time to "babysitting" (not necessarily in the literal sense, but in the sense that a newbie doesn't have historical context or familiarity with processes and workflows, even if they are by all other measures bright individuals w/ actual experience under their belts). Either you spend inordinate amounts of time setting up and maintaining a contrived bubble where a candidate can operate cleanly for a very short period of time, or you're looking at very long evaluation times (requiring a week or more of time from a candidate, who generally already is gainfully employed). It's also worth pointing out that this process is incredibly expensive. One hour of time from a full time employee doesn't really cost anything more than the few minutes lost to context switching (ie. not really that much worse than the person going out to buy a coffee). Literally setting up a paid one week period for a senior level candidate would cost, optimistically, a few hundred dollars for evaluating a single candidate. It's completely unworkable in a large company that conducts dozens of interviews or more per week.
• paxys 5y ago
  Data structures and algorithms questions are a perfect proxy for (1) is the candidate smart and (2) can the candidate write moderately complex code (arrays, hash maps, pointers, nested loops). 90% of candidates will fail 2, and you will get a good idea of the rest with 1. Unless your company can afford a month-long interview process for every candidate which the author suggests, this is the best we have.
  • tharne 5y ago
    > Unless your company can afford a month-long interview process for every candidate which the author suggests, this is the best we have. If the current leetcode style of interviewing is the best that SV can come up with, despite having some of the best engineers and thinkers in the world and billions of dollars to spend, that's really sad.
    • amscanne 5y ago
      Why is this sad? Standardized IQ tests are notoriously difficult to get right and may be legally questionable (Griggs v. Duke Power Co.). At some point you have to accept that there is no ideal way to interview, and that there are fundamental time/precision/recall trade offs. Big tech optimizes for low time and high precision, which means you certainly will end up with low recall (false negatives). I’m sure there are improvements to me made (and maybe the specific leetcode style is not ideal), but I don’t think the billions of dollars is relevant. It’s like saying it’s sad that tech companies haven’t improving on O(n log(n)) sorting despite their billions — it’s not possible.
      • dragonwriter 5y ago
        > Standardized IQ tests are notoriously difficult to get right and may be legally questionable (Griggs v. Duke Power Co.). Griggs applies to any hiring criteria or practice that has a negative impact on a protected class compared to people outside the class (given the set of protected classes, this is essentially any hiring criteria or practice) without sufficient evidence of probative value on job performance. It is neither limited to things very much like IQ tests, nor are IQ tests any harder to justify than any other element of a hiring process.
• axiosgunnar 5y ago
  Good article but > MAANG It‘s FAANG. Don't let Facebook whitewash it's rightfully tainted name.
• tharne 5y ago
  The leetcode style of interviewing is essentially a legal way to practice age discrimination. When you're young and just starting your career, you typically have nothing but time, and doing programming challenges on evenings and weekends can even be fun. As you get older, life happens and you have children to take care of, older relatives to help out, a spouse who'd like to see you every so often, etc. Practicing leetcode-style problems in your 30's and 40's means less time with your family, offloading more of the childcare and housework onto your spouse, putting off household chores and repairs, and so on. So you end up with a hiring process that's incredibly hostile to older workers while being a relatively easy lift for younger workers with fewer responsibilities.
  • frankbreetz 5y ago
    It's almost a way to see how much time and effort someone will spend preparing for something. If someone spends a lot of time preparing for a job interview, it is not a stretch to think they will spend a lot of time on work assignments. And, as someone in there 30s with two young children, I find time for interview prep easier then any other point in my life. As you get older you should get better at learning. Also, who doesn't have chores and jobs when they are in their early twenties, you should get better at managing your time as you get older.
• teg4n_ 5y ago
  I am a frontend web developer and I still get these types of questions. It's incredibly stupid. I don't think I've passed any of them yet I'm making 250k still so whatever. The most I have to think about data structures is deciding when to use a map, set, or array.
• Tycho 5y ago
  If interview tests were truly useful, wouldn't companies want to assign them continually to their existing employees? Constantly keeping their skills sharp, proving their competence without any confounding variables.
• Scene_Cast2 5y ago
  One amusing thing I've noticed about MAANG companies (with perhaps the exception of Google) is that despite passing the technical interview, the employees do not actually do any difficult or tricky implementations when something like that is actually needed. I can't figure out if it's a strong aversion or an inability - the end result being the same in both cases, at best some approximation gets implemented instead, and the analysis / feature / etc just doesn't get done in the worst case.
  • robbiep 5y ago
    This is highly off topic so I apologise to everyone - but this is the second time I’ve seen MAANG and it’s obvious what it is, but why have the body public decided to replace the F with an M, when we haven’t replaced then G with an A?
    • Scene_Cast2 5y ago
      I'm just going with what the article used. I think MANGA is another acronym and is more pleasant to write or pronounce. Why not Alphabet - just a guess, but they haven't really pushed Alphabet branding.
• mavelikara 5y ago
  > I’ve never heard of an ex ICPC competitor go on to build a great piece of software, yet I’m certain every ex-ICPC competitor would outperform the best software creators on interview-style questions. Are there any great pieces of software written by ex-ICP champions?
• maerF0x0 5y ago
  > software development is not data structures and algorithms. Software development is almost entirely about understanding what others want and then translating them into Data structures and algorithms. Author could have meant software development is not about implementing already implemented data structures and algorithms. imo better questions would have a candidate using the DSAs than implementing them. eg: questions that require them to use binary search, bitsets etc
• tantalor 5y ago
  I think "invert a binary tree on a whiteboard" was meant to be a joke, not taken literally. This was on Twitter, after all.
• grogers 5y ago
  I recently went through two interview loops at big tech companies and the DSA questions are not very hard. The typical question starts super easy and then layers on more complexity as time and skill allows. If you are nervous about interviewing because of that, just try a few "easy" leetcode questions or the first ~5 days of advent of code, it really is about that level. Don't be intimidated, you do NOT have to be perfect to get an offer. If you know what a priority queue/heap is you will be fine. Yes practice, but don't slack on preparing for the 1-2 system design interviews and 1-2 behavioral interviews (for senior candidates). System design especially I personally think is way harder because it's (typically) candidate driven and there's no fixed finish line, just tradeoffs and more tradeoffs.
• tacostakohashi 5y ago
  I used have a similar perspective as the author, that the leetcode style questions / data structure / algorithm stuff they ask in interviews but seems to have little to do with day-to-day corporate software development is all a big waste of everyone's time. I'm not sure I still do, though. It's true that this stuff rarely comes up and often doesn't matter for most real-world applications... but when it does, it matters a lot. The difference between O(n2), O(n) and O(log n) is huge for big values of n. Being about to go straight to a (nearly) optimal solution, instead of hacking together something, waiting for it to burst a the seams, and then having to fix it... or being able to spot someone else doing that, can save weeks or months of development at a time. It's a bit like saying it's silly for pilots to have to know what to do in the case of engine failure or stall from memory and regularly practice in a simulator, because that stuff rarely happens and most of the time they're just using autopilot. That's true, it usually doesn't matter, but when it does... it matters a lot.
  • YZF 5y ago
    When it comes up do you need to code it up in 20 minutes or is looking the algorithm up in a textbook ok? If all you want to test that sort of knowledge just ask the knowledge question. I don't need to memorize Floyd-Warshal and be able to code it on demand to be able to recognize when it's needed and look it up. I'm sure I remembered it back in school, haven't used it since, but if the day comes I'll look it up...
    • Hermitian909 5y ago
      I do think there's a real place for interview questions where you identify algorithmic opportunities in real world problem scenarios quickly, but you also may be missing some of the incentives for testing the way it's currently tested: Coding floyd-warshal importantly shows that you can code, and generally signals at least one of two things: 1. You studied hard for the interview, you're willing to do difficult and unpleasant intellectual work for rewards and do so in a structured enough manner to get results. 2. You have such a deep understanding of graphs and graph algorithms you were able to re-construct floyd-warshal in 45 minutes. This implies time will not be an issue if you see an opportunity for these problems, and you can probably handle much harder problems as well. Filtering for these two traits might work very well depending on the hiring needs of a particular company.
• vp8989 5y ago
  Its shocking to me how people don't get bored of rehashing this same, tired opinion over and over and over again and discussing it with hundreds of comments. There is a blog post like this every other day on the front page of HN and it always generates the exact same discussion.
  • tubby12345 5y ago
    I have said this before: it's group therapy for entitled devs that feel they're entitled to oversized salaries just because they have tenure. Young grads accept the process as is. People that are either happy with their comp or comfortable putting in the work do not lament the process - they just accept it and proceed. Only devs that have been doing it a while, whose salaries have plateaued, that either never learned data structures or can't muster the will to review, bemoan the LC grind. The most utterly absurd instantiation is the dev that's 100% sure that FAANG is going to fail soon because they're passing on people like him/her that are just so talented that they don't need to prove it with any assessment. There's also the MGTOW-esque subgroup that eschews such interviews and will proclaim it proudly, along with their salary. Personally I'd be embarrassed if I was making a less than obscene dev salary, which is still upper middle class anywhere in the world, and all the while complaining that I deserve more for no added effort. What's funny is there are other professions where the same sort of multimodal distribution in compensation exists (big law vs everyone else, competitive specialties vs general practice in medicine, IB and PE vs everyone else in finance) and I have never seen members of those communities complain as much as developers do. Edit: I have a friend that's trying to break into FAANG (as a DS) who I've been coaching. He has a stats PhD and basically no software training whatsoever, outside of R. Since DS loops still have LC questions it's been very hard for him. I have never heard him once express feelings of resentment over the process - he fails an interview and just goes back to practicing. He's also a first gen immigrant from a very poor part of the middle east. Admittedly it's hard to assign/attribute his perseverance to any particular thing but naive intuition dictates that some of it must be his humility - something that software as an industry seriously lacks.
    • majormajor 5y ago
      From friends and acquaintances in other industries with similar income distributions, "knock out 4-to-6 programming problems in a short amount of time" is one of the least-arbitrary and most accessible ways to jump from one group to the more highly paid one. Not to mention that if you can knock out those problems, you may not even be required to have a degree or other formal credentials. That's not even getting into how hilariously unrealistic "trial periods", for instance, are. Why am I going to leave a job for just a trial period without any more thorough attempt to check fit before hand? Why would I invest that much time in side work if I'm doing it before quitting, when I could get a FAANG offer instead for single-digit-days-amount of brushing up?
• waynecochran 5y ago
  "but software development is not data structures and algorithms" Now I know why there is so much bad software out there... you just gutted the core of what SD is.
• mdip 5y ago
  I agree with the author, mostly, though I don't have as much of a problem with the deeply technical questions he's suggesting one shy away from -- my problem is with how the responses are handled by the interviewer. And as far as the whole "give them the actual work" approach, I love it -- I do it, and I stand by it, but don't abuse the candidates time in the process. For the most part, I avoid the data structures-style questions. We have a limited time as interviewers and while these show off a candidates knowledge in areas we may need very rarely, I'd rather have someone who is productive at using what the framework we write code in provides. I stick with "when is it appropriate to use 'X' vs 'Y'" style questions but usually don't have to ask them; the candidate reveals that knowledge (or lack thereof) through other questions and I hate whiteboard interviews. I don't penalize the candidate for lacking knowledge they're never going to use on the job, but I like to know what areas they have deep understanding of. Before I ask these questions, I'm careful to point out "I don't care if you know the answer to this or not -- you might encounter that once in 10 years, here -- but if we do encounter it, it's helpful to know if you have a deeper understanding than others on the team." For the most part, though, it's a waste of time and I'd rather spend that time on the next two points. My preference is to ask the candidate (very carefully) to provide a link to anything they've written that they can legally[0] share openly. My explanation includes all of the following: I am not interested in judging them on the code they send me -- in fact, I want their worst. I want the code that they don't care about, that they wrote in a hurry to solve some random problem that they had and that "just needed to work and do that one thing". I'll use that code as a starting point to discuss refactoring -- i.e. "What if you wanted to take this code and make it worthy of a product you'd sell". If there's absolutely nothing that they are comfortable[1] sharing, I ask them to take a few minutes and find a project in the framework we're interviewing for on GitHub that we can use for that purpose. As for the "give them the actual work", we have a single-sheet project we give to every candidate, Junior, Mid or Senior. It's a "to do" app with basic requirements: Must use a database of some kind for storing/retrieving data (Sqlite is called out as an option, but anything relational is allowed), it must allow adding, marking complete, unmarking complete and deleting. We additional coach Senior/Mid developers to include things that would be important for a completely released application with examples (none of which are required), such as documentation, instructions for starting/setting it up, installers (with advice not to take this too far as that's one hell of a rabbit hole). Depending on the outcome of the "actual work", we might follow-up in a similar manner to ask them questions about their choices. I've seen everything from a To Do app that used SignalR to perform its operations, allowing multiple users to watch the check-list get updated in real-time (complete with a WiX installer, OWIN hosted web service running as a Windows Service with everything wired up and a doc generation project[2]) to a simple HTML-template based solution that is done in four files. Of course, we check the usual sources for obvious evidence of whole-sale copying, but we're not terribly strict here, either (this point is made on the spec sheet -- it's such a simple app, it shouldn't require a lot of research/copy-pasta to build). [0] The first time I did this, I left off that "legally" part and the candidate shared code that -- while not representing anything their employer would care about (basic library code that probably every programmer as written from time to time) and certainly not representing anything the company I worked for was interested in "stealing", it carried a copyright that they did not own. Unfortunately, it disqualified the candidate (not my choice in this case, but the company I was working for at the time had concerns that the employee might introduce liability we didn't want). And I don't want any candidate to think I'm asking them to break their contractual obligations in order to get a job -- we operate ethically and don't want to imply otherwise. [1] I'm always careful to say "comfortable" so as to allow them to not have to feel bad about not having any code written "outside of work". I have met many software developers (albiet often in the Junior/mid-level skill levels) who simply don't care to write software outside of work. At the senior level, I've met several software developers who haven't written anything publicly shared in a while -- they're working too much at their current job, have families, and might not have the time to devote to things outside of that. Interviewing sucks and anything I can do to disarm the person and get them talking/excited will help me to evaluate them better. [2] The dude really wanted the job. OK, I lied, the dude was me, but it was rare that I wouldn't include these things in an app I was writing for my current employer, so those parts were automatic (and easy at that point) for me -- it felt wrong not to include them. I used the documentation to explain my choices, mostly, which avoided a second technical interview.
• KerryJones 5y ago
  As a previous CTO and hiring manager, we did do trials, and it worked great. At the time I found an article showing that traditional interviews resulted in an 18% true-positive expectation of how they would perform, while a week-long trial got you to something like 73%. We paid them for the week like a contractor, and made an evaluation at the end of the week. It weeded people out who were great at code but bad culturally, or people who worked great. Sometimes we couldn't get a whole week to commit, but just a few days -- but these days made a huge difference. Now, I work at a FAANG company and _hate_ the interview process. Give me 3 months and I think I could teach just about any CS interview how to pass it (without being specific to any particular FAANG company), but that doesn't mean they would be good engineers. They are loosely related but different skills.
  • stonogo 5y ago
    > but bad culturally Like people who are already employed?
    • KerryJones 5y ago
      We did a trial week with someone who ended up technically excellent, but the feedback I got from existing employees who worked with the candidate really didn't like the candidate (so we passed). This didn't come out in the culture screen we did prior to the week.
• andrewflnr 5y ago
  So a lot of the more complicated DS&A questions seem ridiculous, but I had to check on "inverting a binary tree" to see if it was really as simple as it sounded. It is. Just swap left and right, recursively. It's one of the easiest possible tests of being able to grok recursion[0]. You should be able to do this, even if you're self-taught or early in a college degree program. Am I missing something? The only excuse I can imagine is interview anxiety, which is a real problem that deserves accommodation (bad explanations of the problem may play a role too, but you should be able to ask good clarifying questions). But if someone is really incapable of this very minor feat of programming, then I'm sorry but an interview process is right to reject them. [0] Even if you don't literally write recursive Java methods, which you probably shouldn't, understanding recursive logic is critical all over the tech stack.
  • dgs_sgd 5y ago
    I agree with this. But asking the candidate to invert a binary tree is one thing and asking the candidate to flawlessly solve back to back leetcode "hard" level problems is another. I think the latter is mostly what people lament, where you end up doing so much prep that it boils down to "I've seen this one before" and you proceed to regurgitate what you memorized.
    • andrewflnr 5y ago
      My point is that people lament the latter while using the former as an example. This makes everyone dumber.
• angarg12 5y ago
  > but the best solution is to have trial work periods. There’s no better way to see how someone performs at the job than having them actually do the job. > I agree trial work periods may not scale Great to see the author uses the Green Lumber fallacy to argue against leetcode-style interviews. Now I'm going to guess he also must have skin in the game, otherwise he would only be an empty suit doing armchair recruiting. Let's say we want to do work trial periods. I tell you what actually happened to us: we opened an internship position and we had ~1600 applicants. Since having 1600 trial periods is impossible, we need a way to weed this down to a manageable number. Congratulations, now you moved the problem of "who do we hire" to "who do we invite for a trial period". The software engineering interview isn't perfect, but sadly trial periods are not the answer.
  • shxdow 5y ago
    I still don't get how OP (and every person that agrees with him) seems to conveniently avoid tackling all the logistics of their proposed solution.
• vrnvu 5y ago
  The biggest problem for me is that this style of interview focuses on theoretical efficiency. And the discussions about the interview process only cares about "real world engineering" vs "interview questions". We are discussing algorithms and data structures. What about actual performance? Your code will run on real hardware. Are we interviewing considering that? Do you know what memory alignment is? Padding? Branching? The implications in performance of a cache miss? These questions are never asked! Two nested for loops O(n2) where you considered memory alignment and your cpu cache size when choosing the data structures and defining your structs will perform better than your O(log n) algo with a high missing rate and branching all over the place. In my day to day I work on a garbage collected language (which I think are great, don't get me wrong) and I'm tired of seeing programmers thinking that memory is free, GC is free, syscalls and networking are magic... Software engineering culture is broken in general. From the education phase to the "real world".
• pidge 5y ago
  Google’s first hire (Craig Silverstein) was an International Collegiate Programming Contest champion. Make of that what you will.
• throwawaygal7 5y ago
  people in software tend to be introverts who sink a lot of their life and self worth into the job. We see the effects of this in the gatekeeping by elite software engineers who firmly believe the vast majority of degree holding, gainfully employed software engineers are complete idiots lucky to have a job - which is ridiculous. The other side of it is the ridiculous notion some developers have of not having to meet these standards to get the best jobs.... two sides of the same coin that starts with an unhealthy attachment to your profession as your identity.