7 ms·
I was going to call out this quote : If you read this far hoping for an answer, then I’m sorry: I don’t have a simple one. This is a difficult problem to solve.
by ChuckMcM 8y ago
I was going to call out this quote : If you read this far hoping for an answer, then I’m sorry: I don’t have a simple one. This is a difficult problem to solve.
And yet there is an answer, it is hire an experienced engineer. And the author glosses over the problem by not realizing they "solved" the problem by hiring him.
It is the bad calculus where one "experienced" engineer is going to cost you maybe $150K - $200K and yet you could get anywhere from 3 to 10 junior engineers for that depending on how far out you go.
It would be awesome of there were some sort of certification for "experience" sort of equivalent to the guild master status, it isn't an education thing it is a "oh yeah I've seen that before, lets not do that here's why" kind of thing.
- yen223 8y agoIf you aren't technical yourself, how do you separate an experienced engineer from one who, well, isn't? This was the fundamental problem that the company failed to solve, and frankly I don't think there's any easy solution around it.
- deleted 8y ago[deleted]
- memset 8y agoAgree with this. Same applies to any expert: how do you identify an experienced marketer? Or accountant?
- yen223 8y ago... or climate change scientist? Or economist? Or president? The problem of separating experts vs non-experts might be the single biggest problem society faces right now.
- gameswithgo 8y agoassuming you have a generally good education and brain to start with, a lot of hard work can usually accomplish it. Think Elon Musk learning about rockets, back before he lost his mind. I did it once for climate change, took a couple months, some math. Got there though.
- chmhsm 8y agoBefore he lost his mind? You mean the recent interview or before that?
- jedharris 8y agoTo hire someone with expertise to solve a problem in a domain where you aren't an expert: You find people who claim to understand the domain, and then discuss the problem with them in depth. Your goal is not to evaluate them in general but to understand how they analyze the problem and how they'd deal with it. You may have to hire them as consultants. You have to dedicate a day or more to discussion with each of them -- maybe considerably more if your problem is complex. You have to reveal information about your plans and/or the condition of your systems that may make you anxious or embarrassed. After you've gone over the problem separately with five or ten putative experts, you will have a pretty good idea which of them understand it (if any), and you will also know how to ask much better questions going forward. If necessary repeat this process, using your better understanding to find better experts and gain a better map of the territory. Once you can rank expertise in your domain reliably, you might hire one (or more) of the known experts, but the best ones will be hardest to hire. Or you may use your new meta-expertise in the domain to look for more junior hires who have an unusually good grasp of your issues. Also, you may be able to get the best of your experts to act as advisors. This takes a lot of time and may cost some money depending on the domain. But not doing it will take more time and cost more money in the medium to long term (and even in the short term sometimes).
- hyperpallium 8y agoAKA become expert. I wonder if the people who become fantastically successful simply have the ability to evaluate expertise, without having any of it. i.e. by "reading" people, in some sense. This relies on the "expert"'s own evaluation of their expertise, which mightn't be accurate, because of Dunning–Kruger, or conversely, self-doubt. So, one would need to "read" those too.
- jedharris 8y agoYou definitely can't trust people's self-evaluation either way. Lots of really skilled people are self-critical. "Reading people" of course helps but it is nowhere near enough. But you don't have to become an expert. If I hire a plumber or carpenter, I cannot do what they can. But with enough work I can understand which plumbers or carpenters will do a good job.
- mathattack 8y agoDepends on the magnitude of the problem to be solved. Small problem: Ask friends for someone they have worked with on a similar problem. Large problem (building an org): Ask 5 VCs/CEOs/VIPs for intros to the best person they know in that role, under the explicit direction of “learning what good looks like.” Interview all 5 with open ended questions. Ask each for one for one more contact. Eventually you will learn to pattern match, and one of the folks you like will volunteer themselves or a friend, and you will know how to interview.
- hyperpallium 8y agoThis is the problem that credentials supposedly solve - degrees, certifications, professional body membership etc. You don't realize the difficulty of the problem until you must hire someone outside your own areas of expertise.
- acjohnson55 8y agoReferences and portfolio, at the end of the day.
- robocat 8y agoThe orange juice test is one person's answer: https://www.intercom.com/blog/the-orange-juice-test/ https://www.intercom.com/blog/the-orange-juice-test/
- deleted 8y ago[deleted]
- kuwze 8y agoI remember somebody on this forum saying the best way to judge someone's technical prowess is to give a simple explanation of a complicated part of their field. If you can't summarize your knowledge you don't know what you are talking about.
- wpietri 8y agoUnfortunately, a non-expert can't tell the difference between a good simple explanation and glib, confident ignorance. So I think that's a good test for expertise, but it's not one that a non-expert can fully judge.
- contravariant 8y agoI guess the assumption is that anyone or at least an inexperienced engineer would be able to point out an experienced engineer, while only experienced engineers are any good at picking less experienced engineers.
- megaremote 8y agoSurely there is a big clue in how long they have been working for, how many projects they have done, what their roles were.
- itronitron 8y agoBased on my experience interviewing over the past several years, the companies that have a 'production' system seem to only value experience with their tech stack.
- pm90 8y agoThat’s because they have set something up and ironed out all the bugs so it will take some convincing to move them to a different stack. There’s no easy way around it. In fact I’ve seen some teams too eager to rip up existing production workflows every time some new technology becomes popular and that’s not productive either.
- itronitron 8y agoI'm not saying that they should change their stack, I'm saying that they should consider that someone having extensive experience with related technologies probably knows the domain well and can come up to speed quickly on the team's shit.
- scarface74 8y agoHire a recruiting company. There are recruiting companies that have a staff of full time consultants that will bring in a whole development team and manage the product. At some point, you want to bring that knowledge in house, the recruiting company will then help you hire people or let you convert some of thier contractors (not thier full time people) to fulltime.
- dlhavema 8y agoWe have a lot of recruiters sending us "senior full stack engineers" that dont knoe the language they are most confident in and end up being dev-ops engineers who want to get into coding. A lot of recruiting firms dont know how to to get candidates either
- scarface74 8y agoI am not referring to recruiting companies that just send developers to a company. I am referring to recruiting companies that have full time staffed developers and project managers that manage the project for you in the beginning and then slowly help you build up your own staff. Some will have a combination of onsite developers, “rural source” developers who are in a cheaper part of the US, contractors and outsourced developers but they manage the entire project. I was offered such a position - the pay is above market but it required too much travel for me now.
- marcinzm 8y agoNow how do you separate out a recruitment company who has solid experience engineers on board from one that doesn't but says they do? I've seen many of the latter and many founders who thought they were getting the former in the deal.
- scarface74 8y agoIf you are starting a company, I would hope that you at least know what your requirements are and you could sit down with the project managers to hash out acceptance criteria and deliverables.
- Spooky23 8y agoUse referrals. Ultimately, if you cannot identify critical staff and don’t know anyone who can, you’re screwed anyway.
- wpietri 8y agoExactly. Somebody running a company can't be an expert in everything. But they can be a good judge of people and have a wide enough network that they can get recommendations from people they trust. A few of those trusted technologists will make a fine hiring/oversight committee. You won't need a ton of their time, so you can pay very generously for the insurance they provide.
- tormeh 8y agoDeloitte has pretty good technical interviews. I'm sure they can provide you interviews as a service. Probably not going to be cheap, though.
- crispyambulance 8y agoProjects fail for all kinds of reasons. Usually many reasons at the same time. While I am sure that _some_ fail because of the abject inexperience of the staff running them, this is hardly ever the complete story. Moreover, projects do fail even with a fully experienced, fully vetted staff running the show. The article just seems like a consultant marketing his acumen. There's nothing wrong with that. But keep in mind, he dropped in over a weekend and spent a few hours detailing problems that needed fixing and the staff subsequently made corrections and released on time. It could easily go the other way too. The "inexperienced" team could have instead brought in another seasoned consultant who, after a few hours, got the wrong idea and led them down the wrong path precipitating an even larger slippage of ship date. THAT HAPPENS TOO and I suspect it is even more common.
- anyzen 8y ago> Moreover, projects do fail even with a fully experienced, fully vetted staff running the show. Sure - but in that case they usually don't fail because of technical decisions (which is what OP was talking about). There are however many other ways a project can fail (bad management mostly).
- HeavyStorm 8y agoI don't really agree. Sure, many projects fail for reasons other than developer experience but in many years of many more projects, developer skill was the most frequent reason. Of course, that's probably more true of my environment (third world country, enterprise software) than others.
- gameswithgo 8y agoThat is sort of a tautology solution. Hire someone who can solve the problem and you won't have the problem. But how do you know when you have done that? Experience alone doesn't necessarily mean they can solve your problem.
- freedomben 8y agoRight, I think that's being missed in this thread. Experience alone doesn't necessarily mean they can solve the problem and ship/write good code. I've worked with plenty of "senior" engineers that have 20+ years experience yet have never dealt with any serious issues. These usually work at companies where a small team of "ninjas" handles those sorts of things, and the run of the mill Dilbert coder shows up and does his thing. This is not a knock on those people. They are important in their own way, truly. But I would not want them to be the responsible party for steering the ship in a new app.
- majormajor 8y agoVery few developers even trust themselves to vet the sorts of things this post is talking about. So our current level of in-industry interview training is going to make that a crapshoot. The general reason given for non-representative algorithmic whiteboard coding problems is "you can't trust anybody's experience." But those questions are themselves not resulting in production-ready code, so you're flying blind on "can figure out a quick way to solve a problem" vs "knows what to do to turn a quick solution into something that will last." And then there's a time-based calculation you have to make: hire fast and hope they'll learn on the fly, or slow down and delay until you find people you hope won't have to. You're hoping either way, after all. :)
- marcinzm 8y agoDefine experienced engineer. I've seen many on-paper experienced engineers who would be worse than a junior engineer on a project. >oh yeah I've seen that before, lets not do that here's why Which can actually be a pretty bad way of going about it imho if you only use your own direct experiences for that. Just like science, basing your decisions on a limited number of anecdotes is problematic because small biased subsets are not representative of the whole.
- pm90 8y agoYou have to make a decision based on something. Experience isn’t a bad heuristic if evaluated correctly: this is why in most serious technical interviews today, you will often get asked about your past experiences.
- marcinzm 8y ago>You have to make a decision based on something. And like science you can base it on the collective experience versus your personal experience. Preferably from rigorous studies but we mostly lack those in CS. So hopefully you're reading books, blog posts, studies, conference presentations, discussing with colleagues, etc. >this is why in most serious technical interviews today, you will often get asked about your past experiences. Interestingly the top companies seem to be focusing a lot less on experience and a lot more on problem solving/system design/etc.
- pm90 8y agoYou bring up a good point. I do think interviewing for developers is not a field that has been studied using the scientific method, which is why we seem to have so many rules of thumbs, cargo cult traditions etc. rather than experiment-backed results. I think a few companies do study this as a science (e.g. Google) but most others are either unwilling or don't have enough resources (or some mix of the 2) to do the same. > Interestingly the top companies seem to be focusing a lot less on experience and a lot more on problem solving/system design/etc. As long as we're making conjectures: this is likely because a lot of them have tools and processes that are very different from what you see "in the wild". e.g. working at Google means you can leverage all their deployment infrastructure, something you likely didn't have access to outside (this is somewhat changing with k8s and cloud services).
- jcadam 8y ago> hire an experienced engineer. But "experienced" and "cultural fit" are mutually exclusive for most startups.
- MrEfficiency 8y agoExperienced will tell you the refuse to release without X Y and Z because if B happens your server will crash. Junior wont even know the problem and hack it together. But hey, thats what reddit /r/askprogramming is for... right?