6 ms·

If you aren't technical yourself, how do you separate an experienced engineer from one who, well, isn't? This was the fundamental problem that the company faile

by yen223 8y ago

If you aren't technical yourself, how do you separate an experienced engineer from one who, well, isn't? This was the fundamental problem that the company failed to solve, and frankly I don't think there's any easy solution around it.

deleted 8y ago
[deleted]
memset 8y ago
Agree with this. Same applies to any expert: how do you identify an experienced marketer? Or accountant?
- yen223 8y ago
  ... or climate change scientist? Or economist? Or president? The problem of separating experts vs non-experts might be the single biggest problem society faces right now.
  - gameswithgo 8y ago
    assuming you have a generally good education and brain to start with, a lot of hard work can usually accomplish it. Think Elon Musk learning about rockets, back before he lost his mind. I did it once for climate change, took a couple months, some math. Got there though.
    - chmhsm 8y ago
      Before he lost his mind? You mean the recent interview or before that?
- jedharris 8y ago
  To hire someone with expertise to solve a problem in a domain where you aren't an expert: You find people who claim to understand the domain, and then discuss the problem with them in depth. Your goal is not to evaluate them in general but to understand how they analyze the problem and how they'd deal with it. You may have to hire them as consultants. You have to dedicate a day or more to discussion with each of them -- maybe considerably more if your problem is complex. You have to reveal information about your plans and/or the condition of your systems that may make you anxious or embarrassed. After you've gone over the problem separately with five or ten putative experts, you will have a pretty good idea which of them understand it (if any), and you will also know how to ask much better questions going forward. If necessary repeat this process, using your better understanding to find better experts and gain a better map of the territory. Once you can rank expertise in your domain reliably, you might hire one (or more) of the known experts, but the best ones will be hardest to hire. Or you may use your new meta-expertise in the domain to look for more junior hires who have an unusually good grasp of your issues. Also, you may be able to get the best of your experts to act as advisors. This takes a lot of time and may cost some money depending on the domain. But not doing it will take more time and cost more money in the medium to long term (and even in the short term sometimes).
  - hyperpallium 8y ago
    AKA become expert. I wonder if the people who become fantastically successful simply have the ability to evaluate expertise, without having any of it. i.e. by "reading" people, in some sense. This relies on the "expert"'s own evaluation of their expertise, which mightn't be accurate, because of Dunning–Kruger, or conversely, self-doubt. So, one would need to "read" those too.
    - jedharris 8y ago
      You definitely can't trust people's self-evaluation either way. Lots of really skilled people are self-critical. "Reading people" of course helps but it is nowhere near enough. But you don't have to become an expert. If I hire a plumber or carpenter, I cannot do what they can. But with enough work I can understand which plumbers or carpenters will do a good job.
- mathattack 8y ago
  Depends on the magnitude of the problem to be solved. Small problem: Ask friends for someone they have worked with on a similar problem. Large problem (building an org): Ask 5 VCs/CEOs/VIPs for intros to the best person they know in that role, under the explicit direction of “learning what good looks like.” Interview all 5 with open ended questions. Ask each for one for one more contact. Eventually you will learn to pattern match, and one of the folks you like will volunteer themselves or a friend, and you will know how to interview.
- hyperpallium 8y ago
  This is the problem that credentials supposedly solve - degrees, certifications, professional body membership etc. You don't realize the difficulty of the problem until you must hire someone outside your own areas of expertise.
- acjohnson55 8y ago
  References and portfolio, at the end of the day.
- robocat 8y ago
  The orange juice test is one person's answer: https://www.intercom.com/blog/the-orange-juice-test/ https://www.intercom.com/blog/the-orange-juice-test/
- deleted 8y ago
  [deleted]
- kuwze 8y ago
  I remember somebody on this forum saying the best way to judge someone's technical prowess is to give a simple explanation of a complicated part of their field. If you can't summarize your knowledge you don't know what you are talking about.
  - wpietri 8y ago
    Unfortunately, a non-expert can't tell the difference between a good simple explanation and glib, confident ignorance. So I think that's a good test for expertise, but it's not one that a non-expert can fully judge.
contravariant 8y ago
I guess the assumption is that anyone or at least an inexperienced engineer would be able to point out an experienced engineer, while only experienced engineers are any good at picking less experienced engineers.
megaremote 8y ago
Surely there is a big clue in how long they have been working for, how many projects they have done, what their roles were.
- itronitron 8y ago
  Based on my experience interviewing over the past several years, the companies that have a 'production' system seem to only value experience with their tech stack.
  - pm90 8y ago
    That’s because they have set something up and ironed out all the bugs so it will take some convincing to move them to a different stack. There’s no easy way around it. In fact I’ve seen some teams too eager to rip up existing production workflows every time some new technology becomes popular and that’s not productive either.
    - itronitron 8y ago
      I'm not saying that they should change their stack, I'm saying that they should consider that someone having extensive experience with related technologies probably knows the domain well and can come up to speed quickly on the team's shit.
scarface74 8y ago
Hire a recruiting company. There are recruiting companies that have a staff of full time consultants that will bring in a whole development team and manage the product. At some point, you want to bring that knowledge in house, the recruiting company will then help you hire people or let you convert some of thier contractors (not thier full time people) to fulltime.
- dlhavema 8y ago
  We have a lot of recruiters sending us "senior full stack engineers" that dont knoe the language they are most confident in and end up being dev-ops engineers who want to get into coding. A lot of recruiting firms dont know how to to get candidates either
  - scarface74 8y ago
    I am not referring to recruiting companies that just send developers to a company. I am referring to recruiting companies that have full time staffed developers and project managers that manage the project for you in the beginning and then slowly help you build up your own staff. Some will have a combination of onsite developers, “rural source” developers who are in a cheaper part of the US, contractors and outsourced developers but they manage the entire project. I was offered such a position - the pay is above market but it required too much travel for me now.
    - marcinzm 8y ago
      Now how do you separate out a recruitment company who has solid experience engineers on board from one that doesn't but says they do? I've seen many of the latter and many founders who thought they were getting the former in the deal.
      - scarface74 8y ago
        If you are starting a company, I would hope that you at least know what your requirements are and you could sit down with the project managers to hash out acceptance criteria and deliverables.
Spooky23 8y ago
Use referrals. Ultimately, if you cannot identify critical staff and don’t know anyone who can, you’re screwed anyway.
- wpietri 8y ago
  Exactly. Somebody running a company can't be an expert in everything. But they can be a good judge of people and have a wide enough network that they can get recommendations from people they trust. A few of those trusted technologists will make a fine hiring/oversight committee. You won't need a ton of their time, so you can pay very generously for the insurance they provide.
tormeh 8y ago
Deloitte has pretty good technical interviews. I'm sure they can provide you interviews as a service. Probably not going to be cheap, though.

by yen223 8y ago

deleted 8y ago

[deleted]

memset 8y ago

Agree with this. Same applies to any expert: how do you identify an experienced marketer? Or accountant?

yen223 8y ago
... or climate change scientist? Or economist? Or president? The problem of separating experts vs non-experts might be the single biggest problem society faces right now.
- gameswithgo 8y ago
  assuming you have a generally good education and brain to start with, a lot of hard work can usually accomplish it. Think Elon Musk learning about rockets, back before he lost his mind. I did it once for climate change, took a couple months, some math. Got there though.
  - chmhsm 8y ago
    Before he lost his mind? You mean the recent interview or before that?
jedharris 8y ago
To hire someone with expertise to solve a problem in a domain where you aren't an expert: You find people who claim to understand the domain, and then discuss the problem with them in depth. Your goal is not to evaluate them in general but to understand how they analyze the problem and how they'd deal with it. You may have to hire them as consultants. You have to dedicate a day or more to discussion with each of them -- maybe considerably more if your problem is complex. You have to reveal information about your plans and/or the condition of your systems that may make you anxious or embarrassed. After you've gone over the problem separately with five or ten putative experts, you will have a pretty good idea which of them understand it (if any), and you will also know how to ask much better questions going forward. If necessary repeat this process, using your better understanding to find better experts and gain a better map of the territory. Once you can rank expertise in your domain reliably, you might hire one (or more) of the known experts, but the best ones will be hardest to hire. Or you may use your new meta-expertise in the domain to look for more junior hires who have an unusually good grasp of your issues. Also, you may be able to get the best of your experts to act as advisors. This takes a lot of time and may cost some money depending on the domain. But not doing it will take more time and cost more money in the medium to long term (and even in the short term sometimes).
- hyperpallium 8y ago
  AKA become expert. I wonder if the people who become fantastically successful simply have the ability to evaluate expertise, without having any of it. i.e. by "reading" people, in some sense. This relies on the "expert"'s own evaluation of their expertise, which mightn't be accurate, because of Dunning–Kruger, or conversely, self-doubt. So, one would need to "read" those too.
  - jedharris 8y ago
    You definitely can't trust people's self-evaluation either way. Lots of really skilled people are self-critical. "Reading people" of course helps but it is nowhere near enough. But you don't have to become an expert. If I hire a plumber or carpenter, I cannot do what they can. But with enough work I can understand which plumbers or carpenters will do a good job.
mathattack 8y ago
Depends on the magnitude of the problem to be solved. Small problem: Ask friends for someone they have worked with on a similar problem. Large problem (building an org): Ask 5 VCs/CEOs/VIPs for intros to the best person they know in that role, under the explicit direction of “learning what good looks like.” Interview all 5 with open ended questions. Ask each for one for one more contact. Eventually you will learn to pattern match, and one of the folks you like will volunteer themselves or a friend, and you will know how to interview.
hyperpallium 8y ago
This is the problem that credentials supposedly solve - degrees, certifications, professional body membership etc. You don't realize the difficulty of the problem until you must hire someone outside your own areas of expertise.
acjohnson55 8y ago
References and portfolio, at the end of the day.
robocat 8y ago
The orange juice test is one person's answer: https://www.intercom.com/blog/the-orange-juice-test/ https://www.intercom.com/blog/the-orange-juice-test/
deleted 8y ago
[deleted]
kuwze 8y ago
I remember somebody on this forum saying the best way to judge someone's technical prowess is to give a simple explanation of a complicated part of their field. If you can't summarize your knowledge you don't know what you are talking about.
- wpietri 8y ago
  Unfortunately, a non-expert can't tell the difference between a good simple explanation and glib, confident ignorance. So I think that's a good test for expertise, but it's not one that a non-expert can fully judge.

contravariant 8y ago

I guess the assumption is that anyone or at least an inexperienced engineer would be able to point out an experienced engineer, while only experienced engineers are any good at picking less experienced engineers.

megaremote 8y ago

Surely there is a big clue in how long they have been working for, how many projects they have done, what their roles were.

itronitron 8y ago
Based on my experience interviewing over the past several years, the companies that have a 'production' system seem to only value experience with their tech stack.
- pm90 8y ago
  That’s because they have set something up and ironed out all the bugs so it will take some convincing to move them to a different stack. There’s no easy way around it. In fact I’ve seen some teams too eager to rip up existing production workflows every time some new technology becomes popular and that’s not productive either.
  - itronitron 8y ago
    I'm not saying that they should change their stack, I'm saying that they should consider that someone having extensive experience with related technologies probably knows the domain well and can come up to speed quickly on the team's shit.

scarface74 8y ago

Hire a recruiting company. There are recruiting companies that have a staff of full time consultants that will bring in a whole development team and manage the product. At some point, you want to bring that knowledge in house, the recruiting company will then help you hire people or let you convert some of thier contractors (not thier full time people) to fulltime.

dlhavema 8y ago
We have a lot of recruiters sending us "senior full stack engineers" that dont knoe the language they are most confident in and end up being dev-ops engineers who want to get into coding. A lot of recruiting firms dont know how to to get candidates either
- scarface74 8y ago
  I am not referring to recruiting companies that just send developers to a company. I am referring to recruiting companies that have full time staffed developers and project managers that manage the project for you in the beginning and then slowly help you build up your own staff. Some will have a combination of onsite developers, “rural source” developers who are in a cheaper part of the US, contractors and outsourced developers but they manage the entire project. I was offered such a position - the pay is above market but it required too much travel for me now.
  - marcinzm 8y ago
    Now how do you separate out a recruitment company who has solid experience engineers on board from one that doesn't but says they do? I've seen many of the latter and many founders who thought they were getting the former in the deal.
    - scarface74 8y ago
      If you are starting a company, I would hope that you at least know what your requirements are and you could sit down with the project managers to hash out acceptance criteria and deliverables.

Spooky23 8y ago

Use referrals. Ultimately, if you cannot identify critical staff and don’t know anyone who can, you’re screwed anyway.

wpietri 8y ago
Exactly. Somebody running a company can't be an expert in everything. But they can be a good judge of people and have a wide enough network that they can get recommendations from people they trust. A few of those trusted technologists will make a fine hiring/oversight committee. You won't need a ton of their time, so you can pay very generously for the insurance they provide.

tormeh 8y ago

Deloitte has pretty good technical interviews. I'm sure they can provide you interviews as a service. Probably not going to be cheap, though.