6 ms·

Author here, thanks for sharing this. Let me know what you think. I'm trying to connect the dots on research on expert advice and our fields 'thought-leaders'.

by agbell 5y ago

Author here, thanks for sharing this. Let me know what you think. I'm trying to connect the dots on research on expert advice and our fields 'thought-leaders'. The connection is a bit tenuous but I think contingent advice can be shown to be better than non-contingent advice. I also think people are too confident in their opinions. Also another submission here: https://news.ycombinator.com/item?id=27462255 https://news.ycombinator.com/item?id=27462255

anguslmm 5y ago
This is pure gold for fresher developers, and something of which more experienced devs could use a reminder. Every fad and every champion of every technique or framework has something to teach you, and they are often very happy to teach it to you at the wrong time. Trying to please everyone at the start of the project is tantamount to design by committee, and is a sure way to kill a project. To a hammer, everything looks like nails. Well written.
- agbell 5y ago
  Thanks for reading it. It was one of the those ideas bouncing around in the back of my head for a while but hard to put into words then I read something about Tetlock and the dots sort of connected for me. Software advice isn't totally a prediction, but it sort of is.
  - E14n 5y ago
    I think this is generally good advice for software engineering, accept when its not. The problem is that some bad ideas become better ideas by virtue of being popular ideas. Write a shitty framework/language/technology and you have nothing, convince a million people to use it and it becomes compelling because it has a lot of users working with it and solving problems. Its the classic stone soup story[1]. You see this especially with software and tools that focus on front load new users making it really easy to do trivial things but failing catastrophically when you need more. You also see the reverse of this, great ideas that don't get bye-in failing by virtue of being too niche. 1. https://en.wikipedia.org/wiki/Stone_Soup https://en.wikipedia.org/wiki/Stone_Soup
Ozzie_osman 5y ago
Great read. One thing I've found (as a person who advises engineering managers and startups) is that recipients of advice seem to value non-contingent advice more. They just want simple answers that don't make them think. When someone asks me a question like "how should I interview candidates?", my default answer is "it depends". Tell me about the role. The company. The culture. The product. Remote or in-person? What's the team like. Then i can give a framework that gives you the answer. But people want answers like "use take-homes" or "do 2 behavioral interviews and 1 coding interview". Same for technical decisions. They don't want to hear "it depends". They want to hear "use Rails and MySQL hosted on Heroku". So I naturally find myself being pushed to give non-contingent advice.
- nradov 5y ago
  Do managers who accept contingent advice from you tend to produce better outcomes?
  - webel0 5y ago
    This is a good question. As much as I find them hard to stand, there are cases where having an ideologue in the back of your mind (or one on each shoulder) makes it easier to play out a situation in one’s mind. In other words: I don’t need a rule per se, just an argument. I’ll work out the contingencies on my own.
- agbell 5y ago
  Yeah, I think that could be true in general. Tetlock found the hedgehogs were more famous and more wrong and I got the idea from him that that was because simple advice is more sticky and works better in sounds bites but it could also work the opposite way -- ie. the more people ask you for advice the more you learn to tell them what they want which might be oversimplified. So you get trained to be more of a hedgehog over time. Interesting idea.
- lostcolony 5y ago
  So consider the position the people asking the questions are in. They're facing a problem; they need it solved. All of us, when we're in that position, desire a solution. I'm not sure what differentiates those who want to fully understand the whole solution space, and all the context that dictates -why- a particular solution may be the 'best' (given a specific set of tradeoffs), but certainly, whether we are like that or not, we all desire the right solution ASAP. I'd be super interested in how you respond to those who ask such questions; do they seem interested in explaining their problem in detail? If you, rather than say "it depends", instead immediately launch into questions, are they engaged in answering them? Can you then finish with a "given what you describe, because X, Y, and Z, I think (solution) would be the best fit for you. It has the downsides of A, B, and C, but those don't apply to you", or whatever. I.e., basically change the tone to always be focusing on solving their problem, while also allowing you to inform them, rather than "it depends" which could imply "there isn't a clear-cut solution to your problem".
rch 5y ago
Get article. I really appreciate how you were able to incorporate Tetlock's findings. I've been surprised by how reluctant sales and marketing people are to Brier Scores when it comes to their forecasting, given their interest in delivery estimates from engineering.
Muley 5y ago
Great article Adam, fyi the LinkedIn link on your website is broken. Was trying to look up your history. Also Ctrl F: "beleive" -> "believe"
- agbell 5y ago
  Thanks - I'll fix that. There aren't too many Adam Gordon Bell's out there though.
gumby 5y ago
A side point: the hedgehog/fox analogy originated with philosopher Isaiah Berlin.
- zimpenfish 5y ago
  Technically, I suppose, it was Archilochus: "The fox knows many things; the hedgehog one great thing." Tetlock seems to have a slightly different interpretation to Berlin - (paraphrased from [1]) "hedgehogs have one grand theory; foxes are skeptical about grand theories". [1] https://longnow.org/seminars/02007/jan/26/why-foxes-are-better-forecasters-than-hedgehogs/ https://longnow.org/seminars/02007/jan/26/why-foxes-are-bett...
  - unhammer 5y ago
    I was immediately reminded of the fable of the Fox and the Cat (in which the Cat wins because following its one heuristic takes no time while the Fox has to deliberate to find the best of its many smart solutions – and the dogs are coming fast). Apparently the tales are related https://en.wikipedia.org/wiki/The_Fox_and_the_Cat_(fable)#The_Hedgehog_and_the_Fox https://en.wikipedia.org/wiki/The_Fox_and_the_Cat_(fable)#Th... (while Aesop's https://aesopsfables.org/F89_The-Fox-and-the-Hedgehog.html https://aesopsfables.org/F89_The-Fox-and-the-Hedgehog.html is completely unrelated, though also interesting!)
- FriedrichN 5y ago
  One thing to note is that Berlin does not consider the fox as all around better than the hedgehog. Usually you'll see that people have a preference for the fox but Berlin considers some great people as hedgehogs such as Plato, Nietzsche, and Dostoevsky. He also stresses the fact that it's merely a metaphor and shouldn't be applied strictly. The book is absolutely worth a read if you're into the subject.
megameter 5y ago
There is a very similar problem in advice-giving for technical questions, the problem of "Why do you want to do that? You should do this instead." I've seen others recommend trying to ask binary yes/no questions ("I think it's like this. Yes/no?") or to turn an open-ended question, when asked, into a set of binaries rather than guess at the intent. The property that seems to be common in addressing both is benchmark-setting. The advice of "kick the can down the road" for less productive advice is premised on knowing that it doesn't fit your success benchmarks, but not wanting the confrontation(since a hedgehog benchmark is going to boil down to a single-issue attachment). Likewise, a battery of narrow binary questions that have a definite pass/fail characteristic constructs a form of fox knowledge - it's pragmatic in how it describes the "potential shape" of the outcome, so it makes for a better holistic benchmark than asking "what's the best way to do this?"
- devchix 5y ago
  > the problem of "Why do you want to do that? You should do this instead IIR, there's a word or idiom that describes this kind of solution, I can't think of it and now it's going to bother me until I do. It's a stackoverflow issue, someone asks "How do I do X?" Someone will counter, "Why do you want to do X?" and upon receiving additional information, answer, "You don't want to do X, or this other thing you're doing before doing X. You want to start this way and go down this path and that way you don't have to do X." Maddening!
  - devchix 5y ago
    I remember now! It is called the XY problem. https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem https://meta.stackexchange.com/questions/66377/what-is-the-x...
nyczomg 5y ago
Great read. Also, it inspired someone to post this super interesting comment: https://news.ycombinator.com/item?id=27468654 https://news.ycombinator.com/item?id=27468654 I'd be interested to hear your thoughts on that take since I thought it was very insightful.
lhorie 5y ago
Nice article, very relatable to my experiences. Having been on both sides of these types of discussions, I have a few thoughts: Advice isn't always unconditionally uncontingent. An infra person saying that something should probably be done in some overly specific preachy "best practice" way is sometimes thinking of things that a product person may not. For example, maybe the data guy told you to use WebScaleDB because scaaale, and you chose to use a simple YourSQL thing instead. But it turns out that in the next semester, a metal team you had never heard of is working on chaos testing and they're making sure WebScaleDB handles datacenter failovers properly (but they don't know about your snowflake YourSQL instance silently chugging along in a forgotten corner of one DC). This sort of stuff can be very tricky to anticipate, especially in large companies with siloed teams. I've found it useful to fully embrace the idea of leveraging technical debt: yes maybe YourSQL won't scaaale and maybe it'll die horribly and without explanation when failovers start happening, but if it can carry us to the next point in the evolution cycle, then we can reevaluate our options then, instead of being trapped in analysis paralysis and getting nothing done for the entire duration of time. As a person giving advice, I feel that I fall in the contingent camp (looking at specifics before giving suggestions), but over the years, I've started to try to be mindful of cognitive overload: saying "it depends because X, Y, Z" often goes over people's heads especially when they're already trying to soak up advice from a million different directions. Sometimes, it's better to just take a stance and spit out the TL;DR. If the stance happens to align with "best practices", you can just point at them and people are usually satisfied; if it doesn't align, you can often sway people to understand that there is nuance with a clever enough soundbite: "no, actually you don't want to enforce 100% coverage, full coverage tells you nothing about test quality, uncovered code is what tells you what you're lacking" (or "you don't need WebScaleDB; a billion db rows can be binary-searched in 10 comparisons"). Even if your dumbed down advice now lacks nuance, there's always the opportunity to course-correct as the team builds more experience on top of that advice. Sometimes, you have to be the thought leader and drive the change you want. At my company, for the longest time, every team was suffering the pains of Jenkins. You can't do X because otherwise Jenkins will not be able to handle it, they'd say. We've invested a lot in Jenkins, they'd say. A scaling solution is coming soon, they'd say. My team couldn't wait anymore and we took the initiative to bring in an off-the-shelf 3rd party solution that had all of the pain points figured out (and then some). This turned out to be a really good call because just a week after we deployed the new solution, our Jenkins cluster - shadowing at this point - completely gave out due to scale limits. This third party solution is now what other teams in the company are adopting - including teams that were investing in jenkins integrations before.
Ensorceled 5y ago
This a good framework for identifying a particular type of bad, or useless, feedback. I've added "non-contingent" to my vocabulary. For example, another type of useless feedback is so general as to be insulting; "this needs to scale" or "it needs to be high quality". "too general" vs "non-contingent" are nice distinct buckets.
mikehollinger 5y ago
You captured this well. I genuinely thought this was just something that happened in my company, which has been around for a bit. ;-) I accidentally stumbled upon your technique independently! A) I’d love to have a coffee with you. Virtual or otherwise! B) What do you think about alignment of priorities -within- a team? I’ve seen some interesting behaviors and misbehaviors in a team, where initiatives that are both trivial and non trivial die a death of a thousand cuts because of various and sundry plausible reasons. If I peel back the onion on it, it seems like those situations are ones that arise because of a fundamental lack of trust. Would you challenge or support that premise? If supported would you consider external stakeholders’ objections to stem from the same root lack of trust? It seems like we get more “hedgehog” like behavior when we don’t trust each other, and more “fox-like” behavior when there’s better trust and communication.

by agbell 5y ago

anguslmm 5y ago

This is pure gold for fresher developers, and something of which more experienced devs could use a reminder. Every fad and every champion of every technique or framework has something to teach you, and they are often very happy to teach it to you at the wrong time. Trying to please everyone at the start of the project is tantamount to design by committee, and is a sure way to kill a project. To a hammer, everything looks like nails. Well written.

agbell 5y ago
Thanks for reading it. It was one of the those ideas bouncing around in the back of my head for a while but hard to put into words then I read something about Tetlock and the dots sort of connected for me. Software advice isn't totally a prediction, but it sort of is.
- E14n 5y ago
  I think this is generally good advice for software engineering, accept when its not. The problem is that some bad ideas become better ideas by virtue of being popular ideas. Write a shitty framework/language/technology and you have nothing, convince a million people to use it and it becomes compelling because it has a lot of users working with it and solving problems. Its the classic stone soup story[1]. You see this especially with software and tools that focus on front load new users making it really easy to do trivial things but failing catastrophically when you need more. You also see the reverse of this, great ideas that don't get bye-in failing by virtue of being too niche. 1. https://en.wikipedia.org/wiki/Stone_Soup https://en.wikipedia.org/wiki/Stone_Soup

Ozzie_osman 5y ago

Great read. One thing I've found (as a person who advises engineering managers and startups) is that recipients of advice seem to value non-contingent advice more. They just want simple answers that don't make them think. When someone asks me a question like "how should I interview candidates?", my default answer is "it depends". Tell me about the role. The company. The culture. The product. Remote or in-person? What's the team like. Then i can give a framework that gives you the answer. But people want answers like "use take-homes" or "do 2 behavioral interviews and 1 coding interview". Same for technical decisions. They don't want to hear "it depends". They want to hear "use Rails and MySQL hosted on Heroku". So I naturally find myself being pushed to give non-contingent advice.

nradov 5y ago
Do managers who accept contingent advice from you tend to produce better outcomes?
- webel0 5y ago
  This is a good question. As much as I find them hard to stand, there are cases where having an ideologue in the back of your mind (or one on each shoulder) makes it easier to play out a situation in one’s mind. In other words: I don’t need a rule per se, just an argument. I’ll work out the contingencies on my own.
agbell 5y ago
Yeah, I think that could be true in general. Tetlock found the hedgehogs were more famous and more wrong and I got the idea from him that that was because simple advice is more sticky and works better in sounds bites but it could also work the opposite way -- ie. the more people ask you for advice the more you learn to tell them what they want which might be oversimplified. So you get trained to be more of a hedgehog over time. Interesting idea.
lostcolony 5y ago
So consider the position the people asking the questions are in. They're facing a problem; they need it solved. All of us, when we're in that position, desire a solution. I'm not sure what differentiates those who want to fully understand the whole solution space, and all the context that dictates -why- a particular solution may be the 'best' (given a specific set of tradeoffs), but certainly, whether we are like that or not, we all desire the right solution ASAP. I'd be super interested in how you respond to those who ask such questions; do they seem interested in explaining their problem in detail? If you, rather than say "it depends", instead immediately launch into questions, are they engaged in answering them? Can you then finish with a "given what you describe, because X, Y, and Z, I think (solution) would be the best fit for you. It has the downsides of A, B, and C, but those don't apply to you", or whatever. I.e., basically change the tone to always be focusing on solving their problem, while also allowing you to inform them, rather than "it depends" which could imply "there isn't a clear-cut solution to your problem".

rch 5y ago

Get article. I really appreciate how you were able to incorporate Tetlock's findings. I've been surprised by how reluctant sales and marketing people are to Brier Scores when it comes to their forecasting, given their interest in delivery estimates from engineering.

Muley 5y ago

Great article Adam, fyi the LinkedIn link on your website is broken. Was trying to look up your history. Also Ctrl F: "beleive" -> "believe"

agbell 5y ago
Thanks - I'll fix that. There aren't too many Adam Gordon Bell's out there though.

gumby 5y ago

A side point: the hedgehog/fox analogy originated with philosopher Isaiah Berlin.

zimpenfish 5y ago
Technically, I suppose, it was Archilochus: "The fox knows many things; the hedgehog one great thing." Tetlock seems to have a slightly different interpretation to Berlin - (paraphrased from [1]) "hedgehogs have one grand theory; foxes are skeptical about grand theories". [1] https://longnow.org/seminars/02007/jan/26/why-foxes-are-better-forecasters-than-hedgehogs/ https://longnow.org/seminars/02007/jan/26/why-foxes-are-bett...
- unhammer 5y ago
  I was immediately reminded of the fable of the Fox and the Cat (in which the Cat wins because following its one heuristic takes no time while the Fox has to deliberate to find the best of its many smart solutions – and the dogs are coming fast). Apparently the tales are related https://en.wikipedia.org/wiki/The_Fox_and_the_Cat_(fable)#The_Hedgehog_and_the_Fox https://en.wikipedia.org/wiki/The_Fox_and_the_Cat_(fable)#Th... (while Aesop's https://aesopsfables.org/F89_The-Fox-and-the-Hedgehog.html https://aesopsfables.org/F89_The-Fox-and-the-Hedgehog.html is completely unrelated, though also interesting!)
FriedrichN 5y ago
One thing to note is that Berlin does not consider the fox as all around better than the hedgehog. Usually you'll see that people have a preference for the fox but Berlin considers some great people as hedgehogs such as Plato, Nietzsche, and Dostoevsky. He also stresses the fact that it's merely a metaphor and shouldn't be applied strictly. The book is absolutely worth a read if you're into the subject.

megameter 5y ago

There is a very similar problem in advice-giving for technical questions, the problem of "Why do you want to do that? You should do this instead." I've seen others recommend trying to ask binary yes/no questions ("I think it's like this. Yes/no?") or to turn an open-ended question, when asked, into a set of binaries rather than guess at the intent. The property that seems to be common in addressing both is benchmark-setting. The advice of "kick the can down the road" for less productive advice is premised on knowing that it doesn't fit your success benchmarks, but not wanting the confrontation(since a hedgehog benchmark is going to boil down to a single-issue attachment). Likewise, a battery of narrow binary questions that have a definite pass/fail characteristic constructs a form of fox knowledge - it's pragmatic in how it describes the "potential shape" of the outcome, so it makes for a better holistic benchmark than asking "what's the best way to do this?"

devchix 5y ago
> the problem of "Why do you want to do that? You should do this instead IIR, there's a word or idiom that describes this kind of solution, I can't think of it and now it's going to bother me until I do. It's a stackoverflow issue, someone asks "How do I do X?" Someone will counter, "Why do you want to do X?" and upon receiving additional information, answer, "You don't want to do X, or this other thing you're doing before doing X. You want to start this way and go down this path and that way you don't have to do X." Maddening!
- devchix 5y ago
  I remember now! It is called the XY problem. https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem https://meta.stackexchange.com/questions/66377/what-is-the-x...

nyczomg 5y ago

Great read. Also, it inspired someone to post this super interesting comment: https://news.ycombinator.com/item?id=27468654 https://news.ycombinator.com/item?id=27468654 I'd be interested to hear your thoughts on that take since I thought it was very insightful.

lhorie 5y ago

Nice article, very relatable to my experiences. Having been on both sides of these types of discussions, I have a few thoughts: Advice isn't always unconditionally uncontingent. An infra person saying that something should probably be done in some overly specific preachy "best practice" way is sometimes thinking of things that a product person may not. For example, maybe the data guy told you to use WebScaleDB because scaaale, and you chose to use a simple YourSQL thing instead. But it turns out that in the next semester, a metal team you had never heard of is working on chaos testing and they're making sure WebScaleDB handles datacenter failovers properly (but they don't know about your snowflake YourSQL instance silently chugging along in a forgotten corner of one DC). This sort of stuff can be very tricky to anticipate, especially in large companies with siloed teams. I've found it useful to fully embrace the idea of leveraging technical debt: yes maybe YourSQL won't scaaale and maybe it'll die horribly and without explanation when failovers start happening, but if it can carry us to the next point in the evolution cycle, then we can reevaluate our options then, instead of being trapped in analysis paralysis and getting nothing done for the entire duration of time. As a person giving advice, I feel that I fall in the contingent camp (looking at specifics before giving suggestions), but over the years, I've started to try to be mindful of cognitive overload: saying "it depends because X, Y, Z" often goes over people's heads especially when they're already trying to soak up advice from a million different directions. Sometimes, it's better to just take a stance and spit out the TL;DR. If the stance happens to align with "best practices", you can just point at them and people are usually satisfied; if it doesn't align, you can often sway people to understand that there is nuance with a clever enough soundbite: "no, actually you don't want to enforce 100% coverage, full coverage tells you nothing about test quality, uncovered code is what tells you what you're lacking" (or "you don't need WebScaleDB; a billion db rows can be binary-searched in 10 comparisons"). Even if your dumbed down advice now lacks nuance, there's always the opportunity to course-correct as the team builds more experience on top of that advice. Sometimes, you have to be the thought leader and drive the change you want. At my company, for the longest time, every team was suffering the pains of Jenkins. You can't do X because otherwise Jenkins will not be able to handle it, they'd say. We've invested a lot in Jenkins, they'd say. A scaling solution is coming soon, they'd say. My team couldn't wait anymore and we took the initiative to bring in an off-the-shelf 3rd party solution that had all of the pain points figured out (and then some). This turned out to be a really good call because just a week after we deployed the new solution, our Jenkins cluster - shadowing at this point - completely gave out due to scale limits. This third party solution is now what other teams in the company are adopting - including teams that were investing in jenkins integrations before.

Ensorceled 5y ago

This a good framework for identifying a particular type of bad, or useless, feedback. I've added "non-contingent" to my vocabulary. For example, another type of useless feedback is so general as to be insulting; "this needs to scale" or "it needs to be high quality". "too general" vs "non-contingent" are nice distinct buckets.

mikehollinger 5y ago

You captured this well. I genuinely thought this was just something that happened in my company, which has been around for a bit. ;-) I accidentally stumbled upon your technique independently! A) I’d love to have a coffee with you. Virtual or otherwise! B) What do you think about alignment of priorities -within- a team? I’ve seen some interesting behaviors and misbehaviors in a team, where initiatives that are both trivial and non trivial die a death of a thousand cuts because of various and sundry plausible reasons. If I peel back the onion on it, it seems like those situations are ones that arise because of a fundamental lack of trust. Would you challenge or support that premise? If supported would you consider external stakeholders’ objections to stem from the same root lack of trust? It seems like we get more “hedgehog” like behavior when we don’t trust each other, and more “fox-like” behavior when there’s better trust and communication.