6 ms·

The Thermocline of Truth

57 points by ivank 5y ago

nickdothutton 5y ago
These "invisible disasters" are an almost daily occurrence in software, even within a single sector or geography. Multi-hundreds million dollar projects that are either severely malformed when launched or that collapse entirely before being placed in the hands of a user. I wrote a little about them in a previous post. Have things improved? Silly question. https://news.ycombinator.com/item?id=17844712 https://news.ycombinator.com/item?id=17844712
skmurphy 5y ago
References and extends this post, may be better to start with it: https://brucefwebster.com/2008/04/15/the-wetware-crisis-the-themocline-of-truth/ https://brucefwebster.com/2008/04/15/the-wetware-crisis-the-...
- ebcode 5y ago
  And if you follow the link at the bottom of that post, you'll find what the thermocline consists of: >There was upper Apple management, basking in the sun; the great mass of Apple employees below; and between the two, a constantly shifting layer of middle managers whose function appeared to be to keep the two groups from mixing.[0] [0] http://andstillipersist.com/2006/08/guerrilla-programming-life-below-the-thermocline/ http://andstillipersist.com/2006/08/guerrilla-programming-li...
jsnell 5y ago
Original post discussed a couple of weeks ago: https://news.ycombinator.com/item?id=27871024 https://news.ycombinator.com/item?id=27871024
abakker 5y ago
"Paula Vennells, who took over as Chief Executive in 2012 with, by all accounts, the best of intentions, found herself surrounded by a senior management team desperate to protect their own reputations and that of the Royal Mail. They told her that there simply couldn’t be problems with the IT system on the scale that was being suggested; it was much more likely that the sub-postmasters were colluding, lying about IT bugs in order to escape responsibility for what was at best incompetence and at worst theft." Hanlon's razor would have been useful here. How could anyone honestly believe that mass collusion amongst a distributed and isolated group of people, that was also perfectly successful, was more likely than an IT failure?
- trhway 5y ago
  She had a choice - either go against surrounding her "senior management team desperate to protect their own reputations and that of the Royal Mail" or blame those lowly sub-postmasters. It isn't about "honestly believing", it is what people to side with - your own who is just like you or the very different others. >They told her that there simply couldn’t be problems with the IT system on the scale that was being suggested and thus they provided her with plausible deniability. Like that Enron CEO who supposedly didn't know the financial details and relied on what is told by his CFO/etc. The whole article sounds like a whitewash of senior management responsibility by blaming it on the lower ranks who reports "rosy picture". No wonder that the author seems to be a successful consultant - i can see how his expert opinion may be helpful for such a whitewashing by establishing the existence of the "thermocline of truth" in the specific situations what he gets brought into and thus is liked by the upper management. As an extra example of "thermocline of truth" - that EBay CEO discussed again just yesterday who supposedly didn't know how his security chief with the gang were criminally harassing the blogger to carry out the CEO's order to “take her down”. I think approaches like RICO or SOX or plain old army principle that the CO is responsible for whatever happens under his command are the examples of the tools to pierce the "thermoclines of truth".
  - AnimalMuppet 5y ago
    If you're "siding", you aren't "honestly believing". You're choosing what you want to hear and believe, rather than choosing truth.
  - thewakalix 5y ago
    The article doesn’t blame the underlings that report a “rosy picture”, but rather blames the culture that heavily incentivizes reporting a rosy picture regardless of the facts on the ground, and punished everything else. And the people with the most power over this culture are the ones at the top, making them the most culpable for the thermocline.
    - trhway 5y ago
      setting aside how top managers weaponize anything into "blame others and excuse themselves", the blame is more clear in the original blog post linked in the article https://brucefwebster.com/2008/04/15/the-wetware-crisis-the-themocline-of-truth/ https://brucefwebster.com/2008/04/15/the-wetware-crisis-the-...
ChrisMarshallNY 5y ago
> They are the inevitable consequence of an organisation that puts more emphasis on reputation than truth; that rewards good news and punishes bad; that has a leadership team disconnected from the delivery of the actual work; that instils fear and compliance with process into its employees, rather than a desire to do the right thing. This is so endemic to all organizations. When you shoot messengers, expect to receive fewer messages.
- JumpCrisscross 5y ago
  > When you shoot messengers, expect to receive fewer messages Most modern organizations don't have pure messengers. The king is his messenger. That can blur the line between accountability and transparency. For example, in this case, the messengers were the senior managers partly responsible for the mess. Good managers clearly delineate that boundary. Bringing bad news isn't punished. But making bad calls alone may be. In practice, this takes discipline to do.
  - fungiblecog 5y ago
    The "Good" descriptor is the problem though isn't it. If everyone was "Good" there's be no thermocline.
jessaustin 5y ago
Under "further reading", TFA links a more complete BBC piece [0], which documents that the sub-masters got screwed twice: In December 2019, at the end of a long-running series of civil cases, the Post Office agreed to settle with 555 claimants. It accepted it had previously "got things wrong in [its] dealings with a number of postmasters", and agreed to pay £58m in damages. The claimants received a share of £12m, after legal fees were paid. Those lawyers abused them worse than the Royal Mail did. [0] https://www.bbc.com/news/business-56718036 https://www.bbc.com/news/business-56718036
ehw3 5y ago
Maybe "aletheiacline?"
glaucon 5y ago
Although this piece refers to Post Office management a lot I seem to remember that one or more judges have laid blame at Fujitsu's door as well. In my mind the behaviour of the Post Office was very, very poor but there are people within Fujitsu who had, and have, a much greater understanding of the modes of failure that might have resulted in the discrepancies that were being seen and yet, as I understand it, they went along with Post Office as they trashed people's reputations, livelihoods and threatened their freedom.
jacques_chester 5y ago
> The danger of the thermocline of truth is that, if you’re above it, it’s difficult to know it. Is the rosy picture of reality you’re hearing from subordinates genuine? Or are you heading for an iceberg that everyone can see but you? There's a simple heuristic: Are you only hearing good news? That's bad news.
seastonATccs 5y ago
Last time I gave an honest assessment of how far behind I was and why I got put on an "improvement plan". I've never been as exact since. I've been as behind as before but I don't call it out.
mleonhard 5y ago
How can one eliminate a thermocline of truth from an organization? Ideas: - Have every exec spend one day a week with leaf-node workers, just learning and asking questions. - Rotate all middle managers regularly. When a project has problems, blame is shared by several managers, making it easier for the current manager to admit failure. When a manager rotates off a project, they lose their influence and ability to under-report problems. The new manager is incentivized to look closely for problems and over-report them. - Do not allow middle managers to determine promotions and bonuses of managers and leaf-node workers. Have the execs do it, or hire someone specifically to do it. Without this influence, middle managers lose some ability to suppress bad news. - Split your company into several smaller companies and eliminate all middle managers. This limits the size of the organization to about 1,000 people: CEO x 10 VPs x 10 managers x 10 leaf-node workers. - Have execs and board members interview exiting leaf-node workers. - Hire an outside company to provide an anonymous whistle-blowing service with big cash rewards for being the first to report false progress reports and covered-up problems. Does a company like this exist?

57 points by ivank 5y ago

These "invisible disasters" are an almost daily occurrence in software, even within a single sector or geography. Multi-hundreds million dollar projects that are either severely malformed when launched or that collapse entirely before being placed in the hands of a user. I wrote a little about them in a previous post. Have things improved? Silly question. https://news.ycombinator.com/item?id=17844712 https://news.ycombinator.com/item?id=17844712

skmurphy 5y ago

References and extends this post, may be better to start with it: https://brucefwebster.com/2008/04/15/the-wetware-crisis-the-themocline-of-truth/ https://brucefwebster.com/2008/04/15/the-wetware-crisis-the-...

ebcode 5y ago
And if you follow the link at the bottom of that post, you'll find what the thermocline consists of: >There was upper Apple management, basking in the sun; the great mass of Apple employees below; and between the two, a constantly shifting layer of middle managers whose function appeared to be to keep the two groups from mixing.[0] [0] http://andstillipersist.com/2006/08/guerrilla-programming-life-below-the-thermocline/ http://andstillipersist.com/2006/08/guerrilla-programming-li...

jsnell 5y ago

Original post discussed a couple of weeks ago: https://news.ycombinator.com/item?id=27871024 https://news.ycombinator.com/item?id=27871024

abakker 5y ago

"Paula Vennells, who took over as Chief Executive in 2012 with, by all accounts, the best of intentions, found herself surrounded by a senior management team desperate to protect their own reputations and that of the Royal Mail. They told her that there simply couldn’t be problems with the IT system on the scale that was being suggested; it was much more likely that the sub-postmasters were colluding, lying about IT bugs in order to escape responsibility for what was at best incompetence and at worst theft." Hanlon's razor would have been useful here. How could anyone honestly believe that mass collusion amongst a distributed and isolated group of people, that was also perfectly successful, was more likely than an IT failure?

trhway 5y ago
She had a choice - either go against surrounding her "senior management team desperate to protect their own reputations and that of the Royal Mail" or blame those lowly sub-postmasters. It isn't about "honestly believing", it is what people to side with - your own who is just like you or the very different others. >They told her that there simply couldn’t be problems with the IT system on the scale that was being suggested and thus they provided her with plausible deniability. Like that Enron CEO who supposedly didn't know the financial details and relied on what is told by his CFO/etc. The whole article sounds like a whitewash of senior management responsibility by blaming it on the lower ranks who reports "rosy picture". No wonder that the author seems to be a successful consultant - i can see how his expert opinion may be helpful for such a whitewashing by establishing the existence of the "thermocline of truth" in the specific situations what he gets brought into and thus is liked by the upper management. As an extra example of "thermocline of truth" - that EBay CEO discussed again just yesterday who supposedly didn't know how his security chief with the gang were criminally harassing the blogger to carry out the CEO's order to “take her down”. I think approaches like RICO or SOX or plain old army principle that the CO is responsible for whatever happens under his command are the examples of the tools to pierce the "thermoclines of truth".
- AnimalMuppet 5y ago
  If you're "siding", you aren't "honestly believing". You're choosing what you want to hear and believe, rather than choosing truth.
- thewakalix 5y ago
  The article doesn’t blame the underlings that report a “rosy picture”, but rather blames the culture that heavily incentivizes reporting a rosy picture regardless of the facts on the ground, and punished everything else. And the people with the most power over this culture are the ones at the top, making them the most culpable for the thermocline.
  - trhway 5y ago
    setting aside how top managers weaponize anything into "blame others and excuse themselves", the blame is more clear in the original blog post linked in the article https://brucefwebster.com/2008/04/15/the-wetware-crisis-the-themocline-of-truth/ https://brucefwebster.com/2008/04/15/the-wetware-crisis-the-...

ChrisMarshallNY 5y ago

> They are the inevitable consequence of an organisation that puts more emphasis on reputation than truth; that rewards good news and punishes bad; that has a leadership team disconnected from the delivery of the actual work; that instils fear and compliance with process into its employees, rather than a desire to do the right thing. This is so endemic to all organizations. When you shoot messengers, expect to receive fewer messages.

JumpCrisscross 5y ago
> When you shoot messengers, expect to receive fewer messages Most modern organizations don't have pure messengers. The king is his messenger. That can blur the line between accountability and transparency. For example, in this case, the messengers were the senior managers partly responsible for the mess. Good managers clearly delineate that boundary. Bringing bad news isn't punished. But making bad calls alone may be. In practice, this takes discipline to do.
- fungiblecog 5y ago
  The "Good" descriptor is the problem though isn't it. If everyone was "Good" there's be no thermocline.

jessaustin 5y ago

Under "further reading", TFA links a more complete BBC piece [0], which documents that the sub-masters got screwed twice: In December 2019, at the end of a long-running series of civil cases, the Post Office agreed to settle with 555 claimants. It accepted it had previously "got things wrong in [its] dealings with a number of postmasters", and agreed to pay £58m in damages. The claimants received a share of £12m, after legal fees were paid. Those lawyers abused them worse than the Royal Mail did. [0] https://www.bbc.com/news/business-56718036 https://www.bbc.com/news/business-56718036

ehw3 5y ago

Maybe "aletheiacline?"

glaucon 5y ago

Although this piece refers to Post Office management a lot I seem to remember that one or more judges have laid blame at Fujitsu's door as well. In my mind the behaviour of the Post Office was very, very poor but there are people within Fujitsu who had, and have, a much greater understanding of the modes of failure that might have resulted in the discrepancies that were being seen and yet, as I understand it, they went along with Post Office as they trashed people's reputations, livelihoods and threatened their freedom.

jacques_chester 5y ago

> The danger of the thermocline of truth is that, if you’re above it, it’s difficult to know it. Is the rosy picture of reality you’re hearing from subordinates genuine? Or are you heading for an iceberg that everyone can see but you? There's a simple heuristic: Are you only hearing good news? That's bad news.

seastonATccs 5y ago

Last time I gave an honest assessment of how far behind I was and why I got put on an "improvement plan". I've never been as exact since. I've been as behind as before but I don't call it out.

mleonhard 5y ago

How can one eliminate a thermocline of truth from an organization? Ideas: - Have every exec spend one day a week with leaf-node workers, just learning and asking questions. - Rotate all middle managers regularly. When a project has problems, blame is shared by several managers, making it easier for the current manager to admit failure. When a manager rotates off a project, they lose their influence and ability to under-report problems. The new manager is incentivized to look closely for problems and over-report them. - Do not allow middle managers to determine promotions and bonuses of managers and leaf-node workers. Have the execs do it, or hire someone specifically to do it. Without this influence, middle managers lose some ability to suppress bad news. - Split your company into several smaller companies and eliminate all middle managers. This limits the size of the organization to about 1,000 people: CEO x 10 VPs x 10 managers x 10 leaf-node workers. - Have execs and board members interview exiting leaf-node workers. - Hire an outside company to provide an anonymous whistle-blowing service with big cash rewards for being the first to report false progress reports and covered-up problems. Does a company like this exist?