14 ms·

How We Institutionalized Incompetence

268 points by scottlocklin 6y ago

gipp 6y ago
There might be some interesting observations in here, but this really feels like a grab-bag of complaints, many tangentially-related at best. The author doesn't really do any work to synthesize them to support his central claim.
- 082349872349872 6y ago
  The https://en.wikipedia.org/wiki/Peter_principle https://en.wikipedia.org/wiki/Peter_principle (1969) provides a "never ascribe to malice" explanation. Another explanation: those who hyperoptimise for efficiency get what they pay for.
  - grasshopperpurp 6y ago
    Unfortunately, it's the slavers who HO for efficiency and the people who suffer
  - wwright 6y ago
    "Never ascribe to malice" itself is specifically Hanlon's razor: https://en.wikipedia.org/wiki/Hanlon%27s_razor https://en.wikipedia.org/wiki/Hanlon%27s_razor
- m52go 6y ago
  "What we've institutionalized is run to failure: we'll just keep doing more of what's failed spectacularly until the entire status quo collapses in a fetid heap of greed, self-interest and gross incompetence." This is buried toward the end of the article but I think it's a big statement with massive consequences for many practical issues.
  - Larrikin 6y ago
    He doesn't tie it to the rather tired complaint from most business leaders that regulation = bad. It was kind of an annoying reveal of his central point since it seemed like the lead up might actually introduce a new idea or atleast a new take on an old one
    - dfilppi 6y ago
      Alas the regulators, as arms of the government, represent the epitome of greed, self interest and mind boggling incompetence outstripping anything dreamt of by private industry.
- eternalban 6y ago
  There is an about page: https://www.oftwominds.com/blog.html https://www.oftwominds.com/blog.html
NikolaNovak 6y ago
Some interesting observations and certainly it meshes with my own corporate/public-sector experience in some aspects; but it ends as a list of complaints and an urge to buy a book. As well, some key words such as the repeated "Feeding through" really push it into realm of rants to elicit a predictable response or wink to the knowing/in-crowd, rather than an attempt to start a conversation or offer a solution :-/ I would also argue that in America specifically there ARE numerous counter-examples, and I'd vouch to say that HN is probably a statistically significant outlier of such. A lot of startups we discuss or celebrate (and even lots of startups that we bash;) work on very different principles and incentives than those listed here.
- samsquire 6y ago
  Think of him as the early warning siren that some thing is wrong. He didn't propose a solution, that comes from discussion. Complaining about a lack of solution is just complaining about what you got for free.
bearjaws 6y ago
I am not sure I 100% agree that compliance is a outcome of incompetence. However I do agree that pretty much every company will hold onto an incompetent person until the money runs out. In my world, compliance is about offsetting risks, sure there are plenty of risks involving incompetence, but that is only one of the issues you need to address. Putting the spotlight on incompetence is going to cause you to miss other significant issues in your organization. Ultimately, lists and checking boxes are a long term solution to the classic "How do I do xyz?" "I dunno ask Bob he usually does that".
mwigdahl 6y ago
What he's getting at here is the same thing Scott Alexander wrote about in _Meditations on Moloch_ (https://slatestarcodex.com/2014/07/30/meditations-on-moloch/ https://slatestarcodex.com/2014/07/30/meditations-on-moloch/). Scott's article goes into it a lot deeper and with more rigor, though.
- petermcneeley 6y ago
  Moloch is ferociously competent! This is Moloch https://www.youtube.com/watch?v=aDaOgu2CQtI https://www.youtube.com/watch?v=aDaOgu2CQtI
  - tartoran 6y ago
    This is also a depiction of Moloch: https://upload.wikimedia.org/wikipedia/commons/d/dd/Rubens_saturn.jpg https://upload.wikimedia.org/wikipedia/commons/d/dd/Rubens_s...
irln 6y ago
His reference to David Graeber led me to 5 Types of BS Jobs https://youtu.be/utdDB10usZg https://youtu.be/utdDB10usZg which seems spectacularly on point.
- amirkdv 6y ago
  I could not believe my eyes when I first saw > the late David Graeber He died on Sept 2 2020, aged 59, unexpectedly while vacationing. RIP
extropy 6y ago
I think this is classic case of things need to get worse to get better. Things have been good enough for long enough that the incentives for running smoothly and efficiently have eroded away. We are closing in to the bottom where the a rather radical solution has obvious and major benefits that it will finally be taken. And then the change will be swift. Note that in previous iterations the solution has often been a large scale military conflict. Edit: spelling.
- zepto 6y ago
  What rather radical solution are you imagining?
Out_of_Characte 6y ago
Failure is a single outcome with many reasons leading up to it. Incompetence being one of them. I have no doubt that the claim could be true, but when a bailout happens you cannot assert that it must have been incompetence. Good startups that start with the best intentions can still be destroyed by many things that are not monopolised greed or incompetence.
- gwd 6y ago
  > Failure is a single outcome with many reasons leading up to it. Jared Diamond called this the "Anna Karenina Principle", from the opening line of Tolstoy's novel of that same name: All happy families are alike; each unhappy family is unhappy in its own way. The idea being that a number of things all need to be correct to be successful; if any one of those things is missing, things will fail. [1] https://en.wikipedia.org/wiki/Anna_Karenina_principle https://en.wikipedia.org/wiki/Anna_Karenina_principle
dfilppi 6y ago
Such is the nature of things. All systems develop vested interests which defend the status quo. They do it until it collapses. Rinse and repeat.
- dane-pgp 6y ago
  That sounds like the Iron Law of Bureaucracy (or a corollary to it). https://www.jerrypournelle.com/reports/jerryp/iron.html https://www.jerrypournelle.com/reports/jerryp/iron.html
ThenAsNow 6y ago
The saddest conclusion that occurs to me (not really addressed in his article) is that mechanisms for accountability aren't working very well, if they exist at all. A corollary is that accountability mechanisms are really gross in nature, so any observer or involved person might ascribe something like a demand for leadership change to causes other than the real underlying incompetence.
DubiousPusher 6y ago
Most of the gripes in the article are things that "feel" true but no supporting data is presented. Do public infrastructures and services cost more in inflation adjusted dollars and achieve less? That's a pretty significant claim which needs serious evidence. The contention that we are in some uniquely incompetent time is also dubious. American history and all history for that matter is loaded with stories of the rise and fall of fakes, cranks and cons. Daniel Sickles, The origin of the Chase Manhattan company and John Brinkley just off the top of my head. https://en.m.wikipedia.org/wiki/Daniel_Sickles https://en.m.wikipedia.org/wiki/Daniel_Sickles https://en.m.wikipedia.org/wiki/JPMorgan_Chase https://en.m.wikipedia.org/wiki/JPMorgan_Chase https://en.m.wikipedia.org/wiki/John_R._Brinkley https://en.m.wikipedia.org/wiki/John_R._Brinkley
- Hemospectrum 6y ago
  > Do public infrastructures and services cost more in inflation adjusted dollars and achieve less? That's a pretty significant claim which needs serious evidence. Yeah, it would have been nicer if claims like this within the article had links to supporting discussions of some kind. On this topic, at least, I've seen numerous discussions on HN, so I googled a little bit and found some articles that look familiar: https://fee.org/articles/the-unbearable-truth-about-infrastructure-and-urban-sprawl/ https://fee.org/articles/the-unbearable-truth-about-infrastr... https://news.ycombinator.com/item?id=22122460 https://news.ycombinator.com/item?id=22122460 EDIT: Replaced the second link with an HN discussion about it from earlier this year. Couldn't find one for the other article.
andrewmutz 6y ago
> You've probably noticed things no longer work as well as they once did. The author assumes these statements are obvious to the reader, but they are definitely not obvious to me. Are things getting worse? The examples he cites, with the possible exception of infrastructure spending, seem to me about the way they have always been.
- seneca 6y ago
  It may have something to do with age. I would wager I'm older than the average HN reader, and the steady decline of competence and quality has been pretty obvious in my lifetime. I've had discussions with friends in my age group about this exact topic and no one seemed to disagree. This is all susceptible to all sorts of cognitive tricks though. We may simply be old curmudgeons.
  - tonyedgecombe 6y ago
    If you work in technology then this shouldn't be that surprising. We have gone from it being a small niche activity to becoming mainstream. When I started everybody I worked with was enthusiastic about it. Now for many it's just a job. Outside of that though I'm not so sure. I wouldn't like to go back to the seventies.
  - andrewmutz 6y ago
    That's interesting. I am 41, and I assumed that I was older and that being older meant you had witnessed things always being mediocre
  - deleted 6y ago
    [deleted]
- jpttsn 6y ago
  I concur and I'd bet it's like saying "you've probably noticed kids these days are no good". It's something everyone in time always experiences.
- achillesheels 6y ago
  Yes things have. For instance, potato chip bags have shrunk since the 2008 financial crisis while pricing has increased. Mass market textiles are also no longer durable like they were even twenty years ago (has anyone purchased Marc Jacobs apparel recently?). I was just reflecting on The Organization Man last night and received from the get-go a nice little gem from a man giving a Yale commencement address in 1908: "Here merit is the sole test. Birth is nothing...Here only a natural order of nobility is recognized, and its motto, without a coat of arms or boat of heraldry, is 'Intelligence and integrity'." (p16 2002 UofPenn Press) And then the author, Walter H. Whyte, remarks: "But the very industrial revolution which this highly serviceable ethic begot in time began to confound it...by the 1880's the corporation had already shown the eventual bureaucratic direction it was going to take...the big organization became a standing taunt to this dream of individual success. Quite obvious to anyone who worked in a big organization, those who survived best were not necessarily the fittest but, in more cases than not, those who by birth and personal connections had the breaks." (pp16-17 ibid) The argument being, America escaped the moral inefficacies of the old world, yet it slowly crept back in once it became more profitable - and above all convenient - to be ambitious on a career ladder than ambitious before the nakedness of the sun and stars. Which means it is common to have to lie and conspire in order to get ahead - and how many people are willing to for a nicer car to drive around the neighborhood! The Visible Hand by Alfred D. Chandler, Jr. is a great read corroborating the growth of agglomerated industry, and Joseph Schumpeter's Capitalism, Socialism, and Democracy also has some good passages relating to this phenomenon of the homestead getting saturated. But I'm an eternal optimist and see this as the price of progress :)
keiferski 6y ago
Much of this modern malaise, I think, is traceable to two related phenomenon: 1. The lack of a real "competitor" in a geopolitical sense for the West. I don't think it is a coincidence that the US arguably had the most competence and scientific development circa ±1940 - 1990, when it had a direct competitor to measure itself against. Incompetence against the Nazis or Soviets got you killed, whereas today, nothing really happens (immediately.) The ancient Greeks (and subsequently Nietzsche) recognized the usefulness of competition toward keeping yourself sharp and called it agon. Ironically the rise of China can be seen as a good thing from this perspective. https://en.wikipedia.org/wiki/Agon https://en.wikipedia.org/wiki/Agon http://www.nietzschecircle.com/v3/2017/03/17/book-review-agon-in-nietzsche/ http://www.nietzschecircle.com/v3/2017/03/17/book-review-ago... 2. The loss of a civilization-wide teleological narrative. For the past ±1,500 years, that's been Christianity, in various forms. These days, there is no real "purpose" to the West, beyond some vague notions of individualism, choice and comfort. Institutions and standards become more and more difficult to maintain as a sense of apathy and the lack of a perceived future grows. Younger people especially seem unconvinced that there is anything waiting for them in 50 years.
- nicoburns 6y ago
  Regarding 2: I would love to see some of the community aspects of christianity/religion brought back in a secular form. It seems like believing in God isn't really the key part. It's interacting with your neighbours, which seems to be something really missing from a lot of the western world over the last few decades.
  - keiferski 6y ago
    For sure, and I think that's a widespread political problem too. I wonder if online hostility and polarization would be reduced if people simply had calm in-person discussions with each other once a week. Social media is optimized for outrage and offensiveness.
    - bilbo0s 6y ago
      "I wonder if online hostility and polarization would be reduced if people simply had calm in-person discussions with each other once a week" That might work for some people? I don't know? But I do know that the Charleston church shooter sat down with a bunch of little old ladies for the duration of a bible study engaging in, what I can only assume was, a calm in-person discussion. Then he blew their heads off. Point is that, for some reason, the internet has a much stronger hold over people than do in person connections. People deeply believe they are more connected to people in their virtual spaces than they are to their neighbors or even physical friends. That sense of connection is real to those people. To some of those people, it's real enough to kill for.
kevincox 6y ago
> You've probably noticed massive cost overruns in public projects. That $1 billion bridge is now $3 billion--oh, sorry, make that $4 billion. If we ever get it finished, better estimate $5 billion. This rings home as I was recently talking to someone at an engineering firm that often bids for government contracts. And they were explaining that you build your timeline back from the deadline in the proposal even if you know it isn't feasible. A lot of companies will then bid under-cost and hope that there was a flaw in the design that requires adjustments. Then since they are already on the site with a bunch of workers they will charge extortionary process for the modification to make money on the whole project. So that $10M dollar bid jumps to $15M because they needed to reroute a gas line before starting the building (not a real example, but representative) Of course a lot of the problem is that the city has strong processes and policies to avoid corruption, however the end result is that it is very difficult to hire the people you know do good work, instead you are stuck with the lowest bidder. Of course there is a process for banning companies however it is long, complicated and very bureaucratic, so even if you could get the ban in place it is easier for any project manager not to bother. I guess it all comes down to lack of trust, and this is why we can't have nice things.
- Pfhreak 6y ago
  Here in Seattle, we've built out a light rail system under budget and ahead of schedule. And of course, we've also really flubbed the cost/schedule for a highway tunnel. I'm not sure that the result is axiomatically over budget for major projects. That said, I'm less worried about budget than I am worried about who benefits (e.g. I'm generally ok with transit going over because it provides such a valuable benefit for the city.)
  - kevincox 6y ago
    I guess my point that at least for the city I was talking about the bids were rarely a good estimate of the cost and time required. I'm not saying that they never make it for around that cost, but it is rare. The bids more reflected some complicated mating ritual to get their food in the door, then they deliver the bad news that the actual cost is higher the first chance they get.
jbb67 6y ago
I don't agree with most of these. As an older British person, I say try going back to 1970's Britain and see if you think things worked better then. They didn't. There is an awful lot that could be improved for sure, but overall things are going pretty well in my opinion.
- marcus_holmes 6y ago
  same. The scary stuff that used to happen all the time. We tend to forget. It's so much better now.
deleted 6y ago
[deleted]
efitz 6y ago
I agree with many (bot not all) of the points in the article. However, the article is not well reasoned and it’s not clear how many of the conclusions about cause follow from the description of the problems.
rs999gti 6y ago
> As a result of the network effect, quasi-monopolies abound in Big Tech. Yeah you're free to make an alternative, you might succeed, but if you get too big then the FAANG's will sue you right out of business. Oh I forgot, or the FAANG's PR team will use the media to accuse you of being a home for trolls, hate, etc.
marcinzm 6y ago
You can just as easily argue that services cost more because we expect more from them. You now need to make sure those with disabilities are covered, that the environment isn't damaged, that workers who build them aren't abused or die, that waste is disposed of into something other than the local river, that the thing looks aesthetically pleasing, that a bidding process took place so the governor's cousin doesn't get all contracts, etc, etc. Things seem cheap when externalities aren't included but that doesn't mean they're actually cheap.
- laksdjfkasljdf 6y ago
  nothing you listed isn't in the law since day one. Expect the law to be followed shouldn't count a "high expectations". Sigh.
  - marcinzm 6y ago
    The Clean Air Act was enacted in 1956. Clean Water Act in 1972 (amended in 1977 and 1987). Americans with Disabilities Act in 1990. Occupational Safety and Health Act was in 1970. How can you follow a law that hasn't even been enacted?
guyzero 6y ago
I find it odd that he heads with the least defensible of all these examples: You've probably noticed things no longer work as well as they once did. For example, the store's online inventory says something is in stock and when you get to the store, it's not on the shelf. A small issue, but telling nonetheless. This has never worked! And widespread online inventory systems date back, what, 10 years? This is a weird place to begin an essay about how good the good old days were.
- s17n 6y ago
  The point he's trying to make is that things used to be more reliable - there were no online inventory systems in the 50's, but whatever systems they did have back then were dependable. He is, of course, totally wrong - back then they couldn't even make a reliable car, much less a reliable global supply chain management system. Computers are orders of magnitude more complicated than anything mankind had built as of 1970 so really it's more amazing that they work at all than that there are bugs. The real difference may be one of intelligibility - in the 50's your car wasn't very reliable, but when it did break, you could understand and quite possibly fix what had gone wrong. Now, the world is way too complicated and nobody can possibly understand more than a small fraction of how anything works.
  - bluGill 6y ago
    Of course you understood and could fix your 1950s car. Anyone who couldn't didn't dare drive. Thus you had incentive to learn. Cars today still have the same carnot cycle engines, with mostly the same parts. The controls are enough better that nobody cares about the air fuel mixture
    - germinalphrase 6y ago
      And many car issues can still be fixed if you’re willing to learn. Maybe this is less true the further up you go on the performance scale, but in the ten years I’ve owned my Honda, I haven’t had a single problem I couldn’t fix.
deleted 6y ago
[deleted]
vikramkr 6y ago
Have I noticed all these things? You can cherrypick examples to fit whatever narrative you want. Our space program is finally running at a more reasonable cost thanks to some private competition. What services are costing more now outside of healthcare? Most things are getting commoditized/cheaper. What enormous investments in infrastructure education etc? The problem right now is a lack of investment into fixing bridges- not that the money we're pouring in is wasted. And - the new laguardia terminal seems pretty great. Isn't that an example of an infra success? Are all the highly paid academics clueless and delusional? In the past few years we got CRISPR, an explosion in AI, the Higgs boson, gravitational waves, etc etc etc. All from academics. If every decision is made to maximize self interest - what about the near eradication of polio and Guinea worm disease by the gates and Carter foundations? Personally, I'm not that compelled by the "everything sucks because you know it just feels that way" arguments. You can make someone feel whatever way you want by picking your examples properly. Isnt America doing so great now? Spacex and CRISPR and a booming tech sector so on? And this is pretty clearly just meant to get you into the mood the author things will make you most likely to buy his book at the end.
- xxpor 6y ago
  >What services are costing more now outside of healthcare? Infrastructure. Our (USA) costs are COMPLETETLY OUTRAGIOUS compared to the global standard. We're talking multiples. Things like the gateway project in NJ/NYC cost 4x what they should for a crappier outcome.
  - temp667 6y ago
    This - I've looked at a number of projects. There's some kind of tipping point with public / govt infra projects at which point they just go crazy in terms of cost, it's like the make work / interia / rent extractors just tip out the get things done folks at some point. NY / SF, even germany for a bit with Berlin's airport. I look at what they did in the past (huge railroad bridges in 100 days) and even now with private money (Tesla in germany is going up fast and sports stadiums go up like weeds despite their scale) the contrasts are remarkable. Space Launch System is $20B with not a single flight. SpaceX comes along, get's no $1B/year base, get's paid a fraction of the "big players" - and has done 80+ F9 launches. Also - success academies in NY seems to have some pretty amazing results relative to local schools. I realize some confounding factors, but if I was a parent in NY - that would be the choice if I lived in an area they served.
austincheney 6y ago
Incompetence is institutionalized merely by the act of setting competing standards while never defining competence. People will get away with do less for higher pay so long as the system allows. That is not a violation of ethics. It’s an absence of ethics in the first place.
chasing 6y ago
He writes as if any of these things are anything new. I think humanity's actually doing quite well these days. Not perfect, of course, so let's keep working on making things better. But I'd rather be alive now than pretty much any time in the past.
- tartoran 6y ago
  Yes, doing so well that they're in disbelief about the catastrophe that is about to destroy it all, or most of it. Only global warming would start a war of survival never seen in history. Sprinkle that with an insect apocalypse and we're doomed. But I understand what you mean though. Violence used to be the way of life and inseparable from it whereas nowadays some parts of the world are doing relatively much better. But it might be short term victory.
MattGaiser 6y ago
How many of these examples are just perception of a rosier past? The "everything is getting worse" narrative ignores substantial improvements in life expectancy, literacy, health, yes, and even homelessness. Homelessness is down over the past decade [0]. So yes, many of these things actually are getting better. [0] https://en.wikipedia.org/wiki/Homelessness_in_the_United_States#/media/File:US_yearly_timeline_of_people_experiencing_homelessness.gif https://en.wikipedia.org/wiki/Homelessness_in_the_United_Sta...
- deeg 6y ago
  Agreed. His base cause appears to be "self interest" but hasn't that always been case? What's different about today that makes self interest a bigger influence? To reparaphrase a popular quote: those who forget about the past are doomed to complain about the present.
wanderr 6y ago
This is not a new phenomenon. This sounds like a rehashing of Systemantics by John Gall (1975), except attributing the causes to things the author sees as problematic rather than recognizing that these are problems inherent in complex systems. "Systems tend to oppose their own proper functions"
quelsolaar 6y ago
This is so true in software development. Everything in software these days are about; "how do i avoid doing any work?" No one implements anything, they just use what someone else has written. Nobody wants to learn of understand how things really work. Nobody cares how well something is written. We write single threaded applications in interpreted languages that could literally fun 1000x faster on the CPUs we have (add a few zeros of you add the GPU). So often when I talk to programmers they excuse their code with "Its just a toy implementation", "Our customers dont need all the features", "its fast enough", "we can re-implement it later". Rule one if you avoid failure: Aspire to be good. Why are we fetishizing mediocracy? Why are so few people aiming to implement things on the beading edge? Why are we trying to avoid anything that is hard?
- 908B64B197 6y ago
  > We write single threaded applications in interpreted languages that could literally fun 1000x faster on the CPUs we have (add a few zeros of you add the GPU). It's cheaper to buy hardware than acquire talent to write good software. By several orders of magnitudes.
  - quelsolaar 6y ago
    In my experience hiering amazing programmers always pays off. I would rather hire one truly great programmer, over 10 mediocre ones. I get that its hard to find great talent, what i dont get is why people dont aspire to be great, and why people discourage people from being great.
    - 908B64B197 6y ago
      > I get that its hard to find great talent It's not. It just cost a lot.
PaulDavisThe1st 6y ago
> This is how we've devolved to doing more of what's failed spectacularly. This was the red flag line to me. I haven't heard anyone ever propose "let's do more of what's failed". What I do hear is people saying "we got that wrong, and here's the minor tweak we need to fix it". You wrote some code: for (int i = 0; i < len; ++i) copy (src[i], dst[i]); It fails to copy the final element of src to dst. It"fails spectacularly" every time. Eventually you realize that you compute "len" as an inclusive count, not exclusive. You suggest: for (int i = 0; i <= len; ++i) copy (src[i], dst[i]); as a way to make it work. This isn't "let's do more of what's failed spectacularly". This is "let's make what we're trying actually work". Sure, someone could come along and suggest that we use Algol or brainfuck to solve the issue instead, and that's not a priori insane. But a clear minded reading of the situation - we've got a lot of stuff right, but made some mistakes - would favor trying to fix what we've done wrong, not stop doing what we're trying to do. The cynicism of this analysis mostly ruined the rest of the piece for me.
- deleted 6y ago
  [deleted]
lbriner 6y ago
A more interesting narrative is that our tools have become too complicated for most people to understand and use efficiently. The cost per km of the first London Underground lines, which were dug largely by hand from the road downwards and built with a lot of manual labour was about 1/10th of the cost of building a similar tunnel in the 21st century with automated machinary and far fewer "expensive" people. How do you dig by hand? Simple, get a load of people with no training and give them a spade and get on with it. How do you do that with a $100M tunnel boring machine? Send people on expensive training courses, pay a load of cash in specialist repairs and maintenance and when it breaks down or you get any other kind of problem off the beaten track, just stop work until you can sort it out or pay someone else even more money to get you out of your hole. Same with software. How easy is it to keep bank records in a physical ledger? Really easy and very quick. Introduce some software and you now have a multi-million dollar complicated recording machine with loads of semi-skilled operatives using it, breaking it (accidentally), and only very slow and expensive teams can investigate and fix. We have accepted that sometimes we need to ask 10 times to get an address changed because "computer says no". Of course, technology brings you benefits that didn't exist before like centralising data, like making some hard physical processes easier to automate and to backup data that could otherwise be lost in flood or fire but are these worth the complexity and cost? In many cases that's a hard "no"!
cryptica 6y ago
>> Institutions protect insiders because every insider must mask their self-interest and the general failure of the institution. Insiders protect other insiders lest transparency reveal the insiders' skims and scams and the failure of the institution to fulfill its purpose That is exactly what I thought was going on. Most big companies definitely give off that vibe and the longer you work there, the more you get promoted, the more dirt you uncover, the more you get paid, and the least value you create... What are the odds that the author of this article came to the exact same (very specific) conclusion as I did? Clearly it's a matter of time before this is going to blow wide open.
nixpulvis 6y ago
Completely agree, but was hoping for some actual advise. This reads as an introduction to what deserves to be a complex analysis on the past and speculation on the future. Now I'm just left equally as irritated as I was when I started reading.
lordnacho 6y ago
I recognize some of the sentiments, but I think I'd put it slightly differently. We live in a society where a heck of a lot of value is created, but the value accrues to people who don't seem to be the ones creating it. If you look at economics, the "who creates value" question has been neatly sidestepped. If you can get someone to pay you, GDP goes up by that amount. There's not a whole lot of quibbling about whether the value paid was correct, although the GDP accounting system has had a number of changes throughout the years (vast simplification). At present if you just don't do something illegal, the dollar you earned is as good as the dollar anyone else earned. This system has the fault that it doesn't seem to connect with ordinary ideas about who contributed. If whoever ends up with the trophy is the person who deserves the trophy, what is the trophy for? There's a kind of nihilism about what work is for if the purpose of climbing the greasy pole is to reach the top of the greasy pole. Why should I bother to write good code if the thing that really has value is to sit in meetings and talk myself up to people who have no idea what good code actually is? Why should I not simply create a compliance tax for my colleagues, and become the troll who extracts payment? I can start with things most people agree with: good code should have unit tests and documentation. And we can move on and add code reviews, from at least 2 other devs, before any new merge is allowed. And we can't have the same two guys that are your buddies, let's rotate it a bit to make sure there's no corruption. And make sure the rotation system is documented. And signed. By an independent auditor. With an industry accreditation. From an organization that is overseen by government committee. A government committee that confers with other government committees to ensure best practice. All those people need to be paid. And they are valuable, you can't deny that documentation is important. So all these people are creating value and need some of the pie.
cryptica 6y ago
It's the result of supply-side economics exacerbated by flaws in our global monetary system. Supply-side economics works by essentially pumping free money into big corporations and institutions so that they produce as much stuff as they possibly can at the lowest possible price while creating as many jobs as possible without worrying at all about profitability, Thanks to all the free credit that they're getting, these big corporations end up driving all competitors out of business, then some employees of the corporations end up getting involved in internal politics for their own personal gain (zero sum games)... Employees who are value creators realize that their productive work is not rewarded, so they either quit the company or start getting involved in zero-sum internal politics... Once enough people have figured out the kind of game they're playing, the quality of services decline as a result.
grishka 6y ago
Imagine what the world would've been like if all software developers were competent enough to understand the abstractions they're using.
lcuff 6y ago
Frankly, the whole article just seems like a shill for his book pitch at the bottom. The arguments rely on anecdotal examples, ignoring depth and complexity. Using bold type on each paragraph's first lines just adds to my sense of this being yellow journalism.
atlgator 6y ago
With the economic expansion over the last 8 ish years, instead of paying skilled employees more, companies hired more employees at the same lower wage. This led to reduced skill overall as more large businesses adopted this strategy and competed for top talent at low pay.
pc86 6y ago
How has this not gotten flagged? It's total nonsense. The title is "how we institutionalized incompetence" and then is just a list of cherry-picked examples, nonsensical bolding and italicization, and oh yeah a note about the author's book at the end. What a joke.
james_pm 6y ago
The more of these types of posts and books I read, the more depressed I become. Work feels very pointless, especially when I start noticing the "bullshit" jobs around me and just how much of my day is spent tending to tasks created by this bloated waste-of-time bureaucracy. It's gotten to the point where it's almost impossible to do anything that actually benefits our users anymore.
mathattack 6y ago
The claims are bold, but are they really all true? Yes there is a lot of incompetence but do we really want to go back to the world of the 50s? Are cars really that much worse today? Computers? Inflation has been modest since the 70s. Yes we lag in some places but this fellow is over the top.
sbierwagen 6y ago
Baumol's cost disease. https://en.wikipedia.org/wiki/Baumol%27s_cost_disease https://en.wikipedia.org/wiki/Baumol%27s_cost_disease Everything he complains about are services, or labor-heavy products. All prices are relative. As manufactured goods become cheaper, services and labor become more expensive.
noiv 6y ago
I think that's nothing new and happens since humans started civilization. It's just now you can realize this within a lifetime.
lmilcin 6y ago
I know this going to sound shocking but half of the population has below average intelligence (yeah, below mean, but shush:) I think what happens is that everybody needs to have a job and source of income and people try to do the best for themselves. But, of course, they don't get more intelligent. As the technology progresses, more and more jobs are high skill or abstract tasks and both require intelligence. What this means is decreasing availability of intelligent people for the jobs and necessary incompetence of people who perform them (or the positions would not be filled).
- TuringNYC 6y ago
  Its actually worse -- advancement at work is often based on emotional and social intelligence, which is not fully correlated to intelligence of your subject matter.
  - chrisweekly 6y ago
    See also "The Peter Principle" https://en.m.wikipedia.org/wiki/Peter_principle https://en.m.wikipedia.org/wiki/Peter_principle
- rdiddly 6y ago
  median my friend
  - lmilcin 6y ago
    Of course you are right. I don't know what I have been thinking...
wskinner 6y ago
https://patrickcollison.com/fast https://patrickcollison.com/fast is a great catalog of similar issues. It is periodically discussed here.

268 points by scottlocklin 6y ago

gipp 6y ago

There might be some interesting observations in here, but this really feels like a grab-bag of complaints, many tangentially-related at best. The author doesn't really do any work to synthesize them to support his central claim.

082349872349872 6y ago
The https://en.wikipedia.org/wiki/Peter_principle https://en.wikipedia.org/wiki/Peter_principle (1969) provides a "never ascribe to malice" explanation. Another explanation: those who hyperoptimise for efficiency get what they pay for.
- grasshopperpurp 6y ago
  Unfortunately, it's the slavers who HO for efficiency and the people who suffer
- wwright 6y ago
  "Never ascribe to malice" itself is specifically Hanlon's razor: https://en.wikipedia.org/wiki/Hanlon%27s_razor https://en.wikipedia.org/wiki/Hanlon%27s_razor
m52go 6y ago
"What we've institutionalized is run to failure: we'll just keep doing more of what's failed spectacularly until the entire status quo collapses in a fetid heap of greed, self-interest and gross incompetence." This is buried toward the end of the article but I think it's a big statement with massive consequences for many practical issues.
- Larrikin 6y ago
  He doesn't tie it to the rather tired complaint from most business leaders that regulation = bad. It was kind of an annoying reveal of his central point since it seemed like the lead up might actually introduce a new idea or atleast a new take on an old one
  - dfilppi 6y ago
    Alas the regulators, as arms of the government, represent the epitome of greed, self interest and mind boggling incompetence outstripping anything dreamt of by private industry.
eternalban 6y ago
There is an about page: https://www.oftwominds.com/blog.html https://www.oftwominds.com/blog.html

NikolaNovak 6y ago

Some interesting observations and certainly it meshes with my own corporate/public-sector experience in some aspects; but it ends as a list of complaints and an urge to buy a book. As well, some key words such as the repeated "Feeding through" really push it into realm of rants to elicit a predictable response or wink to the knowing/in-crowd, rather than an attempt to start a conversation or offer a solution :-/ I would also argue that in America specifically there ARE numerous counter-examples, and I'd vouch to say that HN is probably a statistically significant outlier of such. A lot of startups we discuss or celebrate (and even lots of startups that we bash;) work on very different principles and incentives than those listed here.

samsquire 6y ago
Think of him as the early warning siren that some thing is wrong. He didn't propose a solution, that comes from discussion. Complaining about a lack of solution is just complaining about what you got for free.

bearjaws 6y ago

I am not sure I 100% agree that compliance is a outcome of incompetence. However I do agree that pretty much every company will hold onto an incompetent person until the money runs out. In my world, compliance is about offsetting risks, sure there are plenty of risks involving incompetence, but that is only one of the issues you need to address. Putting the spotlight on incompetence is going to cause you to miss other significant issues in your organization. Ultimately, lists and checking boxes are a long term solution to the classic "How do I do xyz?" "I dunno ask Bob he usually does that".

mwigdahl 6y ago

What he's getting at here is the same thing Scott Alexander wrote about in _Meditations on Moloch_ (https://slatestarcodex.com/2014/07/30/meditations-on-moloch/ https://slatestarcodex.com/2014/07/30/meditations-on-moloch/). Scott's article goes into it a lot deeper and with more rigor, though.

petermcneeley 6y ago
Moloch is ferociously competent! This is Moloch https://www.youtube.com/watch?v=aDaOgu2CQtI https://www.youtube.com/watch?v=aDaOgu2CQtI
- tartoran 6y ago
  This is also a depiction of Moloch: https://upload.wikimedia.org/wikipedia/commons/d/dd/Rubens_saturn.jpg https://upload.wikimedia.org/wikipedia/commons/d/dd/Rubens_s...

irln 6y ago

His reference to David Graeber led me to 5 Types of BS Jobs https://youtu.be/utdDB10usZg https://youtu.be/utdDB10usZg which seems spectacularly on point.

amirkdv 6y ago
I could not believe my eyes when I first saw > the late David Graeber He died on Sept 2 2020, aged 59, unexpectedly while vacationing. RIP

extropy 6y ago

I think this is classic case of things need to get worse to get better. Things have been good enough for long enough that the incentives for running smoothly and efficiently have eroded away. We are closing in to the bottom where the a rather radical solution has obvious and major benefits that it will finally be taken. And then the change will be swift. Note that in previous iterations the solution has often been a large scale military conflict. Edit: spelling.

zepto 6y ago
What rather radical solution are you imagining?

Out_of_Characte 6y ago

Failure is a single outcome with many reasons leading up to it. Incompetence being one of them. I have no doubt that the claim could be true, but when a bailout happens you cannot assert that it must have been incompetence. Good startups that start with the best intentions can still be destroyed by many things that are not monopolised greed or incompetence.

gwd 6y ago
> Failure is a single outcome with many reasons leading up to it. Jared Diamond called this the "Anna Karenina Principle", from the opening line of Tolstoy's novel of that same name: All happy families are alike; each unhappy family is unhappy in its own way. The idea being that a number of things all need to be correct to be successful; if any one of those things is missing, things will fail. [1] https://en.wikipedia.org/wiki/Anna_Karenina_principle https://en.wikipedia.org/wiki/Anna_Karenina_principle

dfilppi 6y ago

Such is the nature of things. All systems develop vested interests which defend the status quo. They do it until it collapses. Rinse and repeat.

dane-pgp 6y ago
That sounds like the Iron Law of Bureaucracy (or a corollary to it). https://www.jerrypournelle.com/reports/jerryp/iron.html https://www.jerrypournelle.com/reports/jerryp/iron.html

ThenAsNow 6y ago

The saddest conclusion that occurs to me (not really addressed in his article) is that mechanisms for accountability aren't working very well, if they exist at all. A corollary is that accountability mechanisms are really gross in nature, so any observer or involved person might ascribe something like a demand for leadership change to causes other than the real underlying incompetence.

DubiousPusher 6y ago

Most of the gripes in the article are things that "feel" true but no supporting data is presented. Do public infrastructures and services cost more in inflation adjusted dollars and achieve less? That's a pretty significant claim which needs serious evidence. The contention that we are in some uniquely incompetent time is also dubious. American history and all history for that matter is loaded with stories of the rise and fall of fakes, cranks and cons. Daniel Sickles, The origin of the Chase Manhattan company and John Brinkley just off the top of my head. https://en.m.wikipedia.org/wiki/Daniel_Sickles https://en.m.wikipedia.org/wiki/Daniel_Sickles https://en.m.wikipedia.org/wiki/JPMorgan_Chase https://en.m.wikipedia.org/wiki/JPMorgan_Chase https://en.m.wikipedia.org/wiki/John_R._Brinkley https://en.m.wikipedia.org/wiki/John_R._Brinkley

Hemospectrum 6y ago
> Do public infrastructures and services cost more in inflation adjusted dollars and achieve less? That's a pretty significant claim which needs serious evidence. Yeah, it would have been nicer if claims like this within the article had links to supporting discussions of some kind. On this topic, at least, I've seen numerous discussions on HN, so I googled a little bit and found some articles that look familiar: https://fee.org/articles/the-unbearable-truth-about-infrastructure-and-urban-sprawl/ https://fee.org/articles/the-unbearable-truth-about-infrastr... https://news.ycombinator.com/item?id=22122460 https://news.ycombinator.com/item?id=22122460 EDIT: Replaced the second link with an HN discussion about it from earlier this year. Couldn't find one for the other article.

andrewmutz 6y ago

> You've probably noticed things no longer work as well as they once did. The author assumes these statements are obvious to the reader, but they are definitely not obvious to me. Are things getting worse? The examples he cites, with the possible exception of infrastructure spending, seem to me about the way they have always been.

seneca 6y ago
It may have something to do with age. I would wager I'm older than the average HN reader, and the steady decline of competence and quality has been pretty obvious in my lifetime. I've had discussions with friends in my age group about this exact topic and no one seemed to disagree. This is all susceptible to all sorts of cognitive tricks though. We may simply be old curmudgeons.
- tonyedgecombe 6y ago
  If you work in technology then this shouldn't be that surprising. We have gone from it being a small niche activity to becoming mainstream. When I started everybody I worked with was enthusiastic about it. Now for many it's just a job. Outside of that though I'm not so sure. I wouldn't like to go back to the seventies.
- andrewmutz 6y ago
  That's interesting. I am 41, and I assumed that I was older and that being older meant you had witnessed things always being mediocre
- deleted 6y ago
  [deleted]
jpttsn 6y ago
I concur and I'd bet it's like saying "you've probably noticed kids these days are no good". It's something everyone in time always experiences.
achillesheels 6y ago
Yes things have. For instance, potato chip bags have shrunk since the 2008 financial crisis while pricing has increased. Mass market textiles are also no longer durable like they were even twenty years ago (has anyone purchased Marc Jacobs apparel recently?). I was just reflecting on The Organization Man last night and received from the get-go a nice little gem from a man giving a Yale commencement address in 1908: "Here merit is the sole test. Birth is nothing...Here only a natural order of nobility is recognized, and its motto, without a coat of arms or boat of heraldry, is 'Intelligence and integrity'." (p16 2002 UofPenn Press) And then the author, Walter H. Whyte, remarks: "But the very industrial revolution which this highly serviceable ethic begot in time began to confound it...by the 1880's the corporation had already shown the eventual bureaucratic direction it was going to take...the big organization became a standing taunt to this dream of individual success. Quite obvious to anyone who worked in a big organization, those who survived best were not necessarily the fittest but, in more cases than not, those who by birth and personal connections had the breaks." (pp16-17 ibid) The argument being, America escaped the moral inefficacies of the old world, yet it slowly crept back in once it became more profitable - and above all convenient - to be ambitious on a career ladder than ambitious before the nakedness of the sun and stars. Which means it is common to have to lie and conspire in order to get ahead - and how many people are willing to for a nicer car to drive around the neighborhood! The Visible Hand by Alfred D. Chandler, Jr. is a great read corroborating the growth of agglomerated industry, and Joseph Schumpeter's Capitalism, Socialism, and Democracy also has some good passages relating to this phenomenon of the homestead getting saturated. But I'm an eternal optimist and see this as the price of progress :)

keiferski 6y ago

Much of this modern malaise, I think, is traceable to two related phenomenon: 1. The lack of a real "competitor" in a geopolitical sense for the West. I don't think it is a coincidence that the US arguably had the most competence and scientific development circa ±1940 - 1990, when it had a direct competitor to measure itself against. Incompetence against the Nazis or Soviets got you killed, whereas today, nothing really happens (immediately.) The ancient Greeks (and subsequently Nietzsche) recognized the usefulness of competition toward keeping yourself sharp and called it agon. Ironically the rise of China can be seen as a good thing from this perspective. https://en.wikipedia.org/wiki/Agon https://en.wikipedia.org/wiki/Agon http://www.nietzschecircle.com/v3/2017/03/17/book-review-agon-in-nietzsche/ http://www.nietzschecircle.com/v3/2017/03/17/book-review-ago... 2. The loss of a civilization-wide teleological narrative. For the past ±1,500 years, that's been Christianity, in various forms. These days, there is no real "purpose" to the West, beyond some vague notions of individualism, choice and comfort. Institutions and standards become more and more difficult to maintain as a sense of apathy and the lack of a perceived future grows. Younger people especially seem unconvinced that there is anything waiting for them in 50 years.

nicoburns 6y ago
Regarding 2: I would love to see some of the community aspects of christianity/religion brought back in a secular form. It seems like believing in God isn't really the key part. It's interacting with your neighbours, which seems to be something really missing from a lot of the western world over the last few decades.
- keiferski 6y ago
  For sure, and I think that's a widespread political problem too. I wonder if online hostility and polarization would be reduced if people simply had calm in-person discussions with each other once a week. Social media is optimized for outrage and offensiveness.
  - bilbo0s 6y ago
    "I wonder if online hostility and polarization would be reduced if people simply had calm in-person discussions with each other once a week" That might work for some people? I don't know? But I do know that the Charleston church shooter sat down with a bunch of little old ladies for the duration of a bible study engaging in, what I can only assume was, a calm in-person discussion. Then he blew their heads off. Point is that, for some reason, the internet has a much stronger hold over people than do in person connections. People deeply believe they are more connected to people in their virtual spaces than they are to their neighbors or even physical friends. That sense of connection is real to those people. To some of those people, it's real enough to kill for.

kevincox 6y ago

> You've probably noticed massive cost overruns in public projects. That $1 billion bridge is now $3 billion--oh, sorry, make that $4 billion. If we ever get it finished, better estimate $5 billion. This rings home as I was recently talking to someone at an engineering firm that often bids for government contracts. And they were explaining that you build your timeline back from the deadline in the proposal even if you know it isn't feasible. A lot of companies will then bid under-cost and hope that there was a flaw in the design that requires adjustments. Then since they are already on the site with a bunch of workers they will charge extortionary process for the modification to make money on the whole project. So that $10M dollar bid jumps to $15M because they needed to reroute a gas line before starting the building (not a real example, but representative) Of course a lot of the problem is that the city has strong processes and policies to avoid corruption, however the end result is that it is very difficult to hire the people you know do good work, instead you are stuck with the lowest bidder. Of course there is a process for banning companies however it is long, complicated and very bureaucratic, so even if you could get the ban in place it is easier for any project manager not to bother. I guess it all comes down to lack of trust, and this is why we can't have nice things.

Pfhreak 6y ago
Here in Seattle, we've built out a light rail system under budget and ahead of schedule. And of course, we've also really flubbed the cost/schedule for a highway tunnel. I'm not sure that the result is axiomatically over budget for major projects. That said, I'm less worried about budget than I am worried about who benefits (e.g. I'm generally ok with transit going over because it provides such a valuable benefit for the city.)
- kevincox 6y ago
  I guess my point that at least for the city I was talking about the bids were rarely a good estimate of the cost and time required. I'm not saying that they never make it for around that cost, but it is rare. The bids more reflected some complicated mating ritual to get their food in the door, then they deliver the bad news that the actual cost is higher the first chance they get.

jbb67 6y ago

I don't agree with most of these. As an older British person, I say try going back to 1970's Britain and see if you think things worked better then. They didn't. There is an awful lot that could be improved for sure, but overall things are going pretty well in my opinion.

marcus_holmes 6y ago
same. The scary stuff that used to happen all the time. We tend to forget. It's so much better now.

deleted 6y ago

[deleted]

efitz 6y ago

I agree with many (bot not all) of the points in the article. However, the article is not well reasoned and it’s not clear how many of the conclusions about cause follow from the description of the problems.

rs999gti 6y ago

> As a result of the network effect, quasi-monopolies abound in Big Tech. Yeah you're free to make an alternative, you might succeed, but if you get too big then the FAANG's will sue you right out of business. Oh I forgot, or the FAANG's PR team will use the media to accuse you of being a home for trolls, hate, etc.

marcinzm 6y ago

You can just as easily argue that services cost more because we expect more from them. You now need to make sure those with disabilities are covered, that the environment isn't damaged, that workers who build them aren't abused or die, that waste is disposed of into something other than the local river, that the thing looks aesthetically pleasing, that a bidding process took place so the governor's cousin doesn't get all contracts, etc, etc. Things seem cheap when externalities aren't included but that doesn't mean they're actually cheap.

laksdjfkasljdf 6y ago
nothing you listed isn't in the law since day one. Expect the law to be followed shouldn't count a "high expectations". Sigh.
- marcinzm 6y ago
  The Clean Air Act was enacted in 1956. Clean Water Act in 1972 (amended in 1977 and 1987). Americans with Disabilities Act in 1990. Occupational Safety and Health Act was in 1970. How can you follow a law that hasn't even been enacted?

guyzero 6y ago

I find it odd that he heads with the least defensible of all these examples: You've probably noticed things no longer work as well as they once did. For example, the store's online inventory says something is in stock and when you get to the store, it's not on the shelf. A small issue, but telling nonetheless. This has never worked! And widespread online inventory systems date back, what, 10 years? This is a weird place to begin an essay about how good the good old days were.

s17n 6y ago
The point he's trying to make is that things used to be more reliable - there were no online inventory systems in the 50's, but whatever systems they did have back then were dependable. He is, of course, totally wrong - back then they couldn't even make a reliable car, much less a reliable global supply chain management system. Computers are orders of magnitude more complicated than anything mankind had built as of 1970 so really it's more amazing that they work at all than that there are bugs. The real difference may be one of intelligibility - in the 50's your car wasn't very reliable, but when it did break, you could understand and quite possibly fix what had gone wrong. Now, the world is way too complicated and nobody can possibly understand more than a small fraction of how anything works.
- bluGill 6y ago
  Of course you understood and could fix your 1950s car. Anyone who couldn't didn't dare drive. Thus you had incentive to learn. Cars today still have the same carnot cycle engines, with mostly the same parts. The controls are enough better that nobody cares about the air fuel mixture
  - germinalphrase 6y ago
    And many car issues can still be fixed if you’re willing to learn. Maybe this is less true the further up you go on the performance scale, but in the ten years I’ve owned my Honda, I haven’t had a single problem I couldn’t fix.

deleted 6y ago

[deleted]

vikramkr 6y ago

Have I noticed all these things? You can cherrypick examples to fit whatever narrative you want. Our space program is finally running at a more reasonable cost thanks to some private competition. What services are costing more now outside of healthcare? Most things are getting commoditized/cheaper. What enormous investments in infrastructure education etc? The problem right now is a lack of investment into fixing bridges- not that the money we're pouring in is wasted. And - the new laguardia terminal seems pretty great. Isn't that an example of an infra success? Are all the highly paid academics clueless and delusional? In the past few years we got CRISPR, an explosion in AI, the Higgs boson, gravitational waves, etc etc etc. All from academics. If every decision is made to maximize self interest - what about the near eradication of polio and Guinea worm disease by the gates and Carter foundations? Personally, I'm not that compelled by the "everything sucks because you know it just feels that way" arguments. You can make someone feel whatever way you want by picking your examples properly. Isnt America doing so great now? Spacex and CRISPR and a booming tech sector so on? And this is pretty clearly just meant to get you into the mood the author things will make you most likely to buy his book at the end.

xxpor 6y ago
>What services are costing more now outside of healthcare? Infrastructure. Our (USA) costs are COMPLETETLY OUTRAGIOUS compared to the global standard. We're talking multiples. Things like the gateway project in NJ/NYC cost 4x what they should for a crappier outcome.
- temp667 6y ago
  This - I've looked at a number of projects. There's some kind of tipping point with public / govt infra projects at which point they just go crazy in terms of cost, it's like the make work / interia / rent extractors just tip out the get things done folks at some point. NY / SF, even germany for a bit with Berlin's airport. I look at what they did in the past (huge railroad bridges in 100 days) and even now with private money (Tesla in germany is going up fast and sports stadiums go up like weeds despite their scale) the contrasts are remarkable. Space Launch System is $20B with not a single flight. SpaceX comes along, get's no $1B/year base, get's paid a fraction of the "big players" - and has done 80+ F9 launches. Also - success academies in NY seems to have some pretty amazing results relative to local schools. I realize some confounding factors, but if I was a parent in NY - that would be the choice if I lived in an area they served.

austincheney 6y ago

Incompetence is institutionalized merely by the act of setting competing standards while never defining competence. People will get away with do less for higher pay so long as the system allows. That is not a violation of ethics. It’s an absence of ethics in the first place.

chasing 6y ago

He writes as if any of these things are anything new. I think humanity's actually doing quite well these days. Not perfect, of course, so let's keep working on making things better. But I'd rather be alive now than pretty much any time in the past.

tartoran 6y ago
Yes, doing so well that they're in disbelief about the catastrophe that is about to destroy it all, or most of it. Only global warming would start a war of survival never seen in history. Sprinkle that with an insect apocalypse and we're doomed. But I understand what you mean though. Violence used to be the way of life and inseparable from it whereas nowadays some parts of the world are doing relatively much better. But it might be short term victory.

MattGaiser 6y ago

How many of these examples are just perception of a rosier past? The "everything is getting worse" narrative ignores substantial improvements in life expectancy, literacy, health, yes, and even homelessness. Homelessness is down over the past decade [0]. So yes, many of these things actually are getting better. [0] https://en.wikipedia.org/wiki/Homelessness_in_the_United_States#/media/File:US_yearly_timeline_of_people_experiencing_homelessness.gif https://en.wikipedia.org/wiki/Homelessness_in_the_United_Sta...

deeg 6y ago
Agreed. His base cause appears to be "self interest" but hasn't that always been case? What's different about today that makes self interest a bigger influence? To reparaphrase a popular quote: those who forget about the past are doomed to complain about the present.

wanderr 6y ago

This is not a new phenomenon. This sounds like a rehashing of Systemantics by John Gall (1975), except attributing the causes to things the author sees as problematic rather than recognizing that these are problems inherent in complex systems. "Systems tend to oppose their own proper functions"

quelsolaar 6y ago

This is so true in software development. Everything in software these days are about; "how do i avoid doing any work?" No one implements anything, they just use what someone else has written. Nobody wants to learn of understand how things really work. Nobody cares how well something is written. We write single threaded applications in interpreted languages that could literally fun 1000x faster on the CPUs we have (add a few zeros of you add the GPU). So often when I talk to programmers they excuse their code with "Its just a toy implementation", "Our customers dont need all the features", "its fast enough", "we can re-implement it later". Rule one if you avoid failure: Aspire to be good. Why are we fetishizing mediocracy? Why are so few people aiming to implement things on the beading edge? Why are we trying to avoid anything that is hard?

908B64B197 6y ago
> We write single threaded applications in interpreted languages that could literally fun 1000x faster on the CPUs we have (add a few zeros of you add the GPU). It's cheaper to buy hardware than acquire talent to write good software. By several orders of magnitudes.
- quelsolaar 6y ago
  In my experience hiering amazing programmers always pays off. I would rather hire one truly great programmer, over 10 mediocre ones. I get that its hard to find great talent, what i dont get is why people dont aspire to be great, and why people discourage people from being great.
  - 908B64B197 6y ago
    > I get that its hard to find great talent It's not. It just cost a lot.

PaulDavisThe1st 6y ago

> This is how we've devolved to doing more of what's failed spectacularly. This was the red flag line to me. I haven't heard anyone ever propose "let's do more of what's failed". What I do hear is people saying "we got that wrong, and here's the minor tweak we need to fix it". You wrote some code: for (int i = 0; i < len; ++i) copy (src[i], dst[i]); It fails to copy the final element of src to dst. It"fails spectacularly" every time. Eventually you realize that you compute "len" as an inclusive count, not exclusive. You suggest: for (int i = 0; i <= len; ++i) copy (src[i], dst[i]); as a way to make it work. This isn't "let's do more of what's failed spectacularly". This is "let's make what we're trying actually work". Sure, someone could come along and suggest that we use Algol or brainfuck to solve the issue instead, and that's not a priori insane. But a clear minded reading of the situation - we've got a lot of stuff right, but made some mistakes - would favor trying to fix what we've done wrong, not stop doing what we're trying to do. The cynicism of this analysis mostly ruined the rest of the piece for me.

deleted 6y ago
[deleted]

lbriner 6y ago

A more interesting narrative is that our tools have become too complicated for most people to understand and use efficiently. The cost per km of the first London Underground lines, which were dug largely by hand from the road downwards and built with a lot of manual labour was about 1/10th of the cost of building a similar tunnel in the 21st century with automated machinary and far fewer "expensive" people. How do you dig by hand? Simple, get a load of people with no training and give them a spade and get on with it. How do you do that with a $100M tunnel boring machine? Send people on expensive training courses, pay a load of cash in specialist repairs and maintenance and when it breaks down or you get any other kind of problem off the beaten track, just stop work until you can sort it out or pay someone else even more money to get you out of your hole. Same with software. How easy is it to keep bank records in a physical ledger? Really easy and very quick. Introduce some software and you now have a multi-million dollar complicated recording machine with loads of semi-skilled operatives using it, breaking it (accidentally), and only very slow and expensive teams can investigate and fix. We have accepted that sometimes we need to ask 10 times to get an address changed because "computer says no". Of course, technology brings you benefits that didn't exist before like centralising data, like making some hard physical processes easier to automate and to backup data that could otherwise be lost in flood or fire but are these worth the complexity and cost? In many cases that's a hard "no"!

cryptica 6y ago

>> Institutions protect insiders because every insider must mask their self-interest and the general failure of the institution. Insiders protect other insiders lest transparency reveal the insiders' skims and scams and the failure of the institution to fulfill its purpose That is exactly what I thought was going on. Most big companies definitely give off that vibe and the longer you work there, the more you get promoted, the more dirt you uncover, the more you get paid, and the least value you create... What are the odds that the author of this article came to the exact same (very specific) conclusion as I did? Clearly it's a matter of time before this is going to blow wide open.

nixpulvis 6y ago

Completely agree, but was hoping for some actual advise. This reads as an introduction to what deserves to be a complex analysis on the past and speculation on the future. Now I'm just left equally as irritated as I was when I started reading.

lordnacho 6y ago

I recognize some of the sentiments, but I think I'd put it slightly differently. We live in a society where a heck of a lot of value is created, but the value accrues to people who don't seem to be the ones creating it. If you look at economics, the "who creates value" question has been neatly sidestepped. If you can get someone to pay you, GDP goes up by that amount. There's not a whole lot of quibbling about whether the value paid was correct, although the GDP accounting system has had a number of changes throughout the years (vast simplification). At present if you just don't do something illegal, the dollar you earned is as good as the dollar anyone else earned. This system has the fault that it doesn't seem to connect with ordinary ideas about who contributed. If whoever ends up with the trophy is the person who deserves the trophy, what is the trophy for? There's a kind of nihilism about what work is for if the purpose of climbing the greasy pole is to reach the top of the greasy pole. Why should I bother to write good code if the thing that really has value is to sit in meetings and talk myself up to people who have no idea what good code actually is? Why should I not simply create a compliance tax for my colleagues, and become the troll who extracts payment? I can start with things most people agree with: good code should have unit tests and documentation. And we can move on and add code reviews, from at least 2 other devs, before any new merge is allowed. And we can't have the same two guys that are your buddies, let's rotate it a bit to make sure there's no corruption. And make sure the rotation system is documented. And signed. By an independent auditor. With an industry accreditation. From an organization that is overseen by government committee. A government committee that confers with other government committees to ensure best practice. All those people need to be paid. And they are valuable, you can't deny that documentation is important. So all these people are creating value and need some of the pie.

cryptica 6y ago

It's the result of supply-side economics exacerbated by flaws in our global monetary system. Supply-side economics works by essentially pumping free money into big corporations and institutions so that they produce as much stuff as they possibly can at the lowest possible price while creating as many jobs as possible without worrying at all about profitability, Thanks to all the free credit that they're getting, these big corporations end up driving all competitors out of business, then some employees of the corporations end up getting involved in internal politics for their own personal gain (zero sum games)... Employees who are value creators realize that their productive work is not rewarded, so they either quit the company or start getting involved in zero-sum internal politics... Once enough people have figured out the kind of game they're playing, the quality of services decline as a result.

grishka 6y ago

Imagine what the world would've been like if all software developers were competent enough to understand the abstractions they're using.

lcuff 6y ago

Frankly, the whole article just seems like a shill for his book pitch at the bottom. The arguments rely on anecdotal examples, ignoring depth and complexity. Using bold type on each paragraph's first lines just adds to my sense of this being yellow journalism.

atlgator 6y ago

With the economic expansion over the last 8 ish years, instead of paying skilled employees more, companies hired more employees at the same lower wage. This led to reduced skill overall as more large businesses adopted this strategy and competed for top talent at low pay.

pc86 6y ago

How has this not gotten flagged? It's total nonsense. The title is "how we institutionalized incompetence" and then is just a list of cherry-picked examples, nonsensical bolding and italicization, and oh yeah a note about the author's book at the end. What a joke.

james_pm 6y ago

The more of these types of posts and books I read, the more depressed I become. Work feels very pointless, especially when I start noticing the "bullshit" jobs around me and just how much of my day is spent tending to tasks created by this bloated waste-of-time bureaucracy. It's gotten to the point where it's almost impossible to do anything that actually benefits our users anymore.

mathattack 6y ago

The claims are bold, but are they really all true? Yes there is a lot of incompetence but do we really want to go back to the world of the 50s? Are cars really that much worse today? Computers? Inflation has been modest since the 70s. Yes we lag in some places but this fellow is over the top.

sbierwagen 6y ago

Baumol's cost disease. https://en.wikipedia.org/wiki/Baumol%27s_cost_disease https://en.wikipedia.org/wiki/Baumol%27s_cost_disease Everything he complains about are services, or labor-heavy products. All prices are relative. As manufactured goods become cheaper, services and labor become more expensive.

noiv 6y ago

I think that's nothing new and happens since humans started civilization. It's just now you can realize this within a lifetime.

lmilcin 6y ago

I know this going to sound shocking but half of the population has below average intelligence (yeah, below mean, but shush:) I think what happens is that everybody needs to have a job and source of income and people try to do the best for themselves. But, of course, they don't get more intelligent. As the technology progresses, more and more jobs are high skill or abstract tasks and both require intelligence. What this means is decreasing availability of intelligent people for the jobs and necessary incompetence of people who perform them (or the positions would not be filled).

TuringNYC 6y ago
Its actually worse -- advancement at work is often based on emotional and social intelligence, which is not fully correlated to intelligence of your subject matter.
- chrisweekly 6y ago
  See also "The Peter Principle" https://en.m.wikipedia.org/wiki/Peter_principle https://en.m.wikipedia.org/wiki/Peter_principle
rdiddly 6y ago
median my friend
- lmilcin 6y ago
  Of course you are right. I don't know what I have been thinking...

wskinner 6y ago

https://patrickcollison.com/fast https://patrickcollison.com/fast is a great catalog of similar issues. It is periodically discussed here.