25 ms·

The Curse of Systems Thinkers

289 points by takiwatanga 4y ago

smcnally 4y ago
There’s much to dive into here — ty — from Steve’s “Systems … where it is all about interfaces and trade-offs” to points you make throughout supporting his conclusion > we define a stable interface > reproduce behaviour reliably > Predictability is great Within being believed and valued for protecting engineers and organizations, so much progress applying these principles relies on individuals’ and teams’ readiness to adopt and advocate for them. Hope to see your experience with that in part II.
thyrsus 4y ago
I work within an organization resembling this, and yet it has remained profitable for multiple decades. I forever wonder when the technical naked short options[0] will get called, but they never have in a serious way. My suspicion is that the chaos is S.O.P. among peer companies, so the market has never experienced a competitor not so hobbled. It may also be that controlling chaos is so expensive it's a non-obvious competitive advantage not to do so. [0] technical debt is when you have a plan to pay it down. A technical naked short option is when your plan is to hope the subsystem goes away before its deficiencies have catastrophic effects.
- andreskytt 4y ago
  Complexity never really goes away, it can merely be transformed from one type to another. This organization has apparently decided it rather deals with the complexity of not having processes rather than the complexity of maintaining them. Which is a valid choice: look at India. A society totally chaotic to a stranger but one that has stuck around for millenia.
  - closedloop129 4y ago
    Is India still the same society from millenia ago when they have been conquered since?
  - sateesh 4y ago
    I don't know why you are bringing references to a country ? A country is not a company, running a country is vastly more complex than any organisation.
  - barnabee 4y ago
    > Complexity never really goes away This is only true if the complexity comes from the underlying problem that must be solved or goal that must be achieved. Complexity that exists for historical reasons, derived from technology or platform choices, solves a problem you don’t really have or don’t need to have, relates to the structure of the organisation, etc. etc. can of course all be made to go away completely. In my experience a small fraction of the complexity in most organisations and software is truly, fundamentally unavoidable.
  - denton-scratch 4y ago
    > the complexity of not having processes rather than the complexity of maintaining them There are always processes. Sometimes they are explicit and well-understood; sometimes they are hidden, and not open to improvement because nobody knows what they are. Implicit processes tend to disadvantage newcomers, as well as reinforcing social assumptions and conventions (that tend to disadvantage those that are already disadvantaged). Explicit processes don't have to be heavyweight and bureaucratic. There was a good essay on the problems caused by informal/implicit processes, from about 20 years ago. I've spent 45 minutes searching for it, but I'm afraid I can't find it.
    - hmhhcycbtsc557 4y ago
      Was the essay you were seeking https://www.jofreeman.com/joreen/tyranny.htm https://www.jofreeman.com/joreen/tyranny.htm?
      - denton-scratch 4y ago
        Yes, it absolutely was! Thank you!
- jseban 4y ago
  I think the underlying reason for why IT companies are so chaotic and inefficient, is because they simply don't need enough people, if they are well structured, do quality work and use simple and powerful tools/solutions. You could run a super high valued company on just a handful of people, and society is just not ready for that. It's a case of technology advancing faster than human organisation and economic system, and the multiples are just too large, it's a big challenge for society to handle the impact. So it's better for the organisations to be less efficient and chaotic, and to use dumbed down tools, overcomplicated solutions etc. This will fill the void created by the advancement in technology, so that the organisation can continue to function by at least resembling the traditional model. You would need to significantly shorten the working hours in order to enable more efficient companies. Otherwise you'd get even more hyper concentration of wealth in a very short time.
  - notpachet 4y ago
    > You could run a super high valued company on just a handful of people, and society is just not ready for that. Here is the hidden truth. So much of the current information sector is just daycare for grownups. Then there's a secret nucleus of people who actually do the real work. The secret to happy employment rests on being able to determine who's actually cutting lumber versus who's just playing dress-up. If we want to gradually move towards a system of universal basic income, maybe we could help sell it by funneling larger sections of society into IT, and just give them a bullshit job where they can fingerpaint all day to get their paycheck. Eventually you can let them stop fingerpainting and just give them the paycheck. > The factory of the future will have only two employees, a man and a dog. The man will be there to feed the dog. The dog will be there to keep the man from touching the equipment.
  - spopejoy 4y ago
    > You could run a super high valued company on just a handful of people, and society is just not ready for that. My version of this is that those handful of people become critical to the organization, which must be avoided at all costs. Every place I've been that's weighed down by huge IT staffs, undervalues tech, absolutely hates to give coders raises, and sees tech as a cost center, even while preaching how important technology is. Sadly, this seems to be the case so often that the only companies that really succeed with good tech have builders in the founder's chair who _still code_, at least until the model is firmly established (but even after is good too). Pretty much everybody else resents software devs and thinks they're overpaid. Weirdly a smaller, better paid team can often be resented _more_ (because of course, you can't see all the money that's being _saved_ by having a small highly-powered team).
- redbar0n 4y ago
  > I forever wonder when the technical naked short options[0] will get called, but they never have in a serious way. What does «called» mean in this analogy? That the subsystem doesn’t go away? So your experience is that: Stuff tends to stick around? That’s my experience. I have never encountered a throw-away MVP that was actually thrown away if the product continued in the same direction (and didn’t pivot).
  - giantg2 4y ago
    How long have you stuck around at a company? Maybe your company or business type is different, but I have certainly seen apps get rewritten or replaced over the course of 10 years. Some are built solely for some migration period. Some have been redesigned as part of a different system after a few years. Some major systems get rewritten after the continuous addition of integrations over 10 years demands a better architecture.
  - thyrsus 4y ago
    In the analogy to the naked short, the option is "called" when the known deficiency causes catastrophic consequences. Example: The file system is so large it takes 60 days for the backup to finish, and thus the "call" would be a multiple drive failure requiring a full restore. Example: NFSv3 with "system" security, i.e., whatever the client says it's uid and gids are, we accept that, and thus the "call" would be a malicious client declaring arbitrary unauthenticated uid/gids. See, e.g., possible publicity stunt for the movie "hackers". These are naked shorts because there are no plans to fix them.
- Aeolun 4y ago
  I think badly run tech organisations still work fine. Most of the time it doesn’t matter whether your problem is solved with spaghetti and 300 hours, or 1 hour and a fantastic design, since the gains are still orders of magnitude higher than the cost. It’s just that such an organisation is unpleasant to work for.
contingencies 4y ago
Added to https://github.com/globalcitizen/taoup https://github.com/globalcitizen/taoup
- LB232323 4y ago
  Interesting fact: The first quote mentioned in that app is from Ecclesiastes.
  - contingencies 4y ago
    You mean I can get another if I break it, so a clay cup trumps a grail? The attribution is to Mirza Asadullah Khan Ghalib, classical Urdu and Persian poet from the Mughal Empire. I can't see the same meme in the bible, although there does seem to be passing mention of clay there somewhere. No doubt all the ancient cultures valued pottery as it was such an amazing invention!
deleted 4y ago
[deleted]
wallacoloo 4y ago
consider a slow-growth company free of a title system (e.g. where every engineer is just a "member of engineering"). the less-credentialed nature means you don't have to be as specialized to float ideas. the slow-growth nature allows the company to prioritize process over chasing chaotic payoffs to stay afloat. YMMV and i'm not sure all slow-growth, titleless engineering organizations turn out this way, but there's some correlation.
zubspace 4y ago
I work in a small organization which is kinda chaotic, meaning we have a lean process. Very agile, short standup each morning, but nearly no sprint planning. I think, it's liberating to work in such a company as an experienced developer. You get difficult tasks sometimes, where you need to be the System Engineer and understand the system fully. But you're free to approach the problem in any way you see fit. This requires a lot of trust from the company and your team, but I believe that's a good thing. It's also a nice for customers, who sometimes have crazy requirements, but they still get results in a reasonable amount of time. On the other hand, this approach completely destroys newcomers. It's nearly impossible for them to approach a System which they can't fully grasp and is in constant flux. We mentor them, give them easy and introductory task, review their code, but still... It takes a massive amount of time. And I think, it's one of the reasons a company like this has a hard time growing. And there's the risk of the 'not so competent programmer', who knows how to fix stuff, without thinking far enough. I don't know, if I'm a System Engineer, but I think it aligns with the description in the article. However, I fear the day, we all agree to overhaul our processes, after which we need to plan and document and review everything. It seems to be a necessary evil for a company to grow, but at the same time we would also destroy a lot of the liberty we currently have and I don't know which side is worse.
- nine_k 4y ago
  If your options are fully vested by that time, there is a fair chance to find another company at the same stage of growth.
- zmgsabst 4y ago
  What do you feel destroys “liberty” about planning and documentation? That hasn’t been my experience (in general; had some bad companies for sure).
  - zubspace 4y ago
    It just takes time, a lot of time. And meetings, endless meetings about what to make and how. Constant reviews of the progress and the code. And documenting everything. I don't think, that this is necessarily bad. I even think, that this is a must for a large team. But it changes the way you code. Right now I have the time and opportunity to 'form' the code the way I want. I can try different approaches, rip out or rewrite bad code and add features as I see fit. That's what I call liberty. I don't feel like a raw coder. Instead I feel like an artist in my own universe I where I'm in charge. This feeling can get lost, when you need to request, plan, control and review every single change in coordination with multiple team members. You're not in charge anymore. You're not the artist, but a cog in the wheel. I think some people like one way or the other. But it's hard to transition.
jamesblonde 4y ago
The author is Irish, and as an anglo-saxon who has lived in Sweden for the last 17 years, I can relate. We get more systems engineers in the Anglo-saxon world because in part because our education at university tends to be more generalist and also in part because of our more hierarchical organization does not involve everybody in decision making, so people at the bottom let their "mother hen" boss take more responsibility. In strong engineering cultures like Sweden and Germany, we tend to have more specialization and consensus building. There will always be systems engineers, but they are more prevalent in some cultures (anglo-saxon).
- DyslexicAtheist 4y ago
  > so people at the bottom let their "mother hen" boss take more responsibility. In strong engineering cultures like Sweden and Germany I'm not sure if I understood it correctly or there was a typo, I am surprised because my experience is almost opposite. my experience with Swedish and German multinationals is that it's all about consensus, to a point where the best decisions would be rejected if it lacks support, or where the off-ramp would be no decision (which is also a decision). this I found in stark contrast to French, Italian, Spanish, UK/US/Australian organizations, that tend to value the ego driven hero who saves the day. Also these former locations seem to do better when dealing with chaos as they know how to "think on their feed". I'm curious from your experience would these companies be start-ups or medium sized firms, or could there be other reasons I'm missing that our experience is so different?
dusklight 4y ago
>Steve recommended investigating Systems Engineering as a distinct subject. Specifically, reading the engineering histories of the Gemini and Apollo projects, and especially about the culture clash between the experimental aircraft guys who built Mercury, and the ICBM teams. Anyone know any good books covering this history?
- dabiged 4y ago
  Not a book, but MIT OCW has a class on Aircraft Systems Engineering 16.885J (focusing on the Space Shuttle) that is an excellent example of this. The class starts by looking at the requirements for the system, then examines every single subsystem on the Shuttle, and explains, in detail, why it was built the way it was. Almost always the answer is "because of the constraints placed on the system by the rest of the vehicle" + it has to be as light as possible. I initially watched the lectures because of an interest in aerospace and it is a fascinating historical series in its own right with some incredible speakers. The lessons for systems engineers are numerous too.
a_c 4y ago
Before clicking, I thought the article is about someone who only thinks in "system" and not able to deliver concrete things. How wrong I was. "Give yourself permission to let the organisation fail". I agree. As someone is similar situation, the job is to let the "design of system" be heard, be debated and be implemented once green-light. You cannot convince the decision making body (be it the CEO, management, or a design committee) that your idea is the right one for 3 reasons IMO - idea could be wrong - People won't know what you are talking about unless they had the first hand experience. (A la you don't know what it is like to be a bat) - Your meritocracy is limited to a small group of decision making body If you see the "system" broad enough, you will see a market. It is essentially preaching your "system" to a wider audience. Your system maybe wrong, for which your start up will die or you devise another system. Or you are reaching whole bunch of people who understands your (the initial niche), and your living does not depend on a single decision body but a market. Convincing one body is hard, but broadcasting whatever you believe, some will respond eventually. This is why start up is great.
gorgoiler 4y ago
”My emails produced only well-worded refutations. They explained quite factually why the setup is the way it is, and implicitly therefore why it could not change” This landed so truly for me, it felt like a punch in the stomach. I wouldn’t dare count the number of times I’ve been told the technical details of why something is the way it is, without anyone ever saying the reason why we actually wanted it to be this way. My thesis was usually: we don’t. In my career I feel like I have seen hundreds of examples of me saying the systems equivalent of “lets put the dining table indoors?” to be told that the dining table is outside because the original budget meant the front door could only be yay wide so we had to leave the table in the yard and put a tent over it. And I’m just left standing there agape at how we eat in a cold wet tent every night instead of fixing it. Except it’s usually more like: why do we have to spend $9k on a commercial dishwasher repair contract? Because we have a commercial dishwasher … to get the rust off the silverware … because we eat outdoors every night … because the front door was too small to get the dining table in the house. Somehow, when the real examples of this stuff are clever engineering around build / docker / polyrepo / release / feature flags / third party bugs, the cleverness makes people think the existence of the workaround should be tolerated. It’s infuriating to join a new team held hostage by years and years of band aids because they never suffer the bigger picture consequences. The whole article was fantastic. I hope the author has the engineering leadership role they deserve. We need more people like this.
- codeflo 4y ago
  > In my career I feel like I have seen hundreds of examples of me saying the systems equivalent of “lets put the dining table indoors?” to be told that the dining table is outside because the original budget meant the front door could only be yay wide so we had to leave the table in the yard and put a tent over it. And I’m just left standing there agape at how we eat in a cold wet tent every night instead of fixing it. Oh wow, that hits home. To be fair, the historical context for the decision can be valuable information, the problem is the next step. Even if you can't fix it right now, you might make steps towards that. Or you might say: Now that we have this heavy table outside, why not attach more things to it?
  - 4y ago
LB232323 4y ago
redbar0n 4y ago
Thought it was a post arguing AGAINST Systems Thinkers. Was slighlty provoked and curious to read it, since I’m a fan of systems thinking. Found out it was arguing FOR Systems Thinkers…
- voiper1 4y ago
  The title is based on a line at the end: "the curse of systems thinkers is to be correct, but never valued."
  - redbar0n 4y ago
    Yup. It’s basically a slightly altered version of The Curse of Cassandra: the prophet/oracle who was correct but not believed. «Cassandra was a Trojan priestess of Apollo in Greek mythology cursed to utter true prophecies, but never to be believed», from wikipedia
nonrandomstring 4y ago
> the curse of systems thinkers is to be correct, but never valued. "Systems" are a mixed blessing, but system thinking is almost always good and useful. We can add value but not bask in it. As a someone invested in the _idea_ here are a few of my favourite quotes that illustrate: - People don't like systems. Especially new ones. - Systems ossify and become the problem themselves. - The ideal system exists only in the mind of its designer. - The ideal systems designer is invisible and can never take credit. "I mistrust all systematizers and avoid them. The will to a system is a lack of integrity." -- Nietzsche "The English have a system, which is *no system*, which is also a system, only better." - (?? British political philosopher c 1900 - does anyone know this one?) "A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system" -- Gall Overall, I think the thing is that systems are brilliant, until you try to actually build them and encounter _people_, who have other ideas. Neither the force of the better argument, nor punishment, reward, bribery, or flattery will move things. This is neither the fault of systems thinkers nor people but the misunderstanding that (outside the immediacy of war) systems can be imposed. Working systems evolve and are, if the individuals are mentally healthy and motivated by good attitude, generally such that people are doing the thing they would naturally be doing anyway were a formal system not there. A good system is like cat that falls off a tall building and by luck lands on its feet in a box of wool, and licks itself as if to say - sure I meant to do that.
- giantg2 4y ago
  " but system thinking is almost always good and useful. We can add value but not bask in it." It's only useful and adds value if your idea actually gets used. Otherwise, it's still pretty crushing.
- samizdis 4y ago
  > "The English have a system, which is no system, which is also a system, only better." - (?? British political philosopher c 1900 - does anyone know this one?) I don't, but that quote was used in a comment [1] about five or six weeks ago, and the commenter's relevant bit was: Nietzche said it best: I mistrust all systematizers and avoid them. The will to a system is a lack of integrity. Or maybe (I think Sidgwick): The English system is "No system", Which is also a system, only better. https://news.ycombinator.com/item?id=30598863 https://news.ycombinator.com/item?id=30598863 [Edit to add: I've been dumb there. You were that commenter, so this won't have helped - sorry.]
langsoul-com 4y ago
Being overly systematic might cause an inability to adapt to changing market systems. Like the trade-off between creativity and efficiency
giantg2 4y ago
Wow, this whole thing sounds exactly like me. I'm definitely at the stage where I've given the organization permission to fail and am just drudging on now. I mean, the system I started working on 2 years ago was using a business text field for processing decisions. That caused changes to the system when the business wants to make changes. If they do make changes, the reporting queries have to be modified to look for all the historical versions of that text for audit reasons. If the team asking that change forgets to tell one of the numerous other systems that also uses that field, then there are errors. I proposed we add a field with a code that represents this field so that the business can change the display text without affecting the systems that currently use it. It's been at least 18 months, and nothing has changed. You would think that this is a basic design best practice that should have been implemented from the beginning...
locallost 4y ago
On the flip side: I am not saying this is the case with the author, but in my experience a great number of believers in systems and processes are people who struggle to solve any problem, and therefore want to have a system where most of the problems will be solved by the system. But that's not really possible, since if you struggle to solve a problem in any way, you will not be able to create a problem solving system, which is a next level problem. Systems thinking is a good idea, but at the end of the day, things need to be done by the people and this can be messy and chaotic, and people are different so making a system that works for everybody is impossible.
- CyberDildonics 4y ago
  This is the defeatist thinking that people get into when they can't solve things in a systematic way.
joeman1000 4y ago
I think cross-discipline training is really under-rated and important. For instance, as a 3.5th year civil engineering student, I’ve been taught systems engineering and project management multiple times and in multiple different contexts. These are integral to ‘physical’ engineering, but seem (to me) to be missing from software engineering. I’m dabbling in programming and software eng now and I’m constantly surprised by the lack of standardisation and the sort of ‘wild west’ approach to things. This is fine for getting things done, but in terms of liability and responsibilities (like what the post talks about), it seems that many jobs are ill-defined and poorly scoped. Overall, I think software and ‘physical’ engineers should swap experience. Physical engineering could use a tech-injection, and software could use a ‘structure’-injection.
- spyremeown 4y ago
  Electronics Engineering student here. My Semiconductor Devices Professor said it best (this is paraphrashing): "What you gotta understand is that in the beggining of Electronics, people were pretty much trying to put two materials together they thought worked and then tried modelling it. It was pretty much trial and error, experimentation..." and then he hits me with the most "holy s*" moment of my academic life: "... much like programming and software engineering is today. You write some code, run it, see if it works. Works, ok, go ahead, make sense of it, explain in the documentation, next task". I had NEVER thought of software like this, it just hit me like an atomic bomb in the head, I felt like I understood where in the history of software engineering we are right now. Structure is coming, slowly but surely.
  - joeman1000 4y ago
    It’s shocking to be able to see it this way. Thank you for sharing this insight. It’s easy to see programming as something uber-sophisticated, because in most senses it is. The reality is that we’ve been programming (we, the wider public) for a few decades of our millions of years of consciousness. Of course it doesn’t have the scientific rigour of something we’ve been doing for millennia, like construction. With that said though, I think programming and computers in general is the closest we will ever come to ‘magic’ and wizardry. The fact that we can now effect reality with a few keystrokes is magical.
hasmanean 4y ago
Systems thinking is great but he way we reason about organizations and systems in a pretty medieval way. Most systems are described in a simple way: box and line charts which are just voodoo. We need to develop a algebraic notation for how to represent the various configurations/states of an organization, along with its operations. With a notation, you can describe what you “feel” and share the knowledge and improve on it. You can also define the organizations desired state and track your progress. You can plop a new manager in it and they will know what to do. You can also do basic engineering like pick a configuration that has the desired cost, throughout and latency that you need.
- Terretta 4y ago
  BPMN 2.0 is a reasonable start.
Josteniok 4y ago
[My colleague] would appear to see time spent in planning and writing documents as essentially wasted time, since he asserts that without process we are faster. We are not faster in reality. We are merely faster to say we're finished. That is not the same thing as actually being finished: we can all think of examples of that. And the uncertainty induced by the unknown magnitude of correction required is, IMHO, the biggest contribution to our inefficiency and ineffectiveness. I've spent my entire career working for a medium sized organization and in the past few years it has tried to become "agile". Most of this push is predicated on the idea that we will be able to go faster this way. As a result we have deconstructed ourselves. What's weird is that now we aren't actually even "faster to say we are finished." No, now we are only faster to say that we are going to be faster to be finished. We still end up taking a long time but as long as the slide deck says we are going to be done in a short amount of time, then all is believed to be running smoothly. It's very strange/depressing/unsettling.
ctvo 4y ago
Please stop highlighting random portions of your blog. It's incredibly distracting to your overall message.
jarl-ragnar 4y ago
This rang so true. I recently left a large multi-national aerospace company where I and my team had developed local processes that put systems engineering first. Unfortunately our main contract was developing a component of a larger system being developed by our parent organisation who didn’t have a concept of systems engineering. We tried for years to educate them and I watched aghast as their program costs and schedule continued to spiral out of control. Basically a bomb burst of engineers all doing what they thought was the right thing but no one owning the system design and saying no to good ideas. In the words of their chief engineer “its like 10 different black boxes, I don’t know what I’m getting and I don’t know when it’ll be finished!”
mbrodersen 4y ago
The most successful companies I have ever worked for hired smart developers and got out of the way. The least successful company I have worked for spent a huge amount of time in meetings, planning and documenting everything, while failing in the market place against competitors that didn’t. The more structure you impose, the less competitive you are. It is OK if you are a monopoly (NASA) but if not then be careful what you wish for.

289 points by takiwatanga 4y ago

There’s much to dive into here — ty — from Steve’s “Systems … where it is all about interfaces and trade-offs” to points you make throughout supporting his conclusion > we define a stable interface > reproduce behaviour reliably > Predictability is great Within being believed and valued for protecting engineers and organizations, so much progress applying these principles relies on individuals’ and teams’ readiness to adopt and advocate for them. Hope to see your experience with that in part II.

thyrsus 4y ago

I work within an organization resembling this, and yet it has remained profitable for multiple decades. I forever wonder when the technical naked short options[0] will get called, but they never have in a serious way. My suspicion is that the chaos is S.O.P. among peer companies, so the market has never experienced a competitor not so hobbled. It may also be that controlling chaos is so expensive it's a non-obvious competitive advantage not to do so. [0] technical debt is when you have a plan to pay it down. A technical naked short option is when your plan is to hope the subsystem goes away before its deficiencies have catastrophic effects.

andreskytt 4y ago
Complexity never really goes away, it can merely be transformed from one type to another. This organization has apparently decided it rather deals with the complexity of not having processes rather than the complexity of maintaining them. Which is a valid choice: look at India. A society totally chaotic to a stranger but one that has stuck around for millenia.
- closedloop129 4y ago
  Is India still the same society from millenia ago when they have been conquered since?
- sateesh 4y ago
  I don't know why you are bringing references to a country ? A country is not a company, running a country is vastly more complex than any organisation.
- barnabee 4y ago
  > Complexity never really goes away This is only true if the complexity comes from the underlying problem that must be solved or goal that must be achieved. Complexity that exists for historical reasons, derived from technology or platform choices, solves a problem you don’t really have or don’t need to have, relates to the structure of the organisation, etc. etc. can of course all be made to go away completely. In my experience a small fraction of the complexity in most organisations and software is truly, fundamentally unavoidable.
- denton-scratch 4y ago
  > the complexity of not having processes rather than the complexity of maintaining them There are always processes. Sometimes they are explicit and well-understood; sometimes they are hidden, and not open to improvement because nobody knows what they are. Implicit processes tend to disadvantage newcomers, as well as reinforcing social assumptions and conventions (that tend to disadvantage those that are already disadvantaged). Explicit processes don't have to be heavyweight and bureaucratic. There was a good essay on the problems caused by informal/implicit processes, from about 20 years ago. I've spent 45 minutes searching for it, but I'm afraid I can't find it.
  - hmhhcycbtsc557 4y ago
    Was the essay you were seeking https://www.jofreeman.com/joreen/tyranny.htm https://www.jofreeman.com/joreen/tyranny.htm?
    - denton-scratch 4y ago
      Yes, it absolutely was! Thank you!
jseban 4y ago
I think the underlying reason for why IT companies are so chaotic and inefficient, is because they simply don't need enough people, if they are well structured, do quality work and use simple and powerful tools/solutions. You could run a super high valued company on just a handful of people, and society is just not ready for that. It's a case of technology advancing faster than human organisation and economic system, and the multiples are just too large, it's a big challenge for society to handle the impact. So it's better for the organisations to be less efficient and chaotic, and to use dumbed down tools, overcomplicated solutions etc. This will fill the void created by the advancement in technology, so that the organisation can continue to function by at least resembling the traditional model. You would need to significantly shorten the working hours in order to enable more efficient companies. Otherwise you'd get even more hyper concentration of wealth in a very short time.
- notpachet 4y ago
  > You could run a super high valued company on just a handful of people, and society is just not ready for that. Here is the hidden truth. So much of the current information sector is just daycare for grownups. Then there's a secret nucleus of people who actually do the real work. The secret to happy employment rests on being able to determine who's actually cutting lumber versus who's just playing dress-up. If we want to gradually move towards a system of universal basic income, maybe we could help sell it by funneling larger sections of society into IT, and just give them a bullshit job where they can fingerpaint all day to get their paycheck. Eventually you can let them stop fingerpainting and just give them the paycheck. > The factory of the future will have only two employees, a man and a dog. The man will be there to feed the dog. The dog will be there to keep the man from touching the equipment.
- spopejoy 4y ago
  > You could run a super high valued company on just a handful of people, and society is just not ready for that. My version of this is that those handful of people become critical to the organization, which must be avoided at all costs. Every place I've been that's weighed down by huge IT staffs, undervalues tech, absolutely hates to give coders raises, and sees tech as a cost center, even while preaching how important technology is. Sadly, this seems to be the case so often that the only companies that really succeed with good tech have builders in the founder's chair who _still code_, at least until the model is firmly established (but even after is good too). Pretty much everybody else resents software devs and thinks they're overpaid. Weirdly a smaller, better paid team can often be resented _more_ (because of course, you can't see all the money that's being _saved_ by having a small highly-powered team).
redbar0n 4y ago
> I forever wonder when the technical naked short options[0] will get called, but they never have in a serious way. What does «called» mean in this analogy? That the subsystem doesn’t go away? So your experience is that: Stuff tends to stick around? That’s my experience. I have never encountered a throw-away MVP that was actually thrown away if the product continued in the same direction (and didn’t pivot).
- giantg2 4y ago
  How long have you stuck around at a company? Maybe your company or business type is different, but I have certainly seen apps get rewritten or replaced over the course of 10 years. Some are built solely for some migration period. Some have been redesigned as part of a different system after a few years. Some major systems get rewritten after the continuous addition of integrations over 10 years demands a better architecture.
- thyrsus 4y ago
  In the analogy to the naked short, the option is "called" when the known deficiency causes catastrophic consequences. Example: The file system is so large it takes 60 days for the backup to finish, and thus the "call" would be a multiple drive failure requiring a full restore. Example: NFSv3 with "system" security, i.e., whatever the client says it's uid and gids are, we accept that, and thus the "call" would be a malicious client declaring arbitrary unauthenticated uid/gids. See, e.g., possible publicity stunt for the movie "hackers". These are naked shorts because there are no plans to fix them.
Aeolun 4y ago
I think badly run tech organisations still work fine. Most of the time it doesn’t matter whether your problem is solved with spaghetti and 300 hours, or 1 hour and a fantastic design, since the gains are still orders of magnitude higher than the cost. It’s just that such an organisation is unpleasant to work for.

contingencies 4y ago

Added to https://github.com/globalcitizen/taoup https://github.com/globalcitizen/taoup

LB232323 4y ago
Interesting fact: The first quote mentioned in that app is from Ecclesiastes.
- contingencies 4y ago
  You mean I can get another if I break it, so a clay cup trumps a grail? The attribution is to Mirza Asadullah Khan Ghalib, classical Urdu and Persian poet from the Mughal Empire. I can't see the same meme in the bible, although there does seem to be passing mention of clay there somewhere. No doubt all the ancient cultures valued pottery as it was such an amazing invention!

deleted 4y ago

[deleted]

wallacoloo 4y ago

consider a slow-growth company free of a title system (e.g. where every engineer is just a "member of engineering"). the less-credentialed nature means you don't have to be as specialized to float ideas. the slow-growth nature allows the company to prioritize process over chasing chaotic payoffs to stay afloat. YMMV and i'm not sure all slow-growth, titleless engineering organizations turn out this way, but there's some correlation.

zubspace 4y ago

I work in a small organization which is kinda chaotic, meaning we have a lean process. Very agile, short standup each morning, but nearly no sprint planning. I think, it's liberating to work in such a company as an experienced developer. You get difficult tasks sometimes, where you need to be the System Engineer and understand the system fully. But you're free to approach the problem in any way you see fit. This requires a lot of trust from the company and your team, but I believe that's a good thing. It's also a nice for customers, who sometimes have crazy requirements, but they still get results in a reasonable amount of time. On the other hand, this approach completely destroys newcomers. It's nearly impossible for them to approach a System which they can't fully grasp and is in constant flux. We mentor them, give them easy and introductory task, review their code, but still... It takes a massive amount of time. And I think, it's one of the reasons a company like this has a hard time growing. And there's the risk of the 'not so competent programmer', who knows how to fix stuff, without thinking far enough. I don't know, if I'm a System Engineer, but I think it aligns with the description in the article. However, I fear the day, we all agree to overhaul our processes, after which we need to plan and document and review everything. It seems to be a necessary evil for a company to grow, but at the same time we would also destroy a lot of the liberty we currently have and I don't know which side is worse.

nine_k 4y ago
If your options are fully vested by that time, there is a fair chance to find another company at the same stage of growth.
zmgsabst 4y ago
What do you feel destroys “liberty” about planning and documentation? That hasn’t been my experience (in general; had some bad companies for sure).
- zubspace 4y ago
  It just takes time, a lot of time. And meetings, endless meetings about what to make and how. Constant reviews of the progress and the code. And documenting everything. I don't think, that this is necessarily bad. I even think, that this is a must for a large team. But it changes the way you code. Right now I have the time and opportunity to 'form' the code the way I want. I can try different approaches, rip out or rewrite bad code and add features as I see fit. That's what I call liberty. I don't feel like a raw coder. Instead I feel like an artist in my own universe I where I'm in charge. This feeling can get lost, when you need to request, plan, control and review every single change in coordination with multiple team members. You're not in charge anymore. You're not the artist, but a cog in the wheel. I think some people like one way or the other. But it's hard to transition.

jamesblonde 4y ago

The author is Irish, and as an anglo-saxon who has lived in Sweden for the last 17 years, I can relate. We get more systems engineers in the Anglo-saxon world because in part because our education at university tends to be more generalist and also in part because of our more hierarchical organization does not involve everybody in decision making, so people at the bottom let their "mother hen" boss take more responsibility. In strong engineering cultures like Sweden and Germany, we tend to have more specialization and consensus building. There will always be systems engineers, but they are more prevalent in some cultures (anglo-saxon).

DyslexicAtheist 4y ago
> so people at the bottom let their "mother hen" boss take more responsibility. In strong engineering cultures like Sweden and Germany I'm not sure if I understood it correctly or there was a typo, I am surprised because my experience is almost opposite. my experience with Swedish and German multinationals is that it's all about consensus, to a point where the best decisions would be rejected if it lacks support, or where the off-ramp would be no decision (which is also a decision). this I found in stark contrast to French, Italian, Spanish, UK/US/Australian organizations, that tend to value the ego driven hero who saves the day. Also these former locations seem to do better when dealing with chaos as they know how to "think on their feed". I'm curious from your experience would these companies be start-ups or medium sized firms, or could there be other reasons I'm missing that our experience is so different?

dusklight 4y ago

>Steve recommended investigating Systems Engineering as a distinct subject. Specifically, reading the engineering histories of the Gemini and Apollo projects, and especially about the culture clash between the experimental aircraft guys who built Mercury, and the ICBM teams. Anyone know any good books covering this history?

dabiged 4y ago
Not a book, but MIT OCW has a class on Aircraft Systems Engineering 16.885J (focusing on the Space Shuttle) that is an excellent example of this. The class starts by looking at the requirements for the system, then examines every single subsystem on the Shuttle, and explains, in detail, why it was built the way it was. Almost always the answer is "because of the constraints placed on the system by the rest of the vehicle" + it has to be as light as possible. I initially watched the lectures because of an interest in aerospace and it is a fascinating historical series in its own right with some incredible speakers. The lessons for systems engineers are numerous too.

a_c 4y ago

Before clicking, I thought the article is about someone who only thinks in "system" and not able to deliver concrete things. How wrong I was. "Give yourself permission to let the organisation fail". I agree. As someone is similar situation, the job is to let the "design of system" be heard, be debated and be implemented once green-light. You cannot convince the decision making body (be it the CEO, management, or a design committee) that your idea is the right one for 3 reasons IMO - idea could be wrong - People won't know what you are talking about unless they had the first hand experience. (A la you don't know what it is like to be a bat) - Your meritocracy is limited to a small group of decision making body If you see the "system" broad enough, you will see a market. It is essentially preaching your "system" to a wider audience. Your system maybe wrong, for which your start up will die or you devise another system. Or you are reaching whole bunch of people who understands your (the initial niche), and your living does not depend on a single decision body but a market. Convincing one body is hard, but broadcasting whatever you believe, some will respond eventually. This is why start up is great.

gorgoiler 4y ago

”My emails produced only well-worded refutations. They explained quite factually why the setup is the way it is, and implicitly therefore why it could not change” This landed so truly for me, it felt like a punch in the stomach. I wouldn’t dare count the number of times I’ve been told the technical details of why something is the way it is, without anyone ever saying the reason why we actually wanted it to be this way. My thesis was usually: we don’t. In my career I feel like I have seen hundreds of examples of me saying the systems equivalent of “lets put the dining table indoors?” to be told that the dining table is outside because the original budget meant the front door could only be yay wide so we had to leave the table in the yard and put a tent over it. And I’m just left standing there agape at how we eat in a cold wet tent every night instead of fixing it. Except it’s usually more like: why do we have to spend $9k on a commercial dishwasher repair contract? Because we have a commercial dishwasher … to get the rust off the silverware … because we eat outdoors every night … because the front door was too small to get the dining table in the house. Somehow, when the real examples of this stuff are clever engineering around build / docker / polyrepo / release / feature flags / third party bugs, the cleverness makes people think the existence of the workaround should be tolerated. It’s infuriating to join a new team held hostage by years and years of band aids because they never suffer the bigger picture consequences. The whole article was fantastic. I hope the author has the engineering leadership role they deserve. We need more people like this.

codeflo 4y ago
> In my career I feel like I have seen hundreds of examples of me saying the systems equivalent of “lets put the dining table indoors?” to be told that the dining table is outside because the original budget meant the front door could only be yay wide so we had to leave the table in the yard and put a tent over it. And I’m just left standing there agape at how we eat in a cold wet tent every night instead of fixing it. Oh wow, that hits home. To be fair, the historical context for the decision can be valuable information, the problem is the next step. Even if you can't fix it right now, you might make steps towards that. Or you might say: Now that we have this heavy table outside, why not attach more things to it?
- 4y ago

LB232323 4y ago

redbar0n 4y ago

Thought it was a post arguing AGAINST Systems Thinkers. Was slighlty provoked and curious to read it, since I’m a fan of systems thinking. Found out it was arguing FOR Systems Thinkers…

voiper1 4y ago
The title is based on a line at the end: "the curse of systems thinkers is to be correct, but never valued."
- redbar0n 4y ago
  Yup. It’s basically a slightly altered version of The Curse of Cassandra: the prophet/oracle who was correct but not believed. «Cassandra was a Trojan priestess of Apollo in Greek mythology cursed to utter true prophecies, but never to be believed», from wikipedia

nonrandomstring 4y ago

> the curse of systems thinkers is to be correct, but never valued. "Systems" are a mixed blessing, but system thinking is almost always good and useful. We can add value but not bask in it. As a someone invested in the _idea_ here are a few of my favourite quotes that illustrate: - People don't like systems. Especially new ones. - Systems ossify and become the problem themselves. - The ideal system exists only in the mind of its designer. - The ideal systems designer is invisible and can never take credit. "I mistrust all systematizers and avoid them. The will to a system is a lack of integrity." -- Nietzsche "The English have a system, which is *no system*, which is also a system, only better." - (?? British political philosopher c 1900 - does anyone know this one?) "A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system" -- Gall Overall, I think the thing is that systems are brilliant, until you try to actually build them and encounter _people_, who have other ideas. Neither the force of the better argument, nor punishment, reward, bribery, or flattery will move things. This is neither the fault of systems thinkers nor people but the misunderstanding that (outside the immediacy of war) systems can be imposed. Working systems evolve and are, if the individuals are mentally healthy and motivated by good attitude, generally such that people are doing the thing they would naturally be doing anyway were a formal system not there. A good system is like cat that falls off a tall building and by luck lands on its feet in a box of wool, and licks itself as if to say - sure I meant to do that.

giantg2 4y ago
" but system thinking is almost always good and useful. We can add value but not bask in it." It's only useful and adds value if your idea actually gets used. Otherwise, it's still pretty crushing.
samizdis 4y ago
> "The English have a system, which is no system, which is also a system, only better." - (?? British political philosopher c 1900 - does anyone know this one?) I don't, but that quote was used in a comment [1] about five or six weeks ago, and the commenter's relevant bit was: Nietzche said it best: I mistrust all systematizers and avoid them. The will to a system is a lack of integrity. Or maybe (I think Sidgwick): The English system is "No system", Which is also a system, only better. https://news.ycombinator.com/item?id=30598863 https://news.ycombinator.com/item?id=30598863 [Edit to add: I've been dumb there. You were that commenter, so this won't have helped - sorry.]

langsoul-com 4y ago

Being overly systematic might cause an inability to adapt to changing market systems. Like the trade-off between creativity and efficiency

giantg2 4y ago

Wow, this whole thing sounds exactly like me. I'm definitely at the stage where I've given the organization permission to fail and am just drudging on now. I mean, the system I started working on 2 years ago was using a business text field for processing decisions. That caused changes to the system when the business wants to make changes. If they do make changes, the reporting queries have to be modified to look for all the historical versions of that text for audit reasons. If the team asking that change forgets to tell one of the numerous other systems that also uses that field, then there are errors. I proposed we add a field with a code that represents this field so that the business can change the display text without affecting the systems that currently use it. It's been at least 18 months, and nothing has changed. You would think that this is a basic design best practice that should have been implemented from the beginning...

locallost 4y ago

On the flip side: I am not saying this is the case with the author, but in my experience a great number of believers in systems and processes are people who struggle to solve any problem, and therefore want to have a system where most of the problems will be solved by the system. But that's not really possible, since if you struggle to solve a problem in any way, you will not be able to create a problem solving system, which is a next level problem. Systems thinking is a good idea, but at the end of the day, things need to be done by the people and this can be messy and chaotic, and people are different so making a system that works for everybody is impossible.

CyberDildonics 4y ago
This is the defeatist thinking that people get into when they can't solve things in a systematic way.

joeman1000 4y ago

I think cross-discipline training is really under-rated and important. For instance, as a 3.5th year civil engineering student, I’ve been taught systems engineering and project management multiple times and in multiple different contexts. These are integral to ‘physical’ engineering, but seem (to me) to be missing from software engineering. I’m dabbling in programming and software eng now and I’m constantly surprised by the lack of standardisation and the sort of ‘wild west’ approach to things. This is fine for getting things done, but in terms of liability and responsibilities (like what the post talks about), it seems that many jobs are ill-defined and poorly scoped. Overall, I think software and ‘physical’ engineers should swap experience. Physical engineering could use a tech-injection, and software could use a ‘structure’-injection.

spyremeown 4y ago
Electronics Engineering student here. My Semiconductor Devices Professor said it best (this is paraphrashing): "What you gotta understand is that in the beggining of Electronics, people were pretty much trying to put two materials together they thought worked and then tried modelling it. It was pretty much trial and error, experimentation..." and then he hits me with the most "holy s*" moment of my academic life: "... much like programming and software engineering is today. You write some code, run it, see if it works. Works, ok, go ahead, make sense of it, explain in the documentation, next task". I had NEVER thought of software like this, it just hit me like an atomic bomb in the head, I felt like I understood where in the history of software engineering we are right now. Structure is coming, slowly but surely.
- joeman1000 4y ago
  It’s shocking to be able to see it this way. Thank you for sharing this insight. It’s easy to see programming as something uber-sophisticated, because in most senses it is. The reality is that we’ve been programming (we, the wider public) for a few decades of our millions of years of consciousness. Of course it doesn’t have the scientific rigour of something we’ve been doing for millennia, like construction. With that said though, I think programming and computers in general is the closest we will ever come to ‘magic’ and wizardry. The fact that we can now effect reality with a few keystrokes is magical.

hasmanean 4y ago

Systems thinking is great but he way we reason about organizations and systems in a pretty medieval way. Most systems are described in a simple way: box and line charts which are just voodoo. We need to develop a algebraic notation for how to represent the various configurations/states of an organization, along with its operations. With a notation, you can describe what you “feel” and share the knowledge and improve on it. You can also define the organizations desired state and track your progress. You can plop a new manager in it and they will know what to do. You can also do basic engineering like pick a configuration that has the desired cost, throughout and latency that you need.

Terretta 4y ago
BPMN 2.0 is a reasonable start.

Josteniok 4y ago

[My colleague] would appear to see time spent in planning and writing documents as essentially wasted time, since he asserts that without process we are faster. We are not faster in reality. We are merely faster to say we're finished. That is not the same thing as actually being finished: we can all think of examples of that. And the uncertainty induced by the unknown magnitude of correction required is, IMHO, the biggest contribution to our inefficiency and ineffectiveness. I've spent my entire career working for a medium sized organization and in the past few years it has tried to become "agile". Most of this push is predicated on the idea that we will be able to go faster this way. As a result we have deconstructed ourselves. What's weird is that now we aren't actually even "faster to say we are finished." No, now we are only faster to say that we are going to be faster to be finished. We still end up taking a long time but as long as the slide deck says we are going to be done in a short amount of time, then all is believed to be running smoothly. It's very strange/depressing/unsettling.

ctvo 4y ago

Please stop highlighting random portions of your blog. It's incredibly distracting to your overall message.

jarl-ragnar 4y ago

This rang so true. I recently left a large multi-national aerospace company where I and my team had developed local processes that put systems engineering first. Unfortunately our main contract was developing a component of a larger system being developed by our parent organisation who didn’t have a concept of systems engineering. We tried for years to educate them and I watched aghast as their program costs and schedule continued to spiral out of control. Basically a bomb burst of engineers all doing what they thought was the right thing but no one owning the system design and saying no to good ideas. In the words of their chief engineer “its like 10 different black boxes, I don’t know what I’m getting and I don’t know when it’ll be finished!”

mbrodersen 4y ago

The most successful companies I have ever worked for hired smart developers and got out of the way. The least successful company I have worked for spent a huge amount of time in meetings, planning and documenting everything, while failing in the market place against competitors that didn’t. The more structure you impose, the less competitive you are. It is OK if you are a monopoly (NASA) but if not then be careful what you wish for.