14 ms·
Don’t Let Architecture Astronauts Scare You (2001)
- daveslash 5y agoThis is a good one. Joel Spolsky's writing has been a huge influence on me over the past 20 years. For anyone who liked this article but may not be familiar with his other writings: I encourage you to look at his "reading lists" (posted on https://www.joelonsoftware.com/ https://www.joelonsoftware.com/) for some more gems.
- codeulike 5y agoI really agree with a lot of this. I'll tell you what though, if you have a programming task that you find boring, over-engineering it and over-architecting it can make it _so_ much more enjoyable.
- systemvoltage 5y agoBut that makes it difficult for others to maintain it.
- codeulike 5y agoOf course. I'm trying to point out one of the causes of over-engineering
- systemvoltage 5y agoSorry for obvious reaction. Already burned by this. :-)
- codeulike 5y agoYes, it's a widespread problem. Crafting code for maximum maintainability is often really important and often overlooked
- lanstin 5y agoMaintainable code can be changed more easily than unmaintainable code. Each change will tend to make code either more or less maintainable. Over time, the probability of maintainable code being refactored into unmaintainable code approaches 1. Most older code based observed were in fact found to be unmaintainable, supporting the theory.
- deleted 5y ago[deleted]
- mateo411 5y agoYes, it does. Better get some more head count for that.
- dkarl 5y agoYou better have the self-delusive ability to love and cherish that over-engineered system, because you will go from "not enough to do" to "can't keep up with the business because every little change requires 40 hours of heads-down work" in no time flat.
- Aeolun 5y agoI think if you end up in that situation because of overengineering your engineering is not very good in the first place. Even overengineered systems should be simple to change.
- codeulike 5y agoEven overengineered systems should be simple to change. Clearly you've never met a 10x over-engineer Over-engineering (for example over-generalising or adding too many layers of abstraction) is typically done with certain sorts of potential future changes in mind, so in the resulting system some changes are easy but some types of change (the ones the over-engineer didn't anticipate) can only be done with massive refactoring.
- galaxyLogic 5y agoThe general problem is that it is hard to estimate the amount of changes needed in the future. You have to make a guess. Sometimes this leads to over-engineering. Sometimes it leads to under-engineering. Depends on the case, the future, and the skill of the engineer.
- csande17 5y agoMy favorite designs are the ones that can't even solve current problems, much less anything that might need to change later, because they spent so much time optimizing for non-existent future scenarios that the present ones suffer. "Sure, our web application might take a minimum of two seconds to handle trivial get requests, and logging in doesn't work half the time, but say the word and I can instantly migrate from Postgres to MySQL."
- cmrdporcupine 5y agoAnd that, my friends, is why the FAANGs need to hire tens of thousands of software engineers to copy protobufs from one place to another...
- paxys 5y agoI'm glad that works for you, but I am also really glad that I don't work with you.
- bigbizisverywyz 5y agoPity those that have to maintain it later though.
- reilly3000 5y agoIs it just me or did this not age entirely well? I really appreciate Joel’s writings, but in a kind of very specific way .Net did actually turn out to be the big deal it set out to be, along with Java. Modern spreadsheets are just message-passing machines, and somehow Messaging companies are gigantically valuable. Yes, astronauts are expensive, perhaps overcelebrated and often wrong… but somebody has to do it right!? I for one am glad we’re not still emailing Master-Spreadsheet-FINAL(1)(1).xls back and forth to get things done. I rather like millennials like Ryan Dahl that saw the potential for Javascript to be more than a browser scripting language. I may have even been something of an astronaut myself (an “idea man”… my team would jeer)- but I’ve come down to the earth and dig my hoe into the soil these days. I think we’re better off with some good astronauts out there, at least sparingly.
- Jensson 5y agoHe isn't talking about all architects, architecture astronauts are specifically the kind of architect who no longer writes code. The guy who wrote node.js clearly still coded when he wrote it, and so on, so your examples aren't architecture astronauts. Edit: And if we take your examples, the Architecture Astronaut would be the person who says "We should use node.js in our backend, it will solve all our problems!", not the person who wrote node.js.
- reilly3000 5y agoFair. The most (semi-recent) controversial astronaut I can think of right now might be Martin Fowler and his work on event sourcing + CQRS. I don’t know if he writes code or not, but I’ve heard a lot of angst from the people who have adopted those patterns. I made a small system using those ideas built on AWS Event Bus and it generally worked quite well, but I can see where making that pattern the law in a larger setting could be excruciating.
- nuclearnice3 5y agoCould you elaborate on your success with AWS Event Bus / event sourcing + CQRS? As well as the excruciating angst you saw from others or anticipated at larger scale?
- mhh__ 5y agoBe wary of "When I used to work at Facebook we did xyz"
- Mc91 5y agoIt's not like the architects at Facebook would do something like scrap Android's gradle build system and replace it with their own custom build system...oh yaa, they did do that.
- Moto7451 5y agoWorse yet is when the people who really didn’t make it at a FAANG start the “What we did at X…” adventure. We had a couple of those wander through the woods on their dream quest to the tune of a few million dollars in wasted work we’re throwing out now that they moved on.
- deleted 5y ago[deleted]
- cmrdporcupine 5y agoI mean, Google does this all the time with things, too. And so I suspect it's partially ex-Googlers influencing things to some degree. From what I hear 'buck' at Facebook is directly modelled on Blaze/Bazel.
- yongjik 5y ago"...and by 'we', of course, I mean a team of developers larger than our entire engineering division, dedicated to developing and maintaining xyz. Why do you ask?"
- Frost1x 5y agoWhat is dubbed as an "architecture astronaut" is, in my opinion, a researcher in software engineering as is the work they do. Largely, software engineering and practices aren't well defined and there's a lot of ways to approach a problem. We haven't found best approaches but in many cases we've slowly began converging on some architectural approaches for applications. These are really frameworks to think of how to approach a specific type of software system, much in the way research provides foundational theory to think of work in its domain. We don't think of it that way, but I personally believe that's really what it is. It's a reductionist (abstracted) approach of developing a framework around a set of observations. Just like research, it may be too far abstracted to be relevant to the problem at hand and may take years before it becomes useful. That's just my perspective, though.
- cmrdporcupine 5y agoI think that's a reasonable way of framing it I guess, but the reality is the industry on the whole doesn't have a place to put people like this. Most companies don't have a need for that kind of research. They just have to ship something. Nor is it the case that everyone who does this kind of "research" is actually qualified to be doing it.
- chrisseaton 5y ago> What is dubbed as an "architecture astronaut" is, in my opinion, a researcher in software engineering as is the work they do. So they're publishing papers with falsifiable hypotheses and subjecting themselves to peer review, right? No? Oh.
- galaxyLogic 5y agoIsn't all software engineering research peer-reviewed? Like Transactions of the ACM, all the great conferences etc. Or was that in the last millennium only?
- chrisseaton 5y agoThat was my point - they aren’t publishing papers so they aren’t doing research. They’re just standing up at conferences and giving opinions with no data to back it up. That's not what a researcher or 'scientist' (as some call themselves) do.
- fmajid 5y agoI remember being invited by IBM to some sort of lunch junket with Grady Booch (IBM had acquired Rational Software). The guy basically spouted the most inane platitudes it has ever been my misfortune to be on the receiving end of, despite my working for a company so large it had not just architects but also IT Urbanists.
- burnished 5y agoWhat in the everliving Son of Fuck is an IT urbanist
- robbedpeter 5y agoCivil engineers with stupid titles put in positions at companies that think a rather lot of themselves.
- brianmcc 5y agoGuessing OP means people who don't just do the software equivalent of "architect buildings" (solutions), but do so on a scale bordering on town planning i.e. entire software landscapes?
- fmajid 5y agoArchitect of architects. In other words not just an astronaut, but someone who has transcended into a parallel universe of abstract irrelevance.
- astronauthere 5y ago"When great thinkers ..." One should give credit to Joel for correctly identifying the issue: great thinkers excel in abstract thought. Many of the comments here, however, seem to misunderstand his point about the propensity of creative minds to continue the refinement process and "not knowing when to stop". One (codeulike) apparently thinks simply being "bored" is what drives abstract thinkers. Let me assure you that is not the case, nor is it an inability to code. Many architecture astronauts started out as wiz coders. Now let's review the gifts given by said astronauts - I'll limit to 2 examples but there are more: - LISP is the mind product of a software astronaut. - UNIX ("everything is a file") is the product of software space exploration. My advice to serious young software engineers is to not accept mediocrity and imprecise thinking as acceptable standards for their chosen vocation. This cult of celebrating mediocrity in software design is a transient phase (20 years to date) and it is mostly a side effect of the introduction of facile soap boxes via the internet and blog sphere. When the dust settles, deep thinking and an ability to conceive powerful abstractions will yet again take center stage. Software, after all, is all about abstraction.
- deleted 5y ago[deleted]
- oroul 5y agoIsn't UNIX a clear example of people not engaging in architecture "astronaut" behavior? It was designed to be as simple as possible - in stark contrast to Multics.
- thewakalix 5y ago"Everything is a file" seems comparable to "everything is a message".
- chii 5y agoThe difference between a file and a message is that a file does not prescribe format of the file - it's usually application dependent, but is understood to be plain text. A "message" tends to have formats associated with it (which is what differentiate it from being just a "file" that's transferred via a protocol). These formats, like JSON, or xml, or whatever new fangled formats kids these days use, now requires architectural astronauting; namely, common field names, or standards, so that different programs would parse them similarly. And now you'd want schemas for those formats, and automatic parser generated for those formats, and more and more...
- aazaa 5y ago> When you go too far up, abstraction-wise, you run out of oxygen. What happens if you don't go far enough up, abstraction wise? You end up repeating yourself - everywhere. You end up with globals everywhere. You end up with God Classes. You end up with epic-length functions nested 15 levels deep. You end up with primitives trying to do the job that well-crafted data structures, named using the terminology of the problem domain, should be doing. It's odd that the author rails against "astronauts" without addressing the very real motivations that lead to good architectures. Like all design, there are tensions in software development, and navigating the forces pulling you in opposite directions regarding architecture is one of the most important contribution you can make. It's also odd that the author never calls out specific architectures that are the work of astronauts, just "the stupendous amount of millennial hype that surrounds them." He seems to hint in the introduction that CORBA is (was) one of them, but even that reference is pretty vague.
- smoldesu 5y agoIf you don't go far enough up abstraction-wise, you probably either don't have a good product or aren't solving anything profound. These 'high level' solutions are attractive to investors because they want 'napster for everything' or 'the iPod for your kitchen' or 'Sonos for your bathrooms' et ad nauseum. They're attractive pitches that make the likes of Mark Cuban's ears perk up. Whether or not you can sell them in the long-term, well, that's a different story. But these 'architecture astronauts' are the ones who get all the VC funding and grant money.
- tonto 5y agoYou can have great novel products...that aren't very abstract
- galaxyLogic 5y agoIn general general solutions are better because they are more reusable. More "adaptable". Think of the formula for the solution of 2nd order polynomials. You might say it is too abstract, you don't need to solve all 2nd order polynomials, just this one and this one. If that kind of thinking had prevailed we would be missing many things the modern world has to offer.
- dnndev 5y ago> They tend to work for really big companies that can afford to have lots of unproductive people with really advanced degrees that don’t contribute to the bottom line. Is is the really big company that makes you die inside or the people themselves? I tend to think its the company... its soo hard for me to work and feel productive at a large company. Something just oozes nobody really cares about you or what your doing. Everyone is just trying to do the bare minimum and go home.
- anthony_r 5y agoBecause doing the bare minimum is a reasonable thing to do. You will not be rewarded for more work unless you play the hierarchy game. The Gervais Principle: https://www.ribbonfarm.com/2009/10/07/the-gervais-principle-or-the-office-according-to-the-office/ https://www.ribbonfarm.com/2009/10/07/the-gervais-principle-...
- baobabKoodaa 5y agoYou can rationalize it as much as you like, but many people will not be happy by spending their working hours doing the bare minimum.
- anthony_r 5y agoIt's not a rationalization, it's a description of reality. Play the hierarchy game or find joy elsewhere, there's no other way really.
- baobabKoodaa 5y agoNot everybody is like you. Not everybody shares your worldview, or what you find happiness in. People are different, and different people find happiness in different places. I'm telling you, many people feel the need to do more than the bare minimum at work, and will feel happier by doing more. Perhaps you are not like that. You can still take my word that people like that exist.
- backtoyoujim 5y agoJoelonsoftware always seems to me to be really, really close to punching someone in the nose.
- clpm4j 5y ago"All they’ll talk about is peer-to-peer this, that, and the other thing. Suddenly you have peer-to-peer conferences, peer-to-peer venture capital funds, and even peer-to-peer backlash with the imbecile business journalists dripping with glee as they copy each other’s stories: “Peer To Peer: Dead!”" Strong blockchain vibes in this and the paragraph that follows it.
- ziftface 5y agoI think that example was interesting because he basically described BitTorrent (a more general Napster that's not just a push-button music player) as the potential brainchild of astronauts, but that actually was really useful to millions of people. So maybe the astronauts do have a point sometimes.
- Kerrick 5y agoAnd yet the _user_ problem he describes Napster solving, “type a song name and hear it play,” is currently solved by Spotify which is WAY more popular than BitTorrent.
- chii 5y ago> currently solved by Spotify which is WAY more popular than BitTorrent. Spotify claims they have 356 million users. I would claim that torrent users are greater - you just don't know or hear of them!
- geuis 5y ago> If Napster wasn’t peer-to-peer but it did let you type the name of a song and then listen to it, it would have been just as popular. This is a very effective statement. Today p2p for music is certainly alive but has greatly been surpassed by services like YouTube and Spotify (and Rdio, sadly dead before it’s time). One is ad supported, the others paid. But they all just let the user search and listen. Popcorn Time has done this for movies and tv. It’s purely p2p. Disliked by content owners, but they’ve largely failed to provide the same kind of universal search and watch functionality that the app provides. Hulu, Netflix, Apple+, Disney+, insert other streaming services. They all basically fail at usable search and recommendation usability and have limited overall content.
- FpUser 5y agoOne of my favorites of all time
- BishoyDemian 5y agoPremature optimisation is a real pain when it comes to software design and architecture. However, abstraction is a powerful and necessary tool that can make our life a lot easier by avoid repetition (DRY - don't repeat yourself). A good architect worth their salt will be able to find the right balance. To distill and simplify a problem down to the absolute minimum where it cannot and should not be abstracted anymore before it lose or miss its core objectives and reason to exist.
- Lio 5y agoA somewhat cheesy aphorism for this is "Dry but not so dry it cracks". Personally I like the rule of thumb that you let 3 instances of something occur before you try to refactor it into a single design. Without examples of real usage it's far too easy to come up with bad abstractions. I'd rather have code that's easy to delete later.
- dzonga 5y agowell, the tragedy now compared to 2001 when Joel wrote this, is the architecture astronauts are no longer limited to big corps. with vc money, early stage startups got money to boot. I don't know how many early stage startups, I have encountered barely out of a seed stage with microservice this, serveless this, all running of course on k8's for unlimited scaling. if frontend was madness then current trends in backend dev are madness ^2. I have completely switched off now from node.js work to on saner boring things.
- keithwhor 5y agojoel nailed it, xml was clearly peak internet
- iamleppert 5y ago“ Another common thing Architecture Astronauts like to do is invent some new architecture and claim it solves something. Java, XML, Soap, XmlRpc, Hailstorm, .NET, Jini, oh lord I can’t keep up. And that’s just in the last 12 months!” Wow, this hasn’t aged well. I always found Joel’s articles to be interesting but pretty limited in actual wisdom. He writes as if to be making profound observations, but in reality they are actually pretty narrow minded and really only apply to a very narrow nitch of computing as a whole and really only apply well to a certain startup type — not technically complex communications, productivity or organizational type software. His advice is invalid for the development of anything more complex, for example you shouldn’t take his advice if you’re building a biotech company or starting a new kind of tech company that isn’t your run of the mill CRUD app. I think it’s rather disingenuous and quite arrogant to make generalizations about the types of people who work at big companies (without even knowing what they do or having ever met them). I’m sure the same traits that make one have disdain for corporations are the same drivers that cause one to become an entrepreneur, and then blog loudly about it to the world.
- Angostura 5y ago> His advice is invalid for the development of anything more complex... I think his advice would be 'does this actually have to be more complex?' - I suspect. If might superficially look complex, but could it be actually be handled in an old-school , simpler manner?
- bartread 5y ago> I think it’s rather disingenuous and quite arrogant to make generalizations about the types of people who work at big companies (without even knowing what they do or having ever met them). I get where you're coming from, with your reply as a whole and not just this snippet, but I think it's worth pointing out that Joel did work at Microsoft for quite a few years during its historically ascendent and dominant period in the 90s so I think he has at least some idea about the types of people that work at big companies. If you want to talk about things that haven't aged well, SOAP would be right up there - you may not remember all the fanfare surrounding it, but I certainly do, and where exactly is it now? I know it's still used in some legacy contexts. I've even seen SOAP responses within the last 5 - 7 years, but it's basically dead (or at least stagnated) tech that isn't used for newer projects and systems, and certainly isn't something we'd even consider using.
- oytis 5y agoThe peer to peer part definitely didn't age well. And the Java part too. Nobody likes bombastic claims, but these likely go from marketing than from architects. It's important to see beyond the marketing and recognize the engineering thinking and potential behind technologies.
- dalbasal 5y agoSo... I think a lot this is two problems, not one. One is definitely climbing the transaction ladder to the point of no oxygen, Astronaut Architecture. But, I don't think that alone explains the amount of bombastic, heroic, utopian, grandiloquence of those Astronaut Architecture quotes. People describing their jobs, their companies, ideas and such seem draw to grandiose & abstract nonsense. "Solutions-talk." Walk around a lot of business-ey trade shows and read plaques. 90% of the time, it is impossible to know what they do, who they do it for, why. They all do "business, people, and technology solutions." Even if you stop with questions, the first answer is always hopelessly abstract. It takes a lot of digging to eventually find out they do custom spreadsheets for dentists. Meaningful statements are limiting. Who wants to limit themselves? Also, it works. Saying something specific enough to be meaningful opens you to criticism, being eliminated by process of elimination, etc. Architecture Astronautary feeds comfortably into sales, marketing, investor relations, recruitment. It's acceptable in boardrooms, AGMs, job descriptions. Media is happy to report on it. Obscurity by abstraction works.
- shockeychap 5y agoMuch of what Joel says here - especially the conclusion at the end - is also mirrored in what Steve Jobs said when confronted over shutting down OpenDoc: "You have to start with the customer experience and work backwards to the technology. You can't start with the technology and figure out where you're gonna sell it." https://youtu.be/oeqPrUmVz-o?t=110 https://youtu.be/oeqPrUmVz-o?t=110