8 ms·

You can't design software you don't work on

296 points by saikatsg 10mo ago

skydhash 10mo ago
"Generic Software Design" as the author called it, is nice for setting the general direction of some implementation. This is why I like to read software engineering books. It's easier to solve a problem if you have some kind of framing to guide you. And it's easier to talk about the solution if everyone share the same terminology. But yes, the map is not the territory, and giving directions is not the same as walking the trail. The actual implementation can deviate from the plan drafted at the beginning of the project. A good explanation is found in Naur's Theory of Programming, where he says the true knowledge of the system is inside the head of the engineers that worked on it. And that knowledge is not easily transferrable.
- DerArzt 10mo ago
  Man I would kill for some direction from my "architects", even if it were a bit wrong. I'm at the point that I can't even get them to review my architectural diagrams demanded by my company to guide me on what's expected.
narag 10mo ago
The article incurs in the very same problem it's describing. It's generic advice that might not be appliable to specific situations.
kayo_20211030 10mo ago
> I don’t know if structural engineering works like this, but I do know that software engineering doesn’t. Structural Engineering (generally construction engineering) does work like that. Following the analogy, the engineers draw; they don't lay bricks. But, all the best engineers have probably been site supervisors at some point and have watched brick being layed, and spoken to the layers of bricks, etc. Construction methods change, but they don't change as quickly as software engineering methods. There is also a very material and applicable "reality" constraint. Most struct's knowledge/heuristics remains valid over long periods of time. The software engineers' body of knowledge can change 52 times in a year. To completely stretch the analogy - the site conditions for construction engineering are better known than the site conditions for a large software project. In the latter case the site itself can be adjusted more easily, and more materially, by the engineering itself i.e. the ground can move under your feet. Site conditioning on steroids! Ultimately, that's why I agree fully with the piece. Generic advise may be helpful, but it always applies to some generic site conditions that are less relevant in practice.
- glitchc 10mo ago
  It sounds like you are making the argument that there is no established way to generate good software. If that's the case, then software isn't engineering, but rather art. The former requires established/best practices to be called a discipline, while the latter is a creative endeavour.
  - kayo_20211030 10mo ago
    That's true. I do. I consider it a creative art, with some disciplinary adjacency to engineering. The creative sculptor has to know the material stone in order to make anything good with it. But, construction engineering is creative too; just different.
  - Scarblac 10mo ago
    I think we know how to reliably make good software. E.g. NASA manages. The problem is that doing it like that is much too expensive and too slow for most businesses.
    - pixl97 10mo ago
      >and too slow for most businesses. A lot of this is because while a 'good' business is waiting for the 'good' software to be written, some crappy business has already written the crappy software and sold it to all the customers you were depending on. In general customers are very bad at knowing the difference between good and bad software and typically buy what looks flashy or the sales people bribe them the most for.
    - kayo_20211030 10mo ago
      NASA is a bit of an outlier. In the 50's through the 70's any failure, particularly a failure involving the loss of a life, would have been a national catastrophe; a blow to national prestige. So, they were super careful that it didn't happen. The spent-cost was irrelevant compared to the reputational value at stake. Honestly, it was a wise investment given the operative quid pro quo in those days. Maybe they still do good software, I don't know, but I suspect that the value at risk today makes them more cost averse, and less sensitive to poor software. "Business" runs the same calculations. I'd posit that, as a practical matter, most businesses don't want "good" software; they want "good enough" software.
michaelt 10mo ago
I've always felt it's unrealistic to separate upfront architecture from implementation, because my experience is a lot of systems turn out to have requirements that are a lot more complex in reality than they might seem at first, even if you think quite hard about the requirements. Imagine if you worked for an online retailer like Amazon, and you were assigned to architect a change so you can add free sample items into customers' orders. Take a moment to think about how you'd architect such a system, and what requirements you'd anticipate fulfilling. In the next paragraph, I'll tell you what the requirements are. Or you can skip the next paragraph, the size of which should tell you the requirements are more complex than they seem. The samples must be items in the basket, so the warehouse knows to pick them. They must be added at the moment of checkout, because that's when the order contents and weight can change. Often a customer should receive a sample only once, even if they check out multiple orders - so a record should be kept of which customers have already been allocated a given sample. It should be possible to assign a customer the same sample multiple times, in which case they should receive it once per order until they've received the assigned number. Some samples go out of stock regularly, so the sample items should not be visible to the customer when they view their order on the website, but if shipped it should appear on their receipt to assure them they haven't been charged for it. Samples should never be charged for, even if their barcode is identical to something we normally charge for. If the warehouse is unable to ship the sample, the customer should not receive a missing-item apology or a separate shipment, and the record saying that customer has had that sample already should be decremented. If the warehouse can't ship anything except the sample, the entire orders should be delayed/cancelled, never shipping the sample alone. If a customer ordered three of an item and was assigned one sample item with the same barcode but the warehouse only had three items with that barcode in stock, something sensible should happen. One key type of 'sample' is first-time-customer gifts; internal documentation should explain that if the first order a customer places is on 14-day delivery and their second order is on faster delivery and arrives first, the first-order gift will be in the second order to arrive but that's expected because it's assigned at checkout. If the first-order-checked-out is cancelled, either by the customer or the warehouse, the new-customer gift should be added to the next order they check out. Some customers will want to opt out of free samples, those who do should not be assigned any samples. But the free sample system is also used by customer services to give out token apology gifts to customers whose orders have had problems, customers who've been promised a gift should receive it even if they've opted out of free samples. No reasonable person can design such a system upfront, because things like 'opt-out mechanism sometimes shouldn't opt you out' and 'more than one definition of a customer's first order' do not occur to reasonable people.
austin-cheney 10mo ago
I completely disagree with almost the entirety of the article. It’s all about prior experience building large things many times yourself, not using some framework or other external abstraction. When you have done this many times you absolutely can design a large application without touching the code. This is part planning and risk analysis experience and part architecture experience. You absolutely need a lot of experience creating large applications multiple times and going through that organizational grind but prior experience in management and writing high level plans is extremely helpful.
- wduquette 10mo ago
  You’re speaking of implementing yet another system of a familiar kind, I.e. a new project. The OP says that generic design works for new projects. He’s mostly talking about designing new features to be added to an existing system, in which case the design has to be contingent on the existing system.
  - austin-cheney 10mo ago
    Software developers like to think they are special. They aren't. Software, from a planning perspective, is not much different than physical construction. When it comes to extending an existing application it really comes down to how well the base application was planned to begin with. In most cases the base application developers have no idea, because they either outsourced the planning to some external artifact or simply pushed through it one line at a time and never looked back. Either way the person writing the extension will be more concerned with the corresponding service data and accessibility than conformance to the base application code if it is not well documented and not well tested in a test automation scheme.
- roguecoder 10mo ago
  Why are you rewriting the same application a second time? I've personally yet to have a situation where that comes up. And every application I've ever worked on has its architecture evolve over time, as behavior changes and new domain concepts are identified. There are recurring patterns (one might even call them Design Patterns), but by the time we've internalized them we have even less need for up-front planning. Why write the doc when you can just implement the code?
eviks 10mo ago
> For instance: In large codebases, consistency is more important than “good design” But this is exactly the type of generic software design advice the article warns us about! And it mostly results in all all the bad software practices we as users know and love remaining unchanged (consistently "bad" is better than being good at least in some areas!)
- karmakaze 10mo ago
  Yeah that line gave me a twitch. Reading on though it's more about the resulting coherence and correctness rather than like the Ralph Waldo Emerson quote: "A foolish consistency is the hobgoblin of little minds, adored by little statesmen and philosophers and divines."
  - kayo_20211030 10mo ago
    I agree. It's only the foolish consistency that's problematic. A sensible consistency does, as you say, provide a coherence. William James, who overlapped Emerson, has a lot to say about positive habits.
- johnfn 10mo ago
  I don’t know. At my place a lot of cowboy engineers decided to do things their own way. So now we have the random 10k lines written in Redux (not used anywhere else) that no one likes working with. Then there’s the part that randomly uses some other query library because they didn’t like the one we use in 95% of the code for some reason, so if you ever want to work with that code you need to keep two libraries in your head instead of one. Yes, the existing query library is out of date. Yes, the new one is better— in isolation. But having both is even worse than having the bad one!
  - shayway 10mo ago
    It's essentially the same problem as https://xkcd.com/927/ https://xkcd.com/927/ [How Standards Proliferate]
augustk 10mo ago
In the best scenario the developers are also active users of the software they produce. Then a design flaw or an error that affects the users will also affect the developers and will (hopefully) motivate the latter to correct it.
- roguecoder 10mo ago
  XP putting a customer on the team was the best thing in the methodology. Replacing those with business representatives is one of Scrum's original sins.
  - bitwize 10mo ago
    > XP putting a customer on the team was the best thing in the methodology. Recently my boss said to me: "Customers want something that WORKS. If you deliver something, and it doesn't work, what's the customer going to think?" The huge drawback to putting a customer on the team is that the customer probably doesn't want to know, let alone be involved with, how the sausage is made. They want a turnkey solution unveiled to them on the delivery date, all ready to go, with no effort on their part. Generally what you want is a customer proxy in that role, who knows or can articulate what the customer needs better than the customer themselves can. Steve Jobs was a fantastic example of someone who filled this role.
  - augustk 10mo ago
    It's also worth noting that a customer is not necessarily a user. As a developer I don't care so much about the customer but I care wholeheartedly about the users.
- octorian 10mo ago
  Its also useful for developers to have a way of bypassing customer support to have direct visibility into what issues the actual users are experiencing. This can come in the form of browsing tickets, online forums, or social media. Often something that's easily brushed off by a support rep will ring a bell in the mind of a developer who has recently worked in the area of the code related to the issue.
watters 10mo ago
The second footnote acknowledges that the post is largely tautological.
dcre 10mo ago
Very impressed at the rate of high-quality interesting posts from this author.
fogleman 10mo ago
> The kinds of topic being discussed are not "is DRY better than WET", but instead "could we put this new behavior in subsystem A? No, because it needs information B, which isn't available to that subsystem in context C, and we can't expose that without rewriting subsystem D, but if we split up subsystem E here and here..." Hmm, sounds familiar... Bingo knows everyone's name-o Papaya & MBS generate session tokens Wingman checks if users are ready to take it to the next level Galactus, the all-knowing aggregator, demands a time range stretching to the end of the universe EKS is deprecated, Omega Star still doesn't support ISO timestamps https://www.youtube.com/watch?v=y8OnoxKotPQ https://www.youtube.com/watch?v=y8OnoxKotPQ
- lkglglgllm 10mo ago
  Wngman. Number of softwares not supporting iso8601, TODAY (no pun), is appalling. For example, git (claiming compatibility, but isn’t).
  - deleted 10mo ago
    [deleted]
- bitwize 10mo ago
  This is the kind of situation you get into when you let programmers design the business information systems, rather than letting systems analysts design the software systems.
  - QuercusMax 10mo ago
    I don't think I've ever worked on a project that had "system analysts". You might as well say "this is what happens when you don't allow sorcerers to peer into the future". Best I've ever had are product managers who maybe have a vague idea of what the customer wants.
    - bitwize 10mo ago
      Well, that's just the problem, innit. In decades past, systems analysts performed a vital function, viewing the business and understanding its information flows as a whole and determining what information systems needed to be implemented or improved. Historically, in well-functioning information-systems departments, the programmer's job was confined to implementation only. Programming was just a translation step, going from human requirements to machine readable code. Beginning in about the 1980s or so, with the rise of PCs and later the internet, the "genius programmer" was lionized and there was a lot of money to be made through programming alone. So systems analysts were slowly done away with and programmers filled that role. These days the systems analyst as a separate profession is, as you say, nearly extinct. The programmers who replaced the analysts applied techniques and philosophies from programming to business information analysis, and that's how we got situations like with Bingo, WNGMAN, and Galactus. Little if any business analysis was done, the program information flows do not mirror the business information flows, and chaos reigns. In reality, 65% of the work should be in systems analysis and design—well before a single line of code is written. The actual programming takes up maybe 15% of the overall work. And with AI, you can get it down to maybe a tenth that: using Milt Bryce's PRIDE methodology for systems analysis and development will yield specs that are precise enough to serve as context that an LLM can use to generate the correct code with few errors or hallucinations.
ilaksh 10mo ago
This is also the type of thing that makes having separate software architects that aren't actually maintaining the software generally a nonsensical idea. There are too many decisions, technical details, and active changes to have someone come in and give direction from on high at intervals. Maybe at the beginning it could make sense sort of, but projects have to evolve and more often than not discover something important early on in the implementation or when adding "easy" features, and if someone is good at doing software design then you may need them even more at that point. But they may easily be detrimental if they are not closely involved and following the rest of the project details.
- o_nate 10mo ago
  I guess I'm lucky not to have worked at a place with a role for software architects who don't actually write code. I honestly don't know how that would work. However, I think I can appreciate the author's point. Any sufficiently complex piece of existing software is kind of like a chess game in progress. There is a place for general principles of chess strategy, but once the game is going, general strategy is much less relevant than specific insights into the current state of play, and a player would probably not appreciate advice from someone who has read a lot of chess books but hasn't looked at the current state of the board.
- roguecoder 10mo ago
  The best "architects" serve as facilitators, rather than deciding themselves how software is built. They have to be reading the code, but they don't themselves have to be coding to be effective. You don't need one until you've got 30-70 engineers, but a strong group of collaborative architects is the most important thing for keeping software development effective and efficient at the 30-1,000 engineer range.
7402 10mo ago
> if you come up with the design for a software project, you ought to be responsible for the project’s success or failure I think this should also apply to people who come up with or choose the software development methodology for a project. Scrum masters just don't have the same skin in the game that lead engineers do.
nullorempty 10mo ago
> You can't design software you don't work on In 30 years in software dev, I am yet to see any significant, detailed and consistent effort to be extended into design and architecture. Most architects do not design, do not architect. Senior devs design and architect and then take their design to the architects for *feedback and approvals*. These senior devs make designs for features and only account for code and systems they've been exposed to. With an average employment term of 2 years most are exposed to a small cut of the system, which affects the depth and correctness of their design. And architects mostly approve, sometimes I think without even reading the docs. At most, you can expect the architects to give generic advice and throw a few buzzwords. At large, they feel comfortable and secure in their positions and mostly don't give a shit!
- makeitdouble 10mo ago
  > 2 years I've been thinking about this a lot. 2~3 years is a long time, long enough to have a pretty good grasp on what a code maintained by 50~100 does in pretty concrete terms, come up with decent improvement ideas, and see at least one or two structural ideas hit production. If the person then stays 1 or 2 more years they get a chance to further refine, but usually will be moved up the ladder Peter Principle style. If they get a chance to lead these architecture changes that company has a chance to be on a decent path technally speaking. I'm totally with you on the gist of it: architects will usually be a central switch arranging these ideas coming from more knowledgeable places. In the best terms I see their role as guaranteeing consistency and making sure teams don't impede each other's designs.
- WillAdams 10mo ago
  John Ousterhout has been addressing this one class at a time at Stanford for a while now, and has scaled up to a book: https://www.goodreads.com/en/book/show/39996759-a-philosophy-of-software-design https://www.goodreads.com/en/book/show/39996759-a-philosophy... Video overview at: https://www.youtube.com/watch?v=bmSAYlu0NcY https://www.youtube.com/watch?v=bmSAYlu0NcY
- mmis1000 10mo ago
  > 2 years I feel it's already enough to rewrite a big part of subsystem or change the whole thing into shit (depends on maintainer). Software today moves quite fast. 2 year is sometimes difference between a new company and a dead company
deleted 10mo ago
[deleted]
jkaptur 10mo ago
There are two extremes here: first, the "architects" that this article rails against. Yes, it's frustrating when a highly-paid non-expert swoops in to offer unhelpful or impossible advice. On the other hand, there are Real Programmers [0] who will happily optimize the already-fast initializer, balk at changing business logic, and write code that, while optimal in some senses, is unnecessarily difficult for a newcomer (even an expert engineer) to understand. These systems have plenty of detail and are difficult to change, but the complexity is non-essential. This is not good engineering. It's important to resist both extremes. Decision makers ultimately need both intimate knowledge of the details and the broader knowledge to put those details in context. 0. http://www.catb.org/jargon/html/story-of-mel.html http://www.catb.org/jargon/html/story-of-mel.html
MagicMoonlight 10mo ago
There is real irony in a blog post saying you can’t trust generic advice, which is itself a generic advice blog post, and links to other generic advice they have written.
Sankozi 10mo ago
"In large codebases, consistency is more important than “good design”" - this is completely opposite from my experience. There is some value in consistency within single module but consistency in a large codebase is a big mistake (unless in extremely rare case that code base consists entirely of very similar modules). Modules with different requirements should not have single consistent codebase. Testing strategy, application architecture, even naming should be different across different modules.
roguecoder 10mo ago
The problem isn't programmers: it is cheap-ass executives obsessed with compliance. Good software designers are facilitators. They don't tell people how to build software, but say "not like that" by making the technical requirements clear. They enable design to constantly change as the needs change. It has been a long time since I've been at a company willing to actually employ someone in that roll. They require that their most senior engineers be focused on writing code themselves, at the expense of the team and skill-building necessary for quality software. Instead we get bullshit like "team topologies" or frameworks that are more about how the company wants to manage teams than they are about how well the software works. We get "design documents" that are considered more important than working code. Even the senior engineers that are around aren't allowed to say "no" if it is going to interfere with some junior project manager's imagined deadline. Software companies are penny-wise and pound foolish, resulting in shittastic spaghetti messes with microservice meatballs.
mashally 10mo ago
True somehow, I see a lot of people who don't understand the business well cannot, absolutely, design a good software.
bitwize 10mo ago
The job of the big-picture software architect is not to give "generic software design advice". It's precisely to see the big picture: understand the information needs and flows of the business and determine WHAT needs to be built in order to serve those precise needs. Let the programmers worry about the details. That's their job and their strength: they are detailists who are fluent in the language of the machine, but their biggest drawback is, they tend to have difficulty seeing the big picture and understanding how those details fit into a greater whole. One does not need to be a programmer in order to be a great systems analyst/architect. Matter of fact it's the opposite: great analysts are good with people, and have a strong intuitive grasp of what people need in order to effectively run the business. Leaving that to programmers is a recipe for disaster, as without documentation of existing business systems and requirements and a solid design, programmers will happily build the wrong thing.
nik282000 10mo ago
And software engineers can't design cars that they don't drive. As evidenced by the millions of tons od e-waste rolling around on city streets these days.
guywithahat 10mo ago
As a personal anecdote in agreement with the article, I've seen design consultants come in with no industry experience and ruin projects. Friends of the VP with no satellite experience saying we need to introduce random exceptions everywhere who don't understand threading or type systems outside of Java. It's frustrating to see potential bugs introduced at the design level with no regard for how the end product operates
neopointer 10mo ago
Didn't read the article yet, but came here to say that I couldn't agree more with that sentence in the sense that the industry should stop making people "software architects", specially those that do not code at all but want to make design decisions. Software architect should be a role and not a position.
- chanux 10mo ago
  Don't want to generalize this to all but based on my experience I see it like this: The material architects build with is hollow Lego blocks. Engineers have to build with blocks that have complex machinery inside it.
wewewedxfgdf 10mo ago
I'm wary of absolute statements about programming.
kevinlearynet 10mo ago
The best applications I've ever been a part of building, measured by user satisfaction, are those where the engineers: 1. Value simple, effective systems 2. Understand all use cases, because they use it 3. Have enough freedom to fix small things as they find them #3 is controversial sometimes, but I believe this flexibility and creative freedom for devs leads to much happier people and much better products.
lunias 10mo ago
In my experience an "architect" is someone that used to write code (maybe 10-20 years ago - maybe only for a few years to begin with) but followed incentives away from the practice. Now they read blogs written by other architects about technologies that none of them have used. They champion the iterative process, but have 20 hard requirements up front that "make sure we can scale" (there are 5 users of this intranet app). They have 30 minutes free per day to produce any deliverables, all other time is spent in meetings. The deliverables produced are diagrams which have a few boxes and arrows pointing between them. Engineers are quick to ask, "Aren't those arrows backwards? We pull data from that service, right?". To which the architect replies without thought, "Ah, yeah, you're right. This is more of a work in progress, something to work off of." The next slide in their presentation is about database normalization and shows some data model that they hallucinated without looking at the data. The engineers chime in again, "Do we care more about the size of the data on disk or read times here?". The architect doesn't understand why you would challenge "best practices". You don't understand why the person presumably leading the initiative is unaware that computer science is fundamentally about trade offs. It's not always like this, but the bigger the company, the more statistically probable it is.

296 points by saikatsg 10mo ago

"Generic Software Design" as the author called it, is nice for setting the general direction of some implementation. This is why I like to read software engineering books. It's easier to solve a problem if you have some kind of framing to guide you. And it's easier to talk about the solution if everyone share the same terminology. But yes, the map is not the territory, and giving directions is not the same as walking the trail. The actual implementation can deviate from the plan drafted at the beginning of the project. A good explanation is found in Naur's Theory of Programming, where he says the true knowledge of the system is inside the head of the engineers that worked on it. And that knowledge is not easily transferrable.

DerArzt 10mo ago
Man I would kill for some direction from my "architects", even if it were a bit wrong. I'm at the point that I can't even get them to review my architectural diagrams demanded by my company to guide me on what's expected.

narag 10mo ago

The article incurs in the very same problem it's describing. It's generic advice that might not be appliable to specific situations.

kayo_20211030 10mo ago

> I don’t know if structural engineering works like this, but I do know that software engineering doesn’t. Structural Engineering (generally construction engineering) does work like that. Following the analogy, the engineers draw; they don't lay bricks. But, all the best engineers have probably been site supervisors at some point and have watched brick being layed, and spoken to the layers of bricks, etc. Construction methods change, but they don't change as quickly as software engineering methods. There is also a very material and applicable "reality" constraint. Most struct's knowledge/heuristics remains valid over long periods of time. The software engineers' body of knowledge can change 52 times in a year. To completely stretch the analogy - the site conditions for construction engineering are better known than the site conditions for a large software project. In the latter case the site itself can be adjusted more easily, and more materially, by the engineering itself i.e. the ground can move under your feet. Site conditioning on steroids! Ultimately, that's why I agree fully with the piece. Generic advise may be helpful, but it always applies to some generic site conditions that are less relevant in practice.

glitchc 10mo ago
It sounds like you are making the argument that there is no established way to generate good software. If that's the case, then software isn't engineering, but rather art. The former requires established/best practices to be called a discipline, while the latter is a creative endeavour.
- kayo_20211030 10mo ago
  That's true. I do. I consider it a creative art, with some disciplinary adjacency to engineering. The creative sculptor has to know the material stone in order to make anything good with it. But, construction engineering is creative too; just different.
- Scarblac 10mo ago
  I think we know how to reliably make good software. E.g. NASA manages. The problem is that doing it like that is much too expensive and too slow for most businesses.
  - pixl97 10mo ago
    >and too slow for most businesses. A lot of this is because while a 'good' business is waiting for the 'good' software to be written, some crappy business has already written the crappy software and sold it to all the customers you were depending on. In general customers are very bad at knowing the difference between good and bad software and typically buy what looks flashy or the sales people bribe them the most for.
  - kayo_20211030 10mo ago
    NASA is a bit of an outlier. In the 50's through the 70's any failure, particularly a failure involving the loss of a life, would have been a national catastrophe; a blow to national prestige. So, they were super careful that it didn't happen. The spent-cost was irrelevant compared to the reputational value at stake. Honestly, it was a wise investment given the operative quid pro quo in those days. Maybe they still do good software, I don't know, but I suspect that the value at risk today makes them more cost averse, and less sensitive to poor software. "Business" runs the same calculations. I'd posit that, as a practical matter, most businesses don't want "good" software; they want "good enough" software.

michaelt 10mo ago

I've always felt it's unrealistic to separate upfront architecture from implementation, because my experience is a lot of systems turn out to have requirements that are a lot more complex in reality than they might seem at first, even if you think quite hard about the requirements. Imagine if you worked for an online retailer like Amazon, and you were assigned to architect a change so you can add free sample items into customers' orders. Take a moment to think about how you'd architect such a system, and what requirements you'd anticipate fulfilling. In the next paragraph, I'll tell you what the requirements are. Or you can skip the next paragraph, the size of which should tell you the requirements are more complex than they seem. The samples must be items in the basket, so the warehouse knows to pick them. They must be added at the moment of checkout, because that's when the order contents and weight can change. Often a customer should receive a sample only once, even if they check out multiple orders - so a record should be kept of which customers have already been allocated a given sample. It should be possible to assign a customer the same sample multiple times, in which case they should receive it once per order until they've received the assigned number. Some samples go out of stock regularly, so the sample items should not be visible to the customer when they view their order on the website, but if shipped it should appear on their receipt to assure them they haven't been charged for it. Samples should never be charged for, even if their barcode is identical to something we normally charge for. If the warehouse is unable to ship the sample, the customer should not receive a missing-item apology or a separate shipment, and the record saying that customer has had that sample already should be decremented. If the warehouse can't ship anything except the sample, the entire orders should be delayed/cancelled, never shipping the sample alone. If a customer ordered three of an item and was assigned one sample item with the same barcode but the warehouse only had three items with that barcode in stock, something sensible should happen. One key type of 'sample' is first-time-customer gifts; internal documentation should explain that if the first order a customer places is on 14-day delivery and their second order is on faster delivery and arrives first, the first-order gift will be in the second order to arrive but that's expected because it's assigned at checkout. If the first-order-checked-out is cancelled, either by the customer or the warehouse, the new-customer gift should be added to the next order they check out. Some customers will want to opt out of free samples, those who do should not be assigned any samples. But the free sample system is also used by customer services to give out token apology gifts to customers whose orders have had problems, customers who've been promised a gift should receive it even if they've opted out of free samples. No reasonable person can design such a system upfront, because things like 'opt-out mechanism sometimes shouldn't opt you out' and 'more than one definition of a customer's first order' do not occur to reasonable people.

austin-cheney 10mo ago

I completely disagree with almost the entirety of the article. It’s all about prior experience building large things many times yourself, not using some framework or other external abstraction. When you have done this many times you absolutely can design a large application without touching the code. This is part planning and risk analysis experience and part architecture experience. You absolutely need a lot of experience creating large applications multiple times and going through that organizational grind but prior experience in management and writing high level plans is extremely helpful.

wduquette 10mo ago
You’re speaking of implementing yet another system of a familiar kind, I.e. a new project. The OP says that generic design works for new projects. He’s mostly talking about designing new features to be added to an existing system, in which case the design has to be contingent on the existing system.
- austin-cheney 10mo ago
  Software developers like to think they are special. They aren't. Software, from a planning perspective, is not much different than physical construction. When it comes to extending an existing application it really comes down to how well the base application was planned to begin with. In most cases the base application developers have no idea, because they either outsourced the planning to some external artifact or simply pushed through it one line at a time and never looked back. Either way the person writing the extension will be more concerned with the corresponding service data and accessibility than conformance to the base application code if it is not well documented and not well tested in a test automation scheme.
roguecoder 10mo ago
Why are you rewriting the same application a second time? I've personally yet to have a situation where that comes up. And every application I've ever worked on has its architecture evolve over time, as behavior changes and new domain concepts are identified. There are recurring patterns (one might even call them Design Patterns), but by the time we've internalized them we have even less need for up-front planning. Why write the doc when you can just implement the code?

eviks 10mo ago

> For instance: In large codebases, consistency is more important than “good design” But this is exactly the type of generic software design advice the article warns us about! And it mostly results in all all the bad software practices we as users know and love remaining unchanged (consistently "bad" is better than being good at least in some areas!)

karmakaze 10mo ago
Yeah that line gave me a twitch. Reading on though it's more about the resulting coherence and correctness rather than like the Ralph Waldo Emerson quote: "A foolish consistency is the hobgoblin of little minds, adored by little statesmen and philosophers and divines."
- kayo_20211030 10mo ago
  I agree. It's only the foolish consistency that's problematic. A sensible consistency does, as you say, provide a coherence. William James, who overlapped Emerson, has a lot to say about positive habits.
johnfn 10mo ago
I don’t know. At my place a lot of cowboy engineers decided to do things their own way. So now we have the random 10k lines written in Redux (not used anywhere else) that no one likes working with. Then there’s the part that randomly uses some other query library because they didn’t like the one we use in 95% of the code for some reason, so if you ever want to work with that code you need to keep two libraries in your head instead of one. Yes, the existing query library is out of date. Yes, the new one is better— in isolation. But having both is even worse than having the bad one!
- shayway 10mo ago
  It's essentially the same problem as https://xkcd.com/927/ https://xkcd.com/927/ [How Standards Proliferate]

augustk 10mo ago

In the best scenario the developers are also active users of the software they produce. Then a design flaw or an error that affects the users will also affect the developers and will (hopefully) motivate the latter to correct it.

roguecoder 10mo ago
XP putting a customer on the team was the best thing in the methodology. Replacing those with business representatives is one of Scrum's original sins.
- bitwize 10mo ago
  > XP putting a customer on the team was the best thing in the methodology. Recently my boss said to me: "Customers want something that WORKS. If you deliver something, and it doesn't work, what's the customer going to think?" The huge drawback to putting a customer on the team is that the customer probably doesn't want to know, let alone be involved with, how the sausage is made. They want a turnkey solution unveiled to them on the delivery date, all ready to go, with no effort on their part. Generally what you want is a customer proxy in that role, who knows or can articulate what the customer needs better than the customer themselves can. Steve Jobs was a fantastic example of someone who filled this role.
- augustk 10mo ago
  It's also worth noting that a customer is not necessarily a user. As a developer I don't care so much about the customer but I care wholeheartedly about the users.
octorian 10mo ago
Its also useful for developers to have a way of bypassing customer support to have direct visibility into what issues the actual users are experiencing. This can come in the form of browsing tickets, online forums, or social media. Often something that's easily brushed off by a support rep will ring a bell in the mind of a developer who has recently worked in the area of the code related to the issue.

watters 10mo ago

The second footnote acknowledges that the post is largely tautological.

dcre 10mo ago

Very impressed at the rate of high-quality interesting posts from this author.

fogleman 10mo ago

> The kinds of topic being discussed are not "is DRY better than WET", but instead "could we put this new behavior in subsystem A? No, because it needs information B, which isn't available to that subsystem in context C, and we can't expose that without rewriting subsystem D, but if we split up subsystem E here and here..." Hmm, sounds familiar... Bingo knows everyone's name-o Papaya & MBS generate session tokens Wingman checks if users are ready to take it to the next level Galactus, the all-knowing aggregator, demands a time range stretching to the end of the universe EKS is deprecated, Omega Star still doesn't support ISO timestamps https://www.youtube.com/watch?v=y8OnoxKotPQ https://www.youtube.com/watch?v=y8OnoxKotPQ

lkglglgllm 10mo ago
Wngman. Number of softwares not supporting iso8601, TODAY (no pun), is appalling. For example, git (claiming compatibility, but isn’t).
- deleted 10mo ago
  [deleted]
bitwize 10mo ago
This is the kind of situation you get into when you let programmers design the business information systems, rather than letting systems analysts design the software systems.
- QuercusMax 10mo ago
  I don't think I've ever worked on a project that had "system analysts". You might as well say "this is what happens when you don't allow sorcerers to peer into the future". Best I've ever had are product managers who maybe have a vague idea of what the customer wants.
  - bitwize 10mo ago
    Well, that's just the problem, innit. In decades past, systems analysts performed a vital function, viewing the business and understanding its information flows as a whole and determining what information systems needed to be implemented or improved. Historically, in well-functioning information-systems departments, the programmer's job was confined to implementation only. Programming was just a translation step, going from human requirements to machine readable code. Beginning in about the 1980s or so, with the rise of PCs and later the internet, the "genius programmer" was lionized and there was a lot of money to be made through programming alone. So systems analysts were slowly done away with and programmers filled that role. These days the systems analyst as a separate profession is, as you say, nearly extinct. The programmers who replaced the analysts applied techniques and philosophies from programming to business information analysis, and that's how we got situations like with Bingo, WNGMAN, and Galactus. Little if any business analysis was done, the program information flows do not mirror the business information flows, and chaos reigns. In reality, 65% of the work should be in systems analysis and design—well before a single line of code is written. The actual programming takes up maybe 15% of the overall work. And with AI, you can get it down to maybe a tenth that: using Milt Bryce's PRIDE methodology for systems analysis and development will yield specs that are precise enough to serve as context that an LLM can use to generate the correct code with few errors or hallucinations.

ilaksh 10mo ago

This is also the type of thing that makes having separate software architects that aren't actually maintaining the software generally a nonsensical idea. There are too many decisions, technical details, and active changes to have someone come in and give direction from on high at intervals. Maybe at the beginning it could make sense sort of, but projects have to evolve and more often than not discover something important early on in the implementation or when adding "easy" features, and if someone is good at doing software design then you may need them even more at that point. But they may easily be detrimental if they are not closely involved and following the rest of the project details.

o_nate 10mo ago
I guess I'm lucky not to have worked at a place with a role for software architects who don't actually write code. I honestly don't know how that would work. However, I think I can appreciate the author's point. Any sufficiently complex piece of existing software is kind of like a chess game in progress. There is a place for general principles of chess strategy, but once the game is going, general strategy is much less relevant than specific insights into the current state of play, and a player would probably not appreciate advice from someone who has read a lot of chess books but hasn't looked at the current state of the board.
roguecoder 10mo ago
The best "architects" serve as facilitators, rather than deciding themselves how software is built. They have to be reading the code, but they don't themselves have to be coding to be effective. You don't need one until you've got 30-70 engineers, but a strong group of collaborative architects is the most important thing for keeping software development effective and efficient at the 30-1,000 engineer range.

7402 10mo ago

> if you come up with the design for a software project, you ought to be responsible for the project’s success or failure I think this should also apply to people who come up with or choose the software development methodology for a project. Scrum masters just don't have the same skin in the game that lead engineers do.

nullorempty 10mo ago

> You can't design software you don't work on In 30 years in software dev, I am yet to see any significant, detailed and consistent effort to be extended into design and architecture. Most architects do not design, do not architect. Senior devs design and architect and then take their design to the architects for *feedback and approvals*. These senior devs make designs for features and only account for code and systems they've been exposed to. With an average employment term of 2 years most are exposed to a small cut of the system, which affects the depth and correctness of their design. And architects mostly approve, sometimes I think without even reading the docs. At most, you can expect the architects to give generic advice and throw a few buzzwords. At large, they feel comfortable and secure in their positions and mostly don't give a shit!

makeitdouble 10mo ago
> 2 years I've been thinking about this a lot. 2~3 years is a long time, long enough to have a pretty good grasp on what a code maintained by 50~100 does in pretty concrete terms, come up with decent improvement ideas, and see at least one or two structural ideas hit production. If the person then stays 1 or 2 more years they get a chance to further refine, but usually will be moved up the ladder Peter Principle style. If they get a chance to lead these architecture changes that company has a chance to be on a decent path technally speaking. I'm totally with you on the gist of it: architects will usually be a central switch arranging these ideas coming from more knowledgeable places. In the best terms I see their role as guaranteeing consistency and making sure teams don't impede each other's designs.
WillAdams 10mo ago
John Ousterhout has been addressing this one class at a time at Stanford for a while now, and has scaled up to a book: https://www.goodreads.com/en/book/show/39996759-a-philosophy-of-software-design https://www.goodreads.com/en/book/show/39996759-a-philosophy... Video overview at: https://www.youtube.com/watch?v=bmSAYlu0NcY https://www.youtube.com/watch?v=bmSAYlu0NcY
mmis1000 10mo ago
> 2 years I feel it's already enough to rewrite a big part of subsystem or change the whole thing into shit (depends on maintainer). Software today moves quite fast. 2 year is sometimes difference between a new company and a dead company

deleted 10mo ago

[deleted]

jkaptur 10mo ago

There are two extremes here: first, the "architects" that this article rails against. Yes, it's frustrating when a highly-paid non-expert swoops in to offer unhelpful or impossible advice. On the other hand, there are Real Programmers [0] who will happily optimize the already-fast initializer, balk at changing business logic, and write code that, while optimal in some senses, is unnecessarily difficult for a newcomer (even an expert engineer) to understand. These systems have plenty of detail and are difficult to change, but the complexity is non-essential. This is not good engineering. It's important to resist both extremes. Decision makers ultimately need both intimate knowledge of the details and the broader knowledge to put those details in context. 0. http://www.catb.org/jargon/html/story-of-mel.html http://www.catb.org/jargon/html/story-of-mel.html

MagicMoonlight 10mo ago

There is real irony in a blog post saying you can’t trust generic advice, which is itself a generic advice blog post, and links to other generic advice they have written.

Sankozi 10mo ago

"In large codebases, consistency is more important than “good design”" - this is completely opposite from my experience. There is some value in consistency within single module but consistency in a large codebase is a big mistake (unless in extremely rare case that code base consists entirely of very similar modules). Modules with different requirements should not have single consistent codebase. Testing strategy, application architecture, even naming should be different across different modules.

roguecoder 10mo ago

The problem isn't programmers: it is cheap-ass executives obsessed with compliance. Good software designers are facilitators. They don't tell people how to build software, but say "not like that" by making the technical requirements clear. They enable design to constantly change as the needs change. It has been a long time since I've been at a company willing to actually employ someone in that roll. They require that their most senior engineers be focused on writing code themselves, at the expense of the team and skill-building necessary for quality software. Instead we get bullshit like "team topologies" or frameworks that are more about how the company wants to manage teams than they are about how well the software works. We get "design documents" that are considered more important than working code. Even the senior engineers that are around aren't allowed to say "no" if it is going to interfere with some junior project manager's imagined deadline. Software companies are penny-wise and pound foolish, resulting in shittastic spaghetti messes with microservice meatballs.

mashally 10mo ago

True somehow, I see a lot of people who don't understand the business well cannot, absolutely, design a good software.

bitwize 10mo ago

The job of the big-picture software architect is not to give "generic software design advice". It's precisely to see the big picture: understand the information needs and flows of the business and determine WHAT needs to be built in order to serve those precise needs. Let the programmers worry about the details. That's their job and their strength: they are detailists who are fluent in the language of the machine, but their biggest drawback is, they tend to have difficulty seeing the big picture and understanding how those details fit into a greater whole. One does not need to be a programmer in order to be a great systems analyst/architect. Matter of fact it's the opposite: great analysts are good with people, and have a strong intuitive grasp of what people need in order to effectively run the business. Leaving that to programmers is a recipe for disaster, as without documentation of existing business systems and requirements and a solid design, programmers will happily build the wrong thing.

nik282000 10mo ago

And software engineers can't design cars that they don't drive. As evidenced by the millions of tons od e-waste rolling around on city streets these days.

guywithahat 10mo ago

As a personal anecdote in agreement with the article, I've seen design consultants come in with no industry experience and ruin projects. Friends of the VP with no satellite experience saying we need to introduce random exceptions everywhere who don't understand threading or type systems outside of Java. It's frustrating to see potential bugs introduced at the design level with no regard for how the end product operates

neopointer 10mo ago

Didn't read the article yet, but came here to say that I couldn't agree more with that sentence in the sense that the industry should stop making people "software architects", specially those that do not code at all but want to make design decisions. Software architect should be a role and not a position.

chanux 10mo ago
Don't want to generalize this to all but based on my experience I see it like this: The material architects build with is hollow Lego blocks. Engineers have to build with blocks that have complex machinery inside it.

wewewedxfgdf 10mo ago

I'm wary of absolute statements about programming.

kevinlearynet 10mo ago

The best applications I've ever been a part of building, measured by user satisfaction, are those where the engineers: 1. Value simple, effective systems 2. Understand all use cases, because they use it 3. Have enough freedom to fix small things as they find them #3 is controversial sometimes, but I believe this flexibility and creative freedom for devs leads to much happier people and much better products.

lunias 10mo ago

In my experience an "architect" is someone that used to write code (maybe 10-20 years ago - maybe only for a few years to begin with) but followed incentives away from the practice. Now they read blogs written by other architects about technologies that none of them have used. They champion the iterative process, but have 20 hard requirements up front that "make sure we can scale" (there are 5 users of this intranet app). They have 30 minutes free per day to produce any deliverables, all other time is spent in meetings. The deliverables produced are diagrams which have a few boxes and arrows pointing between them. Engineers are quick to ask, "Aren't those arrows backwards? We pull data from that service, right?". To which the architect replies without thought, "Ah, yeah, you're right. This is more of a work in progress, something to work off of." The next slide in their presentation is about database normalization and shows some data model that they hallucinated without looking at the data. The engineers chime in again, "Do we care more about the size of the data on disk or read times here?". The architect doesn't understand why you would challenge "best practices". You don't understand why the person presumably leading the initiative is unaware that computer science is fundamentally about trade offs. It's not always like this, but the bigger the company, the more statistically probable it is.