5 ms·
I'm of a mind to reduce documentation as much as possible, or keep it as vague as possible. That's because I came from an environment that insisted on incredib
by ChrisMarshallNY 6y ago
I'm of a mind to reduce documentation as much as possible, or keep it as vague as possible.
That's because I came from an environment that insisted on incredibly detailed, formal, approved-by-everyone-including-the-mens-room-attendant, documents.
These became "concrete galoshes"[0] that turned what should have been an agile, iterative project into a waterfall behemoth that cost a mint, took forever to make, and delivered a result that no one wanted.
I suspect that my current process[1] would not be acceptable for many organizations, as it basically requires that everyone involved be extremely experienced, and kept together as a team for years.
I've actually come to realize that "tribal knowledge" is not the boogeyman that its made out to be; but is a way to get very high-quality, rapid, adaptive, development done. The main issue is that it does require good, empathetic management, long-term retention of experienced, capable people, and an "apprenticeship" model. These are not popular concepts in today's corporate world (at least, in the US).
[0] https://medium.com/chrismarshallny/concrete-galoshes-a5798a55af2a https://medium.com/chrismarshallny/concrete-galoshes-a5798a5...
[1] https://medium.com/chrismarshallny/forensic-design-documentation-54541925d30e https://medium.com/chrismarshallny/forensic-design-documenta...
- lhorie 6y agoI think something people need to be aware of is that the engineering culture at places like Google is extremely different than, say, a bank. Design docs work well for Google because it has decentralized, independent engineering teams composed mostly of reasonable people, with few or no mandates from outside their teams. They tend to fail at companies with underperforming, micro-managing, and/or overreaching teams (e.g. unionized project managers that have literally nothing to do, bizdev telling developers what features to implement (down to specifics), QA teams who are incentivized to find "flaws" ad-infinitum to look busy, etc). But I see this as a symptom of a fundamental dissonance in goals. Design documents are used by organizations whose goal is to build/change things. Many organizations are averse to change, and their processes reflect that, through overly microscopic documentation, convoluted approval chains, etc.
- ChrisMarshallNY 6y agoYeah, you are correct. But I have become rather cynical about the motivations of a lot of organizations, these days. Everyone seems to be promoting an environment where there's a constant circulation of relatively inexperienced (not always low-paid, but inexperienced) developers, staying at companies for short periods of time, then moving on. This requires a codified, ingrained structure that needs to be documented and supported. It applies to modern, "agile" companies, as much as it does to more traditional, "hidebound" corporations. People that come in need to be onboarded quickly, squeezed for every ounce of productivity possible, then let go, when they are no longer useful. This is supported by the employees, as much as by the managers. The salaries can be quite high; especially for people that are quick to adapt and come up to speed. But there is absolutely no substitute for an experienced, cohesive team that has developed a shared vocabulary and focus. I was fortunate to be a member of such a team. The person with the least seniority on the team had a decade. Everyone had over 30 years' experience in software development, and we got some fairly intense stuff done. Sadly, we were still subject to the kind of structure I mentioned above, and a lot of our product ended up being housed in substandard wrappers (we did "engine" code).
- bigbizisverywyz 6y ago>I think something people need to be aware of is that the engineering culture at places like Google is extremely different than, say, a bank. I have to say that this is not uniformly true, I've worked for a few financial institutions, and usually follow some sort of design process before embarking on any sizable work (as mentioned above, I find the RUP process to be a really useful framework). More often that not the design process is usually welcomed because people are happy that someone is willing to create a holistic view of the problem (from stakeholders down to db schema and class diagrams). The process helps with project management, identifies stakeholders and describes the support requirements after transition to prod. Banks actually love somebody to detail all that. In a few cases I'm aware of, my design docs are still referred to years after I've moved on since they capture the most important thing that everybody forgets - basically the 'why' that certain decisions were made, and the alternatives rejected. A good architecture diagram never goes out of date either :)
- shadowgovt 6y ago> basically requires that everyone involved be extremely experienced, and kept together as a team for years Yep, that's the issue. Businesses can't codify that approach into a process because they can't make any guarantees around that. Your approach is "conclave of wizards." It's a high-output approach that is the right thing for some problem domains, but (a) it isn't repeatable and (b) it isn't sustainable (if a critical mass of wizards leave the conclave, what is the company to do? Give up and shut down?)
- ChrisMarshallNY 6y ago> Give up and shut down? In some cases, yes. That's exactly what needs to happen. If the business model is dependent upon the "conclave of wizards," then they are a critical path resource, just like a major supplier or business partner, which, if withdrawn, could doom the company just as certainly. If the corporation refuses to treat that resource as a valuable, critical resource, then they don't deserve to stay in business. Managing that type of team is not something that is really taught in school. It comes from experience, and also requires a level of empathy that is, quite honestly, almost nonexistent in today's business culture.
- shadowgovt 6y agoIndeed. So the correct thing for a business to do for both its short- and long-term survival (since we can't know a-priori whether the conclave of wizards will be stable) is to minimize the odds of forming one in the first place. Businesses are structures that minimize risk. A core set of specialists who can't be replaced without tanking the business's performance is a risk. And a hard-to-quantify one at that. Unless, of course, the business is not setting out to minimize risk, but instead to shield some less-risky structure from outsized risk exposure. Like, say, a startup company funded by angel investors trying out experimental technologies in the hopes of hitting on a jackpot new system that a big corp could then buy. If they fail, only the angel investor money is gone. If they succeed, a risk-averse business can buy a valuable asset. But the first thing that business will want to do to maintain its long-term risk profile is fire the wizards. ;)
- smichel17 6y agoMeta: Why do you link to medium when the articles are also posted on your website, which I assume is the canonical version if there were a discrepancy? Also, I reviewed your privacy policy and while I am not a lawyer, I believe your cookie popup is not unnecessary. As far as I understand GDPR, it is not necessary to ask for consent separately if the cookies you're setting are strictly used to fulfil an explicit request. Eg, if a user checks "stay logged in", you don't need to ask separately to set a "wordpress_logged_in_xxxxx" cookie; the user already gave their consent by checking the box. Same with a session cookie when they log in, or a cookie for persisting settings. Not to mention, I there's no prominent log in link, so I'm guessing you are the only one who does that, in which case you'll never be setting cookies on other people's computers anyway :) These things still need to be described in your privacy policy, but you don't need to bother users with them; you only need the consent box when you're using cookies for something the user doesn't explicitly request, like tracking (regardless of whether it's a discrete tracking cookie or you're using their login cookie to do it). I didn't see any of these things in your website privacy policy. Not needing to bother your readers with consent popups is a great advantage that the GDPR gives privacy-respecting websites like yours!
- ChrisMarshallNY 6y agoThanks! That tip about the popup is good. I'll remove it. I would be happy to link to my regular site, but I assume that people prefer Medium; I guess because of the standardized layout (which I don't love), and the ability to use apps and semantic API hooks. There are a few series on my site that are not available on Medium [0][1][2][3][4][5] [0] https://littlegreenviper.com/series/swiftwater/ https://littlegreenviper.com/series/swiftwater/ (I republished one of these on Medium). [1] https://littlegreenviper.com/series/spm/ https://littlegreenviper.com/series/spm/ [2] https://littlegreenviper.com/series/bluetooth/ https://littlegreenviper.com/series/bluetooth/ (Which was actually #1 on HN for a little while -St. Paddy's Day) [3] https://littlegreenviper.com/series/bluetooth-2/ https://littlegreenviper.com/series/bluetooth-2/ [4] https://littlegreenviper.com/series/streaming/ https://littlegreenviper.com/series/streaming/ [5] https://littlegreenviper.com/series/stylist/ https://littlegreenviper.com/series/stylist/ (Which is a decade old)
- skipants 6y ago> The main issue is that it does require good, empathetic management, long-term retention of experienced, capable people, and an "apprenticeship" model. To me this sounds like an idealistic environment. I doubt this environment could fail with any sort of process (or lack thereof). Documents don't have to be "incredibly detailed, formal, approved-by-everyone-including-the-mens-room-attendant". Those are probably the worst kind. I do think that OP's/Googles design docs does seem to lean towards that, unfortunately. I wonder if that's how it is in practice? Especially given the length -- 10 to 20 pages sounds like it's too detailed for me. Though I suppose on bigger projects the sum of all the implementations of the moving parts can get quite large. One criticism I especially have of OP's "Review" section is they mention opening it up to a large audience. I am skeptical whether that is how it actually goes down. In my experience, opening something up to review by many people just ends up being a too-many-cooks situation. Pretty much what you're afraid of with "approved-by-everyone-including-the-mens-room-attendant". Those situations just turn into bikeshedding. In reality, I bet it gets opened up to everyone but only a couple people are actually expected to be the ones who give feedback on it. I still think I'd take the over-formalized process over tribal knowledge. What you don't see behind the scenes of tribal knowledge are people with a lot of it who constantly get interrupted to answer those questions. It ends up being very inefficient at scale. Not to mention the overhead that comes with the person trying to answer the question -- like finding old breadcrumbs about the problem like code or emails. Along with the worst enemy of work for a developer: context switching. I've been the person with tribal knowledge and it basically becomes your full time job. It was great for job security but I lament the amount of time I lost because I wasn't more proactive about writing documentation.
- ChrisMarshallNY 6y ago> What you don't see behind the scenes of tribal knowledge are people with a lot of it who constantly get interrupted to answer those questions. It ends up being very inefficient at scale. Not to mention the overhead that comes with the person trying to answer the question -- like finding old breadcrumbs about the problem like code or emails. Absolutely correct, which is where that "good management" comes in. I managed the team that I mentioned, earlier, and I'd like to think that I did it well. My single biggest task was to shield people from corporate shenanigans, take on as much of the structural overhead as possible, and be a "gatekeeper." I managed the team for 25 years. We got a lot done, in that time, but also suffered a lot of "lessons learned." I certainly made my share of mistakes; most of which were about me deprecating my "management shield" responsibilities, in misguided attempts to be a working team member. I kept my tech chops up in extracurricular open-source work. No one wanted to pay me for my tech skills.
- bo1024 6y agoThat environment reminds me of Conway's law[1] - code structure reflects organizational structure. It sounds like the problem was decisionmaking power was spread out among far too many people. Detailed documentation was a side effect. Removing the documentation likely wouldn't have fixed the problem. [1] https://en.wikipedia.org/wiki/Conway%27s_law https://en.wikipedia.org/wiki/Conway%27s_law