12 ms·

> "Software Is about Developing Knowledge More than Writing Code" The company I work for uses Scrum. They consider the User Stories + the code to be everything

> "Software Is about Developing Knowledge More than Writing Code" The company I work for uses Scrum. They consider the User Stories + the code to be everything you need. I struggle with this, but my manager says they don't want to get tied up doing documentation "because it goes out of date". Beside, they are being Agile which "prefers working code over comprehensive documentation". I am wondering what other companies do to capture this "distilled knowledge". The backend services I rely on are undocumented beside some paltry swagger that leaves much to be desired. The front end has no product-level "spec", if you want to rebuild the thing from scratch. There isn't even a data dictionary, so everyone calls the same thing by different terms (in code, and conversation). There are just user stories (thousands) and code. Does anyone have any suggestions on how to fix this?

gnaritas 7y ago
Knowledge is captured in people, not code. The code is byproduct of the people, the people who are the asset.
tannhaeuser 7y ago
This is not necessarily in opposition to what TFA states: that programming is about knowledge engineering. It just happens that knowledge flows through user stories and code. This might work well or not so well for your team, and there's nothing to fix here. Managers should be aware, though, that this means you no longer have a project, but a line organization put up indefinitely, and where when the teams is disbanded, no "product" as such does remain; nor would it be possible to hand-over the "project" to an offshore team.
PaulRobinson 7y ago
"prefers working code over comprehensive documentation" does not mean "don't do documentation". Documentation is essential. How things work is an important thing to document. Ideally it should be in version control and be generated from the code, because then it's less likely to go out of date. It still has problems (What do you do when the code and the documentation disagree? Which is correct?), but they're not as severe as the problems that arise when there is no documentation at all. What is less useful is having comprehensive documentation for those things that are yet to exist. Writing a few hundred pages of specification and handing it over to the dev team is waterfall, and it is _this_ that the Agile manifesto signatories were interested in making clear. I'd fix it with strategic DDD - I'd develop at least a "ubiquitous language" (or a UL): I'd get others to work with me on having clear terminology and making sure that is used consistently both in user stories and in the code base. That's table stakes. I'd then event storm the contexts I'm working in and start to develop high level documentation. Even at this point relationships between systems emerge, and you get to draw circles around things and name them (domains, contexts), and the UL gets a bit better. At this point you can start to think about describing some of your services using the UL and the language of domains and contexts. By that point, people should start to click that this makes life easier - there is less confusion, and you're now all working together to get a shared understand of the design and the point of DDD is that the design and the code match. The first part (all 100+ pages of it), of the Millet and Tune book on DDD will pay you dividends here. If that doesn't work, look around for somewhere else to work that understands that software development is often a team sport and is committed to making that happen.
- clumsysmurf 7y ago
  Thanks for the thoughtful response, this is helpful. > Ideally it should be in version control and be generated from the code .. May I ask if you have suggestions for tooling to capture the high level documentation. We use javadoc a little, but it seems best for lower level reference. Also for diagrams, like sequence diagram and/or state machines, how do you capture this? Thanks.
  - konstmonst 7y ago
    Use graphviz (dot tool for example) for state machines. It is a text format where you list state machine transitions and it generates a visual representation. Or better yet: generate your state machines from same format you would use to generate visual representation.
  - PaulRobinson 7y ago
    Don't be afraid of building a product specification, and doing it in Markdown and auto-generating a mini-website out of it. Just build a product specification for how the product works (which is useful documentation), not how the product will work (which is waterfall). We're experimenting with this a little, and I'm getting into document-driven development a little: if the product spec is in markdown, why not create a pull request on it as part of your story/project planning that shows the changes that would happen as a consequence of your work. Once the story is done, you can merge the pull request, even. We're not quite there with this yet, but I'm optimistic. Putting design assets into your repo is also acceptable, and also paying time and attention to commit messages can be really, really helpful. I love this talk, for example: https://brightonruby.com/2018/a-branch-in-time-tekin-suleyman/ https://brightonruby.com/2018/a-branch-in-time-tekin-suleyma...
- JohnBooty 7y ago
  Documentation is essential. How things work is an important thing to document. Ideally it should be in version control and be generated from the code, because then it's less likely to go out of date. My solution to this is old and fairly unpopular, but I stand by it: anything in the codebase that's not obvious to a new maintainer should have a brief, explanatory code comment. Generally, this falls into two categories. 1. Hacks/kludges to get around bugs in hardware, external services, or included libraries. These manifest in code as incomprehensible, ugly bits of code that are difficult to distinguish from code that is simply "sloppy" or uninformed. More importantly, they represent hard-won knowledge. It often takes many programmer-hours to discover that knowledge, and therefore many dollars. Why throw it away? (Tip: include the version of the dependency in the comment, ie) # work around bug in libfoo 2.3, see blahblahblah.com/issues/libfoo/48987 for info # should go away once we can upgrade to libfoo 3.. if error_code == 42 reset_buffer() ...so that future programmers (including you) can more easily judge whether the kludge is still needed in the future. 2. Business logic. This too is difficult/impossible to discern from looking at code. Often, one's git commit history is sufficient. But there are any number of scenarios where version control history can become divorced from the code, or require a fair bit of git/hg/svn/whatever spelunking to access. And this of course becomes increasingly onerous as a module grows. If there are 200 lines of code in a given module, it is a significant time investment to go git spelunking for the origins of all 200 lines of code. Some concise internal documentation in the form of code comments can save an order of magnitude or two of effort. It still has problems (What do you do when the code and the documentation disagree? Which is correct?), but they're not as severe as the problems that arise when there is no documentation at all. This is pretty easy to enforce at code review time, prior to merging. In the first place, only a true maniac would intentionally update # no sales tax in Kerplakistan on Mondays return nil if country_code==56 and day_of_week==1 ...without updating the associated comment. If they do neglect to update it, that's an easy catch at review time.
jacquesm 7y ago
> Does anyone have any suggestions on how to fix this? Not to be rude, but yes: switch employers. This is not something you can fix on the employee level, it is a management issue.
- JohnBooty 7y ago
  Yes. I've never seen unenlightened management somehow become enlightened.
- ikll 7y ago
  Sometimes it's even more complex than a management issue. It might be a team culture or company culture issue, and even radical changes in the management are not enough to fix it.
ivanhoe 7y ago
I've been on a couple of projects like that, and in my experience the real reason behind this logic is that manager and product owner this way may make themselves indispensable, they can't ever get fired without practically killing the project (until it becomes completely unmaintainable and slowly dies off)
- ldng 7y ago
  And then they leave
- potta_coffee 7y ago
  It could be worse, the product owner could be killing the project by their presence instead.
goatinaboat 7y ago
prefers working code over comprehensive documentation This is funny because “working code” might just mean that it doesn’t crash. But does it actually do what it’s supposed to do or does it reliably deliver the wrong results? How would you know without documentation? The software in the Therac didn’t crash, it quite reliably killed people with its “working code”.
- dawidw 7y ago
  Using that logic, what would be "working car"? When wheels don't fall apart? So I think "working code/application/program" is when it does what it is supposed to do. Including not crashing.
  - deathanatos 7y ago
    > So I think "working code/application/program" is when it does what it is supposed to do. And the point of the comment you're replying to is to ask what is "what it is supposed to do". How do you know what the answer to that question is, without documentation or a specification? And if you try to rely on just verbal communication, in a group of people probably larger than about 1, they're going to have different ideas about what the software is supposed to do. Some of the most challenging problems I've encountered have been looking at code that does something. What it does is clear enough from the code. But why it does it, or should it do that, that is much harder to answer, particularly if the person who wrote it has left the company or it's been >6 months and they just don't remember.
- frobozz 7y ago
  I can't picture a single scenario in which "doesn't crash" is the sole criterion by which code is evaluated as "working"
tr3ndyBEAR 7y ago
Get a new manager
einhverfr 7y ago
My experience is you want to move fast by reusing good software. This means that well understood components should be well engineered and documented components, while poorly understood problems might reserve documentation for the future. Agility requires a stable foundation. And a lot of places forget that.
- jkepler 7y ago
  Isn't this the old Unix tools idea, where programs are tools that ideally do one thing well, and with good inter-program communications developers can combine basic tools to build more complex programs?
  - einhverfr 7y ago
    This was also the original idea of Object Oriented Programming before people thought the real world was a good analogy.
meerita 7y ago
I've worked in a company where I enforced as a PM to write Technical Document of every project we did along with the PID, the Functional Document (design and interactions). That Technical Document was all about to describe the essentials, and not go deeper into every single class written. It was intended for those developers who wanted to join the team and update that old project. With it was useful document and barely updated.
madeofpalk 7y ago
> They consider the User Stories + the code to be everything you need If you have sufficiently detailed user stories, they can be.
- arethuza 7y ago
  When does a "sufficiently detailed user story" become "documentation"?
  - sailfast 7y ago
    When it's only part of the ticket? Some teams call everything a "user story" when it might actually be it starts with the story format then adds a whole bunch of detailed acceptance criteria, the background for the story, etc.
- mjlangiii 7y ago
  The user story is a promise to have a conversation. I think that is usually well understood. From there I think you can fall into two camps: that conversation should result in a Jira/whatever ticket with all the requisite documentation for an agile team versus that conversation IS the essential information required to properly build the expected valuable working software. Back to the question - what do you do about poor knowledge transfer in a project? I think a moderate de-emphasis on thinking of the user story text and the additional info like acceptance criteria etc. as self-sufficient documentation and adding more emphasis on that close relationship between developer, user, and maybe a tester, can help fill in big knowledge gaps.
dkersten 7y ago
Many years ago, I worked for a company where we were writing complex distributed telecom software and they had a wiki for documenting the system. I spent a few weeks meticulously documenting everything I did and anything that was touched by it (including defining all of the industry jargon and such). It was a great way to get a quick understanding of any part of the system, but I was the only person keeping it up to date so after a while I ran out of steam and stopped doing it. :( I've come across the "documentation becomes quickly outdated" argument a lot, but nobody has ever been able to suggest a good alternative. The best I've found is to write design logs for proposed changes (which you then let other team members/stakeholders can review/comment on before it gets implemented) and decision logs for any decisions that are made. This way, them going out of date is expected and ok, as they become a history of ideas and decisions with their context and outcomes laid out. You don't necessarily have a snapshot of "the system right now" but you have a log of all the ideas and decisions that lead up to the current system.
- sethammons 7y ago
  We create "blueprints" that have system design and goals on new projects and how they fit into the ecosystem. We are supposed to update them after they go into production as things change, but seldom do. Still, going back and reviewing a design document helps.
- whydoyoucare 7y ago
  I believe it is equally important to determine the level of abstraction for the documentation, such that updating it is an infrequent task (essentially, every line of code change should not trigger a document change). It is easier said than done, but that's the best compromise I have arrived it. At the very least, the document should capture high-level (again this is relative term) design, possibly an architecture diagram of major interacting functional units. The success measure should be the relative ability to build a mental model of the system by looking at this document for any newbie. That design document would be a start, and most likely not "quickly outdated". My personal beef with the agile camp is precisely this: when they let go of documentation, they don't do the design doc as well, and all that remains of the system is thousands of incoherent stories and huge amount of code.
erikpukinskis 7y ago
In Scrum, you don’t write code based off User Stories. The scrum team agrees to a set of Stories for the sprint, and then the scrum team breaks those stories into a set of Tasks which are the actual work that must be done. The User Stories are just something the product owner uses to show stakeholders that the project was successful.
- eropple 7y ago
  1) In the Hermes Conrad sense, this is technically correct. 2) In my experience, this basically never happens. Your comment encapsulates a lot of what I have come to call "Scrumbutt." It's Scrum, but. And while I have no idea if it's intended on your part, the sentiment is a fantastic way for a Scrum consultant--only some shade thrown; I've been a "DevOps consultant" before, after all--to come in and pull from deep in their Scrumbutt something to the effect of "you're doing it wrong, Scrum has not failed, you have failed Scrum." Within epsilon of nobody does Scrum "as prescribed"--because the amount of responsibility that must be undertaken at all levels is virtually impossible to get full buy-in on--and as such the boil on our collective behind that it is persists because criticism is immediately bedeviled by Scotsmen of unknown provenance.
  - erikpukinskis 7y ago
    You’re projecting “shade thrown”. People might be interested in what Scrum is. I know I am. That’s why I pointed out the error. It was a shock to me to learn I wasn’t doing anything close. Readers can do with the info what they want. I’m not sure if I can say the same of your comment. You seem to be trying to make me feel bad for commenting? Or accusing me of hawking pointless info for consulting fees? I really can’t tell.
wolco 7y ago
Change managers/leads
sailfast 7y ago
Traceability is where I'd start. First, ensure your stories are being linked to your code / pull requests / issues. That way you can figure out why something was changed in the future. This is key to determining whether you can change something down the road. Stories directly traceable to code can be powerful for capturing knowledge. I might also recommend creating user stories for non-feature development like infrastructure and tech debt paydown (if you don't already). That way, all of the value flow is captured in one place and you're not just leading managers to see new features only. Second, in addition to the user stories I'd advocate for strong background information about the context of the story as well as detailed acceptance criteria if you don't have that in place already.
mindcrime 7y ago
They consider the User Stories + the code to be everything you need Then they are Doing It Wrong™. Note that there's nothing in the Agile Manifesto OR the Scrum Guide that says "don't write documentation." The closest you get is in the AM where it says "We have come to value ... Working software over comprehensive documentation". But note that immediately after that it says "That is, while there is value in the items on the right, we value the items on the left more." IOW, the Agile Manifesto explicitly endorses the value of documentation! Remember this the next time somebody tries to tell you that "we don't do documentation because we're Agile." Anybody running that line is Full Of Shit™.
Viliam1234 7y ago
> Does anyone have any suggestions on how to fix this? Have a product wiki (e.g. MediaWiki). Have documentation in source code that compiles to HTML code, which can be linked to/from the product wiki (e.g. JavaDoc in Java, Natural Docs for languages that do not directly support compilable documentation). Make building and publishing this documentation a part of the continuous integration. When you have this, make it a part of code reviews to ask "where is this documented?" for those kinds of things that are easy to remember today, but no one will remember it a few months later. In other words, make it a "code+doc review". (Don't be dogmatic about whether the information should go to code documentation, unit test documentation, or wiki. Use common sense. If it only related to one method, it's probably the code; if it related to a use case, it's probably the unit test that verifies that use case; if it is a general topic that has an impact on many parts of the program, it probably deserves a separate wiki page.)
- jkepler 7y ago
  > Have documentation in source code that compiles to HTML code, which can be linked to/from the product wiki (e.g. JavaDoc in Java, Natural Docs for languages that do not directly support compilable documentation). Make building and publishing this documentation a part of the continuous integration. Are you referring to something like Knuth's Literate Programming (en.m.wikipedia.org/wiki/Literate_programming)? As a non-professional who's learning to develop on the side, something that follows more of a natural language approach appeals to me, as sometimes I have a few months between working on my project, and comments on my source code help me not to forget why I do certain things in the code. However, I'm not doing Literate Programming, just python with comments.
  - Viliam1234 7y ago
    No. I think one should organize the code as the code needs to be organized, and the documentation can either follow along (if it describes parts of code contained within the same file) or be placed separately (if it relates to multiple files), where "separately" could still be a package-level JavaDoc, or an external wiki. I have never tried the Literal Programming, so perhaps I am out of my depth here, but I strongly suspect it only works after one has already mastered the usual ways of programming. That you do not have to structure the code qua code, because you can already do it in your head. But it's hard to imagine what one has never done before. For example, if you never tried programming the usual way, how do you know when and why to put "Header files to include" in your Literal code? It's only because you can imagine the constructed code, you know where the header files go in the result, so you know where to place them in the Literal version. Otherwise, it would look quite arbitrarily. I don't know about documentation in Python, but the JavaDoc (and Natural Docs) work like this: You put comments to classes and methods, or packages (and files), along with the code. So you can read them and write them while you are looking at the code. But then you run a "documentation compiler" that extracts the comments and builds a separate HTML website out of them. Here you can browse and read about what the individual classes and methods do. The idea is to make this a part of the continuous integration, so that whenever you update the source code and the related comment, the HTML website also gets updated. Java supports this out of the box. When you install the Java compiler, you also install the Java documentation compiler. When you read the official documentation to the standard Java classes, those were made using exactly the same tools you are encouraged to use. I don't know whether Python has something like this. If yes, go ahead and use it. If not, look at Natural Docs -- it is a system to provide this functionality to languages that do not support it out of the box. Just try it: document a part of your existing project, compile the docs, and see whether you can imagine any value in reading that.

gnaritas 7y ago

Knowledge is captured in people, not code. The code is byproduct of the people, the people who are the asset.

tannhaeuser 7y ago

This is not necessarily in opposition to what TFA states: that programming is about knowledge engineering. It just happens that knowledge flows through user stories and code. This might work well or not so well for your team, and there's nothing to fix here. Managers should be aware, though, that this means you no longer have a project, but a line organization put up indefinitely, and where when the teams is disbanded, no "product" as such does remain; nor would it be possible to hand-over the "project" to an offshore team.

PaulRobinson 7y ago

"prefers working code over comprehensive documentation" does not mean "don't do documentation". Documentation is essential. How things work is an important thing to document. Ideally it should be in version control and be generated from the code, because then it's less likely to go out of date. It still has problems (What do you do when the code and the documentation disagree? Which is correct?), but they're not as severe as the problems that arise when there is no documentation at all. What is less useful is having comprehensive documentation for those things that are yet to exist. Writing a few hundred pages of specification and handing it over to the dev team is waterfall, and it is _this_ that the Agile manifesto signatories were interested in making clear. I'd fix it with strategic DDD - I'd develop at least a "ubiquitous language" (or a UL): I'd get others to work with me on having clear terminology and making sure that is used consistently both in user stories and in the code base. That's table stakes. I'd then event storm the contexts I'm working in and start to develop high level documentation. Even at this point relationships between systems emerge, and you get to draw circles around things and name them (domains, contexts), and the UL gets a bit better. At this point you can start to think about describing some of your services using the UL and the language of domains and contexts. By that point, people should start to click that this makes life easier - there is less confusion, and you're now all working together to get a shared understand of the design and the point of DDD is that the design and the code match. The first part (all 100+ pages of it), of the Millet and Tune book on DDD will pay you dividends here. If that doesn't work, look around for somewhere else to work that understands that software development is often a team sport and is committed to making that happen.

clumsysmurf 7y ago
Thanks for the thoughtful response, this is helpful. > Ideally it should be in version control and be generated from the code .. May I ask if you have suggestions for tooling to capture the high level documentation. We use javadoc a little, but it seems best for lower level reference. Also for diagrams, like sequence diagram and/or state machines, how do you capture this? Thanks.
- konstmonst 7y ago
  Use graphviz (dot tool for example) for state machines. It is a text format where you list state machine transitions and it generates a visual representation. Or better yet: generate your state machines from same format you would use to generate visual representation.
- PaulRobinson 7y ago
  Don't be afraid of building a product specification, and doing it in Markdown and auto-generating a mini-website out of it. Just build a product specification for how the product works (which is useful documentation), not how the product will work (which is waterfall). We're experimenting with this a little, and I'm getting into document-driven development a little: if the product spec is in markdown, why not create a pull request on it as part of your story/project planning that shows the changes that would happen as a consequence of your work. Once the story is done, you can merge the pull request, even. We're not quite there with this yet, but I'm optimistic. Putting design assets into your repo is also acceptable, and also paying time and attention to commit messages can be really, really helpful. I love this talk, for example: https://brightonruby.com/2018/a-branch-in-time-tekin-suleyman/ https://brightonruby.com/2018/a-branch-in-time-tekin-suleyma...
JohnBooty 7y ago
Documentation is essential. How things work is an important thing to document. Ideally it should be in version control and be generated from the code, because then it's less likely to go out of date. My solution to this is old and fairly unpopular, but I stand by it: anything in the codebase that's not obvious to a new maintainer should have a brief, explanatory code comment. Generally, this falls into two categories. 1. Hacks/kludges to get around bugs in hardware, external services, or included libraries. These manifest in code as incomprehensible, ugly bits of code that are difficult to distinguish from code that is simply "sloppy" or uninformed. More importantly, they represent hard-won knowledge. It often takes many programmer-hours to discover that knowledge, and therefore many dollars. Why throw it away? (Tip: include the version of the dependency in the comment, ie) # work around bug in libfoo 2.3, see blahblahblah.com/issues/libfoo/48987 for info # should go away once we can upgrade to libfoo 3.. if error_code == 42 reset_buffer() ...so that future programmers (including you) can more easily judge whether the kludge is still needed in the future. 2. Business logic. This too is difficult/impossible to discern from looking at code. Often, one's git commit history is sufficient. But there are any number of scenarios where version control history can become divorced from the code, or require a fair bit of git/hg/svn/whatever spelunking to access. And this of course becomes increasingly onerous as a module grows. If there are 200 lines of code in a given module, it is a significant time investment to go git spelunking for the origins of all 200 lines of code. Some concise internal documentation in the form of code comments can save an order of magnitude or two of effort. It still has problems (What do you do when the code and the documentation disagree? Which is correct?), but they're not as severe as the problems that arise when there is no documentation at all. This is pretty easy to enforce at code review time, prior to merging. In the first place, only a true maniac would intentionally update # no sales tax in Kerplakistan on Mondays return nil if country_code==56 and day_of_week==1 ...without updating the associated comment. If they do neglect to update it, that's an easy catch at review time.

jacquesm 7y ago

> Does anyone have any suggestions on how to fix this? Not to be rude, but yes: switch employers. This is not something you can fix on the employee level, it is a management issue.

JohnBooty 7y ago
Yes. I've never seen unenlightened management somehow become enlightened.
ikll 7y ago
Sometimes it's even more complex than a management issue. It might be a team culture or company culture issue, and even radical changes in the management are not enough to fix it.

ivanhoe 7y ago

I've been on a couple of projects like that, and in my experience the real reason behind this logic is that manager and product owner this way may make themselves indispensable, they can't ever get fired without practically killing the project (until it becomes completely unmaintainable and slowly dies off)

ldng 7y ago
And then they leave
potta_coffee 7y ago
It could be worse, the product owner could be killing the project by their presence instead.

goatinaboat 7y ago

prefers working code over comprehensive documentation This is funny because “working code” might just mean that it doesn’t crash. But does it actually do what it’s supposed to do or does it reliably deliver the wrong results? How would you know without documentation? The software in the Therac didn’t crash, it quite reliably killed people with its “working code”.

dawidw 7y ago
Using that logic, what would be "working car"? When wheels don't fall apart? So I think "working code/application/program" is when it does what it is supposed to do. Including not crashing.
- deathanatos 7y ago
  > So I think "working code/application/program" is when it does what it is supposed to do. And the point of the comment you're replying to is to ask what is "what it is supposed to do". How do you know what the answer to that question is, without documentation or a specification? And if you try to rely on just verbal communication, in a group of people probably larger than about 1, they're going to have different ideas about what the software is supposed to do. Some of the most challenging problems I've encountered have been looking at code that does something. What it does is clear enough from the code. But why it does it, or should it do that, that is much harder to answer, particularly if the person who wrote it has left the company or it's been >6 months and they just don't remember.
frobozz 7y ago
I can't picture a single scenario in which "doesn't crash" is the sole criterion by which code is evaluated as "working"

tr3ndyBEAR 7y ago

Get a new manager

einhverfr 7y ago

My experience is you want to move fast by reusing good software. This means that well understood components should be well engineered and documented components, while poorly understood problems might reserve documentation for the future. Agility requires a stable foundation. And a lot of places forget that.

jkepler 7y ago
Isn't this the old Unix tools idea, where programs are tools that ideally do one thing well, and with good inter-program communications developers can combine basic tools to build more complex programs?
- einhverfr 7y ago
  This was also the original idea of Object Oriented Programming before people thought the real world was a good analogy.

meerita 7y ago

I've worked in a company where I enforced as a PM to write Technical Document of every project we did along with the PID, the Functional Document (design and interactions). That Technical Document was all about to describe the essentials, and not go deeper into every single class written. It was intended for those developers who wanted to join the team and update that old project. With it was useful document and barely updated.

madeofpalk 7y ago

> They consider the User Stories + the code to be everything you need If you have sufficiently detailed user stories, they can be.

arethuza 7y ago
When does a "sufficiently detailed user story" become "documentation"?
- sailfast 7y ago
  When it's only part of the ticket? Some teams call everything a "user story" when it might actually be it starts with the story format then adds a whole bunch of detailed acceptance criteria, the background for the story, etc.
mjlangiii 7y ago
The user story is a promise to have a conversation. I think that is usually well understood. From there I think you can fall into two camps: that conversation should result in a Jira/whatever ticket with all the requisite documentation for an agile team versus that conversation IS the essential information required to properly build the expected valuable working software. Back to the question - what do you do about poor knowledge transfer in a project? I think a moderate de-emphasis on thinking of the user story text and the additional info like acceptance criteria etc. as self-sufficient documentation and adding more emphasis on that close relationship between developer, user, and maybe a tester, can help fill in big knowledge gaps.

dkersten 7y ago

Many years ago, I worked for a company where we were writing complex distributed telecom software and they had a wiki for documenting the system. I spent a few weeks meticulously documenting everything I did and anything that was touched by it (including defining all of the industry jargon and such). It was a great way to get a quick understanding of any part of the system, but I was the only person keeping it up to date so after a while I ran out of steam and stopped doing it. :( I've come across the "documentation becomes quickly outdated" argument a lot, but nobody has ever been able to suggest a good alternative. The best I've found is to write design logs for proposed changes (which you then let other team members/stakeholders can review/comment on before it gets implemented) and decision logs for any decisions that are made. This way, them going out of date is expected and ok, as they become a history of ideas and decisions with their context and outcomes laid out. You don't necessarily have a snapshot of "the system right now" but you have a log of all the ideas and decisions that lead up to the current system.

sethammons 7y ago
We create "blueprints" that have system design and goals on new projects and how they fit into the ecosystem. We are supposed to update them after they go into production as things change, but seldom do. Still, going back and reviewing a design document helps.
whydoyoucare 7y ago
I believe it is equally important to determine the level of abstraction for the documentation, such that updating it is an infrequent task (essentially, every line of code change should not trigger a document change). It is easier said than done, but that's the best compromise I have arrived it. At the very least, the document should capture high-level (again this is relative term) design, possibly an architecture diagram of major interacting functional units. The success measure should be the relative ability to build a mental model of the system by looking at this document for any newbie. That design document would be a start, and most likely not "quickly outdated". My personal beef with the agile camp is precisely this: when they let go of documentation, they don't do the design doc as well, and all that remains of the system is thousands of incoherent stories and huge amount of code.

erikpukinskis 7y ago

In Scrum, you don’t write code based off User Stories. The scrum team agrees to a set of Stories for the sprint, and then the scrum team breaks those stories into a set of Tasks which are the actual work that must be done. The User Stories are just something the product owner uses to show stakeholders that the project was successful.

eropple 7y ago
1) In the Hermes Conrad sense, this is technically correct. 2) In my experience, this basically never happens. Your comment encapsulates a lot of what I have come to call "Scrumbutt." It's Scrum, but. And while I have no idea if it's intended on your part, the sentiment is a fantastic way for a Scrum consultant--only some shade thrown; I've been a "DevOps consultant" before, after all--to come in and pull from deep in their Scrumbutt something to the effect of "you're doing it wrong, Scrum has not failed, you have failed Scrum." Within epsilon of nobody does Scrum "as prescribed"--because the amount of responsibility that must be undertaken at all levels is virtually impossible to get full buy-in on--and as such the boil on our collective behind that it is persists because criticism is immediately bedeviled by Scotsmen of unknown provenance.
- erikpukinskis 7y ago
  You’re projecting “shade thrown”. People might be interested in what Scrum is. I know I am. That’s why I pointed out the error. It was a shock to me to learn I wasn’t doing anything close. Readers can do with the info what they want. I’m not sure if I can say the same of your comment. You seem to be trying to make me feel bad for commenting? Or accusing me of hawking pointless info for consulting fees? I really can’t tell.

wolco 7y ago

Change managers/leads

sailfast 7y ago

Traceability is where I'd start. First, ensure your stories are being linked to your code / pull requests / issues. That way you can figure out why something was changed in the future. This is key to determining whether you can change something down the road. Stories directly traceable to code can be powerful for capturing knowledge. I might also recommend creating user stories for non-feature development like infrastructure and tech debt paydown (if you don't already). That way, all of the value flow is captured in one place and you're not just leading managers to see new features only. Second, in addition to the user stories I'd advocate for strong background information about the context of the story as well as detailed acceptance criteria if you don't have that in place already.

mindcrime 7y ago

They consider the User Stories + the code to be everything you need Then they are Doing It Wrong™. Note that there's nothing in the Agile Manifesto OR the Scrum Guide that says "don't write documentation." The closest you get is in the AM where it says "We have come to value ... Working software over comprehensive documentation". But note that immediately after that it says "That is, while there is value in the items on the right, we value the items on the left more." IOW, the Agile Manifesto explicitly endorses the value of documentation! Remember this the next time somebody tries to tell you that "we don't do documentation because we're Agile." Anybody running that line is Full Of Shit™.

Viliam1234 7y ago

> Does anyone have any suggestions on how to fix this? Have a product wiki (e.g. MediaWiki). Have documentation in source code that compiles to HTML code, which can be linked to/from the product wiki (e.g. JavaDoc in Java, Natural Docs for languages that do not directly support compilable documentation). Make building and publishing this documentation a part of the continuous integration. When you have this, make it a part of code reviews to ask "where is this documented?" for those kinds of things that are easy to remember today, but no one will remember it a few months later. In other words, make it a "code+doc review". (Don't be dogmatic about whether the information should go to code documentation, unit test documentation, or wiki. Use common sense. If it only related to one method, it's probably the code; if it related to a use case, it's probably the unit test that verifies that use case; if it is a general topic that has an impact on many parts of the program, it probably deserves a separate wiki page.)

jkepler 7y ago
> Have documentation in source code that compiles to HTML code, which can be linked to/from the product wiki (e.g. JavaDoc in Java, Natural Docs for languages that do not directly support compilable documentation). Make building and publishing this documentation a part of the continuous integration. Are you referring to something like Knuth's Literate Programming (en.m.wikipedia.org/wiki/Literate_programming)? As a non-professional who's learning to develop on the side, something that follows more of a natural language approach appeals to me, as sometimes I have a few months between working on my project, and comments on my source code help me not to forget why I do certain things in the code. However, I'm not doing Literate Programming, just python with comments.
- Viliam1234 7y ago
  No. I think one should organize the code as the code needs to be organized, and the documentation can either follow along (if it describes parts of code contained within the same file) or be placed separately (if it relates to multiple files), where "separately" could still be a package-level JavaDoc, or an external wiki. I have never tried the Literal Programming, so perhaps I am out of my depth here, but I strongly suspect it only works after one has already mastered the usual ways of programming. That you do not have to structure the code qua code, because you can already do it in your head. But it's hard to imagine what one has never done before. For example, if you never tried programming the usual way, how do you know when and why to put "Header files to include" in your Literal code? It's only because you can imagine the constructed code, you know where the header files go in the result, so you know where to place them in the Literal version. Otherwise, it would look quite arbitrarily. I don't know about documentation in Python, but the JavaDoc (and Natural Docs) work like this: You put comments to classes and methods, or packages (and files), along with the code. So you can read them and write them while you are looking at the code. But then you run a "documentation compiler" that extracts the comments and builds a separate HTML website out of them. Here you can browse and read about what the individual classes and methods do. The idea is to make this a part of the continuous integration, so that whenever you update the source code and the related comment, the HTML website also gets updated. Java supports this out of the box. When you install the Java compiler, you also install the Java documentation compiler. When you read the official documentation to the standard Java classes, those were made using exactly the same tools you are encouraged to use. I don't know whether Python has something like this. If yes, go ahead and use it. If not, look at Natural Docs -- it is a system to provide this functionality to languages that do not support it out of the box. Just try it: document a part of your existing project, compile the docs, and see whether you can imagine any value in reading that.