8 ms·

I have a similar advice, but I will go one step further: add README.md to other folders as well. It is dope to have a map of your whole system in an Architectur

by baby 6y ago

I have a similar advice, but I will go one step further: add README.md to other folders as well. It is dope to have a map of your whole system in an Architecture.md (or a README if it's not too long), but it's even more dope to be able to click through it and have submaps of how other components are structured. Displaying the folder/file structure and explaining what is what is a must. An example from Diem[1]: consensus ├── src │ ├── block_storage # In-memory storage of blocks and related data structures │ ├── consensusdb # Database interaction to persist consensus data for safety and liveness │ ├── liveness # RoundState, proposer, and other liveness related code │ └── test_utils # Mock implementations that are used for testing only └── consensus-types # Consensus data types (i.e. quorum certificates) └── safety-rules # Safety (voting) rules I recently digged into dependabot's code, and I found it extremely well structured. For example you have an Architecture section in the first README[2] with a diagram (how awesome is that!) and with links to the README of all the sub components, which themselves live in the subfolders[3]. What I dread the most is going through a new codebase and not seeing any documentation in internal packages. Like how the fuck am I supposed to understand anything in there? By reading all the code? [1]: https://github.com/diem/diem/tree/master/consensus#how-is-this-module-organized https://github.com/diem/diem/tree/master/consensus#how-is-th... [2]: https://github.com/dependabot/dependabot-core#architecture https://github.com/dependabot/dependabot-core#architecture [3]: https://github.com/dependabot/dependabot-core/blob/main/common/lib/dependabot/file_updaters/README.md https://github.com/dependabot/dependabot-core/blob/main/comm...

xyzzy_plugh 6y ago
> Like how the fuck am I supposed to understand anything in there? By reading all the code? This is one of the superpowers of Go: for most Go projects, this is exactly what I'd do. Just read the code. It's easy to follow, it's all formatted the same, very little implicit behavior, and I don't need an IDE to do it. Few languages were designed to be read by others. Thankfully Go is one of them.
- systemvoltage 6y ago
  I am afraid, this doesn't quite cut the mustard. Code just can't replace a human readable Architecture diagram + explanations of whys of choices and hows of the system. Programming language doesn't replace this no matter how clear and modular it is. While Go is nice, you're vastly under-appreciating architectural documentation.
  - macintux 6y ago
    My standard argument regarding the criticality of documentation: no amount of code can ever explain what isn't there. What optimizations have you tried that failed? What 3rd party tool used to be integrated but now isn’t because the maintenance burden was too high? Why is it safe to ignore this exception?
- mssundaram 6y ago
  > very little implicit behavior Actually I find it quite implicit the way Go structures packages. You definitely need an IDE (I use neovim + coc.nvim) to jump to a definition unless you want to grep the folder for where a struct is defined...
  - baby 6y ago
    There's one issue I can think of, if the struct is defined in the same package then it's a bit of a pain to find which file actually defines it (in Rust you have to be explicit, each file is a module). But that's it I believe, if it's in a different package then you'd have to explicitly write the import
- baby 6y ago
  I agree that Golang has the easiest code to read, and jumping into a Golang project (even massive) is always a happy experience. But still, you can't replace documentation.
michael_j_ward 6y ago
When I first started programming, I thought the description next to the folder / file name on github was actually describing the item - as you did above, and not just the message from the last commit that altered the file. Many years later, I still believe that's how it should be.
- baby 6y ago
  Agree! I gave that feedback many times, who cares about commit message? I want documentation here! I think Github should really start taking a stance on convention, and come up with project structure conventions that would help project navigation. For example, youtube does that with chapters in videos, if you write a description with timestamps it will display these chapters in the timeline of the video[1]. IMO Github should do the same if you have a README in your folder, describing the folder with a certain syntax, then it should extract this info and show it next to folders. If anyone at Github is reading this, pretty please :D? [1]: https://www.youtube.com/watch?v=XfP862hCrDM https://www.youtube.com/watch?v=XfP862hCrDM
  - bombcar 6y ago
    I’ve recommitted a number of times just because the last commit message was dumb or useless or worse - wrong.
  - benjaminjackman 6y ago
    This makes me think we should be able to put something like a standard tag for a 1-line summary (maybe embedded in some markdown in a comment) in a fashion the VCS's can detect & extract and then render in their UXs. e.g. something very simple would probably work like the first line found that matches something like: [Ignore_Non_Words]Summary[IgnorePunctuation&Spaces](Capture)
    - PoignardAzur 6y ago
      I think you could get a lot of mileage from using existing formats in code and comments. So for instance, if save_parser.py starts with """ Parse content loaded from save files ... """ and math_backend.rs starts with //! List of data structures necessary for maths operations //! //! ... then your source tree UX should look like: src ├── save_parser.py # Parse content loaded from save files ├── math_backend.rs # List of data structures necessary for maths operations
jlg23 6y ago
This will help people browsing code in github, but folks browsing code in their IDE will most likely be lost, as the documentation you advice to put into "README.md"s should, IMHO, be in the module/package documentation in the source. One can still link to those from the global README.md
- snicker7 6y ago
  READMEs and API docs aren't mutually exclusive. But they cover different scopes. READMEs are more suited for high-level documentation, focusing on the /why/ instead of the /how/.
- baby 6y ago
  I'm not sure I understand why someone in their IDE would be lost, you can't read README.md and see the file structure in your IDE? Documentation in the source is different from a directory organization documentation IMO. Rust has a lot of tools to write good documentation in source, and it is here to help produce good documentation for users of a library, not to explain how to contribute to the code or understand how everything is architected.
  - jlg23 6y ago
    > I'm not sure I understand why someone in their IDE would be lost, you can't read README.md and see the file structure in your IDE? Of course one can see the files. But I rarely deal with "files" in my IDE, I deal with packages/namespaces/modules/functions/classes/methods. The natural place to put architecture documentation would therefore be the top-level code unit (e.g. package/module/namespace), to which one can easily navigate when viewing docs. > Documentation in the source is different from a directory organization documentation. Is it, given that directories tend to be organized around code units (again: packages/namespaces/modules/classes)? It is thinkable to store code in databases (see Dylan's IDE, either implemented or it was on the roadmap, I don't remember) and all the architecture documentation not directly attached to semantic units of your code would be lost.
    - hashhar 6y ago
jzer0cool 6y ago
Any well known tools which print above like folder path trees command-line?
- pbkhrv 6y ago
  On Linux there is a nice little utility called “tree” https://packages.debian.org/stable/tree https://packages.debian.org/stable/tree Not sure about Mac or Windows.
- house9-2 6y ago
  tree brew tree (if you are on Mac)
- deleted 6y ago
  [deleted]
avodonosov 6y ago
I also use such hierarchical style, with lower level REAODMEs refferring iddividual source files. So that user continues from READMEs to code comments. Example: https://github.com/cl-test-grid/cl-test-grid#system-architecture https://github.com/cl-test-grid/cl-test-grid#system-architec... On every level I first explain the task solved by this module and then go into the implementation details. Knowing the purpose first helps reader to understand impl details.

by baby 6y ago

xyzzy_plugh 6y ago

> Like how the fuck am I supposed to understand anything in there? By reading all the code? This is one of the superpowers of Go: for most Go projects, this is exactly what I'd do. Just read the code. It's easy to follow, it's all formatted the same, very little implicit behavior, and I don't need an IDE to do it. Few languages were designed to be read by others. Thankfully Go is one of them.

systemvoltage 6y ago
I am afraid, this doesn't quite cut the mustard. Code just can't replace a human readable Architecture diagram + explanations of whys of choices and hows of the system. Programming language doesn't replace this no matter how clear and modular it is. While Go is nice, you're vastly under-appreciating architectural documentation.
- macintux 6y ago
  My standard argument regarding the criticality of documentation: no amount of code can ever explain what isn't there. What optimizations have you tried that failed? What 3rd party tool used to be integrated but now isn’t because the maintenance burden was too high? Why is it safe to ignore this exception?
mssundaram 6y ago
> very little implicit behavior Actually I find it quite implicit the way Go structures packages. You definitely need an IDE (I use neovim + coc.nvim) to jump to a definition unless you want to grep the folder for where a struct is defined...
- baby 6y ago
  There's one issue I can think of, if the struct is defined in the same package then it's a bit of a pain to find which file actually defines it (in Rust you have to be explicit, each file is a module). But that's it I believe, if it's in a different package then you'd have to explicitly write the import
baby 6y ago
I agree that Golang has the easiest code to read, and jumping into a Golang project (even massive) is always a happy experience. But still, you can't replace documentation.

michael_j_ward 6y ago

When I first started programming, I thought the description next to the folder / file name on github was actually describing the item - as you did above, and not just the message from the last commit that altered the file. Many years later, I still believe that's how it should be.

baby 6y ago
Agree! I gave that feedback many times, who cares about commit message? I want documentation here! I think Github should really start taking a stance on convention, and come up with project structure conventions that would help project navigation. For example, youtube does that with chapters in videos, if you write a description with timestamps it will display these chapters in the timeline of the video[1]. IMO Github should do the same if you have a README in your folder, describing the folder with a certain syntax, then it should extract this info and show it next to folders. If anyone at Github is reading this, pretty please :D? [1]: https://www.youtube.com/watch?v=XfP862hCrDM https://www.youtube.com/watch?v=XfP862hCrDM
- bombcar 6y ago
  I’ve recommitted a number of times just because the last commit message was dumb or useless or worse - wrong.
- benjaminjackman 6y ago
  This makes me think we should be able to put something like a standard tag for a 1-line summary (maybe embedded in some markdown in a comment) in a fashion the VCS's can detect & extract and then render in their UXs. e.g. something very simple would probably work like the first line found that matches something like: [Ignore_Non_Words]Summary[IgnorePunctuation&Spaces](Capture)
  - PoignardAzur 6y ago
    I think you could get a lot of mileage from using existing formats in code and comments. So for instance, if save_parser.py starts with """ Parse content loaded from save files ... """ and math_backend.rs starts with //! List of data structures necessary for maths operations //! //! ... then your source tree UX should look like: src ├── save_parser.py # Parse content loaded from save files ├── math_backend.rs # List of data structures necessary for maths operations

jlg23 6y ago

This will help people browsing code in github, but folks browsing code in their IDE will most likely be lost, as the documentation you advice to put into "README.md"s should, IMHO, be in the module/package documentation in the source. One can still link to those from the global README.md

snicker7 6y ago
READMEs and API docs aren't mutually exclusive. But they cover different scopes. READMEs are more suited for high-level documentation, focusing on the /why/ instead of the /how/.
baby 6y ago
I'm not sure I understand why someone in their IDE would be lost, you can't read README.md and see the file structure in your IDE? Documentation in the source is different from a directory organization documentation IMO. Rust has a lot of tools to write good documentation in source, and it is here to help produce good documentation for users of a library, not to explain how to contribute to the code or understand how everything is architected.
- jlg23 6y ago
  > I'm not sure I understand why someone in their IDE would be lost, you can't read README.md and see the file structure in your IDE? Of course one can see the files. But I rarely deal with "files" in my IDE, I deal with packages/namespaces/modules/functions/classes/methods. The natural place to put architecture documentation would therefore be the top-level code unit (e.g. package/module/namespace), to which one can easily navigate when viewing docs. > Documentation in the source is different from a directory organization documentation. Is it, given that directories tend to be organized around code units (again: packages/namespaces/modules/classes)? It is thinkable to store code in databases (see Dylan's IDE, either implemented or it was on the roadmap, I don't remember) and all the architecture documentation not directly attached to semantic units of your code would be lost.
  - hashhar 6y ago

jzer0cool 6y ago

Any well known tools which print above like folder path trees command-line?

pbkhrv 6y ago
On Linux there is a nice little utility called “tree” https://packages.debian.org/stable/tree https://packages.debian.org/stable/tree Not sure about Mac or Windows.
house9-2 6y ago
tree brew tree (if you are on Mac)
deleted 6y ago
[deleted]

avodonosov 6y ago

I also use such hierarchical style, with lower level REAODMEs refferring iddividual source files. So that user continues from READMEs to code comments. Example: https://github.com/cl-test-grid/cl-test-grid#system-architecture https://github.com/cl-test-grid/cl-test-grid#system-architec... On every level I first explain the task solved by this module and then go into the implementation details. Knowing the purpose first helps reader to understand impl details.