7 ms·

As a novice programmer, I was absolutely stunned that this was not standard practice. A typical source file provides zero context, background on the subject, po

by jml7c5 8y ago

As a novice programmer, I was absolutely stunned that this was not standard practice. A typical source file provides zero context, background on the subject, pointers to reference material/blog posts/books explaining the concepts, information on how it fits into the program's 'bigger picture', or (most importantly) the thought process that resulted in the file (i.e., why the choice was made to do _this_ rather than _that_, challenges faced, trade-offs, etc.). It still baffles me. Every programmer has to start from scratch dealing with a new codebase, and it makes improving any non-trivial program impossible unless one is willing to spend hours of archaeological examination. To open-source developers: if you want to get people contributing to a project (and make everyone's effort much more enjoyable!), these sorts of comments are essential. Not to mention they'll save everyone boatloads of time; it's a shame that every programmer has to piece together knowledge from scratch, rather than being 'tutored' by their peers' comments.

adjkant 8y ago
I think your desire is right, but think about this every time you create a file, and how much slower your work would be. The question then becomes: "how much commenting exactly is needed before this becomes more time than the technical debt it creates? I think this type of summary should not be per source file but per package/folder/module/project. A high-level developer overview with sufficient depth will also help stop repetition of philosophy in subsequent files. In this case, I do agree with its existence though because of both the length and complexity of the file. For many files, neither are the case.
- zeroimpl 8y ago
  I think this commenting only makes sense if the design is atypical. Probably 90% of the code I write is following design patterns already used throughout the code base.
  - AstralStorm 8y ago
    And the 10% is unmaintained. Design patterns can tap you easily in bad designs. Usually the lost thing is performance, big way, or actual simplicity.
  - AstralStorm 8y ago
    Since comnenter announce does not understand what I meant, it does not matter if design is typical. Functionality still has to be described, so the pattern itself buys you nothing on its own. Then the overarching idea and reasoning also has to be described to make it easier for everybody else to understand. I've met more than one codebase which tried to use a big hammer pattern to open doors. Heavy indirection to do simple things because of design choices, usually caused by a forced framework - instead of easily factorizable simplicity. Patterns should be emergent not forced. Or more specifically, the choice of appropriate pattern should be based on the problem being solved, rarely one pattern can cleanly solve all problems. Similarity is very seductive but end result is often complex and hard to reason about when followed religiously.
- nailer 8y ago
  How long does it take you to write a sentence?
  - james_s_tayler 8y ago
    Twice the time it takes you to write half a sentence.
    - hexane360 8y ago
      Hard disagree. You still need to think of the sentence to write the first half.
      - james_s_tayler 8y ago
        Beg to differ. I consider the time taken to think and write as the aggregate time to write the sentence.
        nailer 8y ago
        You've had to think of the design whether you write the sentence or not.
        james_s_tayler 8y ago
        Invoke gauge theory to set the floor from which we measure, so that the tautology always holds.
        nailer 8y ago
        I don't understand what this means.
        james_s_tayler 8y ago
        Well people keep picking arbitrary starting points in the coordinate system. I'm saying for any arbitrarily picked starting point in the coordinate system, the time taken is always twice that of getting from the arbitrarily picked starting point to the halfway mark. A tautology is a tautology. How long is a piece of string?
qlk1123 8y ago
> As a novice programmer, I was absolutely stunned that this was not standard practice. I agree with your point, and I will be benefit from this style if it were the standard, too. But don't you think a good community culture can make people maintain a good git history for this purpose? My daily job is a Linux kernel developer. I found that source codes are only the "What" part, git comments can and should state the "How/Why" part, and if all those still make no sense to me, I look for the original mailing list for the deeper "Why". Most of the time the information is sufficient.
- nerflad 8y ago
  Not all organizations have such robust communication structures, low levels of dysfunction, or git hygiene. Also, as mentioned, in a corporate environment where the software is proprietary and certain source files may sit dormant long enough for the original authors to move on from the company, it's important that they don't take the "how/why" part with them, so to speak, and leave the next maintainer SOL.
- busterarm 8y ago
  As much as I would implore you to document the why as a comment in the code after your experience, I have also learned the hard way that 'why' can be a moving target given enough time.
- TeMPOraL 8y ago
  I've never seen a git commit comment describing the "why" of code. "Added foo.\n\nImplemented az Bar because of blorgz." isn't nearly enough of a rationale, and that's the best description I see people making. Also, the whole point of putting something in a comment is that you have to read it when working through code around that comment. Putting a note in a commit log instead ensures that crucial information important to the code will not be visible, and most likely not read at all, when working on that code.
  - jnurmine 8y ago
    Look at projects with high discipline and experience, such as the Linux kernel. You will find plenty of "why" examples. It seems to be easy to find plenty of projects with bad Git commits though. IMO the big architectural guidelines and structuring and other highest level things and API contracts etc. should be in an external file (not code). The high-level details around a certain implementation in the commit logs for that file/files. Relevant implementation details and important things in the code. The lower you go, the closer to a comment line in the code you should get. This reflects also the rigidity of the code: the high-level architecture should not change often; if it does, then the architecture is not really ready. Things should be certainly documented, but not in great detail inside the code files. The lack of documentation is in my opinion OK, the code is always the last word for how things actually work anyway. Misleading or wrong documentation are the absolute worst.
hibikir 8y ago
There's plenty of good reasons to not write 95% of code with big walls of explanation. The first is a matter of cost: Writing a good explanation around everything is very expensive to do at first. A whole lot of the custom code you find in random companies, from the shiny SV startup to the old enterprise, is unimportant, cobbled together pieces. We have no idea of whether we are writing code that will be thrown away in a week, a month, a year or whether it will last two decades. Context can change fast enough that the comments become worse than useless, as the terminology might have changed, or had been misguided in the first place, turning the long comments into outright unintended deception. This gets even worse as we do not evaluate all the comments in all the files whenever there's a code change: It's crazy how a large comment block in one place can become harmful after it's forgotten, and someone else makes a correct, business critical change in another file. No matter where I am working, it's rare for me to not find multiple examples every year where the code and the comments provide very different impressions of what is going on, and it's the code that is accurate. This is not to say that there aren't reasons to write large comment blocks, or architecture documents, but that they are often better written not while the system is being first built, but later, in a maintenance cycle, when someone already had wished for the comments, and has regained the knowledge the hard way. By then, it's clear which part of the system are dangerous, suspicious and unclear. Where there's more need for high quality error handling, and where thousands of lines of error handling never get hit, because the failing case that was originally considered doesn't really happen in this dimension anymore. Writing code so that someone, even a future you, can pick it back up and improve it when it's needed, while still delivering the code at a good pace is a kind of skill that many just don't learn, either because they are always in greenfield teams that never pay for their mistakes, or have an approach to maintenance that involves not becoming intimate familiar with a system, and instead either hack or rewrite. But nobody looks great in a resume by saying that they are a specialist in software maintenance.
- db48x 8y ago
  Always say Software Archaeology instead; it sounds better.
- TeMPOraL 8y ago
  > This is not to say that there aren't reasons to write large comment blocks, or architecture documents, but that they are often better written not while the system is being first built, but later, in a maintenance cycle, when someone already had wished for the comments, and has regained the knowledge the hard way. I don't think the "Lean Manufacturing" approach works here. By the time someone "pulls" you for a comment, you've already lost the most important knowledge that should go into the comments - why the code is the way it is. Maybe you'll recall it when asked, hopefully not missing anything crucial. Meanwhile, comments are extremely cheap to write as you're writing the code, and even before you're writing the code (you did spend the time thinking about what you'll write, and aren't just "coding from the hip", right?).
matte_black 8y ago
The information you are talking about is called documentation and it should exist outside of actual code files. If you need to read comments to understand code, you don’t know how to read code. Comments can lie, only code is the source of truth, once you gain experience you won’t bother with comments.
- jniedrauer 8y ago
  > The information you are talking about is called documentation and it should exist outside of actual code files. The file in the OP is Go. In Go, you are encouraged to write documentation and code in one place[0]. The end result is very nice, auto-generated, human readable documentation[1] which is tightly coupled to the code it documents. [0]: https://golang.org/src/net/http/doc.go https://golang.org/src/net/http/doc.go [1]: https://golang.org/pkg/net/http/ https://golang.org/pkg/net/http/
zachberger 8y ago
You may want to checkout the python standard library files. heapq.py, for example, has some amazing documentation in it: https://svn.python.org/projects/python/trunk/Lib/heapq.py https://svn.python.org/projects/python/trunk/Lib/heapq.py
tapland 8y ago
> unless one is willing to spend hours of archaeological examination Currently the only person in-office over Christmas on my first software dev job. Debugging a 50k LOC COBOL beast that digs into three other beast programs and ends up in a final 20k LOC uncommented piece where things are supposed to happen and be returned back. Nothing is commented, the programs are huge and one can only debug one program at a time, requiring me to submit untested changes in one of them to the shared dev environment since I'm making changes in two, before debugging the other program. If it just said somewhere what half of the stuff is I would save an insane amount of time getting to know the system.
- vonseel 8y ago
  my first software dev job. Debugging a 50k LOC COBOL beast We found the real MVP. tapland, what were you thinking when taking this job!?
  - tapland 8y ago
    Heard about COBOL for ages, always been into retro computing and setting up z/OS on the Hercules emulator at home I was offered COBOL-training from a company and two years of guaranteed employment (with pay even if they decide I'm not needed) to join a team that force-retired 40% of their developers this year. I have no real previous development experience, only some Java and Python, since I chose an industrial engineering program at uni, and I got the offer the semester before my CS-classes (my chosen specialization) started. Changed my university classes to equivalent distance-classes and took six months of school to get into training and a job.

by jml7c5 8y ago

adjkant 8y ago

I think your desire is right, but think about this every time you create a file, and how much slower your work would be. The question then becomes: "how much commenting exactly is needed before this becomes more time than the technical debt it creates? I think this type of summary should not be per source file but per package/folder/module/project. A high-level developer overview with sufficient depth will also help stop repetition of philosophy in subsequent files. In this case, I do agree with its existence though because of both the length and complexity of the file. For many files, neither are the case.

zeroimpl 8y ago
I think this commenting only makes sense if the design is atypical. Probably 90% of the code I write is following design patterns already used throughout the code base.
- AstralStorm 8y ago
  And the 10% is unmaintained. Design patterns can tap you easily in bad designs. Usually the lost thing is performance, big way, or actual simplicity.
- AstralStorm 8y ago
  Since comnenter announce does not understand what I meant, it does not matter if design is typical. Functionality still has to be described, so the pattern itself buys you nothing on its own. Then the overarching idea and reasoning also has to be described to make it easier for everybody else to understand. I've met more than one codebase which tried to use a big hammer pattern to open doors. Heavy indirection to do simple things because of design choices, usually caused by a forced framework - instead of easily factorizable simplicity. Patterns should be emergent not forced. Or more specifically, the choice of appropriate pattern should be based on the problem being solved, rarely one pattern can cleanly solve all problems. Similarity is very seductive but end result is often complex and hard to reason about when followed religiously.
nailer 8y ago
How long does it take you to write a sentence?
- james_s_tayler 8y ago
  Twice the time it takes you to write half a sentence.
  - hexane360 8y ago
    Hard disagree. You still need to think of the sentence to write the first half.
    - james_s_tayler 8y ago
      Beg to differ. I consider the time taken to think and write as the aggregate time to write the sentence.
      - nailer 8y ago
        You've had to think of the design whether you write the sentence or not.
        james_s_tayler 8y ago
        Invoke gauge theory to set the floor from which we measure, so that the tautology always holds.
        nailer 8y ago
        I don't understand what this means.
        james_s_tayler 8y ago
        Well people keep picking arbitrary starting points in the coordinate system. I'm saying for any arbitrarily picked starting point in the coordinate system, the time taken is always twice that of getting from the arbitrarily picked starting point to the halfway mark. A tautology is a tautology. How long is a piece of string?

qlk1123 8y ago

> As a novice programmer, I was absolutely stunned that this was not standard practice. I agree with your point, and I will be benefit from this style if it were the standard, too. But don't you think a good community culture can make people maintain a good git history for this purpose? My daily job is a Linux kernel developer. I found that source codes are only the "What" part, git comments can and should state the "How/Why" part, and if all those still make no sense to me, I look for the original mailing list for the deeper "Why". Most of the time the information is sufficient.

nerflad 8y ago
Not all organizations have such robust communication structures, low levels of dysfunction, or git hygiene. Also, as mentioned, in a corporate environment where the software is proprietary and certain source files may sit dormant long enough for the original authors to move on from the company, it's important that they don't take the "how/why" part with them, so to speak, and leave the next maintainer SOL.
busterarm 8y ago
As much as I would implore you to document the why as a comment in the code after your experience, I have also learned the hard way that 'why' can be a moving target given enough time.
TeMPOraL 8y ago
I've never seen a git commit comment describing the "why" of code. "Added foo.\n\nImplemented az Bar because of blorgz." isn't nearly enough of a rationale, and that's the best description I see people making. Also, the whole point of putting something in a comment is that you have to read it when working through code around that comment. Putting a note in a commit log instead ensures that crucial information important to the code will not be visible, and most likely not read at all, when working on that code.
- jnurmine 8y ago
  Look at projects with high discipline and experience, such as the Linux kernel. You will find plenty of "why" examples. It seems to be easy to find plenty of projects with bad Git commits though. IMO the big architectural guidelines and structuring and other highest level things and API contracts etc. should be in an external file (not code). The high-level details around a certain implementation in the commit logs for that file/files. Relevant implementation details and important things in the code. The lower you go, the closer to a comment line in the code you should get. This reflects also the rigidity of the code: the high-level architecture should not change often; if it does, then the architecture is not really ready. Things should be certainly documented, but not in great detail inside the code files. The lack of documentation is in my opinion OK, the code is always the last word for how things actually work anyway. Misleading or wrong documentation are the absolute worst.

hibikir 8y ago

There's plenty of good reasons to not write 95% of code with big walls of explanation. The first is a matter of cost: Writing a good explanation around everything is very expensive to do at first. A whole lot of the custom code you find in random companies, from the shiny SV startup to the old enterprise, is unimportant, cobbled together pieces. We have no idea of whether we are writing code that will be thrown away in a week, a month, a year or whether it will last two decades. Context can change fast enough that the comments become worse than useless, as the terminology might have changed, or had been misguided in the first place, turning the long comments into outright unintended deception. This gets even worse as we do not evaluate all the comments in all the files whenever there's a code change: It's crazy how a large comment block in one place can become harmful after it's forgotten, and someone else makes a correct, business critical change in another file. No matter where I am working, it's rare for me to not find multiple examples every year where the code and the comments provide very different impressions of what is going on, and it's the code that is accurate. This is not to say that there aren't reasons to write large comment blocks, or architecture documents, but that they are often better written not while the system is being first built, but later, in a maintenance cycle, when someone already had wished for the comments, and has regained the knowledge the hard way. By then, it's clear which part of the system are dangerous, suspicious and unclear. Where there's more need for high quality error handling, and where thousands of lines of error handling never get hit, because the failing case that was originally considered doesn't really happen in this dimension anymore. Writing code so that someone, even a future you, can pick it back up and improve it when it's needed, while still delivering the code at a good pace is a kind of skill that many just don't learn, either because they are always in greenfield teams that never pay for their mistakes, or have an approach to maintenance that involves not becoming intimate familiar with a system, and instead either hack or rewrite. But nobody looks great in a resume by saying that they are a specialist in software maintenance.

db48x 8y ago
Always say Software Archaeology instead; it sounds better.
TeMPOraL 8y ago
> This is not to say that there aren't reasons to write large comment blocks, or architecture documents, but that they are often better written not while the system is being first built, but later, in a maintenance cycle, when someone already had wished for the comments, and has regained the knowledge the hard way. I don't think the "Lean Manufacturing" approach works here. By the time someone "pulls" you for a comment, you've already lost the most important knowledge that should go into the comments - why the code is the way it is. Maybe you'll recall it when asked, hopefully not missing anything crucial. Meanwhile, comments are extremely cheap to write as you're writing the code, and even before you're writing the code (you did spend the time thinking about what you'll write, and aren't just "coding from the hip", right?).

matte_black 8y ago

The information you are talking about is called documentation and it should exist outside of actual code files. If you need to read comments to understand code, you don’t know how to read code. Comments can lie, only code is the source of truth, once you gain experience you won’t bother with comments.

jniedrauer 8y ago
> The information you are talking about is called documentation and it should exist outside of actual code files. The file in the OP is Go. In Go, you are encouraged to write documentation and code in one place[0]. The end result is very nice, auto-generated, human readable documentation[1] which is tightly coupled to the code it documents. [0]: https://golang.org/src/net/http/doc.go https://golang.org/src/net/http/doc.go [1]: https://golang.org/pkg/net/http/ https://golang.org/pkg/net/http/

zachberger 8y ago

You may want to checkout the python standard library files. heapq.py, for example, has some amazing documentation in it: https://svn.python.org/projects/python/trunk/Lib/heapq.py https://svn.python.org/projects/python/trunk/Lib/heapq.py

tapland 8y ago

> unless one is willing to spend hours of archaeological examination Currently the only person in-office over Christmas on my first software dev job. Debugging a 50k LOC COBOL beast that digs into three other beast programs and ends up in a final 20k LOC uncommented piece where things are supposed to happen and be returned back. Nothing is commented, the programs are huge and one can only debug one program at a time, requiring me to submit untested changes in one of them to the shared dev environment since I'm making changes in two, before debugging the other program. If it just said somewhere what half of the stuff is I would save an insane amount of time getting to know the system.

vonseel 8y ago
my first software dev job. Debugging a 50k LOC COBOL beast We found the real MVP. tapland, what were you thinking when taking this job!?
- tapland 8y ago
  Heard about COBOL for ages, always been into retro computing and setting up z/OS on the Hercules emulator at home I was offered COBOL-training from a company and two years of guaranteed employment (with pay even if they decide I'm not needed) to join a team that force-retired 40% of their developers this year. I have no real previous development experience, only some Java and Python, since I chose an industrial engineering program at uni, and I got the offer the semester before my CS-classes (my chosen specialization) started. Changed my university classes to equivalent distance-classes and took six months of school to get into training and a job.