5 ms·

The issue with all the ideas to make programs easier to understand is that they require time and work. If I had more time and energy there are plenty of things

The issue with all the ideas to make programs easier to understand is that they require time and work. If I had more time and energy there are plenty of things I could do: refactor, make comments, change variable names or try literate programming. But given the limited time and energy which is best? Put another way, how can you beat thoughtful variable or function names and judicious comments?

blattimwind 8y ago
Program design.
ljm 8y ago
Literate programming might be my least favourite approach when writing normal code, because you're going to read the prose only once or twice and then find that it gets in the way of the implementation you're concerned about. Maintaining something done with literate programming must be an absolute nightmare if you've got any kind of refactoring, because you basically have to rewrite at least a few paragraphs so the prose continues to make sense. The code is more important than the prose. But when it comes to writing a tutorial that you can also execute to get the final result, or interact with along the way, then there is nothing better because, really, you're more interested in the prose that lends explanation to the code you're showcasing. Although this article's about emacs and I've taken the literate approach to my own config too. That's less for the benefit of adding paragraphs of explanation to my config, but because org mode offers a fantastic way to organise code in one file (which is more often than not what an emacs config tends to be). In either case I don't think you can do better than being thoughtful when writing code, thinking about the bigger picture and not just the individual functions and variables in isolation. For others and also yourself. This is also true for emacs itself, considering the idiosyncrasies it presents when most of us learn elisp by copying someone else, rather than reading the language docs.
- ModernMech 8y ago
  > The code is more important than the prose. This is not the position of someone programming in a literate fashion. Their contention is that prose and code are complimentary. Code explains what the program does, prose explains why it does it that way. They work together to paint a coherent view of your program. With just code, a newcomer reading your program and understanding it 100% is still left with many questions unanswered.
- Retra 8y ago
  The main problem I have with literate programming for tutorials is that it forces your writing order to match the code order. You get stuff stuff like this: "Ignore these for now" ... import statements and setup "Now lets talk about X" ... some code "Remember those import statements? Lets talk about them, but scroll up to see the code because I can't repeat it." ... "Oh yeah, and we could have done this on X, but again, can't show any code." ...
  - bennofs 8y ago
    With the noweb syntax, I think you can just write: <<imports>> code you want to show and then later define what `imports` should be.
    - andreareina 8y ago
      Indeed. You can also define several `imports` blocks, and the contents of each block will be concatenated during expansion. So your literate source can have the imports located near where they're used, but in the tangled code they'll be up top where most people expect them to be.
  - Jtsummers 8y ago
    That's not an issue with org-mode itself though people may use it that way. That's one thing I love about it. All the boiler plate crap? Tucked away in an appendix or an unexported heading (so not displayed in HTML or PDF form).
  - detaro 8y ago
    Literate programming tools generally support ordering blocks differently in the human-version of the document, exactly so what you describe is not necessary. And one of the reasons why special tools for exist, instead of just putting many comments in a file.
  - EliRivers 8y ago
    That doesn't sound right. You absolutely don't have to have the literate document follow the order of the code. Absolutely not. One writes the sections and the pieces in a sensible order in the literate document, and the processing of the document into pure code reorders appropriately. Here's something I wrote before, based on my own experiences. A key phrase from what follows is When the text was munged, a beautiful pdf document containing all the code and all the commentary laid out in a sensible order was created for humans to read, and the source code was also created for the compiler to eat. People seemed to find it useful then, so maybe they will now as well: A previous employer (a subdivision of a global top ten defence company) used literate programming. The project I worked on was a decade-long piece for a consortium of defence departments from various countries. We wrote in objective-C, targeting Windows and Linux. All code was written in a noweb-style markup, such that a top level of a code section would look something like this: <<Initialise hardware>> <<Establish networking>> and so on, and each of those variously break out into smaller chunks <<Fetch next data packet>> <<Decode data packet>> <<Store information from data packet>> <<Create new message based on new information>> The layout of the chunks often ended up matching functions in the source code and other such code constructs, but that wasn't by design; the intention of the chunks was to tell a sensible story of design for the human to understand. Some groups of chunks would get commentary, discussing at a high level the design that they were meeting. Ultimately, the actual code of a bottom-level chunk would be written with accompanying text commentary. Commentary, though, not like the kind of comments you put inside the code. These were sections of proper prose going above each chunk (at the bottom level, chunks were pretty small and modular). They would be more a discussion of the purpose of this section of the code, with some design (and sometimes diagrams) bundled with it. When the text was munged, a beautiful pdf document containing all the code and all the commentary laid out in a sensible order was created for humans to read, and the source code was also created for the compiler to eat. The only time anyone looked directly at the source code was to check that the munging was working properly, and when debugging; there was no point working directly on a source code file, of course, because the next time you munged the literate text the source code would be newly written from that. It worked. It worked well. But it demanded discipline. Code reviews were essential (and mandatory), but every code review was thus as much a design review as a code review, and the text and diagrams were being reviewed as much as the design; it wasn't enough to just write good code - the text had to make it easy for someone fresh to it to understand the design and layout of the code. The chunks helped a lot. If you had a chunk you'd called <<Initialise hardware>>, that's all you'd put in it. There was no sneaking not-quite-relevant code in. The top-level design was easy to see in how the chunks were laid out. If you found that you couldn't quite fit what was needed into something, the design needed revisiting. It forced us to keep things clean, modular and simple. It meant doing everything took longer the first time, but at the point of actually writing the code, the coder had a really good picture of exactly what it had to do and exactly where it fitted in to the grander scheme. There was little revisiting or rewriting, and usually the first version written was the last version written. It also made debugging a lot easier. Over the four years I was working there, we made a number of deliveries to the customers for testing and integration, and as I recall they never found a single bug (which is not to say it was bug free, but they never did anything with it that we hadn't planned for and tested). The testing was likewise very solid and very thorough (tests were rightly based on the requirements and the interfaces as designed), but I like to think that the literate programming style enforced a high quality of code (and it certainly meant that the code did meet the design, which did meet the requirements). Of course, we did have the massive advantage that the requirements were set clearly, in advance, and if they changed it was slowly and with plenty of warning. If you've not worked with requirements like that, you might be surprised just how solid you can make the code when you know before touching the keyboard for the first time exactly what the finished product is meant to do. Why don't I see it elsewhere? I suspect lots of people have simply never considered coding in a literate style - never knew it existed. If forces a change to how a lot of people code. Big design, up front. Many projects, especially small projects (by which I mean less than a year from initial ideas to having something in the hands of customers) in which the final product simply isn't known in advance (and thus any design is expected to change, a lot, quickly) are probably not suited - the extra drag literate programming would put on it would lengthen the time of iterative periods. It required a lot of discipline, at lots of levels. It goes against the still popular narrative of some genius coder banging out something as fast as he can think it. Every change beyond the trivial has to be reviewed, and reviewed properly. All our reviews were done on the printed PDFs, marked up with pen. Front sheets stapled to them, listing code comments which the coder either dealt with or, in discussion, they agreed with the reviewer that the comment would be withdrawn. A really good days' work might be a half-dozen code reviews for some other coders, and touching your own keyboard only to print out the PDFs. Programmers who gathered a reputation for doing really good thorough reviews with good comments and the ability to critique people's code without offending anyone's precious sensibilities (we've all met them; people who seem to lose their sense of objectivity completely when it comes to their own code) were in demand, and it was a valued and recognised skill (being an ace at code reviews should be something we all want to put on our CVs, but I suspect a lot of employers basically never see it there) - I have definitely worked in some places in which, if a coder isn't typing, they're seen as not working, so management would have to be properly on board. I don't think literate programming is incompatible with the original agile manifesto, but I think it wouldn't survive in what that seems to have turned into.
- chrissoundz 8y ago
  You mentioned writing a tutorial and being able to execute the result, so thought I'd mention a project I'm working on to do exactly this. It started when I got annoyed with how difficult it was to write a 'code' tutorial. Essentially it allows you commit a markdown file (or any other format) alongside your source code and you can embed source code snippets / shell commands into this file - which then gets rendered into the main 'output' (the tutorial / article). https://github.com/chrissound/GitChapter https://github.com/chrissound/GitChapter
antt 8y ago
>Put another way, how can you beat thoughtful variable or function names and judicious comments? Writing why and how you did something. The setup for org lets you write the equivalent of jupyter notebooks for every language, which means that you can show a toy implementation of what you're doing before the real code. This toy code is live, completely independent of the rest of the project and can be poked at by anyone opening the org file without screwing anything else up. I have programs written in org-mode which tangle and weave not only the code but the devops. Chapter 1 is the setup for the system, chapter 2-N is code, chapter N+1 launches the app. I have only really done it with python and C so far. But I managed to get Scala setup today in less time than it would take to get an ide working. The tools are still immature, tangle especially needs much finer, and better documented control. But even in this state it's the only tool I can use to write programs which I can pickup three years later and grok in an afternoon. The only downside is that the development for the tool is stuck in the 90s with emails for bugs and patches.
- jf 8y ago
  I'd love to see your org files, do you have any online that you can share?
  - antt 8y ago
    Sure: https://github.com/ant-t/LiterateHelloWorld https://github.com/ant-t/LiterateHelloWorld I've gotten interest in running a workshop on org mode for literate programming so expect for that to be filled in the next week or two.
Gandria 8y ago
I think most professional cameras have a voice annotation button, for recording scene and subject notes. At least my D2X did, which was the last pro DSLR I could justify. I keep wishing for the same button, mid line, every time I'm working around some kind of"proprietary" serialisation* I find in in house projects. *I tried to coin the acronym OBSEC to mean security by obscurity, fifteen years ago.. but I have since figured how much pointed derogatory comment serves no enduring purpose unless it at least, like SNAFU and FUBAR releases the frustration felt by the observer.

blattimwind 8y ago

Program design.

ljm 8y ago

Literate programming might be my least favourite approach when writing normal code, because you're going to read the prose only once or twice and then find that it gets in the way of the implementation you're concerned about. Maintaining something done with literate programming must be an absolute nightmare if you've got any kind of refactoring, because you basically have to rewrite at least a few paragraphs so the prose continues to make sense. The code is more important than the prose. But when it comes to writing a tutorial that you can also execute to get the final result, or interact with along the way, then there is nothing better because, really, you're more interested in the prose that lends explanation to the code you're showcasing. Although this article's about emacs and I've taken the literate approach to my own config too. That's less for the benefit of adding paragraphs of explanation to my config, but because org mode offers a fantastic way to organise code in one file (which is more often than not what an emacs config tends to be). In either case I don't think you can do better than being thoughtful when writing code, thinking about the bigger picture and not just the individual functions and variables in isolation. For others and also yourself. This is also true for emacs itself, considering the idiosyncrasies it presents when most of us learn elisp by copying someone else, rather than reading the language docs.

ModernMech 8y ago
> The code is more important than the prose. This is not the position of someone programming in a literate fashion. Their contention is that prose and code are complimentary. Code explains what the program does, prose explains why it does it that way. They work together to paint a coherent view of your program. With just code, a newcomer reading your program and understanding it 100% is still left with many questions unanswered.
Retra 8y ago
The main problem I have with literate programming for tutorials is that it forces your writing order to match the code order. You get stuff stuff like this: "Ignore these for now" ... import statements and setup "Now lets talk about X" ... some code "Remember those import statements? Lets talk about them, but scroll up to see the code because I can't repeat it." ... "Oh yeah, and we could have done this on X, but again, can't show any code." ...
- bennofs 8y ago
  With the noweb syntax, I think you can just write: <<imports>> code you want to show and then later define what `imports` should be.
  - andreareina 8y ago
    Indeed. You can also define several `imports` blocks, and the contents of each block will be concatenated during expansion. So your literate source can have the imports located near where they're used, but in the tangled code they'll be up top where most people expect them to be.
- Jtsummers 8y ago
  That's not an issue with org-mode itself though people may use it that way. That's one thing I love about it. All the boiler plate crap? Tucked away in an appendix or an unexported heading (so not displayed in HTML or PDF form).
- detaro 8y ago
  Literate programming tools generally support ordering blocks differently in the human-version of the document, exactly so what you describe is not necessary. And one of the reasons why special tools for exist, instead of just putting many comments in a file.
- EliRivers 8y ago
  That doesn't sound right. You absolutely don't have to have the literate document follow the order of the code. Absolutely not. One writes the sections and the pieces in a sensible order in the literate document, and the processing of the document into pure code reorders appropriately. Here's something I wrote before, based on my own experiences. A key phrase from what follows is When the text was munged, a beautiful pdf document containing all the code and all the commentary laid out in a sensible order was created for humans to read, and the source code was also created for the compiler to eat. People seemed to find it useful then, so maybe they will now as well: A previous employer (a subdivision of a global top ten defence company) used literate programming. The project I worked on was a decade-long piece for a consortium of defence departments from various countries. We wrote in objective-C, targeting Windows and Linux. All code was written in a noweb-style markup, such that a top level of a code section would look something like this: <<Initialise hardware>> <<Establish networking>> and so on, and each of those variously break out into smaller chunks <<Fetch next data packet>> <<Decode data packet>> <<Store information from data packet>> <<Create new message based on new information>> The layout of the chunks often ended up matching functions in the source code and other such code constructs, but that wasn't by design; the intention of the chunks was to tell a sensible story of design for the human to understand. Some groups of chunks would get commentary, discussing at a high level the design that they were meeting. Ultimately, the actual code of a bottom-level chunk would be written with accompanying text commentary. Commentary, though, not like the kind of comments you put inside the code. These were sections of proper prose going above each chunk (at the bottom level, chunks were pretty small and modular). They would be more a discussion of the purpose of this section of the code, with some design (and sometimes diagrams) bundled with it. When the text was munged, a beautiful pdf document containing all the code and all the commentary laid out in a sensible order was created for humans to read, and the source code was also created for the compiler to eat. The only time anyone looked directly at the source code was to check that the munging was working properly, and when debugging; there was no point working directly on a source code file, of course, because the next time you munged the literate text the source code would be newly written from that. It worked. It worked well. But it demanded discipline. Code reviews were essential (and mandatory), but every code review was thus as much a design review as a code review, and the text and diagrams were being reviewed as much as the design; it wasn't enough to just write good code - the text had to make it easy for someone fresh to it to understand the design and layout of the code. The chunks helped a lot. If you had a chunk you'd called <<Initialise hardware>>, that's all you'd put in it. There was no sneaking not-quite-relevant code in. The top-level design was easy to see in how the chunks were laid out. If you found that you couldn't quite fit what was needed into something, the design needed revisiting. It forced us to keep things clean, modular and simple. It meant doing everything took longer the first time, but at the point of actually writing the code, the coder had a really good picture of exactly what it had to do and exactly where it fitted in to the grander scheme. There was little revisiting or rewriting, and usually the first version written was the last version written. It also made debugging a lot easier. Over the four years I was working there, we made a number of deliveries to the customers for testing and integration, and as I recall they never found a single bug (which is not to say it was bug free, but they never did anything with it that we hadn't planned for and tested). The testing was likewise very solid and very thorough (tests were rightly based on the requirements and the interfaces as designed), but I like to think that the literate programming style enforced a high quality of code (and it certainly meant that the code did meet the design, which did meet the requirements). Of course, we did have the massive advantage that the requirements were set clearly, in advance, and if they changed it was slowly and with plenty of warning. If you've not worked with requirements like that, you might be surprised just how solid you can make the code when you know before touching the keyboard for the first time exactly what the finished product is meant to do. Why don't I see it elsewhere? I suspect lots of people have simply never considered coding in a literate style - never knew it existed. If forces a change to how a lot of people code. Big design, up front. Many projects, especially small projects (by which I mean less than a year from initial ideas to having something in the hands of customers) in which the final product simply isn't known in advance (and thus any design is expected to change, a lot, quickly) are probably not suited - the extra drag literate programming would put on it would lengthen the time of iterative periods. It required a lot of discipline, at lots of levels. It goes against the still popular narrative of some genius coder banging out something as fast as he can think it. Every change beyond the trivial has to be reviewed, and reviewed properly. All our reviews were done on the printed PDFs, marked up with pen. Front sheets stapled to them, listing code comments which the coder either dealt with or, in discussion, they agreed with the reviewer that the comment would be withdrawn. A really good days' work might be a half-dozen code reviews for some other coders, and touching your own keyboard only to print out the PDFs. Programmers who gathered a reputation for doing really good thorough reviews with good comments and the ability to critique people's code without offending anyone's precious sensibilities (we've all met them; people who seem to lose their sense of objectivity completely when it comes to their own code) were in demand, and it was a valued and recognised skill (being an ace at code reviews should be something we all want to put on our CVs, but I suspect a lot of employers basically never see it there) - I have definitely worked in some places in which, if a coder isn't typing, they're seen as not working, so management would have to be properly on board. I don't think literate programming is incompatible with the original agile manifesto, but I think it wouldn't survive in what that seems to have turned into.
chrissoundz 8y ago
You mentioned writing a tutorial and being able to execute the result, so thought I'd mention a project I'm working on to do exactly this. It started when I got annoyed with how difficult it was to write a 'code' tutorial. Essentially it allows you commit a markdown file (or any other format) alongside your source code and you can embed source code snippets / shell commands into this file - which then gets rendered into the main 'output' (the tutorial / article). https://github.com/chrissound/GitChapter https://github.com/chrissound/GitChapter

antt 8y ago

>Put another way, how can you beat thoughtful variable or function names and judicious comments? Writing why and how you did something. The setup for org lets you write the equivalent of jupyter notebooks for every language, which means that you can show a toy implementation of what you're doing before the real code. This toy code is live, completely independent of the rest of the project and can be poked at by anyone opening the org file without screwing anything else up. I have programs written in org-mode which tangle and weave not only the code but the devops. Chapter 1 is the setup for the system, chapter 2-N is code, chapter N+1 launches the app. I have only really done it with python and C so far. But I managed to get Scala setup today in less time than it would take to get an ide working. The tools are still immature, tangle especially needs much finer, and better documented control. But even in this state it's the only tool I can use to write programs which I can pickup three years later and grok in an afternoon. The only downside is that the development for the tool is stuck in the 90s with emails for bugs and patches.

jf 8y ago
I'd love to see your org files, do you have any online that you can share?
- antt 8y ago
  Sure: https://github.com/ant-t/LiterateHelloWorld https://github.com/ant-t/LiterateHelloWorld I've gotten interest in running a workshop on org mode for literate programming so expect for that to be filled in the next week or two.

Gandria 8y ago

I think most professional cameras have a voice annotation button, for recording scene and subject notes. At least my D2X did, which was the last pro DSLR I could justify. I keep wishing for the same button, mid line, every time I'm working around some kind of"proprietary" serialisation* I find in in house projects. *I tried to coin the acronym OBSEC to mean security by obscurity, fifteen years ago.. but I have since figured how much pointed derogatory comment serves no enduring purpose unless it at least, like SNAFU and FUBAR releases the frustration felt by the observer.