10 ms·

Writing a Book with Unix

Insanity 8y ago
I will give bat a try. That looks nice :)
- O1111OOO 8y ago
  > I will give bat a try. That looks nice :) Two of the three comments already referenced bat. It's also what impressed me the most. It looks great. Link below: https://github.com/sharkdp/bat https://github.com/sharkdp/bat
numbers 8y ago
I like that you mentioned `bat` and `ag`, two of many favorite CLI tools. Others you might want to checkout not necessarily for writing a book but general CLI pleasantness: - fzf (https://github.com/junegunn/fzf https://github.com/junegunn/fzf) - autojump (https://github.com/wting/autojump https://github.com/wting/autojump) - jq (https://stedolan.github.io/jq/ https://stedolan.github.io/jq/) - fd (https://github.com/sharkdp/fd https://github.com/sharkdp/fd)
- grimgrin 8y ago
  Many interested in autojump could probably get what they want out of nice, pure https://github.com/rupa/z https://github.com/rupa/z Here's an alternative to fzf, for comparison's sake: https://github.com/jhawthorn/fzy https://github.com/jhawthorn/fzy
- dredmorbius 8y ago
  jq's a useful utility, but I'm curious as to how you're using a JSON query tool in writing.
- ipozgaj 8y ago
  If you use and like `ag`, I suggest taking a look at ripgrep (`rg`). It seems to be by far the fastest out of three (`ack`, `ag`, `rg`). And it has a pretty interesting codebase (written in Rust).
  - Myrmornis 8y ago
    If you're working in a git repository then IMO the most appropriate search tool is simply `git grep`. I don't think there's any reason to use ripgrep, ag, ack etc in that situation. (Personally, if I'm working with text files, then I'm nearly always in a git repo.)
    - burntsushi 8y ago
      (author of ripgrep here) Well at least one reason is because ripgrep is faster. On simple literal queries they'll have comparable speed, but beyond that, `git grep` is _a lot_ slower. Here's an example on a checkout of the Linux kernel: $ time rg '\w+_PM_RESUME' | wc -l 8 real 0.127 user 0.689 sys 0.589 maxmem 19 MB faults 0 $ time LC_ALL=C git grep -E '\w+_PM_RESUME' | wc -l 8 real 4.607 user 28.059 sys 0.442 maxmem 63 MB faults 0 $ time LC_ALL=en_US.UTF-8 git grep -E '\w+_PM_RESUME' | wc -l 8 real 21.651 user 2:09.54 sys 0.413 maxmem 64 MB faults 0 ripgrep supports Unicode by default, so it's actually comparable to the LC_ALL=en_US.UTF-8 variant. There are other reasons. It is nice to use a single tool for searching in all circumstances. ripgrep can fit that role. Maybe you don't know, but ripgrep respects your .gitignore file.
      - Myrmornis 8y ago
        Thanks! I knew ripgrep was praised in particular for its performance but I didn't know the difference was that large. The repo I usually work in has 8.7M lines of code and I had been finding `git grep` performance very adequate (I use it in combination with the Emacs helm library where it forms part of an incremental search UI, and hence gets called multiple times in quick succession in response to changing search input.) It looks like it will be fun to try swapping in ripgrep as the helm search backend; I'll try it.
freedman1611 8y ago
Misleading title, I thought we has writing his book on an ancient AT&T UNIX mainframe. He's using Linux or MacOS, bfd. Why do people throw around the word Unix to describe Linux is beyond me. MacOS claims it's Unix too, they payed some organization to get them a Unix cert, but we all know it's BSD/Mach/Darwin rewrite. Nobody is really using the original Unix.
- coldtea 8y ago
  It's because most people don't confuse the etymology or history of a term with its actual use and current meaning -- and aren't stuck up with BS pedantic distinctions. There's no "original Unix" (except the first Unix back in the day). There's lineage of operating systems. Heck, even the people who actually created UNIX in the 70s and early 80s don't have such stickups.
timClicks 8y ago
Was hoping that this would be using the original UNIX typesetting tools like troff. Also, surprised to see that asciidoc wasn't deployed over markdown.
- svnpenn 8y ago
  i was recently turned on to asciidoc from seeing that its supported at github https://github.com/github/markup https://github.com/github/markup and ironically development seems to have stalled on markdown with the advent of commonmark https://github.com/commonmark/CommonMark/issues/558 https://github.com/commonmark/CommonMark/issues/558 https://github.com/commonmark/CommonMark/issues/559 https://github.com/commonmark/CommonMark/issues/559 https://github.com/commonmark/CommonMark/issues/560 https://github.com/commonmark/CommonMark/issues/560
  - snazz 8y ago
    Emacs Org-mode is also supported on GitHub and can be integrated into some pretty fancy workflows involving inline LaTeX rendered in Emacs. Org-Babel allows for a “notebook”-esque interface with embedded code samples, which show up non-interactively on GitHub.
  - nitemice 8y ago
    I don't understand how these links indicate that markdown development has stalled. All these issues were only created recently, by one person, and are actually not in the right place for the type of discussion they're trying to prompt. CommonMark have a separate [forum](http://talk.commonmark.org/ http://talk.commonmark.org/) for feature discussion, which seems quite active.
    - jabl 8y ago
      Looking at https://spec.commonmark.org/ https://spec.commonmark.org/ , it does seem it has stalled. They'll need to put in another gear if they're ever going to reach 1.0 (if that's a goal..?).
emgee_1 8y ago
Looks like everything he does can easily accomplished using emacs orgmode and pandoc Ripgrep is maybe an alternative for ag
- omaranto 8y ago
  Or even just Org Mode.
- _emacsomancer_ 8y ago
  Any sufficiently advanced text management system contains an ad-hoc, informally-specified, bug-ridden, slow implementation of half of Org mode.
- coldtea 8y ago
  Is a better ag.
tombert 8y ago
I will not name the company, but a book deal I was working on fell through because I refused to install MS Office (since I didn't have a Mac and I refused to install Windows), they refused to accept markdown or LaTeX, and I couldn't get their template working in LibreOffice. The part I find funny was that the book was about doing network server development with Haskell...on Linux.
- yingw787 8y ago
  Did you end up publishing the manuscript elsewhere?
  - tombert 8y ago
    No; I never finished the book as a consequence of the deal falling through. The work I have is probably hidden somewhere on my NAS, though I suspect now the work would make me cringe since I have improved in my programming skills immensely in the last four years. If you're asking because you want help with some network programming you are doing in Haskell then you can PM me on some kind of social media.
    - cmurf 8y ago
      I suggest publishing it. Or publishing something else that's more relevant to your current or future work. And self-publish. A $30 book that you give to a prospective client is like a $30 business card. There's a wide range of self-publishing methods and services, but there are books on that subject that can help you make that decision, how to price it, and market it, etc. If you can provide a sane PDF, you've done most of the work a printing company cares about; the companies that assist in the self-publishing process should be able to accept almost anything, that's their job. Write it as a reference that even you'd find useful, with emphasis on defining the basics and getting them right, and increasingly lighter when it comes to intermediate and advanced concepts. Those can either form future books or consulting or both. Since you've started writing, it's worth it to go through the whole process and publish. I did it with a conventional big publisher some time ago, but I wouldn't do that again today. The idea a big publisher would require you use Word is very familiar to me, and I think it's ridiculous.
dcchambers 8y ago
I love articles like this. I write quick daily notes on my computer in markdown and back them up in a GitHub repo. (Using this fun little script: https://github.com/dcchambers/note-keeper https://github.com/dcchambers/note-keeper) It's worked really well for me and helps me easily synchronize my notes between systems. I love the elegance and simplicity of plain-text notes.
- noir_lord 8y ago
  vscode has these two phenomenal plugins[1] that together convert vscode into a true journal with basic check lists and the ability to add arbitrary markdown notes to any particular entry. It works wonderfully as a programmer journal since I generally have vscode open anyway (for gitlens even when I'm working in intellij) the friction is close to zero. [1] https://marketplace.visualstudio.com/items?itemName=pajoma.vscode-journal https://marketplace.visualstudio.com/items?itemName=pajoma.v... and https://marketplace.visualstudio.com/items?itemName=Gruntfuggly.vscode-journal-view https://marketplace.visualstudio.com/items?itemName=Gruntfug...
akandiah 8y ago
I'm glad there's always LaTeX. For any serious writing, Microsoft Word is one of the worst pieces of tools out there. It starts showing its ugly side when start using anything remotely advanced.
- _emacsomancer_ 8y ago
  Word is horrible, but then word-processors are generally a horrible paradigm for anything significant. (They can handle basic things, but they're such a hostile environment for creating text, I don't understand how so many novelists work in them.)
  - vinodkd 8y ago
    Maybe some people actually like a graphical UI that does not require learning arcane syntax or keyboard shortcuts, provides a passing simile to actual paper and allows them to be productive (if not proficient)? As for "anything significant", even OP shows one file per chapter. That's very likely what most users of word processors do, I'd imagine, even though Word (for example) had master/child documents support in the late nineties.
    - _emacsomancer_ 8y ago
      No. I get that people think that, but it's a mistaken belief. Word processors are at least as arcane, but they disguise and hide their arcane bits. That is, word processors give an illusion of being easier, but really they end up much more complicated. Graphical UI? There are dozens of text editors that work with TeX and most of them provide graphical UIs, with menus and buttons similar to a word processor. In fact, as soon you want to do anything evenly mildly 'advanced', word processors end up being visibly more complicated. In a word processor, I can click the 'bold' button, or press Ctrl-b to switch into 'bold mode', or highlight some text, and use the button/keyboard shortcut to bold that text. In a TeX editor, I can also click the 'bold' button or press a shortcut to auto-create a LaTeX bold environment `\textbf{}` with my cursor placed in-between the {}s. Alternatively, I can highlight text, e.g 'my text', and click the bold button or press the shortcut and the editor will wrap `\textbf{}' arount the text, producing '\textbf{my text]'. Up to this point the two approaches are equivalent. But now say that I want to make all instances of 'important phrase' bold. With the TeX/editor approach, it's just like any other search and replace, I tell the editor to replace all instances of 'important phrase' with '\textbf{important phrase}'. In the word processor, I have to figure out how to click into an advanced search-and-replace and choose something about replace/add styles etc. In LaTeX, for something complicated, I can figure it out and write my function(s) for it, which are easily re-usable. In a word processor, what one 'knows' in the case of doing something complicated is a series of mouse clicks through menus - which is not only more arcane than an explicit function, but is likely to be disrupted by version changes.
dredmorbius 8y ago
I've been writing on Linux/Unix systems since the late 1980s, including now a few book-length projects. Tools have varied over the years, including some uni work in nroff, HTML, and more recently, LaTeX and Markdown, among other markup languages. LaTeX strikes me as the ultimate tool, and far less intimidating than most people seem to think (Laport's book is an excellent intro), though for most purposes, Markdown is more than sufficient. In practice, I tend to use Markdown and either include inline LaTeX as needed, or convert to LaTeX and continue editing in that where finer-grained control is necessary. I'd add pandoc to the toolkit, as well as GNU Make. With the two, I've got a standard makefile that can output a wide range of formats (I refer to them as "endpoints") ranging from ASCII text to standalone or snippets of HTML, PDFs, PS, MS Word, OSX, Mediawiki, and others. Adding a new endpoint is a simple matter of tweaking the makefile. One piece of organisational advice: Do NOT apply your chapter numbers to your filenames. Instead, allow your principle document outline, using an include structure, to define the flow of the text. Depending on the project size and complexity, I'll either directly include chapters within that level, or have a top level of parts with chapters specified (as second-level includes) within those. As you decide you need to re-work flow, this becomes much easier to manage and rearrange than if you'd pre-labled the files themselves.
- User23 8y ago
  TeX is necessary when you want to go to print and have a high level of quality, especially if any math is involved. For screen display Markdown is sufficient.
  - dredmorbius 8y ago
    In addition to HTML for online viewing, I tend to produce PDFs or ePub formats for reading, either onscreen, on a tablet, or (very rarely) printed. I've discovered that consistent pagination really matters to me for content retention, an argument which still favours PDFs over many alternatives (though Postscript and DJVU offer similar capabilities). Markdown is very nearly fully sufficient, and for almost any nontechnical work with minimal art or layout, will suffice. It falls flat in some interesting areas: There's no underbar/underline markup. There is no native colour markup. There is no formula support -- not something typically encountered in most texts, but when you need it, you need it. There is no fine-grained placement control for callouts, boxes, figures, images, etc. They simply appear where they happen to be dropped on a page. (In several of these cases, you can revert to embedded HTML or styles, which are fine when rendering to HTML, but this won't be picked up by all Pandoc endpoints.) Mind, if a work consists of nothing more than text, bold, italic, strikethrough, super/sub-script, lists, tables, sections, footnotes/endnotes, and images whose placement is not critical, Markdown is entirely sufficient. But if you find you need more control, exporting to LaTeX and doing your final editing there will buy you a great deal more control.
thangalin 8y ago
Of possible interest is my open-source, Java-based desktop Markdown editor with live preview and variable interpolation. * https://github.com/DaveJarvis/scrivenvar https://github.com/DaveJarvis/scrivenvar * https://github.com/DaveJarvis/scrivenvar/blob/master/USAGE.md https://github.com/DaveJarvis/scrivenvar/blob/master/USAGE.m... The software provides a simple way to include variables in technical documentation. It also integrates with an R engine for editing R Markdown files, which can also use variables sourced from an external YAML file. (Editing XML documents that have stylesheets is possible, too.) My authoring workflow involves Scrivenvar, Markdown, pandoc, knitr, and ConTeXt. As Markdown separates content from presentation, I prefer ConTeXt to LaTeX for the same reason.
nategri 8y ago
Was hoping to see some sed tricks, and would have settled for some vim, but I guess I need to stop being such a gatekeeping grump about stuff. Hell, maybe I'd like SublimeText if I tried it.
contras1970 8y ago
i wonder why he has the "wrapper over wc" which does just what wc does? #!/bin/sh total=0 for FILE in `find . -type f -name "*.txt"` do wc -w $FILE words=`wc -w < $FILE | tr -d ' '` total=$(($total + $words)) done printf "%'d" $total echo " words" all this achieves is wc -w $(find . type f -name "*.txt") | sed '$s/total/words/' and frankly, i'm not sure the total->words substitution is worth the trouble. then there's the inefficiency of running wc twice per file. while this is not exactly bitcoin-level disaster, it rubs me the wrong way... wc -w $(find . type f -name "*.txt") | awk -v t=0 ' { print; t += $1 } END { print t, "words"; } ' personally i'd just do this (in zsh): wc -w **/*.txt(.D) the (.D) is two "glob qualifiers": the . (dot) limits the result to plain files, the D turns GLOB_DOTS on for the pattern.
flocial 8y ago
Org mode and pandoc work quite well for a similar workflow. The ability to move around chapter trees in org mode is a godsend. It's crazy to see how far the art of "word processing" deviated from WordStar days. MS Word's proprietary doc binary didn't help either (people would mess up formatting and lose entire documents). It's nice to see the focus come back to content and streamlining production with reproducible formatting.
boomlinde 8y ago
On the "txt" script I want to note that this does the job for files like those in the example that have no whitespace: wc -w $(find . -type f -name "*.txt")
nils-m-holm 8y ago
Looks like the author is just using some apps that happen to run on Unix. I write all my books using vi (not vim!), troff and friends, make, and ghostview (gv) for the layout. Plus a couple of shell/awk/sed scripts for making the TOC, index, etc. I cannot imagine any better tools for the job. I tried LateX, which only got me into trouble, and Lout, which was fun, but too complex in the end. After 20-something books, above turns out to be the sweet spot.
boazbarak 8y ago
I am writing a book on introduction to theoretical computer science in markdown and use pandoc to transforming it into HTML, Latex (and from there to PDF) and MS Word. (The latter format is rather buggy at the moment, but I am including it because I've heard from visually impaired students that it is often the easiest format to read as you can control the font size.) I've now put my scripts on https://github.com/boazbk/tcs/tree/master/scripts https://github.com/boazbk/tcs/tree/master/scripts in case anyone finds them useful. (This is not a "plug and play" package that you can install and use, but people that are better programmers than me might be able to adapt it and improve on it.)
Myrmornis 8y ago
> I could use a git repo to keep a backup of the book and ... ag (basically a faster grep) IMO if you are using git then you should use `git grep` rather than ag/ripgrep/ack/grep etc.

I will give bat a try. That looks nice :)

O1111OOO 8y ago
> I will give bat a try. That looks nice :) Two of the three comments already referenced bat. It's also what impressed me the most. It looks great. Link below: https://github.com/sharkdp/bat https://github.com/sharkdp/bat

I like that you mentioned `bat` and `ag`, two of many favorite CLI tools. Others you might want to checkout not necessarily for writing a book but general CLI pleasantness: - fzf (https://github.com/junegunn/fzf https://github.com/junegunn/fzf) - autojump (https://github.com/wting/autojump https://github.com/wting/autojump) - jq (https://stedolan.github.io/jq/ https://stedolan.github.io/jq/) - fd (https://github.com/sharkdp/fd https://github.com/sharkdp/fd)

grimgrin 8y ago
Many interested in autojump could probably get what they want out of nice, pure https://github.com/rupa/z https://github.com/rupa/z Here's an alternative to fzf, for comparison's sake: https://github.com/jhawthorn/fzy https://github.com/jhawthorn/fzy
dredmorbius 8y ago
jq's a useful utility, but I'm curious as to how you're using a JSON query tool in writing.
ipozgaj 8y ago
If you use and like `ag`, I suggest taking a look at ripgrep (`rg`). It seems to be by far the fastest out of three (`ack`, `ag`, `rg`). And it has a pretty interesting codebase (written in Rust).
- Myrmornis 8y ago
  If you're working in a git repository then IMO the most appropriate search tool is simply `git grep`. I don't think there's any reason to use ripgrep, ag, ack etc in that situation. (Personally, if I'm working with text files, then I'm nearly always in a git repo.)
  - burntsushi 8y ago
    (author of ripgrep here) Well at least one reason is because ripgrep is faster. On simple literal queries they'll have comparable speed, but beyond that, `git grep` is _a lot_ slower. Here's an example on a checkout of the Linux kernel: $ time rg '\w+_PM_RESUME' | wc -l 8 real 0.127 user 0.689 sys 0.589 maxmem 19 MB faults 0 $ time LC_ALL=C git grep -E '\w+_PM_RESUME' | wc -l 8 real 4.607 user 28.059 sys 0.442 maxmem 63 MB faults 0 $ time LC_ALL=en_US.UTF-8 git grep -E '\w+_PM_RESUME' | wc -l 8 real 21.651 user 2:09.54 sys 0.413 maxmem 64 MB faults 0 ripgrep supports Unicode by default, so it's actually comparable to the LC_ALL=en_US.UTF-8 variant. There are other reasons. It is nice to use a single tool for searching in all circumstances. ripgrep can fit that role. Maybe you don't know, but ripgrep respects your .gitignore file.
    - Myrmornis 8y ago
      Thanks! I knew ripgrep was praised in particular for its performance but I didn't know the difference was that large. The repo I usually work in has 8.7M lines of code and I had been finding `git grep` performance very adequate (I use it in combination with the Emacs helm library where it forms part of an incremental search UI, and hence gets called multiple times in quick succession in response to changing search input.) It looks like it will be fun to try swapping in ripgrep as the helm search backend; I'll try it.

freedman1611 8y ago

Misleading title, I thought we has writing his book on an ancient AT&T UNIX mainframe. He's using Linux or MacOS, bfd. Why do people throw around the word Unix to describe Linux is beyond me. MacOS claims it's Unix too, they payed some organization to get them a Unix cert, but we all know it's BSD/Mach/Darwin rewrite. Nobody is really using the original Unix.

coldtea 8y ago
It's because most people don't confuse the etymology or history of a term with its actual use and current meaning -- and aren't stuck up with BS pedantic distinctions. There's no "original Unix" (except the first Unix back in the day). There's lineage of operating systems. Heck, even the people who actually created UNIX in the 70s and early 80s don't have such stickups.

timClicks 8y ago

Was hoping that this would be using the original UNIX typesetting tools like troff. Also, surprised to see that asciidoc wasn't deployed over markdown.

svnpenn 8y ago
i was recently turned on to asciidoc from seeing that its supported at github https://github.com/github/markup https://github.com/github/markup and ironically development seems to have stalled on markdown with the advent of commonmark https://github.com/commonmark/CommonMark/issues/558 https://github.com/commonmark/CommonMark/issues/558 https://github.com/commonmark/CommonMark/issues/559 https://github.com/commonmark/CommonMark/issues/559 https://github.com/commonmark/CommonMark/issues/560 https://github.com/commonmark/CommonMark/issues/560
- snazz 8y ago
  Emacs Org-mode is also supported on GitHub and can be integrated into some pretty fancy workflows involving inline LaTeX rendered in Emacs. Org-Babel allows for a “notebook”-esque interface with embedded code samples, which show up non-interactively on GitHub.
- nitemice 8y ago
  I don't understand how these links indicate that markdown development has stalled. All these issues were only created recently, by one person, and are actually not in the right place for the type of discussion they're trying to prompt. CommonMark have a separate [forum](http://talk.commonmark.org/ http://talk.commonmark.org/) for feature discussion, which seems quite active.
  - jabl 8y ago
    Looking at https://spec.commonmark.org/ https://spec.commonmark.org/ , it does seem it has stalled. They'll need to put in another gear if they're ever going to reach 1.0 (if that's a goal..?).

emgee_1 8y ago

Looks like everything he does can easily accomplished using emacs orgmode and pandoc Ripgrep is maybe an alternative for ag

omaranto 8y ago
Or even just Org Mode.
_emacsomancer_ 8y ago
Any sufficiently advanced text management system contains an ad-hoc, informally-specified, bug-ridden, slow implementation of half of Org mode.
coldtea 8y ago
Is a better ag.

tombert 8y ago

I will not name the company, but a book deal I was working on fell through because I refused to install MS Office (since I didn't have a Mac and I refused to install Windows), they refused to accept markdown or LaTeX, and I couldn't get their template working in LibreOffice. The part I find funny was that the book was about doing network server development with Haskell...on Linux.

yingw787 8y ago
Did you end up publishing the manuscript elsewhere?
- tombert 8y ago
  No; I never finished the book as a consequence of the deal falling through. The work I have is probably hidden somewhere on my NAS, though I suspect now the work would make me cringe since I have improved in my programming skills immensely in the last four years. If you're asking because you want help with some network programming you are doing in Haskell then you can PM me on some kind of social media.
  - cmurf 8y ago
    I suggest publishing it. Or publishing something else that's more relevant to your current or future work. And self-publish. A $30 book that you give to a prospective client is like a $30 business card. There's a wide range of self-publishing methods and services, but there are books on that subject that can help you make that decision, how to price it, and market it, etc. If you can provide a sane PDF, you've done most of the work a printing company cares about; the companies that assist in the self-publishing process should be able to accept almost anything, that's their job. Write it as a reference that even you'd find useful, with emphasis on defining the basics and getting them right, and increasingly lighter when it comes to intermediate and advanced concepts. Those can either form future books or consulting or both. Since you've started writing, it's worth it to go through the whole process and publish. I did it with a conventional big publisher some time ago, but I wouldn't do that again today. The idea a big publisher would require you use Word is very familiar to me, and I think it's ridiculous.

dcchambers 8y ago

I love articles like this. I write quick daily notes on my computer in markdown and back them up in a GitHub repo. (Using this fun little script: https://github.com/dcchambers/note-keeper https://github.com/dcchambers/note-keeper) It's worked really well for me and helps me easily synchronize my notes between systems. I love the elegance and simplicity of plain-text notes.

noir_lord 8y ago
vscode has these two phenomenal plugins[1] that together convert vscode into a true journal with basic check lists and the ability to add arbitrary markdown notes to any particular entry. It works wonderfully as a programmer journal since I generally have vscode open anyway (for gitlens even when I'm working in intellij) the friction is close to zero. [1] https://marketplace.visualstudio.com/items?itemName=pajoma.vscode-journal https://marketplace.visualstudio.com/items?itemName=pajoma.v... and https://marketplace.visualstudio.com/items?itemName=Gruntfuggly.vscode-journal-view https://marketplace.visualstudio.com/items?itemName=Gruntfug...

akandiah 8y ago

I'm glad there's always LaTeX. For any serious writing, Microsoft Word is one of the worst pieces of tools out there. It starts showing its ugly side when start using anything remotely advanced.

_emacsomancer_ 8y ago
Word is horrible, but then word-processors are generally a horrible paradigm for anything significant. (They can handle basic things, but they're such a hostile environment for creating text, I don't understand how so many novelists work in them.)
- vinodkd 8y ago
  Maybe some people actually like a graphical UI that does not require learning arcane syntax or keyboard shortcuts, provides a passing simile to actual paper and allows them to be productive (if not proficient)? As for "anything significant", even OP shows one file per chapter. That's very likely what most users of word processors do, I'd imagine, even though Word (for example) had master/child documents support in the late nineties.
  - _emacsomancer_ 8y ago
    No. I get that people think that, but it's a mistaken belief. Word processors are at least as arcane, but they disguise and hide their arcane bits. That is, word processors give an illusion of being easier, but really they end up much more complicated. Graphical UI? There are dozens of text editors that work with TeX and most of them provide graphical UIs, with menus and buttons similar to a word processor. In fact, as soon you want to do anything evenly mildly 'advanced', word processors end up being visibly more complicated. In a word processor, I can click the 'bold' button, or press Ctrl-b to switch into 'bold mode', or highlight some text, and use the button/keyboard shortcut to bold that text. In a TeX editor, I can also click the 'bold' button or press a shortcut to auto-create a LaTeX bold environment `\textbf{}` with my cursor placed in-between the {}s. Alternatively, I can highlight text, e.g 'my text', and click the bold button or press the shortcut and the editor will wrap `\textbf{}' arount the text, producing '\textbf{my text]'. Up to this point the two approaches are equivalent. But now say that I want to make all instances of 'important phrase' bold. With the TeX/editor approach, it's just like any other search and replace, I tell the editor to replace all instances of 'important phrase' with '\textbf{important phrase}'. In the word processor, I have to figure out how to click into an advanced search-and-replace and choose something about replace/add styles etc. In LaTeX, for something complicated, I can figure it out and write my function(s) for it, which are easily re-usable. In a word processor, what one 'knows' in the case of doing something complicated is a series of mouse clicks through menus - which is not only more arcane than an explicit function, but is likely to be disrupted by version changes.

dredmorbius 8y ago

I've been writing on Linux/Unix systems since the late 1980s, including now a few book-length projects. Tools have varied over the years, including some uni work in nroff, HTML, and more recently, LaTeX and Markdown, among other markup languages. LaTeX strikes me as the ultimate tool, and far less intimidating than most people seem to think (Laport's book is an excellent intro), though for most purposes, Markdown is more than sufficient. In practice, I tend to use Markdown and either include inline LaTeX as needed, or convert to LaTeX and continue editing in that where finer-grained control is necessary. I'd add pandoc to the toolkit, as well as GNU Make. With the two, I've got a standard makefile that can output a wide range of formats (I refer to them as "endpoints") ranging from ASCII text to standalone or snippets of HTML, PDFs, PS, MS Word, OSX, Mediawiki, and others. Adding a new endpoint is a simple matter of tweaking the makefile. One piece of organisational advice: Do NOT apply your chapter numbers to your filenames. Instead, allow your principle document outline, using an include structure, to define the flow of the text. Depending on the project size and complexity, I'll either directly include chapters within that level, or have a top level of parts with chapters specified (as second-level includes) within those. As you decide you need to re-work flow, this becomes much easier to manage and rearrange than if you'd pre-labled the files themselves.

User23 8y ago
TeX is necessary when you want to go to print and have a high level of quality, especially if any math is involved. For screen display Markdown is sufficient.
- dredmorbius 8y ago
  In addition to HTML for online viewing, I tend to produce PDFs or ePub formats for reading, either onscreen, on a tablet, or (very rarely) printed. I've discovered that consistent pagination really matters to me for content retention, an argument which still favours PDFs over many alternatives (though Postscript and DJVU offer similar capabilities). Markdown is very nearly fully sufficient, and for almost any nontechnical work with minimal art or layout, will suffice. It falls flat in some interesting areas: There's no underbar/underline markup. There is no native colour markup. There is no formula support -- not something typically encountered in most texts, but when you need it, you need it. There is no fine-grained placement control for callouts, boxes, figures, images, etc. They simply appear where they happen to be dropped on a page. (In several of these cases, you can revert to embedded HTML or styles, which are fine when rendering to HTML, but this won't be picked up by all Pandoc endpoints.) Mind, if a work consists of nothing more than text, bold, italic, strikethrough, super/sub-script, lists, tables, sections, footnotes/endnotes, and images whose placement is not critical, Markdown is entirely sufficient. But if you find you need more control, exporting to LaTeX and doing your final editing there will buy you a great deal more control.

thangalin 8y ago

Of possible interest is my open-source, Java-based desktop Markdown editor with live preview and variable interpolation. * https://github.com/DaveJarvis/scrivenvar https://github.com/DaveJarvis/scrivenvar * https://github.com/DaveJarvis/scrivenvar/blob/master/USAGE.md https://github.com/DaveJarvis/scrivenvar/blob/master/USAGE.m... The software provides a simple way to include variables in technical documentation. It also integrates with an R engine for editing R Markdown files, which can also use variables sourced from an external YAML file. (Editing XML documents that have stylesheets is possible, too.) My authoring workflow involves Scrivenvar, Markdown, pandoc, knitr, and ConTeXt. As Markdown separates content from presentation, I prefer ConTeXt to LaTeX for the same reason.

nategri 8y ago

Was hoping to see some sed tricks, and would have settled for some vim, but I guess I need to stop being such a gatekeeping grump about stuff. Hell, maybe I'd like SublimeText if I tried it.

contras1970 8y ago

i wonder why he has the "wrapper over wc" which does just what wc does? #!/bin/sh total=0 for FILE in `find . -type f -name "*.txt"` do wc -w $FILE words=`wc -w < $FILE | tr -d ' '` total=$(($total + $words)) done printf "%'d" $total echo " words" all this achieves is wc -w $(find . type f -name "*.txt") | sed '$s/total/words/' and frankly, i'm not sure the total->words substitution is worth the trouble. then there's the inefficiency of running wc twice per file. while this is not exactly bitcoin-level disaster, it rubs me the wrong way... wc -w $(find . type f -name "*.txt") | awk -v t=0 ' { print; t += $1 } END { print t, "words"; } ' personally i'd just do this (in zsh): wc -w **/*.txt(.D) the (.D) is two "glob qualifiers": the . (dot) limits the result to plain files, the D turns GLOB_DOTS on for the pattern.

flocial 8y ago

Org mode and pandoc work quite well for a similar workflow. The ability to move around chapter trees in org mode is a godsend. It's crazy to see how far the art of "word processing" deviated from WordStar days. MS Word's proprietary doc binary didn't help either (people would mess up formatting and lose entire documents). It's nice to see the focus come back to content and streamlining production with reproducible formatting.

boomlinde 8y ago

On the "txt" script I want to note that this does the job for files like those in the example that have no whitespace: wc -w $(find . -type f -name "*.txt")

nils-m-holm 8y ago

Looks like the author is just using some apps that happen to run on Unix. I write all my books using vi (not vim!), troff and friends, make, and ghostview (gv) for the layout. Plus a couple of shell/awk/sed scripts for making the TOC, index, etc. I cannot imagine any better tools for the job. I tried LateX, which only got me into trouble, and Lout, which was fun, but too complex in the end. After 20-something books, above turns out to be the sweet spot.

boazbarak 8y ago

I am writing a book on introduction to theoretical computer science in markdown and use pandoc to transforming it into HTML, Latex (and from there to PDF) and MS Word. (The latter format is rather buggy at the moment, but I am including it because I've heard from visually impaired students that it is often the easiest format to read as you can control the font size.) I've now put my scripts on https://github.com/boazbk/tcs/tree/master/scripts https://github.com/boazbk/tcs/tree/master/scripts in case anyone finds them useful. (This is not a "plug and play" package that you can install and use, but people that are better programmers than me might be able to adapt it and improve on it.)

Myrmornis 8y ago

> I could use a git repo to keep a backup of the book and ... ag (basically a faster grep) IMO if you are using git then you should use `git grep` rather than ag/ripgrep/ack/grep etc.