9 ms·

Tell HN: Groff needs your help

67 points by cogburnd02 12y ago

GNU Groff (which is probably doing the behind-the-scenes formatting work when you use man pages on GNU/Linux) currently has no maintainer, and the most recent stable release was in 2013. Who would like to step up and help?

immita 12y ago
In which the fundamental problem with not paying for your open source software is revealed.
- gress 12y ago
  This is a fundamental problem with open source software.
  - immita 12y ago
    I'd limit it to FOSS that people don't pay for (directly or indirectly). When money is changing hands with the expectation that software is maintained in good working order, you won't see abandonment like this. But when nobody is paying or getting paid, maintenance stops, regardless of whether the software is a key component of crypto on the internet or a fundamental part of the documentation of most Linux installations.
    - lollipop331 12y ago
      oh you mean things like windows XP which people have paid for? I rest my case.
      - cmdrfred 12y ago
        operative word 'have'.
      - geofft 12y ago
        You're talking about the same Windows XP which people are currently paying for and receiving support for? http://www.theguardian.com/technology/2014/apr/07/uk-government-microsoft-windows-xp-public-sector http://www.theguardian.com/technology/2014/apr/07/uk-governm...
  - leereeves 12y ago
    But preferable to what happens to abandoned closed source software.
  - mcguire 12y ago
    This is a fundamental problem with software. The fact that someone might possibly be able to do something about it is a fundamental feature of open source software.
- jasonlotito 12y ago
  Paying for software is no assurance that the software will be maintained.
  - 0942v8653 12y ago
    Exactly. If groff were closed-source, the last stable build would be long before 2013.
DanBC 12y ago
The Groff Mission Statement looks interesting. http://www.gnu.org/software/groff/groff-mission-statement.html http://www.gnu.org/software/groff/groff-mission-statement.ht...
- danieldk 12y ago
  The question is - do we really need the typesetting part? I had a troff phase (bought some books) ten years ago. But the community of users is small, its capabilities are eclipsed by TeX, and the language is quite arcane. One could even go a step further and argue that writing man pages would be much easier (and presumably more man pages would be written) if the markup language was switched to e.g. Markdown by default. For my own software, I am already using Pandoc to generate nroff input from Markdown.
  - cogburnd02 12y ago
    > do we really need the typesetting part? I think so; producing high-quality printed manpages is useful, at least to me. > the community of users is small So tell your friends to use groff! :-D > its capabilities are eclipsed by TeX In what way? Yes, TeX can do complex mathematics better, but most documents that people want to publish don't terribly need complex math that much. > and the language is quite arcane. What specifically is 'arcane' about it, versus something like TeX or markdown? > One could...argue that writing man pages would be much easier...if the markup language was switched to e.g. Markdown by default. But then someone would have to rewrite the existing corpus of manpages, and as others have pointed out, using markdown loses some semantics. I think keeping that functionality in pandoc is the right way to go.
stonogo 12y ago
GNU Groff is terrible and the only reason it hasn't been replaced with a superior implementation is that all the better versions are politically incompatible with GNU. It's no surprise they're having a hard time finding someone interested in effectively code-laundering and slapping a GPL on it.
- thristian 12y ago
  Really? What are these better versions?
  - anjbe 12y ago
    There are a couple of other troff implementations in use today. Mandoc is very new and focuses on providing a complete solution for a system that uses manpages: it renders manpages in the terminal or to HTML on the fly, provides a database for semantic search of manuals, and is probably the second most common manpage renderer in common use (after Groff). It’s the default renderer on OpenBSD, FreeBSD, DragonFly BSD, NetBSD, Illumos, and Minix. Unlike the others, it’s not a complete troff implementation; it focuses on manpages (“mdoc”, “man”, “eqn”, and “tbl” macro sets). Heirloom troff is descended from Sun’s troff, and focuses on nice typesetting supporting various PDF and OpenType features. Plan 9 troff is, well, Plan 9’s troff, and is used by plan9port to render its own manpages. There are a few other troff implementations (Neatroff, etc.), but these are the most widely distributed ones these days. I personally use Mandoc for everything except PDF output, for which I use Groff.
    - cogburnd02 12y ago
      I've looked at the kind of PDFs heirloom troff can make, on an iPad, and my eyes, just, wow. There aren't words. If Debian would put heirloom troff in its base-install (along with, say, E.B Garamond or Junicode) instead of groff, there'd probably be more people (1) making beautiful man pages, and (2) viewing man pages graphically (with evince or another PDF viewer) than ever before. It's not like there's a licensing problem, AFAICT. I have not seen a PDF made by mandoc that made me react that way. :-|
djeps 12y ago
it doesn't look abandoned: http://git.savannah.gnu.org/cgit/groff.git http://git.savannah.gnu.org/cgit/groff.git
lucb1e 12y ago
I'm going to be frank: why? I've recently looked into making my own manpage and it's a pretty old looking system. The docs are not really clear but using some examples and trial and error I got there. My point is though, why does it need a maintainer? The system feels old enough to get deprecated instead of keeping it alive, let alone bring out new releases. I haven't spent more than a day working with groff though, perhaps many people still use it (every website with info just looked about as old as your average man page), so I may be completely missing the point here. If so, tell me!
- immita 12y ago
  Well... manpages are still very much alive and widely used, and they're so prevalent that moving to something new is a nearly ridiculous requirement. Half the things with manpages in the wild are probably themselves not maintained. Now you have two problems. We can move to something new, but getting everyone to agree on what that is, then implementing and maintaining it... isn't that more work than just maintaining the working solution everyone already agrees on, and that solves the problem?
  - tomjen3 12y ago
    Why not format manpages as markdown? As a consumer you wouldn't need to care about the format behind the scene and the format is already well known by programmers, there are plenty of libraries that solve can read markdown and plenty of people who know it.
    - immita 12y ago
      I agree that might be a good choice for a more modern alternative, but it still doesn't solve the problem of existing manpages. You could cleverly upconvert on the fly of course, but that's basically the same thing as keeping groff around, except with the added friction of informing everyone that they should be using Markdown now, with new tooling and conventions. Don't underestimate the marketing costs of successfully convincing the diverse Linux ecosystems they should jump onboard. I only see this working if some big entity slams the gavel and decrees this as the new manpage standard, but I also don't see any such entity having much incentive to do so.
tomjen3 12y ago
Does it currently have any important bugs? Because the troff format was designed with the limitations on early 70's computers in mind and seem to be used exclusively for man pages. Why not upgrade to something that better correspond to how we actually code now (ie markdown)?
- immita 12y ago
  If nobody is maintaining it, how can you tell it doesn't currently have any important bugs? Remember that the worst bugs in the last couple of years have been in the code that nobody was looking at.
  - chm 12y ago
    Some people must have reported bugs since 2013. If not, then either it's unused or bug free?
    - voltagex_ 12y ago
      For one example: https://bugs.debian.org/cgi-bin/pkgreport.cgi?dist=unstable;package=groff https://bugs.debian.org/cgi-bin/pkgreport.cgi?dist=unstable;... I haven't compared this with upstream though.
- kazinator 12y ago
  Does markdown have macros equal in power to those of troff? "The great strength of troff is the flexibility of the basic language and its programmability -- it can be made to do almost any formatting task." -- The UNIX Programming Environment, Brian W. Kernighan and Rob Pike.
justincormack 12y ago
OpenBSD replaced groff with mandoc[1] which is a much simpler program, once you known that it is just for manpages not a typesetting system. [1] http://undeadly.org/cgi?action=article&sid=20110314142734 http://undeadly.org/cgi?action=article&sid=20110314142734
- cogburnd02 12y ago
  > not a typesetting system. I like that groff is a typesetting system, it means that I can generate pretty postscript versions of manpages from the command-line and print them off, all in one go. It's really handy that way.
  - anjbe 12y ago
    You can do that with Mandoc too, although the quality of the PDF output is not as nice. What makes Mandoc less of a typesetting system than Groff is that it supports fewer macro sets (mdoc, man, eqn, and tbl, the macro sets commonly used in manpages); Groff supports more (ms, me, mom…) because it’s a more general typesetting system.
zokier 12y ago
I think the obsession with "being maintained" is somewhat unhealthy phenomenon in the FOSS world. If the code works and does what you want then why would it need constant fiddling and consistent stream of releases?
- hanaganics 12y ago
  Software doesn't exist in a vacuum, and a change in the kernel or the C library could break something that used to work. Worse, you will probably discover this only in production because nobody is keeping tabs or running tests. Worse, you could end up with another Heartbleed. Nobody bothered to look at that code for two years and the result was one of the worst security bugs in history.
- 101914 12y ago
  I would go further and call this an obsession with using software that is literally changing by the day, in a constant state of flux. There seems a desire among those users who comment in forums to see recent commits to the software they use as if infrequent commits or no commits in years suggests there is something wrong with the software. To me, it is the constant changes and updates that give me pause when choosing software. I am actually more skeptical of large, complex software that requires constant updating. I prefer software which can deliver reliable performance year after year without changes, e.g., daemontools. But hey, what do I know? In any event, I am glad to see this comment. I wish more folks would call out this silly obsession. As for groff, I think there is more to roff that just printing manpages. http://heirloom.sourceforge.net/doctools.html http://heirloom.sourceforge.net/doctools.html
  - takluyver 12y ago
    > as if infrequent commits or no commits in years suggests there is something wrong with the software. I would be reluctant to start using a package that wasn't actively developed. Libraries, APIs and even languages evolve, and I don't want to be stuck relying on old versions of other things because of one package that isn't being updated. You might say that all the other pieces should take version compatibility much more seriously, to avoid this problem, but that's not the world we're in. More importantly, since all non trivial software has bugs, if there have been no commits for years, either no-one's using it and finding the bugs, or the maintainer isn't merging people's patches. Both options sound bad for me as a user. I understand that a program could theoretically be 'done', not requiring any further changes, but I think there's hardly any software that could conceivably be so 'done' it wouldn't have any commits in months or years. I've just looked up a few of the most stable major projects I can think of: the Linux kernel, Apache and Subversion all have multiple commits within the last day. GNU coreutils only has five in the last week.
kazinator 12y ago
I am the maintainer of a man page that turns into a 260+ page PDF document in letter size. I haven't hacked on groff, but I recently I did a whole bunch of work on the man2html program from the man tools. That code is extremely hacky, like you wouldn't believe! http://www.kylheku.com/cgit/man/ http://www.kylheku.com/cgit/man/ I have it so that a man page can detect whether it's being compiled by groff or by man2html and re-target some of its macros. That aforementioned large man page is here; the macros are upfront: http://www.kylheku.com/cgit/txr/tree/txr.1 http://www.kylheku.com/cgit/txr/tree/txr.1 HTML and PDF here: http://sourceforge.net/projects/txr/files/txr-104/ http://sourceforge.net/projects/txr/files/txr-104/ (The index and hyperlinks in the HTML are due to a post-processing pass, implemented in the "genman.txr" script.)
- cogburnd02 12y ago
  Keep in mind, groff can itself generate HTML. I think it may need some pre- and post-processing to do it, though too, because (I think) what are links in the HTML version end up being footnotes in the pdf/ps version. Could be wrong about that, though.
  - kazinator 12y ago
    The HTML from groff's HTML back-end is pretty much useless garbage.
    - cogburnd02 12y ago
      Well, get into that code, and improve it! :-)
      - kazinator 12y ago
        No thanks; that is broken by design. HTML is treated as a typesetter device, more or less, when what is needed is a semantic translation of the high level document structure. As an analogy to another software system, you wouldn't want to generate HTML from LaTeX by processing the DVI file. What is needed is the high level macros of a specific package being recognized and translated to HTML at a high level. However hacky, the man2html program does that (and I made it work better: it has better support for handling more sophisticated macros, and is less buggy. I likely won't invest any more time into it, however, and I'm not going anywhere near groff).

67 points by cogburnd02 12y ago

immita 12y ago

In which the fundamental problem with not paying for your open source software is revealed.

gress 12y ago
This is a fundamental problem with open source software.
- immita 12y ago
  I'd limit it to FOSS that people don't pay for (directly or indirectly). When money is changing hands with the expectation that software is maintained in good working order, you won't see abandonment like this. But when nobody is paying or getting paid, maintenance stops, regardless of whether the software is a key component of crypto on the internet or a fundamental part of the documentation of most Linux installations.
  - lollipop331 12y ago
    oh you mean things like windows XP which people have paid for? I rest my case.
    - cmdrfred 12y ago
      operative word 'have'.
    - geofft 12y ago
      You're talking about the same Windows XP which people are currently paying for and receiving support for? http://www.theguardian.com/technology/2014/apr/07/uk-government-microsoft-windows-xp-public-sector http://www.theguardian.com/technology/2014/apr/07/uk-governm...
- leereeves 12y ago
  But preferable to what happens to abandoned closed source software.
- mcguire 12y ago
  This is a fundamental problem with software. The fact that someone might possibly be able to do something about it is a fundamental feature of open source software.
jasonlotito 12y ago
Paying for software is no assurance that the software will be maintained.
- 0942v8653 12y ago
  Exactly. If groff were closed-source, the last stable build would be long before 2013.

DanBC 12y ago

The Groff Mission Statement looks interesting. http://www.gnu.org/software/groff/groff-mission-statement.html http://www.gnu.org/software/groff/groff-mission-statement.ht...

danieldk 12y ago
The question is - do we really need the typesetting part? I had a troff phase (bought some books) ten years ago. But the community of users is small, its capabilities are eclipsed by TeX, and the language is quite arcane. One could even go a step further and argue that writing man pages would be much easier (and presumably more man pages would be written) if the markup language was switched to e.g. Markdown by default. For my own software, I am already using Pandoc to generate nroff input from Markdown.
- cogburnd02 12y ago
  > do we really need the typesetting part? I think so; producing high-quality printed manpages is useful, at least to me. > the community of users is small So tell your friends to use groff! :-D > its capabilities are eclipsed by TeX In what way? Yes, TeX can do complex mathematics better, but most documents that people want to publish don't terribly need complex math that much. > and the language is quite arcane. What specifically is 'arcane' about it, versus something like TeX or markdown? > One could...argue that writing man pages would be much easier...if the markup language was switched to e.g. Markdown by default. But then someone would have to rewrite the existing corpus of manpages, and as others have pointed out, using markdown loses some semantics. I think keeping that functionality in pandoc is the right way to go.

stonogo 12y ago

GNU Groff is terrible and the only reason it hasn't been replaced with a superior implementation is that all the better versions are politically incompatible with GNU. It's no surprise they're having a hard time finding someone interested in effectively code-laundering and slapping a GPL on it.

thristian 12y ago
Really? What are these better versions?
- anjbe 12y ago
  There are a couple of other troff implementations in use today. Mandoc is very new and focuses on providing a complete solution for a system that uses manpages: it renders manpages in the terminal or to HTML on the fly, provides a database for semantic search of manuals, and is probably the second most common manpage renderer in common use (after Groff). It’s the default renderer on OpenBSD, FreeBSD, DragonFly BSD, NetBSD, Illumos, and Minix. Unlike the others, it’s not a complete troff implementation; it focuses on manpages (“mdoc”, “man”, “eqn”, and “tbl” macro sets). Heirloom troff is descended from Sun’s troff, and focuses on nice typesetting supporting various PDF and OpenType features. Plan 9 troff is, well, Plan 9’s troff, and is used by plan9port to render its own manpages. There are a few other troff implementations (Neatroff, etc.), but these are the most widely distributed ones these days. I personally use Mandoc for everything except PDF output, for which I use Groff.
  - cogburnd02 12y ago
    I've looked at the kind of PDFs heirloom troff can make, on an iPad, and my eyes, just, wow. There aren't words. If Debian would put heirloom troff in its base-install (along with, say, E.B Garamond or Junicode) instead of groff, there'd probably be more people (1) making beautiful man pages, and (2) viewing man pages graphically (with evince or another PDF viewer) than ever before. It's not like there's a licensing problem, AFAICT. I have not seen a PDF made by mandoc that made me react that way. :-|

djeps 12y ago

it doesn't look abandoned: http://git.savannah.gnu.org/cgit/groff.git http://git.savannah.gnu.org/cgit/groff.git

lucb1e 12y ago

I'm going to be frank: why? I've recently looked into making my own manpage and it's a pretty old looking system. The docs are not really clear but using some examples and trial and error I got there. My point is though, why does it need a maintainer? The system feels old enough to get deprecated instead of keeping it alive, let alone bring out new releases. I haven't spent more than a day working with groff though, perhaps many people still use it (every website with info just looked about as old as your average man page), so I may be completely missing the point here. If so, tell me!

immita 12y ago
Well... manpages are still very much alive and widely used, and they're so prevalent that moving to something new is a nearly ridiculous requirement. Half the things with manpages in the wild are probably themselves not maintained. Now you have two problems. We can move to something new, but getting everyone to agree on what that is, then implementing and maintaining it... isn't that more work than just maintaining the working solution everyone already agrees on, and that solves the problem?
- tomjen3 12y ago
  Why not format manpages as markdown? As a consumer you wouldn't need to care about the format behind the scene and the format is already well known by programmers, there are plenty of libraries that solve can read markdown and plenty of people who know it.
  - immita 12y ago
    I agree that might be a good choice for a more modern alternative, but it still doesn't solve the problem of existing manpages. You could cleverly upconvert on the fly of course, but that's basically the same thing as keeping groff around, except with the added friction of informing everyone that they should be using Markdown now, with new tooling and conventions. Don't underestimate the marketing costs of successfully convincing the diverse Linux ecosystems they should jump onboard. I only see this working if some big entity slams the gavel and decrees this as the new manpage standard, but I also don't see any such entity having much incentive to do so.

tomjen3 12y ago

Does it currently have any important bugs? Because the troff format was designed with the limitations on early 70's computers in mind and seem to be used exclusively for man pages. Why not upgrade to something that better correspond to how we actually code now (ie markdown)?

immita 12y ago
If nobody is maintaining it, how can you tell it doesn't currently have any important bugs? Remember that the worst bugs in the last couple of years have been in the code that nobody was looking at.
- chm 12y ago
  Some people must have reported bugs since 2013. If not, then either it's unused or bug free?
  - voltagex_ 12y ago
    For one example: https://bugs.debian.org/cgi-bin/pkgreport.cgi?dist=unstable;package=groff https://bugs.debian.org/cgi-bin/pkgreport.cgi?dist=unstable;... I haven't compared this with upstream though.
kazinator 12y ago
Does markdown have macros equal in power to those of troff? "The great strength of troff is the flexibility of the basic language and its programmability -- it can be made to do almost any formatting task." -- The UNIX Programming Environment, Brian W. Kernighan and Rob Pike.

justincormack 12y ago

OpenBSD replaced groff with mandoc[1] which is a much simpler program, once you known that it is just for manpages not a typesetting system. [1] http://undeadly.org/cgi?action=article&sid=20110314142734 http://undeadly.org/cgi?action=article&sid=20110314142734

cogburnd02 12y ago
> not a typesetting system. I like that groff is a typesetting system, it means that I can generate pretty postscript versions of manpages from the command-line and print them off, all in one go. It's really handy that way.
- anjbe 12y ago
  You can do that with Mandoc too, although the quality of the PDF output is not as nice. What makes Mandoc less of a typesetting system than Groff is that it supports fewer macro sets (mdoc, man, eqn, and tbl, the macro sets commonly used in manpages); Groff supports more (ms, me, mom…) because it’s a more general typesetting system.

zokier 12y ago

I think the obsession with "being maintained" is somewhat unhealthy phenomenon in the FOSS world. If the code works and does what you want then why would it need constant fiddling and consistent stream of releases?

hanaganics 12y ago
Software doesn't exist in a vacuum, and a change in the kernel or the C library could break something that used to work. Worse, you will probably discover this only in production because nobody is keeping tabs or running tests. Worse, you could end up with another Heartbleed. Nobody bothered to look at that code for two years and the result was one of the worst security bugs in history.
101914 12y ago
I would go further and call this an obsession with using software that is literally changing by the day, in a constant state of flux. There seems a desire among those users who comment in forums to see recent commits to the software they use as if infrequent commits or no commits in years suggests there is something wrong with the software. To me, it is the constant changes and updates that give me pause when choosing software. I am actually more skeptical of large, complex software that requires constant updating. I prefer software which can deliver reliable performance year after year without changes, e.g., daemontools. But hey, what do I know? In any event, I am glad to see this comment. I wish more folks would call out this silly obsession. As for groff, I think there is more to roff that just printing manpages. http://heirloom.sourceforge.net/doctools.html http://heirloom.sourceforge.net/doctools.html
- takluyver 12y ago
  > as if infrequent commits or no commits in years suggests there is something wrong with the software. I would be reluctant to start using a package that wasn't actively developed. Libraries, APIs and even languages evolve, and I don't want to be stuck relying on old versions of other things because of one package that isn't being updated. You might say that all the other pieces should take version compatibility much more seriously, to avoid this problem, but that's not the world we're in. More importantly, since all non trivial software has bugs, if there have been no commits for years, either no-one's using it and finding the bugs, or the maintainer isn't merging people's patches. Both options sound bad for me as a user. I understand that a program could theoretically be 'done', not requiring any further changes, but I think there's hardly any software that could conceivably be so 'done' it wouldn't have any commits in months or years. I've just looked up a few of the most stable major projects I can think of: the Linux kernel, Apache and Subversion all have multiple commits within the last day. GNU coreutils only has five in the last week.

kazinator 12y ago

I am the maintainer of a man page that turns into a 260+ page PDF document in letter size. I haven't hacked on groff, but I recently I did a whole bunch of work on the man2html program from the man tools. That code is extremely hacky, like you wouldn't believe! http://www.kylheku.com/cgit/man/ http://www.kylheku.com/cgit/man/ I have it so that a man page can detect whether it's being compiled by groff or by man2html and re-target some of its macros. That aforementioned large man page is here; the macros are upfront: http://www.kylheku.com/cgit/txr/tree/txr.1 http://www.kylheku.com/cgit/txr/tree/txr.1 HTML and PDF here: http://sourceforge.net/projects/txr/files/txr-104/ http://sourceforge.net/projects/txr/files/txr-104/ (The index and hyperlinks in the HTML are due to a post-processing pass, implemented in the "genman.txr" script.)

cogburnd02 12y ago
Keep in mind, groff can itself generate HTML. I think it may need some pre- and post-processing to do it, though too, because (I think) what are links in the HTML version end up being footnotes in the pdf/ps version. Could be wrong about that, though.
- kazinator 12y ago
  The HTML from groff's HTML back-end is pretty much useless garbage.
  - cogburnd02 12y ago
    Well, get into that code, and improve it! :-)
    - kazinator 12y ago
      No thanks; that is broken by design. HTML is treated as a typesetter device, more or less, when what is needed is a semantic translation of the high level document structure. As an analogy to another software system, you wouldn't want to generate HTML from LaTeX by processing the DVI file. What is needed is the high level macros of a specific package being recognized and translated to HTML at a high level. However hacky, the man2html program does that (and I made it work better: it has better support for handling more sophisticated macros, and is less buggy. I likely won't invest any more time into it, however, and I'm not going anywhere near groff).