6 ms·

Is there a BNF grammar of the TeX language? (2010)

61 points by nequo 2y ago

ginko 2y ago
No wonder Knuth hasn't gotten around to writing the parsing chapter of TAOCP.
behnamoh 2y ago
[flagged]
- bradrn 2y ago
  I don’t think anyone disputes that it is possible to typeset documents like this in Microsoft Word, given sufficient effort. But in LaTeX, I can create documents which look far better than this with far less work. It’s not simply a matter of ‘disliking mouse-and-click’: LaTeX is designed for high-quality typesetting, in ways which Word is not. (To take just two examples, it enables kerning and ligatures by default.)
  - behnamoh 2y ago
    Kerning and ligatures in Word: Simply CMD+D (fonts) and enable anything you want. My point is the marginal benefit of using Latex to make documents slightly prettier is negligible. The focus of the researcher should be on the content. Don't use Latex for 6 months and see if you still remember all the necessary syntax.
    - bradrn 2y ago
      > Simply CMD+D (fonts) and enable anything you want. Like I said, you can do it. That’s why I said ‘by default’: the defaults clearly show how LaTeX is optimised for high-quality typesetting to a greater extent than Word is. > My point is the marginal benefit of using Latex to make documents slightly prettier is negligible. I disagree with this: it’s not just a matter of being ‘slightly prettier’. I will make the claim that, if you care about typography, LaTeX is easier to use than Word. Given the same amount of time and effort, a document created in LaTeX will look better than one made in Word. For instance, take the paper you linked. You mentioned that, in order to create it in Word, you needed to make ‘lots of Word styles’, etc. etc. By contrast, simply retyping the text into a minimal LaTeX document will result in an output which is as good, if not better. I’m not going to bother wrestling with Word styles and whatnot, if I can create a far better-looking document in LaTeX with less effort. > The focus of the researcher should be on the content. And it is, when I use LaTeX. The whole point of LaTeX was always to separate design from content, and I feel it succeeds pretty well in this.
      - behnamoh 2y ago
        > You mentioned that, in order to create it in Word, you needed to make ‘lots of Word styles’, etc. etc. All of that is done just once (one fixed cost). I've been using the template for years now. I think it's a matter of taste as well. I prefer WYSIWYG; Tex users prefer to see this instead: \documentclass{article} \begin{document} \begin{itemize} \item \textbf{TEXT} \begin{itemize} \item TEXT \textit{TEXT} TEXT \item TEXT \textit{TEXT},$\Phi(\lambda$) \textit{TEXT} \end{itemize} \end{itemize} \end{document}
    - compsciphd 2y ago
      I was a systems phd, so the amount of math I used in latex, was minimal. In practice, i used very little of latex's features, and what I did, was kept in a small text notes file that I could copy/paste from as needed (i.e. inserting an image / table mostly).
- anta40 2y ago
  That's why I use Lyx. WYSWYIG-enough like MS Word, but still can resort to good old Tex when needed. Sure, some documents are easier to write in Word. My personal experience with TeX is kinda similar to HTML: you still remember some parts even if you don't write it for many years...
- mrweasel 2y ago
  Because you can put your (La)TeX documents in source control, split them up like you would with code and assign different authors to different chapters or section, without disturbing each other or having to worry about merging the document later. One of Words big issues is that very few people know how to use it. It's such a massive program, with an absolutely insane feature set. Still you see people manually doing stuff like setting font size to do headings, rather than actually using the feature to set and style headings, but now they can't use the automatic index, so they also do that manually. It's also funny to me that companies will complain about UI changes, because that will force them to retrain staff, but most companies don't do training on Windows, Word or Excel to begin with. LaTeX is stable, most people I know who still use it have templates they made at college and they've been using them for 20 years. They can just sit down and start typing, not having to worry about leaving their preferred editor or styling. Anything can be looked up in their old worn out copy of "The LaTeX Companion" (To one with the St. Bernard on the cover. Retraining to use Word would be a major hassle.
  - behnamoh 2y ago
    > [Word] It's such a massive program, with an absolutely insane feature set. Latex books (even those that only talk about one aspect of it such as graphics) are +900 pages! https://www.latex-project.org/help/books/ https://www.latex-project.org/help/books/
  - MrJohz 2y ago
    > One of Words big issues is that very few people know how to use it. It's such a massive program, with an absolutely insane feature set. This seems equally applicable to LaTeX, though, at least once you start adding in a handful of modules. Like with Word, most people I know seem to learn the subset of LaTeX that solves their immediate problems, and then have to Google random snippets for anything outside of that range. You say people have templates that they've used with no changes for the last 20 years, but in my experience that's because they don't remember how half the things they put in there work, and so any change would take days to implement. I think LaTeX is a great example of the difference between inherent complexity and added complexity in software development. Typesetting is an inherently complex problem, and LaTeX is excellent in that solves this problem very well, although to a certain extent users of LaTeX will always need to understand that complexity somewhat - it is inherent in the problem they are dealing with. But it also creates added complexity by providing this solution in the form of the TeX language and the various macros and components that make up LaTeX. This added complexity is not needed to solve the inherently complex core task, but the best tool to solve the inherently complex task includes this added complexity. I don't mean this comment to be a defence of Word, which has its own issues. And I love reading a well-typeset document, and LaTeX does an excellent job of that. But it is also the sort of software that is long overdue a replacement.
- SassyBird 2y ago
  1. Justified text without hyphenation is a quick recipe for very odd-looking uneven spacing, like in your example. 2. The inset used by the headings is jarringly big. 3. The text is too wide for comfortable reading. 4. The fonts used in the body and in the figures don’t match. Other than that, it’s probably the best-formatted Word document I’ve seen. I wouldn’t want to use Word, because in my experience even outside of maths its UI is working against me. In particular it’s hard to recreate the same document in two different major versions of Word. The spacing, the sizes are always off. Moreover, I don’t like how formatting is very sticky based on where you position your text cursor. I constantly need to disable styling copied from some text nearby.
- senkora 2y ago
  See also WordTex: https://tomwildenhain.com/wordtex/ https://tomwildenhain.com/wordtex/
choeger 2y ago
It's a shame that TeX the great typesetting system comes with TeX the ungodly complex language. There's literally no chance to create good tooling for TeX because of this. There's very little chance to optimize rendering of large documents because of it (e.g., by automatic parallelization). Finally, there's never going to be a readable error message from TeX. TeX the language is the prime example of gatekeeping. OTOH, no one has ever succeeded in creating and popularizing a context-free language that compiles to TeX.
- SassyBird 2y ago
  > Finally, there's never going to be a readable error message from LaTeX. FTFY. If you learn TeX from Knuth’s TeX book, along with the typographical concepts it works with, the error messages generated by documents written in raw TeX are understandable. LaTeX on the other hand is what the metaprogramming-heavy Boost/STL libraries are to C or C++. Abandon all hope.
  - breck 2y ago
    > If you learn TeX from Knuth’s TeX book https://visualmatheditor.equatheque.net/doc/texbook.pdf https://visualmatheditor.equatheque.net/doc/texbook.pdf +1 on this. That book is fantastic, one of my favorites. It's been a couple of years since I last read it, and your comment reminded me that I should probably skim again.
    - WillAdams 2y ago
      Except that PDF shouldn't exist since it was generated in flagrant disregard for the copyright/comment: % This manual is copyright (C) 1984 by the American Mathematical Society. % All rights are reserved! % The file is distributed only for people to see its examples of TeX input, % not for use in the preparation of books like The TeXbook. % Permission for any other use of this file must be obtained in writing % from the copyright holder and also from the publisher (Addison-Wesley). \loop\iftrue \errmessage{This manual is copyrighted and should not be TeXed}\repeat If you want a free PDF on TeX try _TeX for the Impatient_: https://tug.ctan.org/info/impatient/book.pdf https://tug.ctan.org/info/impatient/book.pdf If you want a copy of _The TeXbook_, buy it.
twoodfin 2y ago
If you were willing to live with a Turing-complete grammar in a general purpose language, I wonder how it would ideally work and what you’d be able to gain in trade. Sort of a supercharged version of Lisp reader macros.
- samatman 2y ago
  We already have that: Forth. The above sentence can be a Forth program. This? Another Forth program. What does my post do? Run it to find out! > OK
  - MaxBarraclough 2y ago
    I'm a Forth dabbler too, but its opposite number in the Lisp world also deserves a mention: Racket. Like Forth, Racket can be extended to describe languages radically different from the 'starting point' language. https://en.wikipedia.org/wiki/Racket_(programming_language)#Integrated_language_extensibility_and_macros https://en.wikipedia.org/wiki/Racket_(programming_language)#...
wslh 2y ago
If it would be LaTeX, tree-sitter includes a paraer for it [1]. Not BNF though. [1] https://tree-sitter.github.io/tree-sitter/ https://tree-sitter.github.io/tree-sitter/
thiagoharry 2y ago
METAFONT is another macro language where everything is context sensitive because macros redefine everything. But METAFONT has a BNF grammar that describes the language after the macro expansion. Perhaps the problem with TeX are the catcodes, that redefine at runtime how every future token is interpreted even after macro expansion?
tylerneylon 2y ago
I'm a fan of most of what Knuth has done, and in particular I love the high quality _output_ of the TeX family. But TeX's language is extremely difficult to use. I suspect this is true for any macro-expansion-like language. If you like deep dives, I suggest two follow-up tools for mathematical typesetting: 1. LuaTeX, which is TeX + Lua scripting support. (luatex.org) An example: https://tex.stackexchange.com/questions/70/what-is-a-simple-example-of-something-you-can-do-with-luatex https://tex.stackexchange.com/questions/70/what-is-a-simple-... This has become my default, and I enjoy it. (I use lualatex.) 2. A friend of mine was inspired to write a modern macOS app for math typesetting. It's called MadHat: https://madhat.design/ https://madhat.design/ One of the coolest features is "no compilation," which I view as a slight exaggeration, but what's true is that it can work quickly and incrementally so you don't have a compile-and-wait cycle, even on long documents.
- leephillips 2y ago
  few examples of things you can do with LuaTeX, and a tutorial: https://lwn.net/Articles/731581/ https://lwn.net/Articles/731581/

61 points by nequo 2y ago

ginko 2y ago

No wonder Knuth hasn't gotten around to writing the parsing chapter of TAOCP.

behnamoh 2y ago

[flagged]

bradrn 2y ago
I don’t think anyone disputes that it is possible to typeset documents like this in Microsoft Word, given sufficient effort. But in LaTeX, I can create documents which look far better than this with far less work. It’s not simply a matter of ‘disliking mouse-and-click’: LaTeX is designed for high-quality typesetting, in ways which Word is not. (To take just two examples, it enables kerning and ligatures by default.)
- behnamoh 2y ago
  Kerning and ligatures in Word: Simply CMD+D (fonts) and enable anything you want. My point is the marginal benefit of using Latex to make documents slightly prettier is negligible. The focus of the researcher should be on the content. Don't use Latex for 6 months and see if you still remember all the necessary syntax.
  - bradrn 2y ago
    > Simply CMD+D (fonts) and enable anything you want. Like I said, you can do it. That’s why I said ‘by default’: the defaults clearly show how LaTeX is optimised for high-quality typesetting to a greater extent than Word is. > My point is the marginal benefit of using Latex to make documents slightly prettier is negligible. I disagree with this: it’s not just a matter of being ‘slightly prettier’. I will make the claim that, if you care about typography, LaTeX is easier to use than Word. Given the same amount of time and effort, a document created in LaTeX will look better than one made in Word. For instance, take the paper you linked. You mentioned that, in order to create it in Word, you needed to make ‘lots of Word styles’, etc. etc. By contrast, simply retyping the text into a minimal LaTeX document will result in an output which is as good, if not better. I’m not going to bother wrestling with Word styles and whatnot, if I can create a far better-looking document in LaTeX with less effort. > The focus of the researcher should be on the content. And it is, when I use LaTeX. The whole point of LaTeX was always to separate design from content, and I feel it succeeds pretty well in this.
    - behnamoh 2y ago
      > You mentioned that, in order to create it in Word, you needed to make ‘lots of Word styles’, etc. etc. All of that is done just once (one fixed cost). I've been using the template for years now. I think it's a matter of taste as well. I prefer WYSIWYG; Tex users prefer to see this instead: \documentclass{article} \begin{document} \begin{itemize} \item \textbf{TEXT} \begin{itemize} \item TEXT \textit{TEXT} TEXT \item TEXT \textit{TEXT},$\Phi(\lambda$) \textit{TEXT} \end{itemize} \end{itemize} \end{document}
  - compsciphd 2y ago
    I was a systems phd, so the amount of math I used in latex, was minimal. In practice, i used very little of latex's features, and what I did, was kept in a small text notes file that I could copy/paste from as needed (i.e. inserting an image / table mostly).
anta40 2y ago
That's why I use Lyx. WYSWYIG-enough like MS Word, but still can resort to good old Tex when needed. Sure, some documents are easier to write in Word. My personal experience with TeX is kinda similar to HTML: you still remember some parts even if you don't write it for many years...
mrweasel 2y ago
Because you can put your (La)TeX documents in source control, split them up like you would with code and assign different authors to different chapters or section, without disturbing each other or having to worry about merging the document later. One of Words big issues is that very few people know how to use it. It's such a massive program, with an absolutely insane feature set. Still you see people manually doing stuff like setting font size to do headings, rather than actually using the feature to set and style headings, but now they can't use the automatic index, so they also do that manually. It's also funny to me that companies will complain about UI changes, because that will force them to retrain staff, but most companies don't do training on Windows, Word or Excel to begin with. LaTeX is stable, most people I know who still use it have templates they made at college and they've been using them for 20 years. They can just sit down and start typing, not having to worry about leaving their preferred editor or styling. Anything can be looked up in their old worn out copy of "The LaTeX Companion" (To one with the St. Bernard on the cover. Retraining to use Word would be a major hassle.
- behnamoh 2y ago
  > [Word] It's such a massive program, with an absolutely insane feature set. Latex books (even those that only talk about one aspect of it such as graphics) are +900 pages! https://www.latex-project.org/help/books/ https://www.latex-project.org/help/books/
- MrJohz 2y ago
  > One of Words big issues is that very few people know how to use it. It's such a massive program, with an absolutely insane feature set. This seems equally applicable to LaTeX, though, at least once you start adding in a handful of modules. Like with Word, most people I know seem to learn the subset of LaTeX that solves their immediate problems, and then have to Google random snippets for anything outside of that range. You say people have templates that they've used with no changes for the last 20 years, but in my experience that's because they don't remember how half the things they put in there work, and so any change would take days to implement. I think LaTeX is a great example of the difference between inherent complexity and added complexity in software development. Typesetting is an inherently complex problem, and LaTeX is excellent in that solves this problem very well, although to a certain extent users of LaTeX will always need to understand that complexity somewhat - it is inherent in the problem they are dealing with. But it also creates added complexity by providing this solution in the form of the TeX language and the various macros and components that make up LaTeX. This added complexity is not needed to solve the inherently complex core task, but the best tool to solve the inherently complex task includes this added complexity. I don't mean this comment to be a defence of Word, which has its own issues. And I love reading a well-typeset document, and LaTeX does an excellent job of that. But it is also the sort of software that is long overdue a replacement.
SassyBird 2y ago
1. Justified text without hyphenation is a quick recipe for very odd-looking uneven spacing, like in your example. 2. The inset used by the headings is jarringly big. 3. The text is too wide for comfortable reading. 4. The fonts used in the body and in the figures don’t match. Other than that, it’s probably the best-formatted Word document I’ve seen. I wouldn’t want to use Word, because in my experience even outside of maths its UI is working against me. In particular it’s hard to recreate the same document in two different major versions of Word. The spacing, the sizes are always off. Moreover, I don’t like how formatting is very sticky based on where you position your text cursor. I constantly need to disable styling copied from some text nearby.
senkora 2y ago
See also WordTex: https://tomwildenhain.com/wordtex/ https://tomwildenhain.com/wordtex/

choeger 2y ago

It's a shame that TeX the great typesetting system comes with TeX the ungodly complex language. There's literally no chance to create good tooling for TeX because of this. There's very little chance to optimize rendering of large documents because of it (e.g., by automatic parallelization). Finally, there's never going to be a readable error message from TeX. TeX the language is the prime example of gatekeeping. OTOH, no one has ever succeeded in creating and popularizing a context-free language that compiles to TeX.

SassyBird 2y ago
> Finally, there's never going to be a readable error message from LaTeX. FTFY. If you learn TeX from Knuth’s TeX book, along with the typographical concepts it works with, the error messages generated by documents written in raw TeX are understandable. LaTeX on the other hand is what the metaprogramming-heavy Boost/STL libraries are to C or C++. Abandon all hope.
- breck 2y ago
  > If you learn TeX from Knuth’s TeX book https://visualmatheditor.equatheque.net/doc/texbook.pdf https://visualmatheditor.equatheque.net/doc/texbook.pdf +1 on this. That book is fantastic, one of my favorites. It's been a couple of years since I last read it, and your comment reminded me that I should probably skim again.
  - WillAdams 2y ago
    Except that PDF shouldn't exist since it was generated in flagrant disregard for the copyright/comment: % This manual is copyright (C) 1984 by the American Mathematical Society. % All rights are reserved! % The file is distributed only for people to see its examples of TeX input, % not for use in the preparation of books like The TeXbook. % Permission for any other use of this file must be obtained in writing % from the copyright holder and also from the publisher (Addison-Wesley). \loop\iftrue \errmessage{This manual is copyrighted and should not be TeXed}\repeat If you want a free PDF on TeX try _TeX for the Impatient_: https://tug.ctan.org/info/impatient/book.pdf https://tug.ctan.org/info/impatient/book.pdf If you want a copy of _The TeXbook_, buy it.

twoodfin 2y ago

If you were willing to live with a Turing-complete grammar in a general purpose language, I wonder how it would ideally work and what you’d be able to gain in trade. Sort of a supercharged version of Lisp reader macros.

samatman 2y ago
We already have that: Forth. The above sentence can be a Forth program. This? Another Forth program. What does my post do? Run it to find out! > OK
- MaxBarraclough 2y ago
  I'm a Forth dabbler too, but its opposite number in the Lisp world also deserves a mention: Racket. Like Forth, Racket can be extended to describe languages radically different from the 'starting point' language. https://en.wikipedia.org/wiki/Racket_(programming_language)#Integrated_language_extensibility_and_macros https://en.wikipedia.org/wiki/Racket_(programming_language)#...

wslh 2y ago

If it would be LaTeX, tree-sitter includes a paraer for it [1]. Not BNF though. [1] https://tree-sitter.github.io/tree-sitter/ https://tree-sitter.github.io/tree-sitter/

thiagoharry 2y ago

METAFONT is another macro language where everything is context sensitive because macros redefine everything. But METAFONT has a BNF grammar that describes the language after the macro expansion. Perhaps the problem with TeX are the catcodes, that redefine at runtime how every future token is interpreted even after macro expansion?

tylerneylon 2y ago

I'm a fan of most of what Knuth has done, and in particular I love the high quality _output_ of the TeX family. But TeX's language is extremely difficult to use. I suspect this is true for any macro-expansion-like language. If you like deep dives, I suggest two follow-up tools for mathematical typesetting: 1. LuaTeX, which is TeX + Lua scripting support. (luatex.org) An example: https://tex.stackexchange.com/questions/70/what-is-a-simple-example-of-something-you-can-do-with-luatex https://tex.stackexchange.com/questions/70/what-is-a-simple-... This has become my default, and I enjoy it. (I use lualatex.) 2. A friend of mine was inspired to write a modern macOS app for math typesetting. It's called MadHat: https://madhat.design/ https://madhat.design/ One of the coolest features is "no compilation," which I view as a slight exaggeration, but what's true is that it can work quickly and incrementally so you don't have a compile-and-wait cycle, even on long documents.

leephillips 2y ago
few examples of things you can do with LuaTeX, and a tutorial: https://lwn.net/Articles/731581/ https://lwn.net/Articles/731581/