7 ms·

LuaTeX Comes of Age (2017)

134 points by goranmoomin 5y ago

amichail 5y ago
Here's something better that doesn't use TeX and is fully WYSIWYG: http://texmacs.org/tmweb/home/videos.en.html http://texmacs.org/tmweb/home/videos.en.html
- resoluteteeth 5y ago
  TeXmacs doesn't use TeX?
  - Snarwin 5y ago
    > Documents can be saved in TeXmacs, Xml or Scheme format and printed as Pdf or Postscript files. Converters exist for TeX/LaTeX and Html/Mathml. Notice that TeXmacs is not based on TeX/LaTeX. From http://texmacs.org/tmweb/home/welcome.en.html http://texmacs.org/tmweb/home/welcome.en.html
  - PaulHoule 5y ago
    It doesn’t use emacs either!
- leephillips 5y ago
  In my judgement, it’s worse.
  - amichail 5y ago
    Why?
  - forgotpwd16 5y ago
    Without any arguments this is an empty statement. And to be fair so is parent comment (the "better" part).
    - leephillips 5y ago
      Indeed. What was it the late Hitchens liked to say? Something that can be asserted without evidence can be denied without evidence. Here’s a clue: TeX has no bugs.
      - yakubin 5y ago
        "dismissed", not "denied". There's a subtle difference. The exact quote: > That which can be asserted without evidence, can be dismissed without evidence.
      - amichail 5y ago
        The current TeXmacs release is stable. Requiring no bugs is unrealistic and impedes progress.
OtomotO 5y ago
The article is from 2017. Would be great if that could be added to the headline here.
coliveira 5y ago
LuaTeX is another way to create LaTeX macros, but using Lua as the language to manipulate text. I think it helps create macros that would be too complicated in TeX, but on the other hand it is another language for LaTeX users to learn. I haven't see much advantage of their approach other than in specialized niches such as creating complex figures.
- mr_mitm 5y ago
  LaTeX isn't just for users, there are also people who are developing packages or classes. And since LaTeX doesn't even offer the luxury of arrays or hash tables natively, LuaTeX is very a refreshing change. My company is using their own LuaLaTeX class extensively to produce hundreds, if not thousands of reports per year, and features like automatically computing the CVSS base score of a CVSS vector will take a couple of minutes in lua, while it will cost you several days and a bunch of gray hairs to do in TeX. It's only the developer who has to know any lua though. It's completely transparent to the user.
  - PaulHoule 5y ago
    I took a class in TeX a long time ago and got marked off a point because I used self-modifying macros to save a variable.
    - leephillips 5y ago
      Sounds like it should have been extra credit.
  - nanna 5y ago
    Appreciate you giving an example of a real world use for the Lua in LuaLaTex. Any others appreciated! Like the parent, i've never much come up with a possible application.
- leephillips 5y ago
  I was tickled to see my 2017 article pop up here. Lua is a somewhat odd but perfectly sane language that you can pick up in a day. With a few hooks into TeX you can be writing complex programs that manipulate your document, insert the result of calculations into the text, etc. Writing these things in TeX is insanely difficult. I’ve been using TeX literally since it became available, and I wouldn’t dare try to write anything complicated in TeX the programming language. But Lua is a breeze. Because of this I’m a bit surprised that I haven’t seen more. One big application is the layout algorithm for network diagrams in tikz.
dhosek 5y ago
LuaTeX has some small differences from TeX that are not present in pdfTeX or XeTeX that I find really annoying. Most notably, it will not break lines after – or — (whether entered as Unicode or with traditional TeX input using -- or ---). And the slowness is on pretty much all documents. It also does some weird mapping of character codes in generating the pdf files that should be invisible to most users but is evident if one digs into the output files. It could cause issues in malformed pdf readers although as far as I know this isn't an issue. Unless someone specifically needs Lua, it's hard to recommend LuaTeX over XeTeX.
- zauguin 5y ago
  LuaTeX does break after -- and ---, see e.g. \hsize 0pt abc--def \bye I also don't know what kind of weird remapping you are talking about, if you are referring to the Unicode mapping then they are required to get proper copy&paste (and are also generated by other engines these days).
svat 5y ago
LuaTeX is what TeX should have been from the start (and would have been, if the technology of the time had permitted it). Apart from modern font-handling and Unicode, the main thing IMO is the hooks it provides (in the form of callbacks). With other TeX engines (mainly: pdfTeX, XeTeX, or the original Knuth TeX), the only "programming" facility is in the form of macros, which were originally added by Knuth only for some simple text substitution to save typing. He never intended to add programming features into TeX, but of course, it doesn't take much to become "accidentally Turing-complete", and that's what TeX macros became. (There's also some rudimentary support for counters/registers, which IIRC Knuth added after he found users were (ab)using macros to encode numbers using unary or Church numerals.) So the only way to influence anything TeX does automatically, whether it's hyphenation or line-breaking or page-breaking or whatever, is to set up some macros whose blind expansion will ultimately—at a remove of several steps—result in the outcome you want, without affecting anything else (e.g. even an accidental space might get typeset, with undesirable results). This of course is cumbersome and error-prone. With LuaTeX you can program these things at the level you actually intend, e.g. while you can still use macros (or use the `process_input_buffer` callback in Lua), you can now also directly, say, influence the page layout, in a Lua callback that is actually aware of data at the relevant level of abstraction (the vboxes on the page, say), rather than trying to do everything at the lowest (text expansion) level. Knuth has said that he never intended for TeX macros to be used as a full-fledged programming language (he expected people would directly edit the SAIL/Pascal code of the TeX program for anything nontrivial), and that he dislikes each tool coming with its own Turing-complete programming language, and that if a standard embeddable programming language had been available he'd of course have used it in TeX — I think Lua would count. You can see "TeX without TeX" page on the LuaTeX wiki for an example of the power of LuaTeX, typesetting with TeX while completely bypassing TeX syntax: http://wiki.luatex.org/index.php/TeX_without_TeX http://wiki.luatex.org/index.php/TeX_without_TeX — a more elaborate production system is Speedata Publisher: https://github.com/speedata/publisher https://github.com/speedata/publisher As for myself, here is the most fancy thing I did with LuaTeX: https://tex.stackexchange.com/questions/401604/book-on-a-single-poster-page/403353#403353 https://tex.stackexchange.com/questions/401604/book-on-a-sin... (a "poster" of Kafka's Metamorphosis done just for that answer, never "productionized" into a package or whatever). Also, something possibly illuminating is this TeX-vs-LuaTeX comparison of something hyphenation-related (https://tex.stackexchange.com/questions/378704/how-to-avoid-line-breaks-that-result-in-short-words-at-line-edges/379802#379802 https://tex.stackexchange.com/questions/378704/how-to-avoid-...), and in this answer (https://tex.stackexchange.com/questions/398310/why-in-2017-does-latex-not-use-tree-like-structures/398372#398372 https://tex.stackexchange.com/questions/398310/why-in-2017-d...) I link to some other times I used LuaTeX, though some of them just use the Lua part of LuaTeX rather than LuaTeX hooks, such as computing digits of pi for generating pretty pictures (https://tex.stackexchange.com/questions/367902/tikz-color-option-for-edges-without-effect/367905#367905 https://tex.stackexchange.com/questions/367902/tikz-color-op...).
lou1306 5y ago
"Modern" TeX processors (LuaTeX or XeTeX) are sadly held back because publishers still cling to PDFLaTex. For instance, ArXiV "fully supports and automatically recognizes PDFLaTeX" and makes no mention of other backends [0]. So, LuaTeX is perfectly fine for personal stuff that you can compile in-house, but basically nobody wants to risk using it in any scenarios where sources are to be sent to a 3rd party for compilation, only to find out that they need to make them PDFLaTeX-compatible. And after all LuaTeX came out of beta in 2016 [1], which is pretty recent in TeX-years. So it kind of makes sense that publishers still prefer the quite battle-tested PDFLaTeX... [0]: https://arxiv.org/help/submit_tex https://arxiv.org/help/submit_tex [1]: https://tug.org/pipermail/luatex/2016-September/006201.html https://tug.org/pipermail/luatex/2016-September/006201.html
thangalin 5y ago
ConTeXt[1] has had Lua integrated since around 2009[2]. KeenWrite[3], the text editor that I'm developing, converts Markdown documents to XML, then can call ConTeXt with a given theme[4] to typeset the document. This allows for complete separation of content from presentation, as described in my Typesetting Markdown series, in which part 8 briefly touches upon Lua[5]. [1]: https://wiki.contextgarden.net/Installation https://wiki.contextgarden.net/Installation [2]: https://tug.org/TUGboat/tb40-1/tb124hagen-lmtx.pdf https://tug.org/TUGboat/tb40-1/tb124hagen-lmtx.pdf [3]: https://github.com/DaveJarvis/keenwrite https://github.com/DaveJarvis/keenwrite [4]: https://github.com/DaveJarvis/keenwrite-themes/ https://github.com/DaveJarvis/keenwrite-themes/ [5]: https://dave.autonoma.ca/blog/2020/04/28/typesetting-markdown-part-8/ https://dave.autonoma.ca/blog/2020/04/28/typesetting-markdow...
- minusf 5y ago
  IIRC context is "just" a macro package on top of luatex (and pdftex before). so whatever luatex has, context has.
dang 5y ago
Discussed at the time: LuaTeX Comes of Age - https://news.ycombinator.com/item?id=15146300 https://news.ycombinator.com/item?id=15146300 - Sept 2017 (28 comments)
strenholme 5y ago
The linked LWN article mentions that it’s difficult to put comments in inline LuaTeX macros because those macros remove newlines. To add a comment to the middle of a line in Lua, use the --[[ comment ]] form: a = 1 --[[ Set a to 1 ]] print(a) If we have a comment that has ]] in it, we can do this instead: a = 1 --[=[ We love [[brackets]] a lot ]=] print(a) The --[[ (or --[=[ ) form also works for multi-line comments a = 1 --[[ Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Dictum fusce ut placerat orci nulla pellentesque dignissim. Quam viverra orci sagittis eu volutpat odio facilisis mauris. ]] print (a)

134 points by goranmoomin 5y ago

Here's something better that doesn't use TeX and is fully WYSIWYG: http://texmacs.org/tmweb/home/videos.en.html http://texmacs.org/tmweb/home/videos.en.html

resoluteteeth 5y ago
TeXmacs doesn't use TeX?
- Snarwin 5y ago
  > Documents can be saved in TeXmacs, Xml or Scheme format and printed as Pdf or Postscript files. Converters exist for TeX/LaTeX and Html/Mathml. Notice that TeXmacs is not based on TeX/LaTeX. From http://texmacs.org/tmweb/home/welcome.en.html http://texmacs.org/tmweb/home/welcome.en.html
- PaulHoule 5y ago
  It doesn’t use emacs either!
leephillips 5y ago
In my judgement, it’s worse.
- amichail 5y ago
  Why?
- forgotpwd16 5y ago
  Without any arguments this is an empty statement. And to be fair so is parent comment (the "better" part).
  - leephillips 5y ago
    Indeed. What was it the late Hitchens liked to say? Something that can be asserted without evidence can be denied without evidence. Here’s a clue: TeX has no bugs.
    - yakubin 5y ago
      "dismissed", not "denied". There's a subtle difference. The exact quote: > That which can be asserted without evidence, can be dismissed without evidence.
    - amichail 5y ago
      The current TeXmacs release is stable. Requiring no bugs is unrealistic and impedes progress.

OtomotO 5y ago

The article is from 2017. Would be great if that could be added to the headline here.

coliveira 5y ago

LuaTeX is another way to create LaTeX macros, but using Lua as the language to manipulate text. I think it helps create macros that would be too complicated in TeX, but on the other hand it is another language for LaTeX users to learn. I haven't see much advantage of their approach other than in specialized niches such as creating complex figures.

mr_mitm 5y ago
LaTeX isn't just for users, there are also people who are developing packages or classes. And since LaTeX doesn't even offer the luxury of arrays or hash tables natively, LuaTeX is very a refreshing change. My company is using their own LuaLaTeX class extensively to produce hundreds, if not thousands of reports per year, and features like automatically computing the CVSS base score of a CVSS vector will take a couple of minutes in lua, while it will cost you several days and a bunch of gray hairs to do in TeX. It's only the developer who has to know any lua though. It's completely transparent to the user.
- PaulHoule 5y ago
  I took a class in TeX a long time ago and got marked off a point because I used self-modifying macros to save a variable.
  - leephillips 5y ago
    Sounds like it should have been extra credit.
- nanna 5y ago
  Appreciate you giving an example of a real world use for the Lua in LuaLaTex. Any others appreciated! Like the parent, i've never much come up with a possible application.
leephillips 5y ago
I was tickled to see my 2017 article pop up here. Lua is a somewhat odd but perfectly sane language that you can pick up in a day. With a few hooks into TeX you can be writing complex programs that manipulate your document, insert the result of calculations into the text, etc. Writing these things in TeX is insanely difficult. I’ve been using TeX literally since it became available, and I wouldn’t dare try to write anything complicated in TeX the programming language. But Lua is a breeze. Because of this I’m a bit surprised that I haven’t seen more. One big application is the layout algorithm for network diagrams in tikz.

dhosek 5y ago

LuaTeX has some small differences from TeX that are not present in pdfTeX or XeTeX that I find really annoying. Most notably, it will not break lines after – or — (whether entered as Unicode or with traditional TeX input using -- or ---). And the slowness is on pretty much all documents. It also does some weird mapping of character codes in generating the pdf files that should be invisible to most users but is evident if one digs into the output files. It could cause issues in malformed pdf readers although as far as I know this isn't an issue. Unless someone specifically needs Lua, it's hard to recommend LuaTeX over XeTeX.

zauguin 5y ago
LuaTeX does break after -- and ---, see e.g. \hsize 0pt abc--def \bye I also don't know what kind of weird remapping you are talking about, if you are referring to the Unicode mapping then they are required to get proper copy&paste (and are also generated by other engines these days).

svat 5y ago

LuaTeX is what TeX should have been from the start (and would have been, if the technology of the time had permitted it). Apart from modern font-handling and Unicode, the main thing IMO is the hooks it provides (in the form of callbacks). With other TeX engines (mainly: pdfTeX, XeTeX, or the original Knuth TeX), the only "programming" facility is in the form of macros, which were originally added by Knuth only for some simple text substitution to save typing. He never intended to add programming features into TeX, but of course, it doesn't take much to become "accidentally Turing-complete", and that's what TeX macros became. (There's also some rudimentary support for counters/registers, which IIRC Knuth added after he found users were (ab)using macros to encode numbers using unary or Church numerals.) So the only way to influence anything TeX does automatically, whether it's hyphenation or line-breaking or page-breaking or whatever, is to set up some macros whose blind expansion will ultimately—at a remove of several steps—result in the outcome you want, without affecting anything else (e.g. even an accidental space might get typeset, with undesirable results). This of course is cumbersome and error-prone. With LuaTeX you can program these things at the level you actually intend, e.g. while you can still use macros (or use the `process_input_buffer` callback in Lua), you can now also directly, say, influence the page layout, in a Lua callback that is actually aware of data at the relevant level of abstraction (the vboxes on the page, say), rather than trying to do everything at the lowest (text expansion) level. Knuth has said that he never intended for TeX macros to be used as a full-fledged programming language (he expected people would directly edit the SAIL/Pascal code of the TeX program for anything nontrivial), and that he dislikes each tool coming with its own Turing-complete programming language, and that if a standard embeddable programming language had been available he'd of course have used it in TeX — I think Lua would count. You can see "TeX without TeX" page on the LuaTeX wiki for an example of the power of LuaTeX, typesetting with TeX while completely bypassing TeX syntax: http://wiki.luatex.org/index.php/TeX_without_TeX http://wiki.luatex.org/index.php/TeX_without_TeX — a more elaborate production system is Speedata Publisher: https://github.com/speedata/publisher https://github.com/speedata/publisher As for myself, here is the most fancy thing I did with LuaTeX: https://tex.stackexchange.com/questions/401604/book-on-a-single-poster-page/403353#403353 https://tex.stackexchange.com/questions/401604/book-on-a-sin... (a "poster" of Kafka's Metamorphosis done just for that answer, never "productionized" into a package or whatever). Also, something possibly illuminating is this TeX-vs-LuaTeX comparison of something hyphenation-related (https://tex.stackexchange.com/questions/378704/how-to-avoid-line-breaks-that-result-in-short-words-at-line-edges/379802#379802 https://tex.stackexchange.com/questions/378704/how-to-avoid-...), and in this answer (https://tex.stackexchange.com/questions/398310/why-in-2017-does-latex-not-use-tree-like-structures/398372#398372 https://tex.stackexchange.com/questions/398310/why-in-2017-d...) I link to some other times I used LuaTeX, though some of them just use the Lua part of LuaTeX rather than LuaTeX hooks, such as computing digits of pi for generating pretty pictures (https://tex.stackexchange.com/questions/367902/tikz-color-option-for-edges-without-effect/367905#367905 https://tex.stackexchange.com/questions/367902/tikz-color-op...).

lou1306 5y ago

"Modern" TeX processors (LuaTeX or XeTeX) are sadly held back because publishers still cling to PDFLaTex. For instance, ArXiV "fully supports and automatically recognizes PDFLaTeX" and makes no mention of other backends [0]. So, LuaTeX is perfectly fine for personal stuff that you can compile in-house, but basically nobody wants to risk using it in any scenarios where sources are to be sent to a 3rd party for compilation, only to find out that they need to make them PDFLaTeX-compatible. And after all LuaTeX came out of beta in 2016 [1], which is pretty recent in TeX-years. So it kind of makes sense that publishers still prefer the quite battle-tested PDFLaTeX... [0]: https://arxiv.org/help/submit_tex https://arxiv.org/help/submit_tex [1]: https://tug.org/pipermail/luatex/2016-September/006201.html https://tug.org/pipermail/luatex/2016-September/006201.html

thangalin 5y ago

ConTeXt[1] has had Lua integrated since around 2009[2]. KeenWrite[3], the text editor that I'm developing, converts Markdown documents to XML, then can call ConTeXt with a given theme[4] to typeset the document. This allows for complete separation of content from presentation, as described in my Typesetting Markdown series, in which part 8 briefly touches upon Lua[5]. [1]: https://wiki.contextgarden.net/Installation https://wiki.contextgarden.net/Installation [2]: https://tug.org/TUGboat/tb40-1/tb124hagen-lmtx.pdf https://tug.org/TUGboat/tb40-1/tb124hagen-lmtx.pdf [3]: https://github.com/DaveJarvis/keenwrite https://github.com/DaveJarvis/keenwrite [4]: https://github.com/DaveJarvis/keenwrite-themes/ https://github.com/DaveJarvis/keenwrite-themes/ [5]: https://dave.autonoma.ca/blog/2020/04/28/typesetting-markdown-part-8/ https://dave.autonoma.ca/blog/2020/04/28/typesetting-markdow...

minusf 5y ago
IIRC context is "just" a macro package on top of luatex (and pdftex before). so whatever luatex has, context has.

dang 5y ago

Discussed at the time: LuaTeX Comes of Age - https://news.ycombinator.com/item?id=15146300 https://news.ycombinator.com/item?id=15146300 - Sept 2017 (28 comments)

strenholme 5y ago

The linked LWN article mentions that it’s difficult to put comments in inline LuaTeX macros because those macros remove newlines. To add a comment to the middle of a line in Lua, use the --[[ comment ]] form: a = 1 --[[ Set a to 1 ]] print(a) If we have a comment that has ]] in it, we can do this instead: a = 1 --[=[ We love [[brackets]] a lot ]=] print(a) The --[[ (or --[=[ ) form also works for multi-line comments a = 1 --[[ Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Dictum fusce ut placerat orci nulla pellentesque dignissim. Quam viverra orci sagittis eu volutpat odio facilisis mauris. ]] print (a)