4 ms·
While the name “Rewriting LaTeX in Pure Rust” is very exciting, there are two problems: • “LaTeX” is a set of macros (ab)using the macro/text-expansion feature
by svat 6y ago
While the name “Rewriting LaTeX in Pure Rust” is very exciting, there are two problems:
• “LaTeX” is a set of macros (ab)using the macro/text-expansion feature of an underlying TeX engine (LuaTeX, XeTeX, pdTeX, original Knuth TeX, or this one), to provide things like cross-references, section numbering, etc. When you as a typical user use “LaTeX”, you're actually using these macros, along with a huge variety of macro “packages” written by many authors, that together add up to orders of magnitude more lines of code than the TeX engine itself (about 25000 lines originally: https://tex.stackexchange.com/a/505664 https://tex.stackexchange.com/a/505664). Most of the incompatibilities and error messages a typical user encounters when using LaTeX are caused not by the TeX engine but by these packages or LaTeX, which this project doesn't touch.
• This project comes about as:
•— tex.web, Knuth's original source code, written in his literate-programming system WEB (a preprocessor on top of Pascal),
•— extended to xetex.web (XeTeX), to provide Unicode support etc,
•— automatically tangled to Pascal code (macros expanded and constants replaced),
•— automatically translated to C code,
•— now automatically translated to Rust code, by wrapping the C code in `unsafe` Rust blocks.
So it has gone through multiple rounds of machine translation. Actually, I'm impressed that this project has made many improvements in undoing and restoring some of the WEB macros, since the last time this was mentioned here. (Compare a random comparison from 1 year ago: https://gist.github.com/627399d0150e66d211a264bc05b33beb https://gist.github.com/627399d0150e66d211a264bc05b33beb with the same comparison today: https://gist.github.com/71e47f2276f9c6c030efe0e3357ef3bf https://gist.github.com/71e47f2276f9c6c030efe0e3357ef3bf) — two of the four differences I had pointed out in my comment then (https://news.ycombinator.com/item?id=21177367 https://news.ycombinator.com/item?id=21177367) are gone, and the only ones to mention are:
• The comments from the original are gone.
• Something like “cur_if:=subtype(p);” becomes “ cur_if = MEM[p].b16.s0 as i16;”
So there's still a long way to go. But even with all that, what would be more useful IMO are (in order of difficulty):
• Translate directly from WEB (xetex.web) to Rust. The original uses only a subset of Pascal so IMO it should be possible to write an automatic translator that preserves more of the original. Something along these lines has been done for translating WEB to CWEB by Martin Ruckert (see https://w3-o.cs.hm.edu/users/ruckert/public_html/web2w/index.html https://w3-o.cs.hm.edu/users/ruckert/public_html/web2w/index... and http://tug.org/TUGboat/Contents/listauthor.html#Ruckert,Martin http://tug.org/TUGboat/Contents/listauthor.html#Ruckert,Mart... and his demo from TUG 2020). Alternatively, one could give up on XeTeX and start with the LuaTeX source code (in C).
• Understand how the TeX program is written/works, and write a more idiomatic translation in Rust: something along these lines is in progress as a one-person hobby project by Emily Eisenberg (one of the original developers of KaTeX): see https://github.com/xymostech/XymosTeX https://github.com/xymostech/XymosTeX This one is most exciting to me, as I also have (not very useful so far) grand plans to understand TeX and to make it more understandable for everybody (https://shreevatsa.net/tex/program https://shreevatsa.net/tex/program).
• As most of the code a user deals with is in the macro/LaTeX layer, modify TeX macro expansion to make it more “debuggable” (https://tex.stackexchange.com/a/384881 https://tex.stackexchange.com/a/384881, https://cstheory.stackexchange.com/a/40282 https://cstheory.stackexchange.com/a/40282): show full stack traces beyond what one gets with \errorcontextlines=\maxdimen (so file names and line numbers, by tracking the provenance of token lists), show arguments scanned (“passed in”) and expected/found types thereof, cleanly separate the macro expansion layer from the typesetting (line-breaking / page-breaking) layer, maybe even (JIT?) “compile” some of the lower-level or frequently-used macros — the eventual goal would be for the user to completely understand what is going on when they compile their LaTeX document, so that (1) they are less surprised by errors and know what to do, (2) they may be induced (if they're programmers) to do less in the (unsuitable) macro layer of the TeX engine and do more at the appropriate layer: either do it in a preprocessor that runs on the .tex file, or do it at the typesetting layer via something like LuaTeX's hooks (pre_linebreak_filter etc).
Anyway, this project could still end up getting there, so let's see! Interacting with the TeX community (the TeX StackExchange, mailing lists, write an article for TUGboat, etc) might also be useful.