8 ms·

I use some form of TeX (LaTeX or XeTeX) almost daily, and I often think about the future of (La)TeX and potential replacements. I think that we desperately nee

by ubavic 6y ago

I use some form of TeX (LaTeX or XeTeX) almost daily, and I often think about the future of (La)TeX and potential replacements. I think that we desperately need some modern language for document preparation, which will replace TeX and its derivatives, but I also don't see anything like that on the horizon. The biggest drawbacks of TeX are lack of proper Unicode support; lack of support of modern font formats; and outdated programming practices. Tools like XeTeX, LuaTeX and ConTeXt tackle these problems with some success, but they are still not perfect. Also, they are too similar to TeX to bring drastic changes to programmers. We need something that is designed from zero specifically as a TeX replacement (including compiler, package manager, popular packages, bibliography manager, converters, etc...). I can only hope for better future... BTW, fun fact: TeX is still actively developed. Knuth fixed some bugs in the kernel last month (after few years of pause).

spekcular 6y ago
When I'm OK with the defaults, LaTeX is a breeze. When I want to change something nontrivial about the formatting or style, it's a nightmare. For example, there are zero mature fonts besides the default computer modern that also have math rendering that's aesthetically acceptable to me. And I forget the details, but typesetting is handled in such a way that the kerning around things like hyphens is broken. I've fixed it manually before in places where it was particularly egregious. Also on this point: an ungodly number of hours are wasted every year by academics figuring out how to contort LaTeX so the resulting document complies with NSF grant specifications, which you must meet or be auto-rejected. In particular, NSF requires one-inch margins, but there's no way to force the typesetting engine to strictly respect the margins you set. So at the end you're left to turn the visual margin guides on and check there are no margin overruns, or you risk getting your proposal set to the trash (theoretically, at least). I've also heard that LaTeX editors are development disasters with a bus factor of one. The NSF really ought to fund them, because if we wake up one day after the maintainers of all the popular editors die and Apple/Microsoft force an operating system update that breaks compatibility, paper writing in the physical sciences will grind to a halt.
- leephillips 6y ago
  “typesetting is handled in such a way that the kerning around things like hyphens is broken” Can you give more details about this? I’ve never heard this complaint before, nor noticed this myself. Also, I’m not sure I understand what you are saying about “LaTeX editors”; I use Vim, and it works quite well.
  - spekcular 6y ago
    The issue that immediately came to mind was that if you change the kerning settings, then you can't input em dashes as --- anymore. See, for example, this Tex.SE question: https://tex.stackexchange.com/questions/315822/unwanted-kerning-inside-em-dash-when-using-microtype https://tex.stackexchange.com/questions/315822/unwanted-kern.... But after a while I remembered the more fundamental point that bothered me: > one might add that it's not just microtype that doesn't offer optical kerning -- it's structural limitations in TeX's typesetting that prevent it. To TeX, each glyph is contained inside a (I guess: black) box. TeX knows the outer dimensions of those boxes, but has no clue (and doesn't care) about what the glyph looks like that's inside it. https://tex.stackexchange.com/questions/139926/optical-kerning-with-microtype https://tex.stackexchange.com/questions/139926/optical-kerni... And this led to bad spacing for hyphens, which led to me experimenting with manual kerning, which led to the issue that I recalled first. Also, I am not really convinced of the merits of Vim for LaTeX. Most of my time writing is spent revising, which requires a lot of "random access edits," and I'm at least twice as fast at jumping to arbitrary locations on screen using a mouse as I am with vim (even with the usual tricks). (I tend to think of vim as something that was purpose-built for programming in C and with usefulness inversely proportional to how far a given task is from that purpose. While I realize that's not entirely accurate, I've found it a good heuristic. Programming: vim works decently for me. General purpose writing: not so much.) Further, the ability to control-click .pdfs and skip to the corresponding place in the .tex file, and vice versa, is an amazing convenience. I understand that there are technically ways to do this in vim, but then at that point I'm using the mouse a ton anyway and I don't really see the point of vim (which is usually proposed to be "hands don't go off keyboard = write faster" in my experience). So, yes, I could survive if forced to use vim. I just wouldn't like it. Given this, and given I feel I'm much more willing to experiment with these things than the average person writing papers in the physical sciences. I can't even imagine how the conversation would go if I tried to tell my 72-year-old advisor that TeXShop's not going to work anymore because he updated his computer and it's time to get friendly with the command line. These people want to do science, they don't want to learn vim shortcuts.
coliveira 6y ago
I think this is not a good reason to replace LaTeX. The big advantage of TeX over other systems is the extensibility of the language. Yes, the language is strange, but you get used to it, and it does what it's supposed to do. So I think that, just like UNIX, replacing TeX will never be completely done, unless something really great shows up.
Mediterraneo10 6y ago
> we desperately need some modern language for document preparation, which will replace TeX and its derivatives Do we? Sadly, appreciation for excellent typesetting has declined now that most people are consuming writing through a screen. And even for printed material there is pressure to cost costs: venerable and respected publishers with long traditions of obsessively good typesetting, realized they could outsource their typesetting to developing-world shops using Word or similar software, and few readers would care. (Similarly, and to the dismay of bibliophiles, even prestige titles these days are likely to get a glued building instead of a sewn one, and fairly low-resolution digital printing).
GiovanniP 6y ago
> I think that we desperately need some modern language for document preparation See my comment below https://news.ycombinator.com/item?id=26498409 https://news.ycombinator.com/item?id=26498409 :-)
- ubavic 6y ago
  TeXmac looks very interesting. I will maybe give it a try. But I must say that TeXmac workflow seems a little bit dependent on GUI. That is not a bad thing per se but seams different form TeX. Thank you for bringing it up :)
  - GiovanniP 6y ago
    In case, for Linux I am using their static binary, if I recall correctly the Ubuntu package wanted to change my Guile to 1.8 and this wasn't ok for me. I have learnt today that Fedora has it in the repositories. The Windows installer works and as far as I know the one for Mac OS too.
dhosek 6y ago
Fixing bugs isn't the same as actively developing. The bugs that are getting fixed these days are in increasingly obscure edge cases (in some cases no actual implementation in use can reach those edges). That said, I've started working on such a project as you propose. The document preparation language will be almost identical to LaTeX, but no macro language and no category codes, UTF-8 as the standard input coding, first-class support for multiple back ends (e.g., use the same source to produce PDF, HTML or ePub), command definition through an embedded scripting language (I'm still debating between Python, Lua or something else entirely), formatting done declaratively, most likely through a YAML file, etc. My musings along the way are at http://finl.xyz http://finl.xyz
- enriquto 6y ago
  > but no macro language Any reason for that? The rest of your description looks very attractive, but why would I prefer a language without macros? I need conditional compilation, setting variables, parametrized notations, etc. This is extremely useful when preparing a scientific document. If anything, I don't really care about the document preparation language itself, but it must have macros to be useful!
  - dhosek 6y ago
    To clarify, there would still be programmability, just not via macro expansion. Instead of macros there would be first-class variables instead of TeX's mix of registers and macros to emulate that. Instead of having to deal with the timing of macro expansions and all the pitfalls therein, an embedded scripting language would be available to handle complex settings (I'm leaning towards Python at the moment, but I'm far from having to make a decision on that topic). First-class flow control will also be available (right now, things like \foreach and \ifXXX are handled via macro expansion which causes a lot of pain beyond simple use cases).
    - enriquto 6y ago
      OK, that's reassuring! What you describe sounds pretty much like a "macro language" to me (i.e., a separate language to describe your document). But this is just a question of terminology, somewhat arbitrary. Being able to generate the text of your document is a very powerful tool and, according to what you say, it will be even more powerful.
vitejose 6y ago
Matthew Butterick is working on a new document processor that borrows some ideas from LaTeX and is built in Racket: https://docs.racket-lang.org/quad/ https://docs.racket-lang.org/quad/
xorcist 6y ago
> lack of proper Unicode support; lack of support of modern font formats Is that really true? I had no trouble writing UTF8 documents using regular TrueType-fonts in LaTeX almost two decades ago and with no problems with output quality. The only thing was you couldn't go through the dvi backend so any tools which manipulated dvi was not possible to use. But I believe all this is default now.
- dkna 6y ago
  > Is that really true? Oh yes. For instance, LuaTeX doesn't print combining diacritics correctly unless you painstakingly swap the underlying codepoints around in some unspecified order (to be determined through trial and error) that doesn't match _any_ of the Unicode normalization forms.
- ubavic 6y ago
  I would be interested in how you did that? I had a lot of problems with Cyrillic scripts until I started using XeTeX three years ago. I think that LaTeX3 made some progress on UTF-8 compatibility in the last two years, but font loading is still a mess.

by ubavic 6y ago

spekcular 6y ago

When I'm OK with the defaults, LaTeX is a breeze. When I want to change something nontrivial about the formatting or style, it's a nightmare. For example, there are zero mature fonts besides the default computer modern that also have math rendering that's aesthetically acceptable to me. And I forget the details, but typesetting is handled in such a way that the kerning around things like hyphens is broken. I've fixed it manually before in places where it was particularly egregious. Also on this point: an ungodly number of hours are wasted every year by academics figuring out how to contort LaTeX so the resulting document complies with NSF grant specifications, which you must meet or be auto-rejected. In particular, NSF requires one-inch margins, but there's no way to force the typesetting engine to strictly respect the margins you set. So at the end you're left to turn the visual margin guides on and check there are no margin overruns, or you risk getting your proposal set to the trash (theoretically, at least). I've also heard that LaTeX editors are development disasters with a bus factor of one. The NSF really ought to fund them, because if we wake up one day after the maintainers of all the popular editors die and Apple/Microsoft force an operating system update that breaks compatibility, paper writing in the physical sciences will grind to a halt.

leephillips 6y ago
“typesetting is handled in such a way that the kerning around things like hyphens is broken” Can you give more details about this? I’ve never heard this complaint before, nor noticed this myself. Also, I’m not sure I understand what you are saying about “LaTeX editors”; I use Vim, and it works quite well.
- spekcular 6y ago
  The issue that immediately came to mind was that if you change the kerning settings, then you can't input em dashes as --- anymore. See, for example, this Tex.SE question: https://tex.stackexchange.com/questions/315822/unwanted-kerning-inside-em-dash-when-using-microtype https://tex.stackexchange.com/questions/315822/unwanted-kern.... But after a while I remembered the more fundamental point that bothered me: > one might add that it's not just microtype that doesn't offer optical kerning -- it's structural limitations in TeX's typesetting that prevent it. To TeX, each glyph is contained inside a (I guess: black) box. TeX knows the outer dimensions of those boxes, but has no clue (and doesn't care) about what the glyph looks like that's inside it. https://tex.stackexchange.com/questions/139926/optical-kerning-with-microtype https://tex.stackexchange.com/questions/139926/optical-kerni... And this led to bad spacing for hyphens, which led to me experimenting with manual kerning, which led to the issue that I recalled first. Also, I am not really convinced of the merits of Vim for LaTeX. Most of my time writing is spent revising, which requires a lot of "random access edits," and I'm at least twice as fast at jumping to arbitrary locations on screen using a mouse as I am with vim (even with the usual tricks). (I tend to think of vim as something that was purpose-built for programming in C and with usefulness inversely proportional to how far a given task is from that purpose. While I realize that's not entirely accurate, I've found it a good heuristic. Programming: vim works decently for me. General purpose writing: not so much.) Further, the ability to control-click .pdfs and skip to the corresponding place in the .tex file, and vice versa, is an amazing convenience. I understand that there are technically ways to do this in vim, but then at that point I'm using the mouse a ton anyway and I don't really see the point of vim (which is usually proposed to be "hands don't go off keyboard = write faster" in my experience). So, yes, I could survive if forced to use vim. I just wouldn't like it. Given this, and given I feel I'm much more willing to experiment with these things than the average person writing papers in the physical sciences. I can't even imagine how the conversation would go if I tried to tell my 72-year-old advisor that TeXShop's not going to work anymore because he updated his computer and it's time to get friendly with the command line. These people want to do science, they don't want to learn vim shortcuts.

coliveira 6y ago

I think this is not a good reason to replace LaTeX. The big advantage of TeX over other systems is the extensibility of the language. Yes, the language is strange, but you get used to it, and it does what it's supposed to do. So I think that, just like UNIX, replacing TeX will never be completely done, unless something really great shows up.

Mediterraneo10 6y ago

> we desperately need some modern language for document preparation, which will replace TeX and its derivatives Do we? Sadly, appreciation for excellent typesetting has declined now that most people are consuming writing through a screen. And even for printed material there is pressure to cost costs: venerable and respected publishers with long traditions of obsessively good typesetting, realized they could outsource their typesetting to developing-world shops using Word or similar software, and few readers would care. (Similarly, and to the dismay of bibliophiles, even prestige titles these days are likely to get a glued building instead of a sewn one, and fairly low-resolution digital printing).

GiovanniP 6y ago

> I think that we desperately need some modern language for document preparation See my comment below https://news.ycombinator.com/item?id=26498409 https://news.ycombinator.com/item?id=26498409 :-)

ubavic 6y ago
TeXmac looks very interesting. I will maybe give it a try. But I must say that TeXmac workflow seems a little bit dependent on GUI. That is not a bad thing per se but seams different form TeX. Thank you for bringing it up :)
- GiovanniP 6y ago
  In case, for Linux I am using their static binary, if I recall correctly the Ubuntu package wanted to change my Guile to 1.8 and this wasn't ok for me. I have learnt today that Fedora has it in the repositories. The Windows installer works and as far as I know the one for Mac OS too.

dhosek 6y ago

Fixing bugs isn't the same as actively developing. The bugs that are getting fixed these days are in increasingly obscure edge cases (in some cases no actual implementation in use can reach those edges). That said, I've started working on such a project as you propose. The document preparation language will be almost identical to LaTeX, but no macro language and no category codes, UTF-8 as the standard input coding, first-class support for multiple back ends (e.g., use the same source to produce PDF, HTML or ePub), command definition through an embedded scripting language (I'm still debating between Python, Lua or something else entirely), formatting done declaratively, most likely through a YAML file, etc. My musings along the way are at http://finl.xyz http://finl.xyz

enriquto 6y ago
> but no macro language Any reason for that? The rest of your description looks very attractive, but why would I prefer a language without macros? I need conditional compilation, setting variables, parametrized notations, etc. This is extremely useful when preparing a scientific document. If anything, I don't really care about the document preparation language itself, but it must have macros to be useful!
- dhosek 6y ago
  To clarify, there would still be programmability, just not via macro expansion. Instead of macros there would be first-class variables instead of TeX's mix of registers and macros to emulate that. Instead of having to deal with the timing of macro expansions and all the pitfalls therein, an embedded scripting language would be available to handle complex settings (I'm leaning towards Python at the moment, but I'm far from having to make a decision on that topic). First-class flow control will also be available (right now, things like \foreach and \ifXXX are handled via macro expansion which causes a lot of pain beyond simple use cases).
  - enriquto 6y ago
    OK, that's reassuring! What you describe sounds pretty much like a "macro language" to me (i.e., a separate language to describe your document). But this is just a question of terminology, somewhat arbitrary. Being able to generate the text of your document is a very powerful tool and, according to what you say, it will be even more powerful.

vitejose 6y ago

Matthew Butterick is working on a new document processor that borrows some ideas from LaTeX and is built in Racket: https://docs.racket-lang.org/quad/ https://docs.racket-lang.org/quad/

xorcist 6y ago

> lack of proper Unicode support; lack of support of modern font formats Is that really true? I had no trouble writing UTF8 documents using regular TrueType-fonts in LaTeX almost two decades ago and with no problems with output quality. The only thing was you couldn't go through the dvi backend so any tools which manipulated dvi was not possible to use. But I believe all this is default now.

dkna 6y ago
> Is that really true? Oh yes. For instance, LuaTeX doesn't print combining diacritics correctly unless you painstakingly swap the underlying codepoints around in some unspecified order (to be determined through trial and error) that doesn't match _any_ of the Unicode normalization forms.
ubavic 6y ago
I would be interested in how you did that? I had a lot of problems with Cyrillic scripts until I started using XeTeX three years ago. I think that LaTeX3 made some progress on UTF-8 compatibility in the last two years, but font loading is still a mess.