5 ms·

Part of what bothers me so much about TeX is how obscure and opaque the language feels. I am familiar with ten programming languages, from C and assembly to Has

by j2kun 11y ago

Part of what bothers me so much about TeX is how obscure and opaque the language feels. I am familiar with ten programming languages, from C and assembly to Haskell, Perl, Lisp. But just the way that people write TeX macros makes everything so unapproachable. The only insight I have gleaned is that it's like one big fat state machine, which makes it extremely hard to code with principle. I understand Knuth wrote a book, but reading through it has only made me wonder more about the bizarre choices.

ufo 11y ago
Keep in min that tex is really old software, from a time when computers weren't as powerful. So it makes a bit of sense that it was architected to make multiple passes over the input instead of storing the whole document in memory.
- j2kun 11y ago
  Surely. I'm just saying modern times call for modern languages, and that makes me uneasy about TeX.
  - christopheraden 11y ago
    What about something ala Neovim? I've only ever looked at Tex from the perspective of a user (I don't program my own macros too often), so I don't know how hard it'd be, but why not a language overhaul?
    - _delirium 11y ago
      XeTeX and LuaTeX are projects you might be interested in.
    - wtbob 11y ago
      The issue is that so many man-centuries (man-millennia?) are invested into TeX & LaTeX that sinking time into something else is very, very expensive (much like trying to build a better CPU than amd64 or arm64). TeX is really, really amazingly powerful. It can do almost anything a typesetter could want to do, fairly easily, and it can do just about everything, one way or another. And its output is heart-achingly beautiful. Sadly, the code necessary to achieve that output ranges from…heart-achingly beautiful to heart-breakingly ugly. There are other projects out there, of course. I do think that TeX & LaTeX are close to a local maximum, if not al the way there. XML, in comparison, is a booger joke.
      - goalieca 11y ago
        > XML, in comparison, is a booger joke. A dried up booger on the floor.
      - dschep 11y ago
        > I do think that TeX & LaTeX are close to a local maximum, if not al the way there. Funny you say that since TeX's version scheme (in part) is that it approaches pi. And IIRC will be pi upon Knuth's death.
      - jandrese 11y ago
        XML isn't even really in the same field as TeX. TeX is for typesetting, that's all it does. XML is for encoding general data. It's like saying PDFs are way better than protocol buffers. TeX is incredibly powerful, but it's also incredibly idiosyncratic. I also personally think Computer Modern is an ugly font.
        leephillips 11y ago
        TeX has long ago gained the power to use any OpenType/TrueType font on your system, so dislike of CM is really not a reason to avoid using it. (I happen to like CM, but I know it's disliked by multitudes.)
        wtbob 11y ago
        > XML is for encoding general data. No, it's really not. XML is a markup language, not a data-encoding language. JSON, S-expressions, ASN.1 &c. are all data encodings; TeX, LaTeX, HTML and XML are markup languages. > TeX is incredibly powerful, but it's also incredibly idiosyncratic. Agreed. > I also personally think Computer Modern is an ugly font. Eh, it's not great on-screen, but it looks pretty good on paper. But TeX & LaTeX have supported multiple fonts since the beginning.
    - CarVac 11y ago
      The trouble is the ecosystem of packages that people would have to redo. If you replaced TeX, LaTeX itself would need replacing, as would every single LaTeX package and class that you ever want to use, ones like microtype and stuff, would have to be rewritten. To be honest, the multi-pass deal isn't that bad, but the macro expansion system is crazy complicated. Every once in a while after working in LaTeX I'll get the feeling I understand it, but that feeling inevitably dissipates after ten minutes or so.
      - e12e 11y ago
        Very much this. As far as I can gather TeX is actually rather simple - considering that it basically does what PostScript does. Now, while coding raw PostScript might be fun as an exercise, most would prefer not to. LaTeX lands somewhere between PostScript, and something higher up. To meaningfully replace Tex/LaTeX/Metafont and even just a selection of "the best" LaTeX packages... would be a herculean task. Making a "new" TeX probably wouldn't be that hard - but it's also something that wouldn't be that useful. I would very much like something that's both simpler and also keeps some of the lessons learned/implemented (word spacing/splitting, page layout, page breaks etc). As for other "tools in the same space", I do like pandoc a lot. I want to like python's ReST (Re-Structured Text) - but that's a package I feel is in need of a rewrite/redesign. Many good ideas there - but figuring out how to take a simple document and produce simple, modern (preferably somewhat semantic) html for example -- or to produce a decent looking PDF without needing all of LaTeX/Texlive on hand isn't easy. Rewriting ReST tools would be a lot of work, but I think if one didn't try for 100% backwards (output, plugin) compatibility it might be worthwhile. The astute reader will notice that ReST/Pandoc deals with structured documents, and not really layout for paper/screen (both use TeX/LaTeX as an output target/pipeline). I don't know of anything that comes close to TeX/LaTeX for "rasterized" output. On the other hand, I also don't know of any package/combination that'll make TeX/LaTeX produce anything but messy, 90s-style html -- that generally looks awful. Even if you were to try and force a modern set of CSS down over the resulting mess. If anyone knows of a modern hypertext package for TeX/LaTeX or some similar tool, I'd be happy to be proven wrong.
        dragonwriter 11y ago
  - mjn 11y ago
    I've seen projects announced on and off to make a next-gen TeX or TeX-like from a clean start, but they never seem to go anywhere. I think writing a good rendering engine is just a ton of work. Two approaches I see as plausible: 1. Use TeX as a "document assembly language" of sorts, a pure back-end rendering target not exposed to the user, and do your document markup/scripting in some other front-end language. Pandoc [1] is a start at building infrastructure for this. You can then build a new renderer too if you want, to remove the dependency on TeX, but at least in the meantime you have a pathway to producing quality output right now. 2. Build something on top of HTML+CSS (maybe also +JS). Advantages of this would be that it'd make it relatively easy to target the web with the same document source, and open-source web browsers have already sunk a ton of development time into rendering. CSS even includes a bunch of print-oriented features that in principle provide markup for most of what you might need here, though browsers for obvious reasons haven't tended to prioritize those parts. wkhtmltopdf [2] is a project aiming to build a to-print or to-PDF document workflow on top of Webkit. However imo the results are still not near TeX-replacement level. I believe the gold standard currently, if you want high-quality print out of HTML+CSS, is the proprietary PrinceXML [3]. [1] http://pandoc.org/ http://pandoc.org/ [2] http://wkhtmltopdf.org/ http://wkhtmltopdf.org/ [3] http://www.princexml.com/ http://www.princexml.com/
    - deleted 11y ago
      [deleted]
  - userbinator 11y ago
    I seriously doubt that implementations of any "modern languages" - or any other languages for that matter - have had anywhere near the astonishingly low defect rate and efficiency of TeX.
    - legulere 11y ago
      The TeX ecosystem (what you actually use) is riddled with tons of bugs. Images that end up somewhere totally wrong, text/graphics overflowing onto the page margins, are just some of the bugs that you regularly have to fight with, but there are also some more higher level systemic bugs like that there's no formal grammar. I don't know what is supposed to be efficient concerning TeX
      - jimhefferon 11y ago
        > Images that end up somewhere totally wrong, text/graphics overflowing onto the page margins, are just some of the bugs that you regularly have to fight with I write a lot of TeX, and I have never found that those things are the fault of the standard software; rather, they are my fault.
      - dragonwriter 11y ago
        > I don't know what is supposed to be efficient concerning TeX The absence of a better alternative in the domain, even though attempts to serve the domain by other tools (including ones using more "modern languages") have come and gone over the years. TeX isn't theoretically ideal, its just practically very good and the expected cost to benefit ratio of a ground-up replacement is very high.
  - sklogic 11y ago
    People are trying to build alternatives, see http://www.patoline.org/ http://www.patoline.org/ for example
- legulere 11y ago
  Yet Tex is slow as hell
  - david_ar 11y ago
    TeX itself is quite fast, it's loading all the extra packages that slows things down.
  - jimhefferon 11y ago
    It doesn't seem slow to me. I have a 450 page book that as part of the compile generates a 200 page answer answer manual. They are full of math, hyperlinked cross references, figures, etc. They use tons of packages including amsmath, hyperref, and many more. Compiling takes perhaps 15 seconds. This means compiling twice, to resolve references.
zvrba 11y ago
LuaTex project (http://www.luatex.org/ http://www.luatex.org/) is working on fixing the language part. Hans Hagen, of Pragma Ade company, is on the team. I know about him and his work because I've used the ConTeXt macro package for TeX for my bachelor's thesis. This was like 15 years ago and already then I wished that Context were standard instead of Latex. Whereas in Latex you have a bunch of (sometimes conflicting) packages to solve various problems, Context is a coherent unit. I didn't need external package for anything, not even for complex stuff like wrapping text around figures. The downside is, and the reason why I couldn't keep using it, is that the markup commands are incompatible with Latex. Submissions to journals accept Latex only :(
- unfamiliar 11y ago
  Unfortunately the LuaTex website is completely opaque and I can't see any examples of the language or even a top level description of how to use it or how the workflow differs from that of LaTeX.
  - jimhefferon 11y ago
    LuaTeX is still under development (note, though, that this is not abandonware or anything like it, it is just a deliberate and careful project that has done amazing work over an extended time). A person who wants a document that will still work in five years should likely go elsewhere. That said, however, if you are interested, a good place to start is by reading the user support list http://www.luatex.org/support.html http://www.luatex.org/support.html .
    - unfamiliar 11y ago
      What's the point of a project that never ships?
      - jimhefferon 11y ago
        It comes with TeX Live. You can get it today, or five years ago, for that matter. But the final version, for which there is a plan, has not shipped or been written. Writing it is a very ambitious task and takes time, and a little money (through support from, e.g., the TeX Users Group). Things take time.
arnsholt 11y ago
The reason TeX can be so unapproachable is the fact that it's a very different language from pretty much any other programming language you've ever used. It's not a Turing machine, and it's not lambda calculus. It's not even Horn clauses (which is the model behind Prolog). It's a string rewriting system; basically a giant cascade of rules like "rewrite this string to this other string" applied over and over until the whole input is consumed. As it happens, this is yet another Turing-equivalent form of computation. The closest analogy I have is programming in Lisp, but with the constraint that you can only use special forms and macros, no functions are allowed.
- versteegen 11y ago
  If you want to compare to abstract models of computation, the closest one might be a multi-stack push-down automaton (which is Turing complete).
- vanderZwan 11y ago
  I was about to say: isn't that just another expression language? I've sometimes wondered if Lisp-like languages wouldn't be a much better way of writing documents than our current markup languages of choice. If I'll ever seriously try to pick up Racket, it'll probably be to give Pollen a spin and see how it is in practice. http://pollenpub.com/ http://pollenpub.com/
- sklogic 11y ago
  The formalism behind it: https://en.wikipedia.org/wiki/Markov_algorithm https://en.wikipedia.org/wiki/Markov_algorithm
smcl 11y ago
It probably doesn't fit exactly what you want, but I found Lout (http://savannah.nongnu.org/projects/lout http://savannah.nongnu.org/projects/lout) to be a suitable enough replacement for a lot of LaTeX-y tasks. I'm not a heavy LaTeX user so it's likely there's a whole lot missing, but I found the document structure and syntax to be a good deal simpler. In addition the installer is (iirc) only a few MB - I'm sure you can get tiny LaTeX\TeX installers or packages but the ones I've seen tend to be very "batteries included, plus a whole lot more" which is maybe a bit excessive.
coliveira 11y ago
All languages you mentioned are general programming languages. TeX is a word processing language, so it makes complete sense that it is different. If you take the time to properly learn TeX you will see that many of the decisions are obvious for what it intends to do, which is to create full-featured document formats.
baldfat 11y ago
I always had HIGH hopes for LxY http://www.lyx.org/ http://www.lyx.org/ for people with issues with LaTex but have just decided that people really need to learn to write code for typesetting and wish that Latex would produce a better syntax. I have high hopes for LuaTeX.

by j2kun 11y ago

ufo 11y ago

Keep in min that tex is really old software, from a time when computers weren't as powerful. So it makes a bit of sense that it was architected to make multiple passes over the input instead of storing the whole document in memory.

j2kun 11y ago
Surely. I'm just saying modern times call for modern languages, and that makes me uneasy about TeX.
- christopheraden 11y ago
  What about something ala Neovim? I've only ever looked at Tex from the perspective of a user (I don't program my own macros too often), so I don't know how hard it'd be, but why not a language overhaul?
  - _delirium 11y ago
    XeTeX and LuaTeX are projects you might be interested in.
  - wtbob 11y ago
    The issue is that so many man-centuries (man-millennia?) are invested into TeX & LaTeX that sinking time into something else is very, very expensive (much like trying to build a better CPU than amd64 or arm64). TeX is really, really amazingly powerful. It can do almost anything a typesetter could want to do, fairly easily, and it can do just about everything, one way or another. And its output is heart-achingly beautiful. Sadly, the code necessary to achieve that output ranges from…heart-achingly beautiful to heart-breakingly ugly. There are other projects out there, of course. I do think that TeX & LaTeX are close to a local maximum, if not al the way there. XML, in comparison, is a booger joke.
    - goalieca 11y ago
      > XML, in comparison, is a booger joke. A dried up booger on the floor.
    - dschep 11y ago
      > I do think that TeX & LaTeX are close to a local maximum, if not al the way there. Funny you say that since TeX's version scheme (in part) is that it approaches pi. And IIRC will be pi upon Knuth's death.
    - jandrese 11y ago
      XML isn't even really in the same field as TeX. TeX is for typesetting, that's all it does. XML is for encoding general data. It's like saying PDFs are way better than protocol buffers. TeX is incredibly powerful, but it's also incredibly idiosyncratic. I also personally think Computer Modern is an ugly font.
      - leephillips 11y ago
        TeX has long ago gained the power to use any OpenType/TrueType font on your system, so dislike of CM is really not a reason to avoid using it. (I happen to like CM, but I know it's disliked by multitudes.)
      - wtbob 11y ago
        > XML is for encoding general data. No, it's really not. XML is a markup language, not a data-encoding language. JSON, S-expressions, ASN.1 &c. are all data encodings; TeX, LaTeX, HTML and XML are markup languages. > TeX is incredibly powerful, but it's also incredibly idiosyncratic. Agreed. > I also personally think Computer Modern is an ugly font. Eh, it's not great on-screen, but it looks pretty good on paper. But TeX & LaTeX have supported multiple fonts since the beginning.
  - CarVac 11y ago
    The trouble is the ecosystem of packages that people would have to redo. If you replaced TeX, LaTeX itself would need replacing, as would every single LaTeX package and class that you ever want to use, ones like microtype and stuff, would have to be rewritten. To be honest, the multi-pass deal isn't that bad, but the macro expansion system is crazy complicated. Every once in a while after working in LaTeX I'll get the feeling I understand it, but that feeling inevitably dissipates after ten minutes or so.
    - e12e 11y ago
      Very much this. As far as I can gather TeX is actually rather simple - considering that it basically does what PostScript does. Now, while coding raw PostScript might be fun as an exercise, most would prefer not to. LaTeX lands somewhere between PostScript, and something higher up. To meaningfully replace Tex/LaTeX/Metafont and even just a selection of "the best" LaTeX packages... would be a herculean task. Making a "new" TeX probably wouldn't be that hard - but it's also something that wouldn't be that useful. I would very much like something that's both simpler and also keeps some of the lessons learned/implemented (word spacing/splitting, page layout, page breaks etc). As for other "tools in the same space", I do like pandoc a lot. I want to like python's ReST (Re-Structured Text) - but that's a package I feel is in need of a rewrite/redesign. Many good ideas there - but figuring out how to take a simple document and produce simple, modern (preferably somewhat semantic) html for example -- or to produce a decent looking PDF without needing all of LaTeX/Texlive on hand isn't easy. Rewriting ReST tools would be a lot of work, but I think if one didn't try for 100% backwards (output, plugin) compatibility it might be worthwhile. The astute reader will notice that ReST/Pandoc deals with structured documents, and not really layout for paper/screen (both use TeX/LaTeX as an output target/pipeline). I don't know of anything that comes close to TeX/LaTeX for "rasterized" output. On the other hand, I also don't know of any package/combination that'll make TeX/LaTeX produce anything but messy, 90s-style html -- that generally looks awful. Even if you were to try and force a modern set of CSS down over the resulting mess. If anyone knows of a modern hypertext package for TeX/LaTeX or some similar tool, I'd be happy to be proven wrong.
      - dragonwriter 11y ago
- mjn 11y ago
  I've seen projects announced on and off to make a next-gen TeX or TeX-like from a clean start, but they never seem to go anywhere. I think writing a good rendering engine is just a ton of work. Two approaches I see as plausible: 1. Use TeX as a "document assembly language" of sorts, a pure back-end rendering target not exposed to the user, and do your document markup/scripting in some other front-end language. Pandoc [1] is a start at building infrastructure for this. You can then build a new renderer too if you want, to remove the dependency on TeX, but at least in the meantime you have a pathway to producing quality output right now. 2. Build something on top of HTML+CSS (maybe also +JS). Advantages of this would be that it'd make it relatively easy to target the web with the same document source, and open-source web browsers have already sunk a ton of development time into rendering. CSS even includes a bunch of print-oriented features that in principle provide markup for most of what you might need here, though browsers for obvious reasons haven't tended to prioritize those parts. wkhtmltopdf [2] is a project aiming to build a to-print or to-PDF document workflow on top of Webkit. However imo the results are still not near TeX-replacement level. I believe the gold standard currently, if you want high-quality print out of HTML+CSS, is the proprietary PrinceXML [3]. [1] http://pandoc.org/ http://pandoc.org/ [2] http://wkhtmltopdf.org/ http://wkhtmltopdf.org/ [3] http://www.princexml.com/ http://www.princexml.com/
  - deleted 11y ago
    [deleted]
- userbinator 11y ago
  I seriously doubt that implementations of any "modern languages" - or any other languages for that matter - have had anywhere near the astonishingly low defect rate and efficiency of TeX.
  - legulere 11y ago
    The TeX ecosystem (what you actually use) is riddled with tons of bugs. Images that end up somewhere totally wrong, text/graphics overflowing onto the page margins, are just some of the bugs that you regularly have to fight with, but there are also some more higher level systemic bugs like that there's no formal grammar. I don't know what is supposed to be efficient concerning TeX
    - jimhefferon 11y ago
      > Images that end up somewhere totally wrong, text/graphics overflowing onto the page margins, are just some of the bugs that you regularly have to fight with I write a lot of TeX, and I have never found that those things are the fault of the standard software; rather, they are my fault.
    - dragonwriter 11y ago
      > I don't know what is supposed to be efficient concerning TeX The absence of a better alternative in the domain, even though attempts to serve the domain by other tools (including ones using more "modern languages") have come and gone over the years. TeX isn't theoretically ideal, its just practically very good and the expected cost to benefit ratio of a ground-up replacement is very high.
- sklogic 11y ago
  People are trying to build alternatives, see http://www.patoline.org/ http://www.patoline.org/ for example
legulere 11y ago
Yet Tex is slow as hell
- david_ar 11y ago
  TeX itself is quite fast, it's loading all the extra packages that slows things down.
- jimhefferon 11y ago
  It doesn't seem slow to me. I have a 450 page book that as part of the compile generates a 200 page answer answer manual. They are full of math, hyperlinked cross references, figures, etc. They use tons of packages including amsmath, hyperref, and many more. Compiling takes perhaps 15 seconds. This means compiling twice, to resolve references.

zvrba 11y ago

LuaTex project (http://www.luatex.org/ http://www.luatex.org/) is working on fixing the language part. Hans Hagen, of Pragma Ade company, is on the team. I know about him and his work because I've used the ConTeXt macro package for TeX for my bachelor's thesis. This was like 15 years ago and already then I wished that Context were standard instead of Latex. Whereas in Latex you have a bunch of (sometimes conflicting) packages to solve various problems, Context is a coherent unit. I didn't need external package for anything, not even for complex stuff like wrapping text around figures. The downside is, and the reason why I couldn't keep using it, is that the markup commands are incompatible with Latex. Submissions to journals accept Latex only :(

unfamiliar 11y ago
Unfortunately the LuaTex website is completely opaque and I can't see any examples of the language or even a top level description of how to use it or how the workflow differs from that of LaTeX.
- jimhefferon 11y ago
  LuaTeX is still under development (note, though, that this is not abandonware or anything like it, it is just a deliberate and careful project that has done amazing work over an extended time). A person who wants a document that will still work in five years should likely go elsewhere. That said, however, if you are interested, a good place to start is by reading the user support list http://www.luatex.org/support.html http://www.luatex.org/support.html .
  - unfamiliar 11y ago
    What's the point of a project that never ships?
    - jimhefferon 11y ago
      It comes with TeX Live. You can get it today, or five years ago, for that matter. But the final version, for which there is a plan, has not shipped or been written. Writing it is a very ambitious task and takes time, and a little money (through support from, e.g., the TeX Users Group). Things take time.

arnsholt 11y ago

The reason TeX can be so unapproachable is the fact that it's a very different language from pretty much any other programming language you've ever used. It's not a Turing machine, and it's not lambda calculus. It's not even Horn clauses (which is the model behind Prolog). It's a string rewriting system; basically a giant cascade of rules like "rewrite this string to this other string" applied over and over until the whole input is consumed. As it happens, this is yet another Turing-equivalent form of computation. The closest analogy I have is programming in Lisp, but with the constraint that you can only use special forms and macros, no functions are allowed.

versteegen 11y ago
If you want to compare to abstract models of computation, the closest one might be a multi-stack push-down automaton (which is Turing complete).
vanderZwan 11y ago
I was about to say: isn't that just another expression language? I've sometimes wondered if Lisp-like languages wouldn't be a much better way of writing documents than our current markup languages of choice. If I'll ever seriously try to pick up Racket, it'll probably be to give Pollen a spin and see how it is in practice. http://pollenpub.com/ http://pollenpub.com/
sklogic 11y ago
The formalism behind it: https://en.wikipedia.org/wiki/Markov_algorithm https://en.wikipedia.org/wiki/Markov_algorithm

smcl 11y ago

It probably doesn't fit exactly what you want, but I found Lout (http://savannah.nongnu.org/projects/lout http://savannah.nongnu.org/projects/lout) to be a suitable enough replacement for a lot of LaTeX-y tasks. I'm not a heavy LaTeX user so it's likely there's a whole lot missing, but I found the document structure and syntax to be a good deal simpler. In addition the installer is (iirc) only a few MB - I'm sure you can get tiny LaTeX\TeX installers or packages but the ones I've seen tend to be very "batteries included, plus a whole lot more" which is maybe a bit excessive.

coliveira 11y ago

All languages you mentioned are general programming languages. TeX is a word processing language, so it makes complete sense that it is different. If you take the time to properly learn TeX you will see that many of the decisions are obvious for what it intends to do, which is to create full-featured document formats.

baldfat 11y ago

I always had HIGH hopes for LxY http://www.lyx.org/ http://www.lyx.org/ for people with issues with LaTex but have just decided that people really need to learn to write code for typesetting and wish that Latex would produce a better syntax. I have high hopes for LuaTeX.