9 ms·

The price of a LaTeX rewrite would be even higher: incompatibility. If one tries to rewrite TeX, the problem gets much much worse, since TeX is, for all intents

The price of a LaTeX rewrite would be even higher: incompatibility. If one tries to rewrite TeX, the problem gets much much worse, since TeX is, for all intents and purposes, bug-free. A new implementation will certainly not be. Joel Spolsky said it very well http://www.joelonsoftware.com/articles/fog0000000069.html http://www.joelonsoftware.com/articles/fog0000000069.html I disagree that the current situation is LaTeX developers' fault. TeX and LaTeX are complex pieces of software, that are developed over time. They were extended as the capabilities of their platforms increased, to take advantage of those capabilities. This requires full access to the OS utilities, and naturally LaTeX environment does that. If anything, this is Apple's fault. For whatever reason, they cannot allow applications to use the existing capabilities of the underlying OS. This goes against the Unix mindset; of course there will be unpleasant consequences, but one cannot hold Unix mindset responsible for these.

X-Istence 15y ago
The big problem on tablet devices is constraints in memory. Yes the unix way is to have various small utilities work together through pipes and interprocess communications. The big challenge at that point is how to do you manage memory? What if a single process in that huge chain uses up all of the memory and another process gets killed by the OS which was part of that chain (do note, no swap on mobile devices)... How is the calling program supposed to handle one of its subprocesses suddenly dying? Do note that the same is true on Google's Android OS. You can run as many threads as you want, but no calling other executables to do work in the background ...
- timmy-turner 15y ago
  A current tablet device, even a cheap one, has about 256Mb of main memory. Android (at least the cyanogen mod kernel) has also the ability to use a part of the main memory to store compressed swap partition data. That leaves about 80 Megabytes to userspace programs (on Android). About the amount a normal desktop computer had in 1998. LaTex dates back to the beginning of the 80s. Given that it has the ability to create large documents, I doubt that the required amount of main memory scales linearly with the size of the document. Also, when you use subprocesses, besides checking their return value, you have some sort of interprocess communication which should detect when one process failed to finish his job. It looks more like the smartphone runtimes and their sandboxes just don't support it at all, because modern GUI applications rarely build upon multiple executables in the background.
  - fpgeek 15y ago
    Android does have IPC mechanisms (Intents and remote methods) that can be used to have another process do work in the background. The difference is that even if you use them, you're still not likely to wind up with something that looks like a set of cooperating command-line utilities. In any case, I absolutely agree that resource constraints aren't keeping LaTeX off of our mobile devices.
  - nieve 15y ago
    I have TeX Live available in the Ubuntu chroot on my HP TouchPad, so you're right that it's not a resource thing. Admittedly in this case webOS provides a pretty standard Linux environment under the hood, so it falls pretty firmly outside the Android model (much less iOS).
pantaloons 15y ago
I'm curious where you get the idea that LaTeX is bug free. In my experience it is rare to find two packages that work together bug free, yet alone the collections of them needed to form even a respectable document.
- deleted 15y ago
  [deleted]
- _delirium 15y ago
  I assume he meant that core TeX (the part written ages ago by Knuth) is in effect bug-free. However you do have a good point that one effect of core TeX being ancient and not really updated is that, in practice, what people actually use is TeX plus giant macro packages whose size is bigger than TeX itself, and those are certainly not bug-free. It's at least plausible that a rewrite wouldn't lead to more bugs total, if the rewrite made it easier to write less-buggy macro packages that interacted more nicely (the TeX macro/module situation is not great for that).
  - patrickg 15y ago
    Nobody uses (Knuth's) TeX anymore. They use PDFTeX, XeTeX and LuaTeX. None of these are as stable as TeX, but in practice a whole lot better than most other software I use.
    - coliveira 15y ago
      All of these versions of TeX are just extensions of the original TeX. The core Knuth's code is still used in each one of them. The stability comes from the core, not just from the work of pdftex, xetex and luatex developers.
      - patrickg 15y ago
        The stability comes from the core, but the problems arise from the extensions. I have pushed LuaTeX to some limits and seen lots of segfaults and other errors.
tolmasky 15y ago
Unless I'm forgetting something from the article, the author never said it was the LaTeX developer's fault -- he simply said Apple was right for not allowing it in its current state. Not all situations are someone's "fault", and thus just because Apple is not at fault doesn't mean the developers are. It's just as you declared it, an unfortunate result of many decisions which were right at the time. That being said, I also think you have a low bar for definitions of "bugs". Terrible performance is very much a "bug" in my book. Even the things that aren't "bugs" in the strictest sense can still make for bad software. The executable described here is 4GB, that's 1/4 thte total space on a low end iPad! Also, if tomorrow someone does discover a bug, it sounds like this would be a mess to fix. The jumble of source simply won't be attracting many new developers (whether it be to fix existing bugs, add new features, or port to new platforms).
- merijnv 15y ago
  The LaTeX executable isn't 4GB, it's 3MB or so (pdflatex/pdftex) or 16MB in the case of XeLaTeX. The 4GB consists of billions of extension/modules/packages for typesetting documents.
  - tolmasky 15y ago
    Yes I noticed that in the comments before I posted which is why I was careful to specifically say "The executable described here is 4GB" so as to refer to whatever it is he was trying to work with. I suppose whether all of the 4GB is needed for what he wanted, whether it could easily be taken apart, or could now be remade in a much smaller package, are all valid questions.
    - JoachimSchipper 15y ago
      4G is a bit over the top. My install (TeXLive 2010 on OpenBSD) is ~2G, and that includes gobs of rarely-used packages and documentation. Really, borrowing MikTeX's install-on-first-use for packages and making documentation optional should result in a much more reasonable size. If you want to get fancy, teach LaTeX how to decompress packages on demand.
idspispopd 15y ago
>This requires full access to the OS utilities, and naturally LaTeX environment does that. If anything, this is Apple's fault. I'm interested to see how this could be justified, instead of "hit'n'run" style. Noting here that other than the source problems, performance was a considerable issue.
dextorious 15y ago
"""The price of a LaTeX rewrite would be even higher: incompatibility.""" Yeah. The only benefit is a codebase that reflects current needs and can grow in new directions and follow current practices, that programmers of today are familiar with. And also be faster, and more flexible. But what are those compared to compatibility right? (Of course people interested in pure compatibility could just continue to use the old version, until the new is compatible enough, or they don't need compatibility anymore). """If one tries to rewrite TeX, the problem gets much much worse, since TeX is, for all intents and purposes, bug-free.""" And your wrong assumption is a) that a problem has to be bug free to be useful or b) that a new effort can't produce a relatively bug free codebase?
- quanticle 15y ago
  And your wrong assumption is a) that a problem has to be bug free to be useful or b) that a new effort can't produce a relatively bug free codebase? How are either of those requirements wrong? I certainly expect my programs to be bug free. LaTeX is a tool that's older than many GNU utilities. It's been reviewed and tested numerous times over by the harshest test suite of all: the real world. Yes, in theory, you could rewrite TeX to be cleaner, while maintaining backwards compatibility and lack of bugs and regressions. In practice, this is very very hard. It's so hard that no one, no individual or company has managed to pull it off. But hey, who knows? You could be the first. The LaTeX codebase is open-source. If you think you can pull off what the most talented software engineers in the world haven't been able to do, go right ahead.
  - dextorious 15y ago
    """How are either of those requirements wrong? I certainly expect my programs to be bug free.""" Good luck with that. What I said, though, was that "a problem has to be bug free to be useful" is a wrong assumption. We all use programs that have bugs, and still found them immensely useful. """Yes, in theory, you could rewrite TeX to be cleaner, while maintaining backwards compatibility and lack of bugs and regressions. In practice, this is very very hard. It's so hard that no one, no individual or company has managed to pull it off.""" What are you talking about? Tons of people have written TeX versions of their own, even in Lua, and most of those are widely used. """But hey, who knows? You could be the first.""" Hardly. There is LuaTex, XeTeX (even more widely used) and others. Are you just talking out of your ass? (Not to mention several other modern typesetting engines that are not TeX based. You make it sound like TeX is an impossible achievement). Now, what I said I'd like to see was not a rewriting of TeX itself, but a modern version of a TeX like engine, with or without compatibility. Programs like XeTeX and Omega for example provide Unicode support and TTF/OT support to TeX, but I'd like to see something more modular, and with a more streamlined markup than TeX+LaTeX. While we're at it, even scriptable. There are also some improvements found since in typesetting, like those employed in the InDesign layout engine concerning edge text and hyphenation.
zerostar07 15y ago
The only think i 'd keep from old tex is part of the mathematical notation. Otherwise, it's bloated, quirky and doesn't handle international text well.
nailer 15y ago
> For whatever reason, they cannot allow applications to use the existing capabilities of the underlying OS. This goes against the Unix mindset; of course there will be unpleasant consequences, but one cannot hold Unix mindset responsible for these. Using the native capabilities of the OS in the sense you mean - allowing TeX to use its ancient, hideous bitmap font system, goes against: - Unix philisophy - do one thing well, link against or pipe to other apps/libraries that do their thing well - Simplicity. The user should not be forced to make an unnecessary choice where one option is almost always superor to the other. Knuth's concept of 'bug free' is one of an academic computer scientist. TeX has many, many issues: unnecessary dependencies, a horrible build process, and many UX issues that stopped TeX from reaching TeX's original goal of 'allowing anyone to create typeset documents'. This is fine: Knuth is a mathematically focused computer scientist from a period where everyone who had access to a computer was a programmer. It was valid in this era.
- hyperbovine 15y ago
  You forgot my favorite TeX non-feature: completely uninformative error messages! All kidding aside, I use (La)TeX almost every day and it's bulletproof. It's not going anywhere, so you might as well learn to love it, worts and all.
  - jawaddeo 15y ago
    At last someone mentioned the error reporting - the cause of so much joy during my grad student days!
- yonran 15y ago
  TeX's original goal of 'allowing anyone to create typeset documents' Actually, according to this interview[1], speaking of the very first version of TeX, "I should mention that I had no idea that I was designing a typesetting system that anybody else in the world would ever use, besides me and my secretary. My idea was that I would teach Phyllis, who was my secretary for almost 20 years… So Phyllis and I were going to be the users of this TeX system. … I tried to design it just good enough to do my books and no other books." Even with the released version, I don't think TeX was ever intended to be the last word on typesetting; it was simply his best attempt at a decent markup. He then froze the features so that previously typeset books could be corrected and reprinted using the same source. He left it up to future programs to innovate. [1]: http://www.webofstories.com/play/17118 http://www.webofstories.com/play/17118
bh42222 15y ago
There's rewrites and there's rewrites. I think what Joel is against is the starting from scratch rewrite, I agree with him on this. But I don't think Joel is against a slow, one line at a time, working on the old code base rewrite. It is not clear what kind of re-write the OP is advocating. But I would be surprised if he is pro the start from scratch kind. But it might be just my biased perception.
- jerf 15y ago
  It is also worth pointing out Joel says "don't rewrite" specifically in the context of a software company, where I strongly believe it is largely correct. In the context of a decades-old open source project the decision may be quite different for legitimate reasons. I find it likely that you'd be better off performing an algorithm extraction and doing a true from-scratch rewrite than trying to incrementally translate TeX and LaTeX. For the reader's convenience and reference: http://www.joelonsoftware.com/articles/fog0000000069.html http://www.joelonsoftware.com/articles/fog0000000069.html

X-Istence 15y ago

The big problem on tablet devices is constraints in memory. Yes the unix way is to have various small utilities work together through pipes and interprocess communications. The big challenge at that point is how to do you manage memory? What if a single process in that huge chain uses up all of the memory and another process gets killed by the OS which was part of that chain (do note, no swap on mobile devices)... How is the calling program supposed to handle one of its subprocesses suddenly dying? Do note that the same is true on Google's Android OS. You can run as many threads as you want, but no calling other executables to do work in the background ...

timmy-turner 15y ago
A current tablet device, even a cheap one, has about 256Mb of main memory. Android (at least the cyanogen mod kernel) has also the ability to use a part of the main memory to store compressed swap partition data. That leaves about 80 Megabytes to userspace programs (on Android). About the amount a normal desktop computer had in 1998. LaTex dates back to the beginning of the 80s. Given that it has the ability to create large documents, I doubt that the required amount of main memory scales linearly with the size of the document. Also, when you use subprocesses, besides checking their return value, you have some sort of interprocess communication which should detect when one process failed to finish his job. It looks more like the smartphone runtimes and their sandboxes just don't support it at all, because modern GUI applications rarely build upon multiple executables in the background.
- fpgeek 15y ago
  Android does have IPC mechanisms (Intents and remote methods) that can be used to have another process do work in the background. The difference is that even if you use them, you're still not likely to wind up with something that looks like a set of cooperating command-line utilities. In any case, I absolutely agree that resource constraints aren't keeping LaTeX off of our mobile devices.
- nieve 15y ago
  I have TeX Live available in the Ubuntu chroot on my HP TouchPad, so you're right that it's not a resource thing. Admittedly in this case webOS provides a pretty standard Linux environment under the hood, so it falls pretty firmly outside the Android model (much less iOS).

pantaloons 15y ago

I'm curious where you get the idea that LaTeX is bug free. In my experience it is rare to find two packages that work together bug free, yet alone the collections of them needed to form even a respectable document.

deleted 15y ago
[deleted]
_delirium 15y ago
I assume he meant that core TeX (the part written ages ago by Knuth) is in effect bug-free. However you do have a good point that one effect of core TeX being ancient and not really updated is that, in practice, what people actually use is TeX plus giant macro packages whose size is bigger than TeX itself, and those are certainly not bug-free. It's at least plausible that a rewrite wouldn't lead to more bugs total, if the rewrite made it easier to write less-buggy macro packages that interacted more nicely (the TeX macro/module situation is not great for that).
- patrickg 15y ago
  Nobody uses (Knuth's) TeX anymore. They use PDFTeX, XeTeX and LuaTeX. None of these are as stable as TeX, but in practice a whole lot better than most other software I use.
  - coliveira 15y ago
    All of these versions of TeX are just extensions of the original TeX. The core Knuth's code is still used in each one of them. The stability comes from the core, not just from the work of pdftex, xetex and luatex developers.
    - patrickg 15y ago
      The stability comes from the core, but the problems arise from the extensions. I have pushed LuaTeX to some limits and seen lots of segfaults and other errors.

tolmasky 15y ago

Unless I'm forgetting something from the article, the author never said it was the LaTeX developer's fault -- he simply said Apple was right for not allowing it in its current state. Not all situations are someone's "fault", and thus just because Apple is not at fault doesn't mean the developers are. It's just as you declared it, an unfortunate result of many decisions which were right at the time. That being said, I also think you have a low bar for definitions of "bugs". Terrible performance is very much a "bug" in my book. Even the things that aren't "bugs" in the strictest sense can still make for bad software. The executable described here is 4GB, that's 1/4 thte total space on a low end iPad! Also, if tomorrow someone does discover a bug, it sounds like this would be a mess to fix. The jumble of source simply won't be attracting many new developers (whether it be to fix existing bugs, add new features, or port to new platforms).

merijnv 15y ago
The LaTeX executable isn't 4GB, it's 3MB or so (pdflatex/pdftex) or 16MB in the case of XeLaTeX. The 4GB consists of billions of extension/modules/packages for typesetting documents.
- tolmasky 15y ago
  Yes I noticed that in the comments before I posted which is why I was careful to specifically say "The executable described here is 4GB" so as to refer to whatever it is he was trying to work with. I suppose whether all of the 4GB is needed for what he wanted, whether it could easily be taken apart, or could now be remade in a much smaller package, are all valid questions.
  - JoachimSchipper 15y ago
    4G is a bit over the top. My install (TeXLive 2010 on OpenBSD) is ~2G, and that includes gobs of rarely-used packages and documentation. Really, borrowing MikTeX's install-on-first-use for packages and making documentation optional should result in a much more reasonable size. If you want to get fancy, teach LaTeX how to decompress packages on demand.

idspispopd 15y ago

>This requires full access to the OS utilities, and naturally LaTeX environment does that. If anything, this is Apple's fault. I'm interested to see how this could be justified, instead of "hit'n'run" style. Noting here that other than the source problems, performance was a considerable issue.

dextorious 15y ago

"""The price of a LaTeX rewrite would be even higher: incompatibility.""" Yeah. The only benefit is a codebase that reflects current needs and can grow in new directions and follow current practices, that programmers of today are familiar with. And also be faster, and more flexible. But what are those compared to compatibility right? (Of course people interested in pure compatibility could just continue to use the old version, until the new is compatible enough, or they don't need compatibility anymore). """If one tries to rewrite TeX, the problem gets much much worse, since TeX is, for all intents and purposes, bug-free.""" And your wrong assumption is a) that a problem has to be bug free to be useful or b) that a new effort can't produce a relatively bug free codebase?

quanticle 15y ago
And your wrong assumption is a) that a problem has to be bug free to be useful or b) that a new effort can't produce a relatively bug free codebase? How are either of those requirements wrong? I certainly expect my programs to be bug free. LaTeX is a tool that's older than many GNU utilities. It's been reviewed and tested numerous times over by the harshest test suite of all: the real world. Yes, in theory, you could rewrite TeX to be cleaner, while maintaining backwards compatibility and lack of bugs and regressions. In practice, this is very very hard. It's so hard that no one, no individual or company has managed to pull it off. But hey, who knows? You could be the first. The LaTeX codebase is open-source. If you think you can pull off what the most talented software engineers in the world haven't been able to do, go right ahead.
- dextorious 15y ago
  """How are either of those requirements wrong? I certainly expect my programs to be bug free.""" Good luck with that. What I said, though, was that "a problem has to be bug free to be useful" is a wrong assumption. We all use programs that have bugs, and still found them immensely useful. """Yes, in theory, you could rewrite TeX to be cleaner, while maintaining backwards compatibility and lack of bugs and regressions. In practice, this is very very hard. It's so hard that no one, no individual or company has managed to pull it off.""" What are you talking about? Tons of people have written TeX versions of their own, even in Lua, and most of those are widely used. """But hey, who knows? You could be the first.""" Hardly. There is LuaTex, XeTeX (even more widely used) and others. Are you just talking out of your ass? (Not to mention several other modern typesetting engines that are not TeX based. You make it sound like TeX is an impossible achievement). Now, what I said I'd like to see was not a rewriting of TeX itself, but a modern version of a TeX like engine, with or without compatibility. Programs like XeTeX and Omega for example provide Unicode support and TTF/OT support to TeX, but I'd like to see something more modular, and with a more streamlined markup than TeX+LaTeX. While we're at it, even scriptable. There are also some improvements found since in typesetting, like those employed in the InDesign layout engine concerning edge text and hyphenation.

zerostar07 15y ago

The only think i 'd keep from old tex is part of the mathematical notation. Otherwise, it's bloated, quirky and doesn't handle international text well.

nailer 15y ago

> For whatever reason, they cannot allow applications to use the existing capabilities of the underlying OS. This goes against the Unix mindset; of course there will be unpleasant consequences, but one cannot hold Unix mindset responsible for these. Using the native capabilities of the OS in the sense you mean - allowing TeX to use its ancient, hideous bitmap font system, goes against: - Unix philisophy - do one thing well, link against or pipe to other apps/libraries that do their thing well - Simplicity. The user should not be forced to make an unnecessary choice where one option is almost always superor to the other. Knuth's concept of 'bug free' is one of an academic computer scientist. TeX has many, many issues: unnecessary dependencies, a horrible build process, and many UX issues that stopped TeX from reaching TeX's original goal of 'allowing anyone to create typeset documents'. This is fine: Knuth is a mathematically focused computer scientist from a period where everyone who had access to a computer was a programmer. It was valid in this era.

hyperbovine 15y ago
You forgot my favorite TeX non-feature: completely uninformative error messages! All kidding aside, I use (La)TeX almost every day and it's bulletproof. It's not going anywhere, so you might as well learn to love it, worts and all.
- jawaddeo 15y ago
  At last someone mentioned the error reporting - the cause of so much joy during my grad student days!
yonran 15y ago
TeX's original goal of 'allowing anyone to create typeset documents' Actually, according to this interview[1], speaking of the very first version of TeX, "I should mention that I had no idea that I was designing a typesetting system that anybody else in the world would ever use, besides me and my secretary. My idea was that I would teach Phyllis, who was my secretary for almost 20 years… So Phyllis and I were going to be the users of this TeX system. … I tried to design it just good enough to do my books and no other books." Even with the released version, I don't think TeX was ever intended to be the last word on typesetting; it was simply his best attempt at a decent markup. He then froze the features so that previously typeset books could be corrected and reprinted using the same source. He left it up to future programs to innovate. [1]: http://www.webofstories.com/play/17118 http://www.webofstories.com/play/17118

bh42222 15y ago

There's rewrites and there's rewrites. I think what Joel is against is the starting from scratch rewrite, I agree with him on this. But I don't think Joel is against a slow, one line at a time, working on the old code base rewrite. It is not clear what kind of re-write the OP is advocating. But I would be surprised if he is pro the start from scratch kind. But it might be just my biased perception.

jerf 15y ago
It is also worth pointing out Joel says "don't rewrite" specifically in the context of a software company, where I strongly believe it is largely correct. In the context of a decades-old open source project the decision may be quite different for legitimate reasons. I find it likely that you'd be better off performing an algorithm extraction and doing a true from-scratch rewrite than trying to incrementally translate TeX and LaTeX. For the reader's convenience and reference: http://www.joelonsoftware.com/articles/fog0000000069.html http://www.joelonsoftware.com/articles/fog0000000069.html