8 ms·

Typst: A Programmable Markup Language for Typesetting [pdf]

116 points by kindaAnIdiot 4y ago

0x000007 4y ago
nice. really nice. if you do take it further i wish you nothing but success
erk__ 4y ago
The program for it seems to be the one here https://typst.app/ https://typst.app/
- sitkack 4y ago
  Their supporting code is available on github and it is all in Rust. https://github.com/typst/ https://github.com/typst/
  - virtualritz 4y ago
    So for sub-pixel accurate font rasterization in Rust there are [1], [2], [3] and now the one from Typist, [4]. Possibly there are more (I know Raph Linus stopped maintaining [5] so that one is out). All of these crates are actively maintained. Are people re-inventing the wheel? [1] https://gitlab.redox-os.org/redox-os/rusttype https://gitlab.redox-os.org/redox-os/rusttype [2] https://github.com/alexheretic/ab-glyph https://github.com/alexheretic/ab-glyph [3] https://github.com/mooman219/fontdue https://github.com/mooman219/fontdue [4] https://github.com/typst/pixglyph https://github.com/typst/pixglyph [5] https://github.com/raphlinus/font-rs https://github.com/raphlinus/font-rs
    - vitorsr 4y ago
      To me, all of these crates reinvent the wheel when we have FreeType and HarfBuzz which are actually used in the real world. I see no net gain in implementing or in using subpar nonreal-world-tested libraries just to do so in Rust.
      - virtualritz 4y ago
        How do you know something is sub-par if it is not tested? You can only make a statement about the former if the latter is true. But nice try.
        vitorsr 4y ago
        The linked libraries are not even close to solving limited subsets of problems solved by FreeType or HarfBuzz. No test is needed if they do not even have a working implementation of particular requisites: Do they work on heterogeneous layouts, directions, languages, locales, scripts, symbols and composites, extensions, variations, legacy, missing, partial or corrupted instructions, standards interpretations, platforms, output devices, nonstandard point structures and grids? They do not. What they solve is almost a toy problem compared to the size, scope and breadth of these libraries. Just because some project is implemented in Rust does not make it comparable never mind superior by default. There is a world out there and it is not homogeneous format and standards-compliant Latin fonts in English LTR text in linear disposition with some generic rectangular subpixel rendering on a regular rectangular grid. I warmly welcome you to browse closed issues of FreeType [1] and also the closed issues of HarfBuzz [2]. If you feel inspired please do also look into mailing lists and discussion pages related to the development, building, tracking and patching of packages of these projects in any of the numerous places it is used. The only argument Rust people have is in relation WASM but if you insist in targeting WASM why not fork FreeType, strip it to the strict subset of features your application needs and target it? Why do it in the first place? Why reinvent the wheel? As such I will restate my view: I see no gain in using any of these subpar libraries. [1] https://gitlab.freedesktop.org/freetype/freetype/-/issues/?state=closed https://gitlab.freedesktop.org/freetype/freetype/-/issues/?s... [2] https://github.com/harfbuzz/harfbuzz/issues?q=is%3Aclosed https://github.com/harfbuzz/harfbuzz/issues?q=is%3Aclosed
        laurmaedje 4y ago
        I would argue that font rendering and text shaping are completely separate problems. The font renderer doesn't need to know about directions, language, locales and scripts and using one of these simple libraries doesn't mean that an application cannot have a capable text layout and shaping implementation. Although Typst uses pixglyph for rendering, it does support BiDi, complex script shaping, etc. For shaping, we use rustybuzz, which is pretty much a 1-1 port of harfbuzz to Rust. Although we would have gladly used harfbuzz, linking C and Rust in WASM is unfortunately not really possible. So we went for the practical choice of helping finish this port and using it.
azangru 4y ago
The syntax in examples in the thesis looks nice.
mjburgess 4y ago
As far as I can tell it compiles directly to pdf, which seems a non-starter for submitting to many journals which accept either latex or word documents. This would need various compiler backends (perhaps via pandoc) to be that useful. Certainly it would help adoption if you could emit, eg., markdown/latex/etc. and others you're working with wouldn't need to adopt your tooling.
- sebzim4500 4y ago
  Markdown is not close to powerful enough to produce all the stuff that Typst can produce, and emitting latex would be a massive technical effort. I think the best hope is that whatever journals currently only accept latex or .docx will start accepting typst files (or at least pdf).
  - Xeophon 4y ago
    One has to bow first and I do not think that journals and large publishers will be the ones. A typst <-> LaTeX converter would be something which would be a huge feature I'd pay for. I am writing papers for journals/publishers after all.
    - mjburgess 4y ago
      My initial hope was that this would be a front-end for latex.
      - WolfOliver 4y ago
        check out MonsterWriter
  - less_less 4y ago
    Supporting multiple formats is a bit problematic though, if you want to process everything in the journal together to prepare it for publication. It makes the tooling far more complicated than supporting only one format. It might be possible to have a subset of typst output to latex though, and then grow that subset. Like you could just evaluate all the functions to get raw text/markdown and then translate that to latex, and at the beginning you just wouldn't support all possible styling choices.
fermigier 4y ago
Impressive, as a Master's thesis, but also as a long-term project. I've been recently working on a book project, and I've reviewed all the tools available. My current conclusion is that I found nothing that would combine the simplicity of Markdown or Asciidoc with the typographical control of LaTeX. This project seems to hit the sweet spot. Is the source code of the thesis available somewhere? Of other works created with Typst? That would help making a more educated bet. Edit: this doesn't seem to be an open source project. So not what I'm looking for.
- deleted 4y ago
  [deleted]
- laurmaedje 4y ago
  Hey, I'm the author of the thesis. Thanks for the kind words! Just wanted to let you know that while Typst isn't open source yet, we are planning to open source it in March. We will also make a free CLI tool available as to not lock anybody into our web app. There isn't much Typst source code out there yet, but you can find some examples and discussion on our Discord server [1] and in our documentation [2]. [1]: https://discord.gg/2uDybryKPe https://discord.gg/2uDybryKPe [2]: https://typst.app/docs/ https://typst.app/docs/
- mgubi 4y ago
  Have you considered TeXmacs (www.texmacs.org)? See here a short video describing its features: https://www.youtube.com/watch?v=H46ON2FB30U https://www.youtube.com/watch?v=H46ON2FB30U and the twitter feed for more examples: https://twitter.com/gnu_texmacs https://twitter.com/gnu_texmacs
- hendrikrassmann 4y ago
  Have you tried org-mode?
tannhaeuser 4y ago
I really like the paper, though I'm not sure the world needs another Turing-complete document language however well-motivated ;) As SGML pedant however, I can't resist commenting on the following: > The second offspring of SGML is XML, specified by the World Wide Web Consortium (W3C) in 1998. It has a reduced feature set compared to SGML (for example, it forbids unclosed tags and concurrent markup). But it retains the most important aspect of SGML, one that HTML is lacking: The ability to define custom structural elements. This lets XML represent documents with much more semantic detail than HTML. As the SGML vocabulary HTML was once envisioned, HTML itself doesn't need extensibility. When used as an SGML application, defining your own elements in HTML is as easy as declaring those in the "internal subset" or in a custom DTD right away. Assuming any wellformed element is accepted as of ISO 8879 Annex K's FEATURES IMPLYDEF ELEMENT rather than rejecting undeclared elements, that's actually only necessary if you want to validate/infer custom content models, or use any of the other things markup declarations provide, such as custom SHORTREF syntax a la markdown. Arguably, HTML5's "custom elements" do provide a facility to define new elements, if incredibly lousy; ie. custom elements can't have content model restrictions (see above) and can't be used with tag omission/inference (important for customized elements), aren't integrated with DOM parsing, and need JavaScript for declaration - the latter point making them completely pointless as a markup feature.
- sebzim4500 4y ago
  >I really like the paper, though I'm not sure the world needs another Turing-complete document language however well-motivated ;) What would you recommend instead? From my experience of the beta it is vastly nicer to use than latex, and I'm not really aware of any other competition.
- chrismorgan 4y ago
  But HTML never was “an SGML application” in practice, and I highly doubt it was ever actually envisioned as that. There may have been some tools out there that processed HTML as SGML, but none of the ones I know of did (most notably browsers). And in fact, in practice you could just use your own custom elements without worrying about validity and it’d mostly just work. This wasn’t even particularly rare. (There was the whole “CSS doesn’t work on them until you call document.createElement("…")” bug in IE, but that’s the only problem I can think of, and it was easily worked around.)
conaclos 4y ago
Note for the authors: I have just taken a look to the syntax. It is pretty close to djot [1]. Of course Typst is more powerful than djot. One noticeable difference is the chosen markup for headers (`=` instead of `#`). This could be nice to reduce differences and make Typst a close-to superset of djot. [1] https://www.djot.net https://www.djot.net
- laurmaedje 4y ago
  We have switched back and forth on this because we also wanted to stay as close to Markdown as possible, just for the familiarity. However, the fact that Typst also uses the hashtag to indicate an inline expression led to too much confusion (`#thing` was a variable access and `# thing` a heading). And the hashtag is really quite nice for these inline expressions, so it won over the headings. Using `=` for them does have some precedent with AsciiDoc.
  - breck 4y ago
    Definitely a lot of precedent for `=` for headers: https://pldb.com/languages/asciidoc.html https://pldb.com/languages/asciidoc.html https://pldb.com/languages/creole.html https://pldb.com/languages/creole.html https://pldb.com/languages/texti.html https://pldb.com/languages/texti.html https://pldb.com/languages/mediawiki.html https://pldb.com/languages/mediawiki.html
luisivan 4y ago
I've been working on something somewhat similar for producing legal documents, called Linked Markdown (https://linked.md https://linked.md). So the section about modules/importing and dynamic references really rang a bell. Very cool stuff, will keep track of progress.
- breck 4y ago
  I'm very interested in computational law. Drop me an email if you ever want to chat.
WolfOliver 4y ago
How do you handle citations and citation styles? Was scanning the thesis but could find any hints?
- laurmaedje 4y ago
  For the purposes of this thesis I coded the citation style and numbering using Typst's introspection system and handwrote the references. However, we have also written a BibLaTeX-compatible citation system called hayagriva [1] for Typst. We haven't yet integrated it, but want to do so before our public beta. [1]: https://github.com/typst/hayagriva https://github.com/typst/hayagriva
charlieyu1 4y ago
Looks very nice, combining the good parts of latex, markdown and other ordinary programming languages.
sebzim4500 4y ago
I'm in the beta and it's really nice. It's still missing some important features (references is the main thing for me) but once they arrive I doubt I will ever go back to latex.
- amichail 4y ago
  Have you tried TeXmacs?
atemerev 4y ago
Wow, impressive. At last, something _new_ and _better_ in the field.
Ghast 4y ago
This is exciting, as I've been waiting for something like this to come along as I'm typesetting an RPG book with LaTeX, and they generally have rather complex layouts: Random example: https://external-content.duckduckgo.com/iu/?u=https%3A%2F%2Fi.pinimg.com%2Foriginals%2F3f%2Fdb%2F5f%2F3fdb5fc8eec3b9b5f9f5994c71bd6c74.jpg&f=1&nofb=1&ipt=adb0170585badb4a1aa7e172abfb2a13e329746df7d4f55ae22456767922e901&ipo=images https://external-content.duckduckgo.com/iu/?u=https%3A%2F%2F... Calculating stat-blocks by hand is a nighmtare for one author, so I need a typesetting language so I can type `\elf`, and have a random elf, then generate all its derived stats correctly (like Attack, Defence, et c.). My example: https://i.redd.it/ng82unzqxru41.png https://i.redd.it/ng82unzqxru41.png Each book has different versions (minimal/ full) so getting reliable output has been a bit of a chore. LaTeX is the only tool I know which can do the job, but it's still a bit of a nightmare at time. Ugly formatting means my code looks whacky, images can't float cleanly around a multicolumn environment, packages conflict, words overlap and it needs constant hand-holding. Typst looks like just what I've been after - something like rmarkdown, with more power. Tables and layouts, and best of all - no packages. What licence is typst under? Will it be able to reference page numbers from a different book dynamically? (so if book-a and book-b are in the same directory, one can reference another's sections) Will there be support for intelligent floating images? Will it accept 'every-page' commands, so I can use that #rect command to show a chapter's name on side-tabs? Will error messages tell me hbox underfull badness 10000 at least 3000 times per compile?
- vitorsr 4y ago
  It is not my intention to overstep what you seem to be very engrossed in but have you tried LuaLaTeX? You can have Lua code that generates valid LaTeX and LaTeX macros that execute Lua code (see [1]). LuaLaTeX also supports TTF and OTF formats which plays well with other publishing software. In relation to formatting, I cannot tell what could be the issue but this does not correspond to mine and others' experiences. Typically you would typeset the book with the generally intended text areas with LaTeX, and then generate the preprint with previously not included graphical elements and fine-tuned text layouts with Adobe InDesign. This is what publishers typically do, again, from mine and others' experiences. [1] http://mirrors.ctan.org/info/luatex/lualatex-doc/lualatex-doc.pdf http://mirrors.ctan.org/info/luatex/lualatex-doc/lualatex-do...
less_less 4y ago
Note for the authors: As someone who writes a lot of math, I'm not sure I would want / as stacked fraction to be the default choice, as it is in Word (but not Latex). Many mathematical (sub)expressions, especially short ones, are more readable when written inline with the slash.
- laurmaedje 4y ago
  This is interesting feedback. I imagine defining stackedness through global styling wouldn't help, as you would like to decide on a case-by-case basis which one is preferrable?
  - less_less 4y ago
    Yeah, it would ideally be per fraction by the author's choice. In Latex notation, inline math is usually $a/b$ especially for simple fractions, because otherwise you expand the line spacing or have really small text. Block math is sometimes $$a/b$$ and sometimes $$\frac{a}{b}$$, depending on what looks cleaner. Exponents are often $x^{a/b}$ to avoid font size / layout issues. It's even fairly common to mix it within a single equation, especially with fractional exponents or fractions of fractions, e.g.: $$\frac{ x^{2/3} + 7 }{ 4y/z + 9 }$$ This makes it easier to read: nested fractions kinda suck no matter what, but with bars on bars on bars they're even worse. Anyway, there should at least be a way to specify which fraction notation you want. I would even suggest // for stacked-frac, but that's a comment. Microsoft Word and powerpoint have copied quite a bit of Latex in the past several years which have made them more usable for math, but their lack of fraction rendering control is one of my pet peeves. Edited to add: Also the slash is used for things other than fractions, like group quotients, which are not conventionally written in the stacked notation. See eg https://www.math.leidenuniv.nl/~psh/ANTproc/04psh.pdf https://www.math.leidenuniv.nl/~psh/ANTproc/04psh.pdf
    - laurmaedje 4y ago
      Double slash for stacked fractions was also my first impulse upon reading your post. Currently, it starts a line comment even within math, but that could be changed. We'll consider it!
blacklion 4y ago
It is known, that TeX algorithm to split long text into strings to make paragraph is globally-optimal, but splitting stack of lines into pages are only locally-optimal due to memory constrains at the time of original TeX implementation. Does this system uses globally-optimal algorithm for second step (page composition)? Does anybody know globally-optimal formatting system?
blacklion 4y ago
I'm reading thesis right now and I could say that it is very well-written text. Markup-based DTP was my interest for many years, and I happy to see new full-featured players on this field. MarkDown, AsciiDoc and others github-README formats are nice but disappointingly limited (and under-specified). Edit: > the table of contents is typically at the start of a book, not the end. In my culture it is typical at the end, though in last 20 years a sheer amount of translated literature which layout is copy'n'pasted form originals (by people who was not trained in classical typesetting, but learned DTP on-the-fly) somewhat change this. If authors want to continue to support & expand their product, they need to study very diverse typesetting traditions of different countries and languages.
blacklion 4y ago
Ok, I've read thesis and have many questions to authors: 1) Is it possible to create new structure elements and rules how they can be nested? Now there are headers of several levels, and it's all? 2) What's about tables? 2.a) Multipage tables? 2.b) Multipage tables which have alternative orientation - like landscape table in portrait book? 3) What's about inserts, which must be placed in exact break in the main text? 4) What's about marginalia? Marginalia which must be synchronized with text? 5) What's about multicolumn layout? 5.a) What about multicolumn layout which is interrupted by single floating object (which belongs to page, not one column) - there could be two styles, when columns above and under floating object are independent and when they are continue after interruption. 6) What's about multiple flows of text, synchronized? Think about bi- and triple-lingual texts, like translations of legal documents or poetry. Different flows could go to different pages (odd-even) or different columns on one page, but each page/column must contain same text, even if one language is more compact than another and one page/column will be systematically underfilled. 7) What's about poetry, and modern poetry especially, where each line can require its own indentation (look up Mayakovsky's poetry). 8) What's about footnotes? 8.a) Long, multi-page footnotes? 8.b) Nested footnotes, more than 2 levels of them? 8.c) Nested multi-page footnotes? 9) Is it possible to create "Problem / Solution" object typical for textbooks? I want to type problems and solutions together and automatically place all problems where they are typed, but move all solutions in designated place (like, last chapter)? 10) Is it possible to have Initials automatically? 11) What's with custom alignment rules? For example, in some typographic traditions last line of aligned paragraph must be not shorter than 1/3 of paragraph's width, and must be aligned too if it is longer than 2/3. 12) What's with layout engine in general? Is it TeX' box-and-glue model? Is orphan/widow control customizable? 13) What's about non-rectangular images? 13.a) Alpha channel support for runaround? 13.b) Additional vector path for runaround? 13.c) Choice should image be trimmed or become background? 14) What's about images and other objects on the double-page spread? 15) What's about layouts typical for magazines, like "Cosmopolitan"? It is weakest point of TeX-like DTP systems :-(
- laurmaedje 4y ago
  You raise many interesting problems. While I have answered each one individually below, the general answer is: Typst is still a young project and while its already possible to create papers and other documents in it, lots of stuff is still missing. In the long term, we definitely want to handle all sorts of complex layout requirements, but for now we're working to get the basics right. That being said, here the individual answers: > 1) Is it possible to create new structure elements and rules how they can be nested? Now there are headers of several levels, and it's all? At the moment you cannot define custom structural elements, but you can define a function that produces some layout and the result is very similar. The structural aspect becomes more important when different parties need to agree on the meaning of something, i.e. you and Typst's standard library or you and a package author, so that the styling system also works with for those elements. We will add support for defining custom structural elements down the road. > 2) What's about tables? 2.a) Multipage tables? 2.b) Multipage tables which have alternative orientation - like landscape table in portrait book? Tables exist, also multipage tables. The table sizing is already quite flexible (and inspired by CSS grid), otherwise the customization is still quite primitive, though. We will improve tables in the future, for sure. See also here: https://typst.app/docs/reference/basics/table/ https://typst.app/docs/reference/basics/table/ > 3) What's about inserts, which must be placed in exact break in the main text? Do you just mean a block between two paragraphs? If so, that's possible, yes. > 4) What's about marginalia? Marginalia which must be synchronized with text? Not implemented at the moment, but planned. > 5) What's about multicolumn layout? 5.a) What about multicolumn layout which is interrupted by single floating object (which belongs to page, not one column) - there could be two styles, when columns above and under floating object are independent and when they are continue after interruption. Typst does support multicolumn layout. Floating objects that interrupt both columns are not possible at the moment, but we're aware of the requirement. > 6) What's about multiple flows of text, synchronized? Think about bi- and triple-lingual texts, like translations of legal documents or poetry. Different flows could go to different pages (odd-even) or different columns on one page, but each page/column must contain same text, even if one language is more compact than another and one page/column will be systematically underfilled. This is obviously a bit more complicated. I can see supporting something like this, but only in the long term. > 7) What's about poetry, and modern poetry especially, where each line can require its own indentation (look up Mayakovsky's poetry). You can create manual line breaks and insert horizontal spacing. Do you imagine a more automatic way? > 8) What's about footnotes? 8.a) Long, multi-page footnotes? 8.b) Nested footnotes, more than 2 levels of them? 8.c) Nested multi-page footnotes? This is one of the basic things that is still missing, unfortunately. But it has relatively high priority and we want to ship them before the public beta launch in March. > 9) Is it possible to create "Problem / Solution" object typical for textbooks? I want to type problems and solutions together and automatically place all problems where they are typed, but move all solutions in designated place (like, last chapter)? This will be possible with Typst's introspection system (which is already implemented, but not yet fully exposed to the user). > 10) Is it possible to have Initials automatically? Not yet, but planned. > 11) What's with custom alignment rules? For example, in some typographic traditions last line of aligned paragraph must be not shorter than 1/3 of paragraph's width, and must be aligned too if it is longer than 2/3. Exposing more control over paragraph layout is something I've wanted to do, but we haven't designed such a system yet. > 12) What's with layout engine in general? Is it TeX' box-and-glue model? Is orphan/widow control customizable? It's not boxes and glue, but we use TeX's optimal paragraph layout algorithm. Orphan/widow handling is currently baked-in, but similarly to exposing more control over paragraph layout, we also want to expose more control over flow (i.e. page or subbox) layout in the future. > 13) What's about non-rectangular images? 13.a) Alpha channel support for runaround? 13.b) Additional vector path for runaround? 13.c) Choice should image be trimmed or become background? Not prioritized, but planned. > 14) What's about layouts typical for magazines, like "Cosmopolitan"? It is weakest point of TeX-like DTP systems :-( This is a hard problem. We want to support content flowing across multiple boxes, which is of course a requirement for such a layout. But overall, I think the hardest part is not making it possible, but making the creation of a magazine in a markup-based system actually hassle-free. When creating a magazine, you just benefit much more from dragging stuff around with the mouse than if you're writing a research paper.

116 points by kindaAnIdiot 4y ago

0x000007 4y ago

nice. really nice. if you do take it further i wish you nothing but success

erk__ 4y ago

The program for it seems to be the one here https://typst.app/ https://typst.app/

sitkack 4y ago
Their supporting code is available on github and it is all in Rust. https://github.com/typst/ https://github.com/typst/
- virtualritz 4y ago
  So for sub-pixel accurate font rasterization in Rust there are [1], [2], [3] and now the one from Typist, [4]. Possibly there are more (I know Raph Linus stopped maintaining [5] so that one is out). All of these crates are actively maintained. Are people re-inventing the wheel? [1] https://gitlab.redox-os.org/redox-os/rusttype https://gitlab.redox-os.org/redox-os/rusttype [2] https://github.com/alexheretic/ab-glyph https://github.com/alexheretic/ab-glyph [3] https://github.com/mooman219/fontdue https://github.com/mooman219/fontdue [4] https://github.com/typst/pixglyph https://github.com/typst/pixglyph [5] https://github.com/raphlinus/font-rs https://github.com/raphlinus/font-rs
  - vitorsr 4y ago
    To me, all of these crates reinvent the wheel when we have FreeType and HarfBuzz which are actually used in the real world. I see no net gain in implementing or in using subpar nonreal-world-tested libraries just to do so in Rust.
    - virtualritz 4y ago
      How do you know something is sub-par if it is not tested? You can only make a statement about the former if the latter is true. But nice try.
      - vitorsr 4y ago
        The linked libraries are not even close to solving limited subsets of problems solved by FreeType or HarfBuzz. No test is needed if they do not even have a working implementation of particular requisites: Do they work on heterogeneous layouts, directions, languages, locales, scripts, symbols and composites, extensions, variations, legacy, missing, partial or corrupted instructions, standards interpretations, platforms, output devices, nonstandard point structures and grids? They do not. What they solve is almost a toy problem compared to the size, scope and breadth of these libraries. Just because some project is implemented in Rust does not make it comparable never mind superior by default. There is a world out there and it is not homogeneous format and standards-compliant Latin fonts in English LTR text in linear disposition with some generic rectangular subpixel rendering on a regular rectangular grid. I warmly welcome you to browse closed issues of FreeType [1] and also the closed issues of HarfBuzz [2]. If you feel inspired please do also look into mailing lists and discussion pages related to the development, building, tracking and patching of packages of these projects in any of the numerous places it is used. The only argument Rust people have is in relation WASM but if you insist in targeting WASM why not fork FreeType, strip it to the strict subset of features your application needs and target it? Why do it in the first place? Why reinvent the wheel? As such I will restate my view: I see no gain in using any of these subpar libraries. [1] https://gitlab.freedesktop.org/freetype/freetype/-/issues/?state=closed https://gitlab.freedesktop.org/freetype/freetype/-/issues/?s... [2] https://github.com/harfbuzz/harfbuzz/issues?q=is%3Aclosed https://github.com/harfbuzz/harfbuzz/issues?q=is%3Aclosed
        laurmaedje 4y ago
        I would argue that font rendering and text shaping are completely separate problems. The font renderer doesn't need to know about directions, language, locales and scripts and using one of these simple libraries doesn't mean that an application cannot have a capable text layout and shaping implementation. Although Typst uses pixglyph for rendering, it does support BiDi, complex script shaping, etc. For shaping, we use rustybuzz, which is pretty much a 1-1 port of harfbuzz to Rust. Although we would have gladly used harfbuzz, linking C and Rust in WASM is unfortunately not really possible. So we went for the practical choice of helping finish this port and using it.

azangru 4y ago

The syntax in examples in the thesis looks nice.

mjburgess 4y ago

As far as I can tell it compiles directly to pdf, which seems a non-starter for submitting to many journals which accept either latex or word documents. This would need various compiler backends (perhaps via pandoc) to be that useful. Certainly it would help adoption if you could emit, eg., markdown/latex/etc. and others you're working with wouldn't need to adopt your tooling.

sebzim4500 4y ago
Markdown is not close to powerful enough to produce all the stuff that Typst can produce, and emitting latex would be a massive technical effort. I think the best hope is that whatever journals currently only accept latex or .docx will start accepting typst files (or at least pdf).
- Xeophon 4y ago
  One has to bow first and I do not think that journals and large publishers will be the ones. A typst <-> LaTeX converter would be something which would be a huge feature I'd pay for. I am writing papers for journals/publishers after all.
  - mjburgess 4y ago
    My initial hope was that this would be a front-end for latex.
    - WolfOliver 4y ago
      check out MonsterWriter
- less_less 4y ago
  Supporting multiple formats is a bit problematic though, if you want to process everything in the journal together to prepare it for publication. It makes the tooling far more complicated than supporting only one format. It might be possible to have a subset of typst output to latex though, and then grow that subset. Like you could just evaluate all the functions to get raw text/markdown and then translate that to latex, and at the beginning you just wouldn't support all possible styling choices.

fermigier 4y ago

Impressive, as a Master's thesis, but also as a long-term project. I've been recently working on a book project, and I've reviewed all the tools available. My current conclusion is that I found nothing that would combine the simplicity of Markdown or Asciidoc with the typographical control of LaTeX. This project seems to hit the sweet spot. Is the source code of the thesis available somewhere? Of other works created with Typst? That would help making a more educated bet. Edit: this doesn't seem to be an open source project. So not what I'm looking for.

deleted 4y ago
[deleted]
laurmaedje 4y ago
Hey, I'm the author of the thesis. Thanks for the kind words! Just wanted to let you know that while Typst isn't open source yet, we are planning to open source it in March. We will also make a free CLI tool available as to not lock anybody into our web app. There isn't much Typst source code out there yet, but you can find some examples and discussion on our Discord server [1] and in our documentation [2]. [1]: https://discord.gg/2uDybryKPe https://discord.gg/2uDybryKPe [2]: https://typst.app/docs/ https://typst.app/docs/
mgubi 4y ago
Have you considered TeXmacs (www.texmacs.org)? See here a short video describing its features: https://www.youtube.com/watch?v=H46ON2FB30U https://www.youtube.com/watch?v=H46ON2FB30U and the twitter feed for more examples: https://twitter.com/gnu_texmacs https://twitter.com/gnu_texmacs
hendrikrassmann 4y ago
Have you tried org-mode?

tannhaeuser 4y ago

I really like the paper, though I'm not sure the world needs another Turing-complete document language however well-motivated ;) As SGML pedant however, I can't resist commenting on the following: > The second offspring of SGML is XML, specified by the World Wide Web Consortium (W3C) in 1998. It has a reduced feature set compared to SGML (for example, it forbids unclosed tags and concurrent markup). But it retains the most important aspect of SGML, one that HTML is lacking: The ability to define custom structural elements. This lets XML represent documents with much more semantic detail than HTML. As the SGML vocabulary HTML was once envisioned, HTML itself doesn't need extensibility. When used as an SGML application, defining your own elements in HTML is as easy as declaring those in the "internal subset" or in a custom DTD right away. Assuming any wellformed element is accepted as of ISO 8879 Annex K's FEATURES IMPLYDEF ELEMENT rather than rejecting undeclared elements, that's actually only necessary if you want to validate/infer custom content models, or use any of the other things markup declarations provide, such as custom SHORTREF syntax a la markdown. Arguably, HTML5's "custom elements" do provide a facility to define new elements, if incredibly lousy; ie. custom elements can't have content model restrictions (see above) and can't be used with tag omission/inference (important for customized elements), aren't integrated with DOM parsing, and need JavaScript for declaration - the latter point making them completely pointless as a markup feature.

sebzim4500 4y ago
>I really like the paper, though I'm not sure the world needs another Turing-complete document language however well-motivated ;) What would you recommend instead? From my experience of the beta it is vastly nicer to use than latex, and I'm not really aware of any other competition.
chrismorgan 4y ago
But HTML never was “an SGML application” in practice, and I highly doubt it was ever actually envisioned as that. There may have been some tools out there that processed HTML as SGML, but none of the ones I know of did (most notably browsers). And in fact, in practice you could just use your own custom elements without worrying about validity and it’d mostly just work. This wasn’t even particularly rare. (There was the whole “CSS doesn’t work on them until you call document.createElement("…")” bug in IE, but that’s the only problem I can think of, and it was easily worked around.)

conaclos 4y ago

Note for the authors: I have just taken a look to the syntax. It is pretty close to djot [1]. Of course Typst is more powerful than djot. One noticeable difference is the chosen markup for headers (`=` instead of `#`). This could be nice to reduce differences and make Typst a close-to superset of djot. [1] https://www.djot.net https://www.djot.net

laurmaedje 4y ago
We have switched back and forth on this because we also wanted to stay as close to Markdown as possible, just for the familiarity. However, the fact that Typst also uses the hashtag to indicate an inline expression led to too much confusion (`#thing` was a variable access and `# thing` a heading). And the hashtag is really quite nice for these inline expressions, so it won over the headings. Using `=` for them does have some precedent with AsciiDoc.
- breck 4y ago
  Definitely a lot of precedent for `=` for headers: https://pldb.com/languages/asciidoc.html https://pldb.com/languages/asciidoc.html https://pldb.com/languages/creole.html https://pldb.com/languages/creole.html https://pldb.com/languages/texti.html https://pldb.com/languages/texti.html https://pldb.com/languages/mediawiki.html https://pldb.com/languages/mediawiki.html

luisivan 4y ago

I've been working on something somewhat similar for producing legal documents, called Linked Markdown (https://linked.md https://linked.md). So the section about modules/importing and dynamic references really rang a bell. Very cool stuff, will keep track of progress.

breck 4y ago
I'm very interested in computational law. Drop me an email if you ever want to chat.

WolfOliver 4y ago

How do you handle citations and citation styles? Was scanning the thesis but could find any hints?

laurmaedje 4y ago
For the purposes of this thesis I coded the citation style and numbering using Typst's introspection system and handwrote the references. However, we have also written a BibLaTeX-compatible citation system called hayagriva [1] for Typst. We haven't yet integrated it, but want to do so before our public beta. [1]: https://github.com/typst/hayagriva https://github.com/typst/hayagriva

charlieyu1 4y ago

Looks very nice, combining the good parts of latex, markdown and other ordinary programming languages.

sebzim4500 4y ago

I'm in the beta and it's really nice. It's still missing some important features (references is the main thing for me) but once they arrive I doubt I will ever go back to latex.

amichail 4y ago
Have you tried TeXmacs?

atemerev 4y ago

Wow, impressive. At last, something _new_ and _better_ in the field.

Ghast 4y ago

This is exciting, as I've been waiting for something like this to come along as I'm typesetting an RPG book with LaTeX, and they generally have rather complex layouts: Random example: https://external-content.duckduckgo.com/iu/?u=https%3A%2F%2Fi.pinimg.com%2Foriginals%2F3f%2Fdb%2F5f%2F3fdb5fc8eec3b9b5f9f5994c71bd6c74.jpg&f=1&nofb=1&ipt=adb0170585badb4a1aa7e172abfb2a13e329746df7d4f55ae22456767922e901&ipo=images https://external-content.duckduckgo.com/iu/?u=https%3A%2F%2F... Calculating stat-blocks by hand is a nighmtare for one author, so I need a typesetting language so I can type `\elf`, and have a random elf, then generate all its derived stats correctly (like Attack, Defence, et c.). My example: https://i.redd.it/ng82unzqxru41.png https://i.redd.it/ng82unzqxru41.png Each book has different versions (minimal/ full) so getting reliable output has been a bit of a chore. LaTeX is the only tool I know which can do the job, but it's still a bit of a nightmare at time. Ugly formatting means my code looks whacky, images can't float cleanly around a multicolumn environment, packages conflict, words overlap and it needs constant hand-holding. Typst looks like just what I've been after - something like rmarkdown, with more power. Tables and layouts, and best of all - no packages. What licence is typst under? Will it be able to reference page numbers from a different book dynamically? (so if book-a and book-b are in the same directory, one can reference another's sections) Will there be support for intelligent floating images? Will it accept 'every-page' commands, so I can use that #rect command to show a chapter's name on side-tabs? Will error messages tell me hbox underfull badness 10000 at least 3000 times per compile?

vitorsr 4y ago
It is not my intention to overstep what you seem to be very engrossed in but have you tried LuaLaTeX? You can have Lua code that generates valid LaTeX and LaTeX macros that execute Lua code (see [1]). LuaLaTeX also supports TTF and OTF formats which plays well with other publishing software. In relation to formatting, I cannot tell what could be the issue but this does not correspond to mine and others' experiences. Typically you would typeset the book with the generally intended text areas with LaTeX, and then generate the preprint with previously not included graphical elements and fine-tuned text layouts with Adobe InDesign. This is what publishers typically do, again, from mine and others' experiences. [1] http://mirrors.ctan.org/info/luatex/lualatex-doc/lualatex-doc.pdf http://mirrors.ctan.org/info/luatex/lualatex-doc/lualatex-do...

less_less 4y ago

Note for the authors: As someone who writes a lot of math, I'm not sure I would want / as stacked fraction to be the default choice, as it is in Word (but not Latex). Many mathematical (sub)expressions, especially short ones, are more readable when written inline with the slash.

laurmaedje 4y ago
This is interesting feedback. I imagine defining stackedness through global styling wouldn't help, as you would like to decide on a case-by-case basis which one is preferrable?
- less_less 4y ago
  Yeah, it would ideally be per fraction by the author's choice. In Latex notation, inline math is usually $a/b$ especially for simple fractions, because otherwise you expand the line spacing or have really small text. Block math is sometimes $$a/b$$ and sometimes $$\frac{a}{b}$$, depending on what looks cleaner. Exponents are often $x^{a/b}$ to avoid font size / layout issues. It's even fairly common to mix it within a single equation, especially with fractional exponents or fractions of fractions, e.g.: $$\frac{ x^{2/3} + 7 }{ 4y/z + 9 }$$ This makes it easier to read: nested fractions kinda suck no matter what, but with bars on bars on bars they're even worse. Anyway, there should at least be a way to specify which fraction notation you want. I would even suggest // for stacked-frac, but that's a comment. Microsoft Word and powerpoint have copied quite a bit of Latex in the past several years which have made them more usable for math, but their lack of fraction rendering control is one of my pet peeves. Edited to add: Also the slash is used for things other than fractions, like group quotients, which are not conventionally written in the stacked notation. See eg https://www.math.leidenuniv.nl/~psh/ANTproc/04psh.pdf https://www.math.leidenuniv.nl/~psh/ANTproc/04psh.pdf
  - laurmaedje 4y ago
    Double slash for stacked fractions was also my first impulse upon reading your post. Currently, it starts a line comment even within math, but that could be changed. We'll consider it!

blacklion 4y ago

It is known, that TeX algorithm to split long text into strings to make paragraph is globally-optimal, but splitting stack of lines into pages are only locally-optimal due to memory constrains at the time of original TeX implementation. Does this system uses globally-optimal algorithm for second step (page composition)? Does anybody know globally-optimal formatting system?

blacklion 4y ago

I'm reading thesis right now and I could say that it is very well-written text. Markup-based DTP was my interest for many years, and I happy to see new full-featured players on this field. MarkDown, AsciiDoc and others github-README formats are nice but disappointingly limited (and under-specified). Edit: > the table of contents is typically at the start of a book, not the end. In my culture it is typical at the end, though in last 20 years a sheer amount of translated literature which layout is copy'n'pasted form originals (by people who was not trained in classical typesetting, but learned DTP on-the-fly) somewhat change this. If authors want to continue to support & expand their product, they need to study very diverse typesetting traditions of different countries and languages.

blacklion 4y ago

Ok, I've read thesis and have many questions to authors: 1) Is it possible to create new structure elements and rules how they can be nested? Now there are headers of several levels, and it's all? 2) What's about tables? 2.a) Multipage tables? 2.b) Multipage tables which have alternative orientation - like landscape table in portrait book? 3) What's about inserts, which must be placed in exact break in the main text? 4) What's about marginalia? Marginalia which must be synchronized with text? 5) What's about multicolumn layout? 5.a) What about multicolumn layout which is interrupted by single floating object (which belongs to page, not one column) - there could be two styles, when columns above and under floating object are independent and when they are continue after interruption. 6) What's about multiple flows of text, synchronized? Think about bi- and triple-lingual texts, like translations of legal documents or poetry. Different flows could go to different pages (odd-even) or different columns on one page, but each page/column must contain same text, even if one language is more compact than another and one page/column will be systematically underfilled. 7) What's about poetry, and modern poetry especially, where each line can require its own indentation (look up Mayakovsky's poetry). 8) What's about footnotes? 8.a) Long, multi-page footnotes? 8.b) Nested footnotes, more than 2 levels of them? 8.c) Nested multi-page footnotes? 9) Is it possible to create "Problem / Solution" object typical for textbooks? I want to type problems and solutions together and automatically place all problems where they are typed, but move all solutions in designated place (like, last chapter)? 10) Is it possible to have Initials automatically? 11) What's with custom alignment rules? For example, in some typographic traditions last line of aligned paragraph must be not shorter than 1/3 of paragraph's width, and must be aligned too if it is longer than 2/3. 12) What's with layout engine in general? Is it TeX' box-and-glue model? Is orphan/widow control customizable? 13) What's about non-rectangular images? 13.a) Alpha channel support for runaround? 13.b) Additional vector path for runaround? 13.c) Choice should image be trimmed or become background? 14) What's about images and other objects on the double-page spread? 15) What's about layouts typical for magazines, like "Cosmopolitan"? It is weakest point of TeX-like DTP systems :-(

laurmaedje 4y ago
You raise many interesting problems. While I have answered each one individually below, the general answer is: Typst is still a young project and while its already possible to create papers and other documents in it, lots of stuff is still missing. In the long term, we definitely want to handle all sorts of complex layout requirements, but for now we're working to get the basics right. That being said, here the individual answers: > 1) Is it possible to create new structure elements and rules how they can be nested? Now there are headers of several levels, and it's all? At the moment you cannot define custom structural elements, but you can define a function that produces some layout and the result is very similar. The structural aspect becomes more important when different parties need to agree on the meaning of something, i.e. you and Typst's standard library or you and a package author, so that the styling system also works with for those elements. We will add support for defining custom structural elements down the road. > 2) What's about tables? 2.a) Multipage tables? 2.b) Multipage tables which have alternative orientation - like landscape table in portrait book? Tables exist, also multipage tables. The table sizing is already quite flexible (and inspired by CSS grid), otherwise the customization is still quite primitive, though. We will improve tables in the future, for sure. See also here: https://typst.app/docs/reference/basics/table/ https://typst.app/docs/reference/basics/table/ > 3) What's about inserts, which must be placed in exact break in the main text? Do you just mean a block between two paragraphs? If so, that's possible, yes. > 4) What's about marginalia? Marginalia which must be synchronized with text? Not implemented at the moment, but planned. > 5) What's about multicolumn layout? 5.a) What about multicolumn layout which is interrupted by single floating object (which belongs to page, not one column) - there could be two styles, when columns above and under floating object are independent and when they are continue after interruption. Typst does support multicolumn layout. Floating objects that interrupt both columns are not possible at the moment, but we're aware of the requirement. > 6) What's about multiple flows of text, synchronized? Think about bi- and triple-lingual texts, like translations of legal documents or poetry. Different flows could go to different pages (odd-even) or different columns on one page, but each page/column must contain same text, even if one language is more compact than another and one page/column will be systematically underfilled. This is obviously a bit more complicated. I can see supporting something like this, but only in the long term. > 7) What's about poetry, and modern poetry especially, where each line can require its own indentation (look up Mayakovsky's poetry). You can create manual line breaks and insert horizontal spacing. Do you imagine a more automatic way? > 8) What's about footnotes? 8.a) Long, multi-page footnotes? 8.b) Nested footnotes, more than 2 levels of them? 8.c) Nested multi-page footnotes? This is one of the basic things that is still missing, unfortunately. But it has relatively high priority and we want to ship them before the public beta launch in March. > 9) Is it possible to create "Problem / Solution" object typical for textbooks? I want to type problems and solutions together and automatically place all problems where they are typed, but move all solutions in designated place (like, last chapter)? This will be possible with Typst's introspection system (which is already implemented, but not yet fully exposed to the user). > 10) Is it possible to have Initials automatically? Not yet, but planned. > 11) What's with custom alignment rules? For example, in some typographic traditions last line of aligned paragraph must be not shorter than 1/3 of paragraph's width, and must be aligned too if it is longer than 2/3. Exposing more control over paragraph layout is something I've wanted to do, but we haven't designed such a system yet. > 12) What's with layout engine in general? Is it TeX' box-and-glue model? Is orphan/widow control customizable? It's not boxes and glue, but we use TeX's optimal paragraph layout algorithm. Orphan/widow handling is currently baked-in, but similarly to exposing more control over paragraph layout, we also want to expose more control over flow (i.e. page or subbox) layout in the future. > 13) What's about non-rectangular images? 13.a) Alpha channel support for runaround? 13.b) Additional vector path for runaround? 13.c) Choice should image be trimmed or become background? Not prioritized, but planned. > 14) What's about layouts typical for magazines, like "Cosmopolitan"? It is weakest point of TeX-like DTP systems :-( This is a hard problem. We want to support content flowing across multiple boxes, which is of course a requirement for such a layout. But overall, I think the hardest part is not making it possible, but making the creation of a magazine in a markup-based system actually hassle-free. When creating a magazine, you just benefit much more from dragging stuff around with the mouse than if you're writing a research paper.