4 ms·

If you want me (as an academic) to use ScholarlyMarkdown, then you can't force my collaborators to use ScholarlyMarkdown as a consequence. There should be a ni

by j2kun 12y ago

If you want me (as an academic) to use ScholarlyMarkdown, then you can't force my collaborators to use ScholarlyMarkdown as a consequence. There should be a nice ScholarlyMarkdown -> LaTeX cross compiler for starting simple documents in TeX and then sharing with collaborators (pick some obvious defaults or allow a config file to get fancy). But more importantly, if I am joining a project that already has a bunch of LaTeX wizardry going on, I should be able to seamlessly and implicitly edit the text parts in ScholarlyMarkdown without my collaborators knowing. Can ScholarlyMarkdown do this? If not then I'm not really interested.

zzleeper 12y ago
My guess is that he is building the PDFs through latex/xetex. I built something similar to that, but instead of forking Pandoc (and investing the time to learn Haskell), I just added filters through Python. It's not at fast, but you can intercept mostly anything. Thus, the flow ends up as: paper.md -> Pandoc -> paper.json + metadata -> paper.tex -> paper.pdf
- j2kun 12y ago
  I want the flow to be paper.tex -> paper.md + metadata ->(merge) paper.tex -> paper.pdf
arnsholt 12y ago
That's probably never going to happen (for any Markdown-based project). As you probably know, TeX is a full programming language (albeit an odd one), which means that to compile (La)TeX to Markdown, Markdown needs to support all the features of TeX, which means that it has to be Turing-complete. Not to mention the fact that parsing TeX is kind of nuts too.
- chriswarbo 12y ago
  There's no need to convert all TeX back to markdown; the converter only needs to recognise the sub-set of TeX that it produces. If anyone edits the TeX to fall outside that sub-set, it's reasonable to just embed those parts verbatim in the resulting Markdown.
  - j2kun 12y ago
    Exactly. And I have a very strong feeling that without the ability to collaborate implicitly (at least on the text/math mode part), no replacement for LaTeX will catch on in my community (mathematics).
timtylin 12y ago
I've actually been wondering if this is possible from a theoretical standpoint. I'm thinking you can use Pandoc's LaTeX to MD conversion mode, save changes to a copy, wdiff to get the total change set, then somehow convert that into a LaTeX patch. Whether you can guarantee that this won't break anything is going to be a really challenging problem, as my head is already swimming with edge-cases. I guess we'll know the day that some genius comes up with automatically generated non format-breaking Critic Markup diffs. At least the problem should be easier than LaTeX --- look how long it took for the latexdiff tool to be what it is today, and it still breaks often when you have anything slightly exotic. I'm currently using a workflow in my thesis where I use ScholarlyMarkdown to write individual chapters for final inclusion into an existing LaTeX book document. I find that ScholarlyMarkdown works quite well this way, and it potentially allows collaborators, since individual parts are isolated.
- j2kun 12y ago
  In any case, my point is plain and simple: I collaborate in almost everything I do, and my collaborators are my primary concern when I choose a tool. If they use notepad and Dropbox, I need to make sure my tools don't conflict with that. There's no way in hell I would ask a collaborator to learn to use git or a new markup language just to work with me. I'd love a tool that works with this philosophy, and I feel certain anything like ScholarlyMarkdown won't catch on in my field (theoretical computer science) without such tools.
  - timtylin 12y ago
    As a thought exercise, assuming for simplicity that you can isolate the parts you're authoring, so that your changes only involve additions to other people's work: what kind of additional metainfo would you need for this workflow? All defined labels and macros? Available bib entries? What else?
    - j2kun 12y ago
      I think it depends on how I'm viewing the file. If I'm just editing in SM and building the tex as usual, then I don't need anything extra. If I'm trying to convert to all sorts of other documents (which I haven't ever really done) then labels, theorem environments, one-line macros, and bib entries cover almost every tex-specific thing I use. Here is an excerpt from a typical paper's macro section [1]. As you can see they're mostly one-liners to remove the need to keep typing textup and mathbb, simple mathoperator definitions and such. [1]: http://pastebin.com/RL1gejEZ http://pastebin.com/RL1gejEZ

by j2kun 12y ago

zzleeper 12y ago

My guess is that he is building the PDFs through latex/xetex. I built something similar to that, but instead of forking Pandoc (and investing the time to learn Haskell), I just added filters through Python. It's not at fast, but you can intercept mostly anything. Thus, the flow ends up as: paper.md -> Pandoc -> paper.json + metadata -> paper.tex -> paper.pdf

j2kun 12y ago
I want the flow to be paper.tex -> paper.md + metadata ->(merge) paper.tex -> paper.pdf

arnsholt 12y ago

That's probably never going to happen (for any Markdown-based project). As you probably know, TeX is a full programming language (albeit an odd one), which means that to compile (La)TeX to Markdown, Markdown needs to support all the features of TeX, which means that it has to be Turing-complete. Not to mention the fact that parsing TeX is kind of nuts too.

chriswarbo 12y ago
There's no need to convert all TeX back to markdown; the converter only needs to recognise the sub-set of TeX that it produces. If anyone edits the TeX to fall outside that sub-set, it's reasonable to just embed those parts verbatim in the resulting Markdown.
- j2kun 12y ago
  Exactly. And I have a very strong feeling that without the ability to collaborate implicitly (at least on the text/math mode part), no replacement for LaTeX will catch on in my community (mathematics).

timtylin 12y ago

I've actually been wondering if this is possible from a theoretical standpoint. I'm thinking you can use Pandoc's LaTeX to MD conversion mode, save changes to a copy, wdiff to get the total change set, then somehow convert that into a LaTeX patch. Whether you can guarantee that this won't break anything is going to be a really challenging problem, as my head is already swimming with edge-cases. I guess we'll know the day that some genius comes up with automatically generated non format-breaking Critic Markup diffs. At least the problem should be easier than LaTeX --- look how long it took for the latexdiff tool to be what it is today, and it still breaks often when you have anything slightly exotic. I'm currently using a workflow in my thesis where I use ScholarlyMarkdown to write individual chapters for final inclusion into an existing LaTeX book document. I find that ScholarlyMarkdown works quite well this way, and it potentially allows collaborators, since individual parts are isolated.

j2kun 12y ago
In any case, my point is plain and simple: I collaborate in almost everything I do, and my collaborators are my primary concern when I choose a tool. If they use notepad and Dropbox, I need to make sure my tools don't conflict with that. There's no way in hell I would ask a collaborator to learn to use git or a new markup language just to work with me. I'd love a tool that works with this philosophy, and I feel certain anything like ScholarlyMarkdown won't catch on in my field (theoretical computer science) without such tools.
- timtylin 12y ago
  As a thought exercise, assuming for simplicity that you can isolate the parts you're authoring, so that your changes only involve additions to other people's work: what kind of additional metainfo would you need for this workflow? All defined labels and macros? Available bib entries? What else?
  - j2kun 12y ago
    I think it depends on how I'm viewing the file. If I'm just editing in SM and building the tex as usual, then I don't need anything extra. If I'm trying to convert to all sorts of other documents (which I haven't ever really done) then labels, theorem environments, one-line macros, and bib entries cover almost every tex-specific thing I use. Here is an excerpt from a typical paper's macro section [1]. As you can see they're mostly one-liners to remove the need to keep typing textup and mathbb, simple mathoperator definitions and such. [1]: http://pastebin.com/RL1gejEZ http://pastebin.com/RL1gejEZ