4 ms·

I'm still skimming the documentation so I apologize if the answer is obvious, but does anyone know offhand why this is implemented as a fork of Pandoc? Pandoc a

by _-_-_-_ 12y ago

I'm still skimming the documentation so I apologize if the answer is obvious, but does anyone know offhand why this is implemented as a fork of Pandoc? Pandoc already has extended markdown features and the creator of pandoc is very much an academic (http://johnmacfarlane.net/ http://johnmacfarlane.net/), so is there are a reason why these contributions aren't part of pandoc proper?

kazagistar 12y ago
This was my first thought as well. Pandoc has a rather excellent infrastructure in place to let you extend and build on it, so if you instead choose to fork, there better be a good reason...
- aaren 12y ago
  Pandoc filters let you do a lot of things provided you stick to the defined types. If you want to add new types of element, like this does, then the only practical way is a fork. (except for really simple cases)
  - zzleeper 12y ago
    However, in most cases you could just RawBlock (or was it BlockQuote?) with specific attributes, and then intercept those in a filter and convert to Latex.
    - timtylin 12y ago
      This was basically my original approach. However, after working out the math syntax I realized that some things, like the double-backtick inline math, just can't be accomplished without a pre-filter. At that point I decided to just start playing with the parser code. I also became super-convinced that some level of AST change was necessary to keep things sane, and since I wasn't able to use the existing Math and Image types anyways (they're not attributed), I ultimately just started a new AST type package namespace called "Scholdoc". Everything just evolved from there.
    - aaren 12y ago
      Yes, BlockQuote can be used as a generic block container. It doesn't have attributes itself but you can put them on a header at the start. For generic inline elements you can use Code. Markdown syntax for generic containers (Div and Span) hasn't been implemented yet but the discussion is fairly mature now: https://github.com/jgm/pandoc/issues/168 https://github.com/jgm/pandoc/issues/168 https://github.com/jgm/pandoc/pull/1791 https://github.com/jgm/pandoc/pull/1791
aaren 12y ago
This is a fork because making these changes to pandoc itself needs a lot of consideration. Internal referencing and attributes on figures are two things that are currently being discussed for pandoc. The discussion has been going on for quite a while though - hence people making forks. Discussion on internal referencing: https://github.com/jgm/pandoc/issues/813 https://github.com/jgm/pandoc/issues/813 Discussion on image attributes: https://github.com/jgm/pandoc/issues/261 https://github.com/jgm/pandoc/issues/261
- _-_-_-_ 12y ago
  Thank you for the information. This raises a couple more questions for me. First, when searching I was able to find Martin Fenner's very interesting blog posts about ideas for a "Scholarly Markdown" and, as those issues and the first link on the scholarlymarkdown.com site reference, he appears to be associated with a separate "scholmd" project, also called "Scholarly Markdown," which is apparently a related project that itself is a fork of the Python markdown science project: scholmd: http://scholmd.org/ http://scholmd.org/ https://github.com/scholmd https://github.com/scholmd Markdown Science: https://github.com/karthik/markdown_science https://github.com/karthik/markdown_science However, it's unclear what all of the relationships are between all of these projects and forks. Secondly, since some (or all?) of the changes are being discussed in the Pandoc issue tracker, are these changes intended to be submitted to Pandoc in pull requests? I don't currently see any.
  - aaren 12y ago
    First: I'm not sure of the exact origins of things. The way I see it, academic markdown is more of an ecosystem of tools with a lot of overlap. There is no one single markdown workflow right now when you want to do do academic writing. I think this is because no one is sure what the final spec should look like and people are trying things out and seeing what sticks. I feel that there has been some convergence in the last 2/3 years though. Second: The PR for image attributes is here: https://github.com/jgm/pandoc/pull/1806 https://github.com/jgm/pandoc/pull/1806 There isn't a PR for internal referencing yet because the implementation hasn't been worked out yet and it isn't a simple change (should there be a native representation, should it be a filter, which syntax should we use, what about the existing citation syntax...).
timtylin 12y ago
Note: I'm the maintainer of this project Basically this is forked because I've been wanting to see how much change to the AST is needed to include most of the academic-specific features. Since Pandoc's AST definition is a separate package from Pandoc itself and could possibly be a dependency of other projects, I thought it would be best to figure most of it out first and end up with just one proposal. Scholdoc has a much more limited number of input/output syntax, so it has much more flexibility when it comes to adding new document element types. Consider this a self-motivated skunkworks project for Pandoc.

by _-_-_-_ 12y ago

kazagistar 12y ago

This was my first thought as well. Pandoc has a rather excellent infrastructure in place to let you extend and build on it, so if you instead choose to fork, there better be a good reason...

aaren 12y ago
Pandoc filters let you do a lot of things provided you stick to the defined types. If you want to add new types of element, like this does, then the only practical way is a fork. (except for really simple cases)
- zzleeper 12y ago
  However, in most cases you could just RawBlock (or was it BlockQuote?) with specific attributes, and then intercept those in a filter and convert to Latex.
  - timtylin 12y ago
    This was basically my original approach. However, after working out the math syntax I realized that some things, like the double-backtick inline math, just can't be accomplished without a pre-filter. At that point I decided to just start playing with the parser code. I also became super-convinced that some level of AST change was necessary to keep things sane, and since I wasn't able to use the existing Math and Image types anyways (they're not attributed), I ultimately just started a new AST type package namespace called "Scholdoc". Everything just evolved from there.
  - aaren 12y ago
    Yes, BlockQuote can be used as a generic block container. It doesn't have attributes itself but you can put them on a header at the start. For generic inline elements you can use Code. Markdown syntax for generic containers (Div and Span) hasn't been implemented yet but the discussion is fairly mature now: https://github.com/jgm/pandoc/issues/168 https://github.com/jgm/pandoc/issues/168 https://github.com/jgm/pandoc/pull/1791 https://github.com/jgm/pandoc/pull/1791

aaren 12y ago

This is a fork because making these changes to pandoc itself needs a lot of consideration. Internal referencing and attributes on figures are two things that are currently being discussed for pandoc. The discussion has been going on for quite a while though - hence people making forks. Discussion on internal referencing: https://github.com/jgm/pandoc/issues/813 https://github.com/jgm/pandoc/issues/813 Discussion on image attributes: https://github.com/jgm/pandoc/issues/261 https://github.com/jgm/pandoc/issues/261

_-_-_-_ 12y ago
Thank you for the information. This raises a couple more questions for me. First, when searching I was able to find Martin Fenner's very interesting blog posts about ideas for a "Scholarly Markdown" and, as those issues and the first link on the scholarlymarkdown.com site reference, he appears to be associated with a separate "scholmd" project, also called "Scholarly Markdown," which is apparently a related project that itself is a fork of the Python markdown science project: scholmd: http://scholmd.org/ http://scholmd.org/ https://github.com/scholmd https://github.com/scholmd Markdown Science: https://github.com/karthik/markdown_science https://github.com/karthik/markdown_science However, it's unclear what all of the relationships are between all of these projects and forks. Secondly, since some (or all?) of the changes are being discussed in the Pandoc issue tracker, are these changes intended to be submitted to Pandoc in pull requests? I don't currently see any.
- aaren 12y ago
  First: I'm not sure of the exact origins of things. The way I see it, academic markdown is more of an ecosystem of tools with a lot of overlap. There is no one single markdown workflow right now when you want to do do academic writing. I think this is because no one is sure what the final spec should look like and people are trying things out and seeing what sticks. I feel that there has been some convergence in the last 2/3 years though. Second: The PR for image attributes is here: https://github.com/jgm/pandoc/pull/1806 https://github.com/jgm/pandoc/pull/1806 There isn't a PR for internal referencing yet because the implementation hasn't been worked out yet and it isn't a simple change (should there be a native representation, should it be a filter, which syntax should we use, what about the existing citation syntax...).

timtylin 12y ago

Note: I'm the maintainer of this project Basically this is forked because I've been wanting to see how much change to the AST is needed to include most of the academic-specific features. Since Pandoc's AST definition is a separate package from Pandoc itself and could possibly be a dependency of other projects, I thought it would be best to figure most of it out first and end up with just one proposal. Scholdoc has a much more limited number of input/output syntax, so it has much more flexibility when it comes to adding new document element types. Consider this a self-motivated skunkworks project for Pandoc.