5 ms·

A Working Mathematician’s Guide to Parsing

79 points by imgabe 8y ago

RiversHaveWings 8y ago
My go-to library for parsing in Python is https://github.com/pyparsing/pyparsing https://github.com/pyparsing/pyparsing, a https://en.wikipedia.org/wiki/Parsing_expression_grammar https://en.wikipedia.org/wiki/Parsing_expression_grammar -based https://en.wikipedia.org/wiki/Parser_combinator https://en.wikipedia.org/wiki/Parser_combinator library. I generally find it much easier to get my head around parser combinators than parser generators.
- vorg 8y ago
  As the author says: > because the parser generator knows about all patterns you want to match in advance, it will match longer terminals before shorter—more ambiguous—terminals If you don't mind manually ordering your choices in your alt operators, then PEG and parser combinators are OK. Oh, and you often need to restructure your grammar to avoid left recursion.
theoh 8y ago
Am I the only person who looks at Jeremy Kun's output and thinks "Nope, this guy is not actually a gifted communicator dedicated to sharing knowledge, he's just a postdoc on an annoyingly self-aggrandizing narcissism trip"?
- pmyteh 8y ago
  I can't see that him being "just" a postdoc has very much to do with anything, especially when covering stuff like this which is foundational rather than cutting-edge.
  - theoh 8y ago
    The "just" was supposed to apply to the whole self-aggrandizing trip. My point is that there are a lot of people with math PhDs, but Kun is in a small unenviable minority that is driven to engage in self-promotion via all these blog post "primers", which of very dubious pedagogical quality.
    - joppy 8y ago
      “Oh no, someone is attempting to teach, how dare they!” is what this reads like. Writing articles like this is good for the author, in terms of finding new ways to explain things, and clarify knowledge they already had. The writing style of this author in particular is quite engaging for myself and (I would assume) many others.
    - algorithmsRcool 8y ago
      I enjoy his content and I have found some of his explanations novel and enlightening. I don't know why him spending untold hours producing detailed content is a bad thing. Even if it is for self-promotion.
- foldr 8y ago
  He's not a postdoc, though?
  - theoh 8y ago
    Presumably you understand what I'm saying ("he's 'just a vain postdoc on an ego trip'"). Of course I know what his professional status is, it's the most prominent information on his website. It's a shame that rather than actually considering the very evident pathology here (i.e. the production of mediocre vanity blog content that serves primarily to improve the author's profile) everyone is either downvoting or nitpicking.
    - imgabe 8y ago
      You might have better luck if you were to expound upon the ways in which you found the content to be mediocre and how it might be improved rather than personally attacking the author for what you perceive his motives to be.
    - pfortuny 8y ago
      OTOH he might just be writing for the pleasure of it and getting self-feedback from it (the more you explain something the better you understand it).
    - foldr 8y ago
      I mean, if you know what his professional status is, why say that he's a postdoc when he isn't?
- mlevental 8y ago
  people are downvoting you because that's one the saltiest opinions I've ever seen on hn. the guy is a narcissist because he writes a blog and other people post it? how does make that sense in your head? or do you think that the people that post links to his blog are shills? not to mention his pedagogy is excellent - these blog posts (which are not his primers) are meant to be introductory and cursory.
- sigstoat 8y ago
  yes, you are.
kevinventullo 8y ago
For converting LaTex to WordPress, I'm a fan of https://lucatrevisan.wordpress.com/latex-to-wordpress/ https://lucatrevisan.wordpress.com/latex-to-wordpress/
OskarS 8y ago
> Our hero, a mathematician, is writing notes in LaTeX and needs to convert it to a format that her blog platform accepts. She’s used to using dollar sign delimiters for math mode, but her blog requires  and \[ \]. Unless our hero mathematician is writing, like, several blog posts a day, using tools like parser generators and writing your grammar in BNF and the like is WAY overkill. Just use search/replace and manually fix it up. Or, like, maybe some kind of editor macro/multiple cursor nonsense in your fully-featured text editor. If she really wants to go nuts, she should write a four-line python script to do the search/replace a tiny bit smarter. Fixing it up with some manual oversight takes like three minutes per blog post at most, it's not worth trying to automate fully. This is one of them "killing an ant with a nuclear bomb" situations.
- acprog42 8y ago
  But killing an ant with a nuclear bomb is moar funnier! Don't stop the fun!
- delinka 8y ago
  `sed` is your friend
- ngcc_hk 8y ago
  I think he use vim. Last paragraph. This is click bait. Hooked myself.
pgtan 8y ago
Sadly, he don't understand what LaTeX is and what TeX is. And $$ is a TeX macro, should not be used in LaTeX, the are much better math display environments.
- joppy 8y ago
  Regardless, this is a very common way of writing mathematics in LaTeX, and so will need to be addressed in any solution to this problem.
satyenr 8y ago
Just use pandoc[1] — assuming the blog platform accepts a well known markup language. [1] https://pandoc.org https://pandoc.org

79 points by imgabe 8y ago

My go-to library for parsing in Python is https://github.com/pyparsing/pyparsing https://github.com/pyparsing/pyparsing, a https://en.wikipedia.org/wiki/Parsing_expression_grammar https://en.wikipedia.org/wiki/Parsing_expression_grammar -based https://en.wikipedia.org/wiki/Parser_combinator https://en.wikipedia.org/wiki/Parser_combinator library. I generally find it much easier to get my head around parser combinators than parser generators.

vorg 8y ago
As the author says: > because the parser generator knows about all patterns you want to match in advance, it will match longer terminals before shorter—more ambiguous—terminals If you don't mind manually ordering your choices in your alt operators, then PEG and parser combinators are OK. Oh, and you often need to restructure your grammar to avoid left recursion.

theoh 8y ago

Am I the only person who looks at Jeremy Kun's output and thinks "Nope, this guy is not actually a gifted communicator dedicated to sharing knowledge, he's just a postdoc on an annoyingly self-aggrandizing narcissism trip"?

pmyteh 8y ago
I can't see that him being "just" a postdoc has very much to do with anything, especially when covering stuff like this which is foundational rather than cutting-edge.
- theoh 8y ago
  The "just" was supposed to apply to the whole self-aggrandizing trip. My point is that there are a lot of people with math PhDs, but Kun is in a small unenviable minority that is driven to engage in self-promotion via all these blog post "primers", which of very dubious pedagogical quality.
  - joppy 8y ago
    “Oh no, someone is attempting to teach, how dare they!” is what this reads like. Writing articles like this is good for the author, in terms of finding new ways to explain things, and clarify knowledge they already had. The writing style of this author in particular is quite engaging for myself and (I would assume) many others.
  - algorithmsRcool 8y ago
    I enjoy his content and I have found some of his explanations novel and enlightening. I don't know why him spending untold hours producing detailed content is a bad thing. Even if it is for self-promotion.
foldr 8y ago
He's not a postdoc, though?
- theoh 8y ago
  Presumably you understand what I'm saying ("he's 'just a vain postdoc on an ego trip'"). Of course I know what his professional status is, it's the most prominent information on his website. It's a shame that rather than actually considering the very evident pathology here (i.e. the production of mediocre vanity blog content that serves primarily to improve the author's profile) everyone is either downvoting or nitpicking.
  - imgabe 8y ago
    You might have better luck if you were to expound upon the ways in which you found the content to be mediocre and how it might be improved rather than personally attacking the author for what you perceive his motives to be.
  - pfortuny 8y ago
    OTOH he might just be writing for the pleasure of it and getting self-feedback from it (the more you explain something the better you understand it).
  - foldr 8y ago
    I mean, if you know what his professional status is, why say that he's a postdoc when he isn't?
mlevental 8y ago
people are downvoting you because that's one the saltiest opinions I've ever seen on hn. the guy is a narcissist because he writes a blog and other people post it? how does make that sense in your head? or do you think that the people that post links to his blog are shills? not to mention his pedagogy is excellent - these blog posts (which are not his primers) are meant to be introductory and cursory.
sigstoat 8y ago
yes, you are.

kevinventullo 8y ago

For converting LaTex to WordPress, I'm a fan of https://lucatrevisan.wordpress.com/latex-to-wordpress/ https://lucatrevisan.wordpress.com/latex-to-wordpress/

OskarS 8y ago

> Our hero, a mathematician, is writing notes in LaTeX and needs to convert it to a format that her blog platform accepts. She’s used to using dollar sign delimiters for math mode, but her blog requires  and \[ \]. Unless our hero mathematician is writing, like, several blog posts a day, using tools like parser generators and writing your grammar in BNF and the like is WAY overkill. Just use search/replace and manually fix it up. Or, like, maybe some kind of editor macro/multiple cursor nonsense in your fully-featured text editor. If she really wants to go nuts, she should write a four-line python script to do the search/replace a tiny bit smarter. Fixing it up with some manual oversight takes like three minutes per blog post at most, it's not worth trying to automate fully. This is one of them "killing an ant with a nuclear bomb" situations.

acprog42 8y ago
But killing an ant with a nuclear bomb is moar funnier! Don't stop the fun!
delinka 8y ago
`sed` is your friend
ngcc_hk 8y ago
I think he use vim. Last paragraph. This is click bait. Hooked myself.

pgtan 8y ago

Sadly, he don't understand what LaTeX is and what TeX is. And $$ is a TeX macro, should not be used in LaTeX, the are much better math display environments.

joppy 8y ago
Regardless, this is a very common way of writing mathematics in LaTeX, and so will need to be addressed in any solution to this problem.

satyenr 8y ago

Just use pandoc[1] — assuming the blog platform accepts a well known markup language. [1] https://pandoc.org https://pandoc.org