5 ms·

> How easy it is to parse doesn't matter. How easy it is to parse does matter, because there’s a definite correlation between how easy it is to parse for the c

> How easy it is to parse doesn't matter. How easy it is to parse does matter, because there’s a definite correlation between how easy it is to parse for the computer and for you. When there are bad corner cases, you either have to learn the rules, or keep on producing erroneous and often-content-destructive formatting. > How easy it is to extend is largely irrelevant. If you’re content with stock CommonMark, it is irrelevant to you. If you want to go beyond that, you’re in for a world of pain and mangled content, content that you often won’t notice is mangled until much later, because there’s generally no meaningful way of sanity-checking stuff. As soon as you interact with more than one Markdown engine—which is extremely likely to happen, your text editor is probably not using the parser your build tool uses, configured as it is configured—it matters a lot. If you have ever tried migrating from one engine to another on anything beyond the basics, you will have encountered problems because of this.

thiht 1y ago
> there’s a definite correlation between how easy it is to parse for the computer and for you I’m not sure that’s true tbh. Exhibit A: natural language. Exhibit B: Polish notation.
- chrismorgan 1y ago
  I don’t see how either of those exhibits demonstrate your point. I believe various research has shown that humans and machines parse natural language in rather similar ways. Garden-path sentences <https://en.wikipedia.org/wiki/Garden-path_sentence https://en.wikipedia.org/wiki/Garden-path_sentence> are a fun demonstration of how human sentence parsing involves speculation and backtracking. Polish notation is easy for both to parse; humans only struggle because they’re not so familiar with it. (By adulthood, human processing biases extremely heavily toward the familiar. Computer parsing has to be implemented from scratch, so there’s not so much concept of familiarity, though libraries can encapsulate elements of parsing.)
  - saghm 1y ago
    > Polish notation is easy for both to parse; humans only struggle because they’re not so familiar with it I think you're downplaying the significance of this. The lack of familiarity is exactly what I'd argue makes a huge difference in practice even if theoretically the way our brains parse things isn't that different. We spend so much time reading and writing words that it requires effort to learn how to parse each specific symbol-oriented thing we might want to learn how to read. To add to the parent comment's examples, I'll throw in Brainfuck, which is an extremely simple language for a machine to learn to parse that's literally named for how impenetrable it looks to people at first glance. "Simple if I spend the time to learn it" is not the same as "simple without having to spend time to learn it", and for some things, the fact that the syntax essentially ignores some of the finer details is the main feature rather than a drawback. When everyone I work with can read and write markdown good enough for us not to have major issues, and junior engineers can get up to basically the same level of competence in it without needing a lot of hand holding, it's just not worth the effort for me to try to convince everyone to use RST even if it is better in theory. The total amount of time I've spent dealing with the minor annoyances in markdown in my life is less than the amount of time it would probably take me to convince even one of my coworkers that we should switch all of our READMEs to RST.
  - thiht 1y ago
    > I don’t see how either of those exhibits demonstrate your point. Natural language is easy to do for a human and a hard computing problem. Polish notation is extremely simple to implement, but relatively "hard" for a human, even knowing the rules and how to read it. See: `+ * - 15 6 / 20 4 ^ 2 3 - + 7 8 * 3 2`
    - fluidcruft 1y ago
      Polish notation looks like a nightmare for expressing something like a partial differential equation. Even combining fractions looks like it's going to be a nightmare.
    - chrismorgan 1y ago
      > Natural language is easy to do for a human and a hard computing problem. You ever see someone learning a new language? They struggle hard on more complex sentences. It’s easy for us because we’ve practised it so much. > + * - 15 6 / 20 4 ^ 2 3 - + 7 8 * 3 2 To begin with, you’re missing an operator. I’ll assume another leading +. + + * - 15 6 / 20 4 ^ 2 3 - + 7 8 * 3 2 Now, if you use infix, you have to have at least some of the parentheses, in this case actually only one pair, given rules of operator precedence, associativity and commutativity: (15 - 6) * 20 / 4 + 2 ^ 3 + 7 + 8 - 3 * 2 But you may well just parenthesise everything, it makes solving easier: ((((15 - 6) * (20 / 4)) + (2 ^ 3)) + ((7 + 8) - (3 * 2))) And you know how you go about solving it? Calculating chunks from the inside out, and replacing them with their values: ((( 9 * 5 ) + 8 ) + ( 15 - 6 )) (( 45 + 8 ) + 9 ) ( 53 + 9 ) 62 Coming back to Polish notation—you know what? It’s exactly the same: (+ (+ (* (- 15 6) (/ 20 4)) (^ 2 3)) (- (+ 7 8) (* 3 2))) (+ (+ (* 9 5) 8) (- 15 6)) (+ (+ 45 8) 9) (+ 53 9) 62 For arithmetic at least, it’s not hard. You’re just not accustomed to it.
Arainach 1y ago
It's miserable to parse C++ and that's fine, because only a few people have to write a parser while 5 orders of magnitude more have to read and write it. Same thing with markdown - the user experience is what matters. Edge cases largely don't matter, because again I'm not trying to make a book. I don't care if my table is off by a few pixels. 50% of the time I'm reading markdown it's not even formatted, it's just in raw format in an editor.
- chrismorgan 1y ago
  If you write C++ in a way that it will misparse, you will normally get a hard error that you have to fix. (Also, the complexity is mostly fairly well encapsulated.) If you write Markdown in a way that your engine will misparse, you may well not notice—the compiler probably doesn’t even have the notion of throwing an error. The fact that you’re normally working with the unformatted text makes it even more likely that you won’t notice. (And the complexity is badly encapsulated, too.) I have seen errors due to Markdown complexity so often. I just dealt with a blog where some images were completely missing because they’d written <img> tags in order to add some class attributes, and the engine didn’t allow raw HTML, and they didn’t notice they were gone (probably because ![]() images were still there). Markdown is so technically unsound it’s sometimes quite distressing. We’re not talking about a table being off by a few pixels. We’re talking about words being mangled, text disappearing, much more severe stuff. Markdown isn’t even capable of table stuff.
  - yawaramin 1y ago
    You are missing the point. We are talking about not even parsing the Markdown. We are talking about reading it raw. Literally raw-dogging it. At that point it doesn't even matter, we just want a format that's brain-dead simple. HTML transformation is a bonus on top of that. If we want that we will mandate a specific Markdown engine with a strict parser.
    - chrismorgan 1y ago
      Actually I think you’re missing the point. “Parsing” is not something that computers alone do; humans do it. You see text and understand it to be text, you see <img> and understand it to be an HTML tag (and hopefully know whether your engine will pass it through, or leave it as text, or strip it), you see **double asterisks** and understand it to be bold or strong emphasis. If you only care about reading it raw, you don’t bother with Markdown. Some of what you write will be the same as Markdown, but not all—for example, no one would use its ridiculous link or image syntax. The reason you write with Markdown syntax is because you want to be able to format it (even if you will normally consume it in plain text yourself). And once you’re using Markdown syntax, you need to know the rules, to a greater or lesser extent. You must be able to parse Markdown syntax mentally. If you don’t know the rules well enough, your mental parse will be incorrect, and you’ll produce errors. Errors that won’t be detected by your computer. That’s the hazard with Markdown compared to C++: your mental parser is buggy and incomplete, but with C++ the computer will generally catch your errors while with Markdown it will never catch your errors.

thiht 1y ago

> there’s a definite correlation between how easy it is to parse for the computer and for you I’m not sure that’s true tbh. Exhibit A: natural language. Exhibit B: Polish notation.

chrismorgan 1y ago
I don’t see how either of those exhibits demonstrate your point. I believe various research has shown that humans and machines parse natural language in rather similar ways. Garden-path sentences <https://en.wikipedia.org/wiki/Garden-path_sentence https://en.wikipedia.org/wiki/Garden-path_sentence> are a fun demonstration of how human sentence parsing involves speculation and backtracking. Polish notation is easy for both to parse; humans only struggle because they’re not so familiar with it. (By adulthood, human processing biases extremely heavily toward the familiar. Computer parsing has to be implemented from scratch, so there’s not so much concept of familiarity, though libraries can encapsulate elements of parsing.)
- saghm 1y ago
  > Polish notation is easy for both to parse; humans only struggle because they’re not so familiar with it I think you're downplaying the significance of this. The lack of familiarity is exactly what I'd argue makes a huge difference in practice even if theoretically the way our brains parse things isn't that different. We spend so much time reading and writing words that it requires effort to learn how to parse each specific symbol-oriented thing we might want to learn how to read. To add to the parent comment's examples, I'll throw in Brainfuck, which is an extremely simple language for a machine to learn to parse that's literally named for how impenetrable it looks to people at first glance. "Simple if I spend the time to learn it" is not the same as "simple without having to spend time to learn it", and for some things, the fact that the syntax essentially ignores some of the finer details is the main feature rather than a drawback. When everyone I work with can read and write markdown good enough for us not to have major issues, and junior engineers can get up to basically the same level of competence in it without needing a lot of hand holding, it's just not worth the effort for me to try to convince everyone to use RST even if it is better in theory. The total amount of time I've spent dealing with the minor annoyances in markdown in my life is less than the amount of time it would probably take me to convince even one of my coworkers that we should switch all of our READMEs to RST.
- thiht 1y ago
  > I don’t see how either of those exhibits demonstrate your point. Natural language is easy to do for a human and a hard computing problem. Polish notation is extremely simple to implement, but relatively "hard" for a human, even knowing the rules and how to read it. See: `+ * - 15 6 / 20 4 ^ 2 3 - + 7 8 * 3 2`
  - fluidcruft 1y ago
    Polish notation looks like a nightmare for expressing something like a partial differential equation. Even combining fractions looks like it's going to be a nightmare.
  - chrismorgan 1y ago
    > Natural language is easy to do for a human and a hard computing problem. You ever see someone learning a new language? They struggle hard on more complex sentences. It’s easy for us because we’ve practised it so much. > + * - 15 6 / 20 4 ^ 2 3 - + 7 8 * 3 2 To begin with, you’re missing an operator. I’ll assume another leading +. + + * - 15 6 / 20 4 ^ 2 3 - + 7 8 * 3 2 Now, if you use infix, you have to have at least some of the parentheses, in this case actually only one pair, given rules of operator precedence, associativity and commutativity: (15 - 6) * 20 / 4 + 2 ^ 3 + 7 + 8 - 3 * 2 But you may well just parenthesise everything, it makes solving easier: ((((15 - 6) * (20 / 4)) + (2 ^ 3)) + ((7 + 8) - (3 * 2))) And you know how you go about solving it? Calculating chunks from the inside out, and replacing them with their values: ((( 9 * 5 ) + 8 ) + ( 15 - 6 )) (( 45 + 8 ) + 9 ) ( 53 + 9 ) 62 Coming back to Polish notation—you know what? It’s exactly the same: (+ (+ (* (- 15 6) (/ 20 4)) (^ 2 3)) (- (+ 7 8) (* 3 2))) (+ (+ (* 9 5) 8) (- 15 6)) (+ (+ 45 8) 9) (+ 53 9) 62 For arithmetic at least, it’s not hard. You’re just not accustomed to it.

Arainach 1y ago

It's miserable to parse C++ and that's fine, because only a few people have to write a parser while 5 orders of magnitude more have to read and write it. Same thing with markdown - the user experience is what matters. Edge cases largely don't matter, because again I'm not trying to make a book. I don't care if my table is off by a few pixels. 50% of the time I'm reading markdown it's not even formatted, it's just in raw format in an editor.

chrismorgan 1y ago
If you write C++ in a way that it will misparse, you will normally get a hard error that you have to fix. (Also, the complexity is mostly fairly well encapsulated.) If you write Markdown in a way that your engine will misparse, you may well not notice—the compiler probably doesn’t even have the notion of throwing an error. The fact that you’re normally working with the unformatted text makes it even more likely that you won’t notice. (And the complexity is badly encapsulated, too.) I have seen errors due to Markdown complexity so often. I just dealt with a blog where some images were completely missing because they’d written <img> tags in order to add some class attributes, and the engine didn’t allow raw HTML, and they didn’t notice they were gone (probably because ![]() images were still there). Markdown is so technically unsound it’s sometimes quite distressing. We’re not talking about a table being off by a few pixels. We’re talking about words being mangled, text disappearing, much more severe stuff. Markdown isn’t even capable of table stuff.
- yawaramin 1y ago
  You are missing the point. We are talking about not even parsing the Markdown. We are talking about reading it raw. Literally raw-dogging it. At that point it doesn't even matter, we just want a format that's brain-dead simple. HTML transformation is a bonus on top of that. If we want that we will mandate a specific Markdown engine with a strict parser.
  - chrismorgan 1y ago
    Actually I think you’re missing the point. “Parsing” is not something that computers alone do; humans do it. You see text and understand it to be text, you see <img> and understand it to be an HTML tag (and hopefully know whether your engine will pass it through, or leave it as text, or strip it), you see **double asterisks** and understand it to be bold or strong emphasis. If you only care about reading it raw, you don’t bother with Markdown. Some of what you write will be the same as Markdown, but not all—for example, no one would use its ridiculous link or image syntax. The reason you write with Markdown syntax is because you want to be able to format it (even if you will normally consume it in plain text yourself). And once you’re using Markdown syntax, you need to know the rules, to a greater or lesser extent. You must be able to parse Markdown syntax mentally. If you don’t know the rules well enough, your mental parse will be incorrect, and you’ll produce errors. Errors that won’t be detected by your computer. That’s the hazard with Markdown compared to C++: your mental parser is buggy and incomplete, but with C++ the computer will generally catch your errors while with Markdown it will never catch your errors.