4 ms·
I disagree with this intensely. For starters, syntax drives how I interact with a language as much as - maybe more than - semantics. How expressions are laid o
by vidarh 10y ago
I disagree with this intensely.
For starters, syntax drives how I interact with a language as much as - maybe more than - semantics. How expressions are laid out is intensely important to me, as it affects how I remember and visualise the code. I can visualise the layout of code I have not worked with in years when the syntax is clear, and the code is well formatted.
I can work around painful semantics and find ways to pretend they don't exist by avoiding features or picking patterns that work better; but painful syntax usually stares me in the face ever moment I work with a language.
I have more than once rejected or picked languages based on syntax. E.g. I can't look at a Python program without getting annoyed with the syntax, and I avoid using the language whenever possible over it, and I work with Ruby whenever I can for the same reason (though the language geek in me wants to cry whenever I think about the Ruby grammar)
I also reject the idea of avoiding hand written parsers to start with. I sympathise a bit with the idea. I can see quickly testing changes with a parser generator. And certainly, if you hand write a parser, you need to avoid the temptation of adding all kinds of awful exceptions.
E.g. I love Ruby as a user of the language, but the MRI parser is beyond awful, and I think the syntax could have had most of the nice aspects and avoided most of the awful syntactical warts with a bit more discipline ("favorite" wart at the moment: '% x ' parses to the literal string "x" - "%" when not preceeded by an operand that makes it the infix operator "%" starts a quote-sequence where the following character indicates what the quote character should be - with the exception of a few special character, most characters will set the quote character to its identity. So in '% x ', the quote character is space).
Though MRI uses a Bison parser, but contains thousands of lines of handwritten exceptions, demonstrating both the bad parts of hand writing irregular exceptions into parsers, as well as how easily you can mess things up even if using a parser generator if you have one that isn't strict enough.
But to me, if your hand written parser becomes big and/or problematic to maintain, you're designing a language that will be problematic to parse cleanly, and it's probably worth revising your grammar (I wish this rule had been adhered to for Ruby).
Nice, regular, clean grammars tend to lend themselves very well to small, compact hand-written parsers. In practice I've never run into a situation where a grammar change required major rewrites of a parser in any project I've worked on for this reason, unless the rule deviated majorly from what I'd consider good practice in language design in ways that would cause problems for most parser generators too.
Modularising a hand written parser along the lines of the grammar rules is easy, and few changes cut so deeply across grammar rules to make this difficult.
But what a hand written parser tends to get you over a parser generator, is better ability to do clean error reporting, and better ease of introspecting how parser changes actually changes the processing in ways that are meaningful to mortals. To me at least, this is a lot more difficult to do with ever parser generator I've tried (and I keep hoping to be proven wrong; I've tried writing my own too, to try to prove myself wrong, and so far I've failed to come up with something I consider a usable replacement to handwritten parsers - you certainly can come up with something expressive enough, but it tends to end up being verbose enough to lose most of the benefit over clean code in the target language that saves you from having to deal with idiosyncracies of the generator).
To me the "solutions" offered demonstrate exactly why syntax matters to me:
I deeply admire Forth and Lisp and descendants on a technical level, but the syntax has always been a massive barrier to me for both language classes. I chose a s-expression inspired syntax to kick off my own compiler project by basically treating it as a serialization format for the parse tree, and first adding a parser on top later, but I did that first to be able to toy with semantics of something I didn't intend to make into its own language, and then to act as the "guts" of my in-progress Ruby compiler, not because I'd be willing to work with it more than that.
If anything, I've found it incredibly painful to work with, and I'd never have "held out" for very long without bolting a more human-friendly parser on top very early on. The experience has made me more insistent - not less - on if not starting with the syntax, then at the very least co-evolving semantics and syntax from the outset.
- wycats 10y ago> "favorite" wart at the moment: '% x ' parses to the literal string "x" - "%" when not preceeded by an operand that makes it the infix operator "%" starts a quote-sequence where the following character indicates what the quote character should be - with the exception of a few special character, most characters will set the quote character to its identity. So in '% x ', the quote character is space. How is this different, in principle, from any other unary/binary operator like plus? In most languages, when `+` is preceded by an expression, it's a binary operator, otherwise it's a unary operator. The same seems true here: when `%` is preceded by an expression it's binary `%`, otherwise it's a unary operator with the semantics you describe.
- vidarh 10y agoIt's not horribly hard to parse. It is however ugly and surprising to almost everyone that sees it. The difference between this and "+" is that if I present you with "+ x ", you know that this represents two tokens: "+" and "x". If you don't know what preceded it, you don't know if "+" is a prefix or infix operator, or what parse to return, but assuming the string starts at a token boundary, you can unambiguously tokenize it lexically without additional knowledge. But for "% x ", you don't know in isolation if it represents the single token representing the literal string "x" or if it represents the infix operator "%" and the identifier "x". It's an example of one of the features that prevents you from doing bottom up lexical analysis of Ruby without doing a full parse and pushing information down from the parser. As I said, it's not hard - in the case of an operator precedence parser, if your value stack is not empty when you see "%", then you need to parse what follows as an expression. In the case it is empty, you need to parse it as a quoted string. There are a variety of ways to do that. But it's relatively uncommon for languages to be impossible to unambiguously tokenize without doing higher level processing. I've also yet to see a single example of Ruby code where this freedom to pick pretty much any quote character has been used in a sensible way - or at all (I certainly have seen cases where "expected" quote characters have been used). Thankfully. So it's an unnecessary wart.
- kazinator 10y ago> But what a hand written parser tends to get you over a parser generator, is better ability to do clean error reporting, and better ease of introspecting how parser changes actually changes the processing in ways that are meaningful to mortals. Here is the thing: in a hand-written parser, we can more or less have a function corresponding to a grammar feature. And we can can trace the entry and exit of that function and put a breakpoint on it. And that's "`nuff said", pretty much.