6 ms·

I'm assuming that uses back-references to handle the matching parens, but I don't like reading regexes (even my own). It's funny that paren matching used to be

by xscott 4y ago

I'm assuming that uses back-references to handle the matching parens, but I don't like reading regexes (even my own). It's funny that paren matching used to be an example of why you needed to use a push down automata instead of a regular language. I like your idea for making a prettier lisp quickly, but depending on the language/library, isn't this going to give the a user a bunch of nested `match` objects instead of nicely nested and printable lists? No free lunch - I guess you trade one ugliness for another.

NoThisIsMe 4y ago
GP calls it a tokenizer (aka lexer) so presumably it doesn't match parens, it just returns a flat array of tokens.
- xscott 4y ago
  That's disappointing. I don't have any stats, but I imagine 90% of beginners writing an interpreter (much less a compiler) start a lexer and parser and quit before getting to the good stuff. Skipping the tedium until you've got "hello world" and "factorial working" makes sense to me. You can always go fix the syntax later.
  - morelisp 4y ago
    If you don't include the parser, "factorial working" is going to be the same evaluating just about any scripting language. You don't see half the value in Lisp's syntax until you're writing the parser.
kazinator 4y ago
> It's funny that paren matching used to be an example of why you needed to use a push down automata instead of a regular language. Expressions with parentheses are not a regular language; expressions which can match them are not regular expressions, even if they extend regular expression syntax. "Regular expression" became the name of a software feature, which retained its name as it was extended beyond regular sets. Just like "web browser" became the name of a kind of program, and that name still sticks even though it's now a monstrous application platform, not just for browsing.
- xscott 4y ago
  I honestly can't tell if you intentionally take stuff out of context so you can argue against it, or if your reading comprehension and logic skills are really just that bad. I said, "instead of a regular language". You even quoted it. You don't even read the bits that you cut and paste to argue against. > expressions which can match them are not regular expressions And that's the damned point. Modern "regular expressions" aren't just "regular" any more. They've had back references and other extensions for a while now: https://www.regular-expressions.info/balancing.html https://www.regular-expressions.info/balancing.html https://www.regular-expressions.info/recurse.html https://www.regular-expressions.info/recurse.html https://www.regular-expressions.info/backref.html https://www.regular-expressions.info/backref.html Please - don't reply to me until you take the time to read and understand what I've said.
  - kazinator 4y ago
    If "paren matching used to be an example of why you needed to use a push down automata instead of a regular language", isn't that intended to say that this is not the case today? If we change "language" to "expression", then I agree and don't have anything to add. The scope of "regular expression" has increased, in informal usage, but (I strongly suspect) "regular language" still means the same thing as before.

by xscott 4y ago

NoThisIsMe 4y ago

GP calls it a tokenizer (aka lexer) so presumably it doesn't match parens, it just returns a flat array of tokens.

xscott 4y ago
That's disappointing. I don't have any stats, but I imagine 90% of beginners writing an interpreter (much less a compiler) start a lexer and parser and quit before getting to the good stuff. Skipping the tedium until you've got "hello world" and "factorial working" makes sense to me. You can always go fix the syntax later.
- morelisp 4y ago
  If you don't include the parser, "factorial working" is going to be the same evaluating just about any scripting language. You don't see half the value in Lisp's syntax until you're writing the parser.

kazinator 4y ago

> It's funny that paren matching used to be an example of why you needed to use a push down automata instead of a regular language. Expressions with parentheses are not a regular language; expressions which can match them are not regular expressions, even if they extend regular expression syntax. "Regular expression" became the name of a software feature, which retained its name as it was extended beyond regular sets. Just like "web browser" became the name of a kind of program, and that name still sticks even though it's now a monstrous application platform, not just for browsing.

xscott 4y ago
I honestly can't tell if you intentionally take stuff out of context so you can argue against it, or if your reading comprehension and logic skills are really just that bad. I said, "instead of a regular language". You even quoted it. You don't even read the bits that you cut and paste to argue against. > expressions which can match them are not regular expressions And that's the damned point. Modern "regular expressions" aren't just "regular" any more. They've had back references and other extensions for a while now: https://www.regular-expressions.info/balancing.html https://www.regular-expressions.info/balancing.html https://www.regular-expressions.info/recurse.html https://www.regular-expressions.info/recurse.html https://www.regular-expressions.info/backref.html https://www.regular-expressions.info/backref.html Please - don't reply to me until you take the time to read and understand what I've said.
- kazinator 4y ago
  If "paren matching used to be an example of why you needed to use a push down automata instead of a regular language", isn't that intended to say that this is not the case today? If we change "language" to "expression", then I agree and don't have anything to add. The scope of "regular expression" has increased, in informal usage, but (I strongly suspect) "regular language" still means the same thing as before.