6 ms·

Regex combinators are a much better solution to that problem, for which I gave various arguments here[1]: - You don't need to remember which regex syntax the l

by Drup 10y ago

Regex combinators are a much better solution to that problem, for which I gave various arguments here[1]: - You don't need to remember which regex syntax the library is using. Is it using the emacs one ? The perl one ? The javascript one ? that new "real language" one ? - It's "self documenting". Your combinators are just functions, so you just expose them and give them type signatures, and the usual documentation/autocompletion/whatevertooling works. - It composes better. You don't have to mash string together to compose your regex, you can name intermediary regexs with normal variables, etc. - Related to the point above: No string quoting hell. - You stay in your home language. No sublanguage involved, just function calls. - Capturing is much cleaner. You don't need to conflate "parenthesis for capture" and "parenthesis for grouping" (since you can use the host's languages parens). [1]: https://news.ycombinator.com/item?id=12293687 https://news.ycombinator.com/item?id=12293687

kazinator 10y ago
> You don't need to remember which regex syntax the library is using. Only for the time being, while only one regex combinator API/implementation exists for your language.
- Drup 10y ago
  Sure, but 1. It's pretty easy to expose a similar API, those are all functions 2. The documentation tools for the language naturally document the combinators. Even if the actual combinators are different, it's still a much better situation than the basilions regex syntaxes with completely different quoting and escaping mechanisms.
  - kazinator 10y ago
    True; you will never have to wonder whether alternatives(x, y, z) has to be \alternatives. :)
- colanderman 10y ago
  Unlike a regex string, using incorrect syntax with a parser combinator generally results in a syntax error. Regex parsers generally treat "unknown" characters as characters to match, hiding bugs.
  - 1ris 10y ago
    And usually not at compile time.
    - deleted 10y ago
      [deleted]
    - junke 10y ago
      Here is an example of an "unusual" case where the compiler can detect errors in statically known regexes. This fails at compilation with Opening paren has no matching closing paren. at position 0 in string "(ab" (defun scan-ab (s) (ppcre:scan "(ab" s)) By the way, here is the alternative syntax: (defun scan-ab (s) (ppcre:scan '(:register "ab") s)) The alternative syntax allows to embed string regexes: (defun wrap-regex (regex) (typecase regex (string `(:regex ,regex)) (t regex))) This is useful for combining regexes: (defun exactly-some (&rest choices) (let ((choices (mapcar #'wrap-regex choices))) `(:sequence :start-anchor ,@(if (rest choices) `((:alternation ,@choices)) choices) :end-anchor))) (exactly-some "t.*") (:SEQUENCE :START-ANCHOR (:REGEX "t.*") :END-ANCHOR) (exactly-some "t.*" "a.a" ':whitespace-char-class) (:SEQUENCE :START-ANCHOR (:ALTERNATION (:REGEX "t.*") (:REGEX "a.a") :WHITESPACE-CHAR-CLASS) :END-ANCHOR)
  - deleted 10y ago
    [deleted]
  - kazinator 10y ago
    A regex language could fix it by some clear prefix convention or whatever for all operators. Though more verbose, it would still be less verbose than alternatives. Suppose all regex operator characters have to be backslashed (and \\ stands for a single \). Then it's clear. No backslash means it's literal; otherwise it's an operator (and a backslash on a nonexistent operator is a parse error). The ambiguities exist because regex aficionados want common operators to be just one character long.
davexunit 10y ago
There's an MIT CS assignment that goes along with exactly what you are saying: "Regexp Disaster" by Gerald Sussman. It's a good read for those that want to learn more about the combinator approach. https://groups.csail.mit.edu/mac/users/gjs/6.945/psets/ps01/ps.txt https://groups.csail.mit.edu/mac/users/gjs/6.945/psets/ps01/...
sillysaurus3 10y ago
Personally, I use regexes all the time. Every day. I'm constantly typing regexes into emacs and vim to do text manipulation. Sometimes a quick :%s/\(foo\|bar\)d/baz\1/g, other times a :%v/function /d. I can't imagine what it would be like if I had to write out "begin with either of (number, letter, ..." I'd never get anything done. For programming it could be nice. It's an interesting idea. But I've found that good highlighting solves most of the problem. When I type "\\(" into emacs, it immediately highlights that \\( in a different color, along with \\| and \\). That way you can differentiate instantly between capture groups vs matching literal parens. Here's an example: http://i.imgur.com/b417O2o.png http://i.imgur.com/b417O2o.png That small snippet would become way, way longer if you use a regex combinator. Maybe good, maybe bad, but it's hard to read a lot of code. Yeah, it's ugly. And there are a bazillion small variations between regex engines. But for raw productivity, it seems hard to beat.
- TAForObvReasons 10y ago
  I find that most of the problems are resolved with interstitial comments a la perl /x or coffeescript heregex. Merely splitting the regular expression into smaller blocks with a short description of what to match makes it easy for people to verify the regular expression.
- foxylad 10y ago
  I, on the other hand, have to build a regex a few times a year. Every darn time I have to look up the difference between .* and .+, and how to do an "or", and why I get matches when I group a set of characters. So SRL sounds like a great idea: us mortals can use it to painfully stitch together our our poor regexes, and when we get good enough we can skip SRL and become fully fledged regex gods. Off topic, if SRL is a regex compiler, I'd love a regex decompiler. To be able to turn an impossible jumble of weird characters into a structured description of what it does would be hugely useful.
  - tokenizerrr 10y ago
    RegexBuddy has a regex explainer, debugger, builder and tester. It works very well and supports exporting code to multiple languages and dialects
pygy_ 10y ago
https://github.com/pygy/compose-regexp.js https://github.com/pygy/compose-regexp.js ^^^ Regexp combinators in JS. 614 bytes minimized and gzipped. Once you're proficient with the standard RegExp syntax, you can write pretty complex ones without external tools, but when the times comes to debug and modify them, using combinators makes the task much much more simple.

by Drup 10y ago

kazinator 10y ago

> You don't need to remember which regex syntax the library is using. Only for the time being, while only one regex combinator API/implementation exists for your language.

Drup 10y ago
Sure, but 1. It's pretty easy to expose a similar API, those are all functions 2. The documentation tools for the language naturally document the combinators. Even if the actual combinators are different, it's still a much better situation than the basilions regex syntaxes with completely different quoting and escaping mechanisms.
- kazinator 10y ago
  True; you will never have to wonder whether alternatives(x, y, z) has to be \alternatives. :)
colanderman 10y ago
Unlike a regex string, using incorrect syntax with a parser combinator generally results in a syntax error. Regex parsers generally treat "unknown" characters as characters to match, hiding bugs.
- 1ris 10y ago
  And usually not at compile time.
  - deleted 10y ago
    [deleted]
  - junke 10y ago
    Here is an example of an "unusual" case where the compiler can detect errors in statically known regexes. This fails at compilation with Opening paren has no matching closing paren. at position 0 in string "(ab" (defun scan-ab (s) (ppcre:scan "(ab" s)) By the way, here is the alternative syntax: (defun scan-ab (s) (ppcre:scan '(:register "ab") s)) The alternative syntax allows to embed string regexes: (defun wrap-regex (regex) (typecase regex (string `(:regex ,regex)) (t regex))) This is useful for combining regexes: (defun exactly-some (&rest choices) (let ((choices (mapcar #'wrap-regex choices))) `(:sequence :start-anchor ,@(if (rest choices) `((:alternation ,@choices)) choices) :end-anchor))) (exactly-some "t.*") (:SEQUENCE :START-ANCHOR (:REGEX "t.*") :END-ANCHOR) (exactly-some "t.*" "a.a" ':whitespace-char-class) (:SEQUENCE :START-ANCHOR (:ALTERNATION (:REGEX "t.*") (:REGEX "a.a") :WHITESPACE-CHAR-CLASS) :END-ANCHOR)
- deleted 10y ago
  [deleted]
- kazinator 10y ago
  A regex language could fix it by some clear prefix convention or whatever for all operators. Though more verbose, it would still be less verbose than alternatives. Suppose all regex operator characters have to be backslashed (and \\ stands for a single \). Then it's clear. No backslash means it's literal; otherwise it's an operator (and a backslash on a nonexistent operator is a parse error). The ambiguities exist because regex aficionados want common operators to be just one character long.

davexunit 10y ago

There's an MIT CS assignment that goes along with exactly what you are saying: "Regexp Disaster" by Gerald Sussman. It's a good read for those that want to learn more about the combinator approach. https://groups.csail.mit.edu/mac/users/gjs/6.945/psets/ps01/ps.txt https://groups.csail.mit.edu/mac/users/gjs/6.945/psets/ps01/...

sillysaurus3 10y ago

Personally, I use regexes all the time. Every day. I'm constantly typing regexes into emacs and vim to do text manipulation. Sometimes a quick :%s/\(foo\|bar\)d/baz\1/g, other times a :%v/function /d. I can't imagine what it would be like if I had to write out "begin with either of (number, letter, ..." I'd never get anything done. For programming it could be nice. It's an interesting idea. But I've found that good highlighting solves most of the problem. When I type "\\(" into emacs, it immediately highlights that \\( in a different color, along with \\| and \\). That way you can differentiate instantly between capture groups vs matching literal parens. Here's an example: http://i.imgur.com/b417O2o.png http://i.imgur.com/b417O2o.png That small snippet would become way, way longer if you use a regex combinator. Maybe good, maybe bad, but it's hard to read a lot of code. Yeah, it's ugly. And there are a bazillion small variations between regex engines. But for raw productivity, it seems hard to beat.

TAForObvReasons 10y ago
I find that most of the problems are resolved with interstitial comments a la perl /x or coffeescript heregex. Merely splitting the regular expression into smaller blocks with a short description of what to match makes it easy for people to verify the regular expression.
foxylad 10y ago
I, on the other hand, have to build a regex a few times a year. Every darn time I have to look up the difference between .* and .+, and how to do an "or", and why I get matches when I group a set of characters. So SRL sounds like a great idea: us mortals can use it to painfully stitch together our our poor regexes, and when we get good enough we can skip SRL and become fully fledged regex gods. Off topic, if SRL is a regex compiler, I'd love a regex decompiler. To be able to turn an impossible jumble of weird characters into a structured description of what it does would be hugely useful.
- tokenizerrr 10y ago
  RegexBuddy has a regex explainer, debugger, builder and tester. It works very well and supports exporting code to multiple languages and dialects

pygy_ 10y ago

https://github.com/pygy/compose-regexp.js https://github.com/pygy/compose-regexp.js ^^^ Regexp combinators in JS. 614 bytes minimized and gzipped. Once you're proficient with the standard RegExp syntax, you can write pretty complex ones without external tools, but when the times comes to debug and modify them, using combinators makes the task much much more simple.