3 ms·

Python's re has nothing on CL-PPCRE [1] though. The ability to build up a "regular expression" from S-expressions is just too useful. [1] http://weitz.de/cl-pp

• nanny 11y ago
  It was also twice as fast as Perl in benchmarks at one point or another.
  • kbenson 11y ago
    That's not exactly hard to do, depending on features. There's a definite trade-off between features and the type of regex engine that can be implemented.
    • DasIch 11y ago
      It seems to me that one should be able to detect which features are actually used in a regular expression and choose one of multiple different underlying implementations based on that.
      • kbenson 11y ago
        Newer versions of Perl actually support a pluggable regex engine system, so you can use specific regex engines for specific tasks.
      • Twirrim 11y ago
        If you hook in libpcre direct it actually comes with its own JIT these days. http://sljit.sourceforge.net/pcre.html http://sljit.sourceforge.net/pcre.html
    • draven 11y ago
      Having the compiler available at runtime kinda helps too!
• junke 11y ago
  For example, here below is a CL version of "Enter The Scanner". Instead of a closure, we could instanciate a class, but this is sufficient: (defun create-tokenizer (rules) (loop for (token rule) in rules for regex = (if (stringp rule) `(:regex ,rule) rule) collect `(:register ,regex) into alternatives collect token into tokens finally (let ((scanner (ppcre:create-scanner `(:alternation ,@alternatives))) (tokens (coerce tokens 'vector))) (return (lambda (string &key (start 0)) ;; generator-like (lambda () (multiple-value-bind (match-start match-end registers) (ppcre:scan scanner string :start start) (cond (match-start (setf start match-end) (values (aref tokens (position-if-not #'null registers)) match-start match-end)) (t (values)))))))))) The above compiles the regex and returns a closure which accepts a string. That closure returns another closure which generates tokens on-demand for the given string, along with the start and end positions of the token inside the string. Here is a test: (loop with tokenizer = (funcall (create-tokenizer '((ws "\\s+") (plus #\+) (minus #\-) (mult #\*) (div #\/) (num :digit-class) (par-open #\() (par-close #\)))) "(1 + 2) * 3") for token = (multiple-value-list (funcall tokenizer)) while token collect token) => ((par-open 0 1) (num 1 2) (ws 2 3) (plus 3 4) (ws 4 5) (num 5 6) (par-close 6 7) (ws 7 8) (mult 8 9) (ws 9 10) (num 10 11))