6 ms·

According to my totally unscientific benchmarks, if the performance of Rust's regular expression module were 1x, Go's was 8x and Ruby's was 32x, and Java's was

According to my totally unscientific benchmarks, if the performance of Rust's regular expression module were 1x, Go's was 8x and Ruby's was 32x, and Java's was 42x. Using Google's Java regex module improved the speed quite a bit but still was at ~18x. I was very impressed to see Rust doing so well. And it was sad to see Java so underperforming in such a typical workload. I know we're measuring libraries not languages, but I think regexes are so prevalent that not optimizing for it would hinder the language's real life performance.

pcwalton 5y ago
By performance you mean elapsed time, right? So by 8x you mean 8x slower?
- xpressvideoz 5y ago
  Yeah that's right. I compiled a pattern and matched it against a huge wall of text, measuring elapsed time.
tapirl 5y ago
convenient to show your benchmarks?
- xpressvideoz 5y ago
  Unfortunately, the test code belongs to the company I work for so I cannot take it out. It was done to determine what language we should use for an internal tool. I hope to conduct another benchmark one day, publicly this time.
- dekelpilli 5y ago
  Not OP, but Benchmarks Game has a performance test based on regex: https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/regexredux.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... The top times for the mentioned languages are: Rust: 0.78s Ruby: 12.33s Java: 5.34s Go: 3.80s Python: 1.34s
  - esprehn 5y ago
    That's interesting that Ruby is as slow as it is, the regex engine (Onigmo) is written in C. I wonder where the bottleneck is compared to the other languages.
  - azth 5y ago
    The Java and golang numbers are not apples to apples. The Java solution is using the standard library code, while golang's is calling out to native code (wrapper C library) as far as I can tell. Same with Python.
    - adgjlsfhk1 5y ago
      To be fair to non-Javas, the java version is using Java code because Java sucks at calling out to faster languages.
      - azth 5y ago
        There's JNI that exists now. And Project Panama will be a big improvement.
      - zmmmmm 5y ago
        Oh yes, that Python code looks so idiomatic: https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/regexredux-python3-2.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
        viraptor 5y ago
        regex=PCRE2.pcre2_compile_8(pattern, c_size_t(len(pattern)), c_uint32(0), byref(c_int()), byref(c_size_t()), None) As far as FFI goes, I would say this looks amazingly simple.
        pmontra 5y ago
        Can't comment on that but why are these people putting spaces after commas (correct) and not to the left and right of the assignment sign = ? It's so weird to read a=b
tyingq 5y ago
Interesting. Looking at this repo, they have Rust -> Ruby -> Java -> Golang https://github.com/mariomka/regex-benchmark https://github.com/mariomka/regex-benchmark Though it appears the numbers are two years old or so, and only for 3 specific regexes.
dpratt 5y ago
For the JVM benchmark, unless you used Pattern.compile() and let the VM warm up (or used the JMH benchmark framework), your numbers are likely wildly off.
- burntsushi 5y ago
  It depends on the goal of the measurement. For example, if you wanted to write a grep in Java, would the total runtime of the program be faster or slower if you "waited for the VM to warm up?"

pcwalton 5y ago

By performance you mean elapsed time, right? So by 8x you mean 8x slower?

xpressvideoz 5y ago
Yeah that's right. I compiled a pattern and matched it against a huge wall of text, measuring elapsed time.

tapirl 5y ago

convenient to show your benchmarks?

xpressvideoz 5y ago
Unfortunately, the test code belongs to the company I work for so I cannot take it out. It was done to determine what language we should use for an internal tool. I hope to conduct another benchmark one day, publicly this time.
dekelpilli 5y ago
Not OP, but Benchmarks Game has a performance test based on regex: https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/regexredux.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... The top times for the mentioned languages are: Rust: 0.78s Ruby: 12.33s Java: 5.34s Go: 3.80s Python: 1.34s
- esprehn 5y ago
  That's interesting that Ruby is as slow as it is, the regex engine (Onigmo) is written in C. I wonder where the bottleneck is compared to the other languages.
- azth 5y ago
  The Java and golang numbers are not apples to apples. The Java solution is using the standard library code, while golang's is calling out to native code (wrapper C library) as far as I can tell. Same with Python.
  - adgjlsfhk1 5y ago
    To be fair to non-Javas, the java version is using Java code because Java sucks at calling out to faster languages.
    - azth 5y ago
      There's JNI that exists now. And Project Panama will be a big improvement.
    - zmmmmm 5y ago
      Oh yes, that Python code looks so idiomatic: https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/regexredux-python3-2.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
      - viraptor 5y ago
        regex=PCRE2.pcre2_compile_8(pattern, c_size_t(len(pattern)), c_uint32(0), byref(c_int()), byref(c_size_t()), None) As far as FFI goes, I would say this looks amazingly simple.
      - pmontra 5y ago
        Can't comment on that but why are these people putting spaces after commas (correct) and not to the left and right of the assignment sign = ? It's so weird to read a=b

tyingq 5y ago

Interesting. Looking at this repo, they have Rust -> Ruby -> Java -> Golang https://github.com/mariomka/regex-benchmark https://github.com/mariomka/regex-benchmark Though it appears the numbers are two years old or so, and only for 3 specific regexes.

dpratt 5y ago

For the JVM benchmark, unless you used Pattern.compile() and let the VM warm up (or used the JMH benchmark framework), your numbers are likely wildly off.

burntsushi 5y ago
It depends on the goal of the measurement. For example, if you wanted to write a grep in Java, would the total runtime of the program be faster or slower if you "waited for the VM to warm up?"