4 ms·

Any guesstimates as to when it starts to become worth it to use hyperscan vs pcre etc? 30 regex and 1 meg of data for example. I'm somewhat curious as I have

Any guesstimates as to when it starts to become worth it to use hyperscan vs pcre etc? 30 regex and 1 meg of data for example. I'm somewhat curious as I have a couple of scraping things I do where I can compile the regex once and keep it hanging around (or save to/load from disk if that's feasible) for 3 to 5 minutes at a time.

glangdale 7y ago
Well, 30 regex is not a great case for libpcre as libpcre really doesn't have a multiple regex mode. Thanks to some excellent work in libpcre2 with accelerated searching, it's no longer the "7-10x slower than Hyperscan on a single regex" case of the past (still frequently slower), but it wouldn't surprise me if HS was 50-100x faster than libpcre once compile time isn't a factor. However, the compile times of HS for 30 regexes might be not entirely trivial (maybe a second, probably less). A megabyte you'd probably see benefits but I think the benefits might not be drastic. Probably the best way to find out would be to fire up hsbench (a tool that comes with Hyperscan now), figure out the slightly weird corpus format (sorry) and get some numbers for yourself on your own workload.