4 ms·

Based on the documentation[1], and not surprisingly, this library has been optimized for Intel processors. Particularly, you can enable AVX2 and AVX512 instruct

Based on the documentation[1], and not surprisingly, this library has been optimized for Intel processors. Particularly, you can enable AVX2 and AVX512 instruction sets in the regex. In addition to that, you can optimize the matching database to a specific Intel processor family (like Haswell or Sandy Bridge). BSD-license, and they accept contributions. I wonder if they accept any flags that would allow optimization on AMD processors. [1]: http://intel.github.io/hyperscan/dev-reference/api_constants.html#cpu-feature-support-flags http://intel.github.io/hyperscan/dev-reference/api_constants...

jchw 7y ago
I wonder if they support allowing AVX2 optimizations on non Intel processors; Intel MKL refuses to. I think for that reason alone I’d never use an upstream Intel library directly...
- glangdale 7y ago
  I built Hyperscan with my team in Australia (we've since moved on, or been moved on, from Intel). There are no shenanigans to prevent Hyperscan from using AVX2 on non-IA processors, although we didn't have any high performance examples of same when we were last at Intel (2017). The mechanism is trivial to control IIRC and would not be difficult to patch. It is, after all and unlike MKL, open source.
  - jchw 7y ago
    Thanks for responding. Please excuse my weariness, then. (I think it’s fair to say it’s pretty well earned at this point, but its certainly not you or your team’s fault.)
  - dmbaggett 7y ago
    Kudos for making one of the most impressive pieces of software ever. The level of optimization you achieved is truly rare these days.
    - gnufx 7y ago
      I don't know anything about Hyperscan, but it surprises me that it has a truly rare level of optimization. How does the optimization compare with typical tuned linear algebra, for instance?
      - gnufx 7y ago
        Rather than just downvoting, could someone answer the genuine question about how the optimization that's been done compares with the work on linear algebra which underpins so much? As a specific example, there's libxsmm for small matrices. There's no aspersions cast on Hyperscan at all, just a query about what makes it "truly rare" for the benefit of people who don't have time to study it. It would also be interesting to know how it compares with hardware regex implementations, of which I haven't heard anything recently in connexion with bioinformatics where the interest was.
        creato 7y ago
        The vast, vast majority of code is not optimized to the level of some linear algebra code. Pointing at MKL or comparable linear algebra packages as an example of why that level of optimization is not "rare" is silly.
        jonstewart 7y ago
        I might be the only other person to solve multipattern regex searching. It’s easy conceptually, and fiendishly tricksy when you try to get the tests to pass. You want to take advantage of commonality between patterns but the matches must still be independent. The engine I wrote uses a classical interpreter backend, optimized well enough to beat TRE and the old PCRE handily. I’d read about the bit parallel algorithms and play with them a bit, but the pattern sets we’d throw at any given BP algorithm would break it, as any given BP algorithm comes with several caveats. The genius of the HyperScan team is that they threw damn near every algorithm at their pattern sets, including some they invented, and divvied up the patterns into subsets that fit the particular algorithms well. Usually getting the tests to pass with one backend is the last act of any regex author—once it ain’t broke, you’re too exhausted to fix it. Contemplating the complexity that comes with a multi-algorithm backend makes me want to cry. But HyperScan did it. So, to put it in perspective with linear algebra, imagine a library that first spent time analyzing given matrices, and then split them into submatrices and operated on them with different algorithms according to the particular patterns detected therein. That’s kind of insane to contemplate in a linear algebra—it’s really not a domain that compares well at all—but it’s how HyperScan works... and that ignores all the crazy ISA tricks it used.
- gnufx 7y ago
  I keep seeing complaints about MKL not supporting other architectures. Why do people care? For instance, for linear algebra, use BLIS (like AMD do), or OpenBLAS. Intel actually support BLIS (at least to some extent), and I'm not aware of libxsmm being Intel-only, though I haven't checked. MKL-DNN (whatever it's called now) also appeared to be fairly generic when I looked.
  - jchw 7y ago
    My complaint with MKL is that Intel is intentionally harming performance by only allowing AVX2 code to run on Intel’s own processors. It does not matter if there are alternatives because having the ability to avoid something doesn’t excuse bad behavior. That said, if you’re relying on a dependency that uses MKL i guess you are probably SOL without hacks.
    - gnufx 7y ago
      I don't know how MKL works, but it's been reported that you can make it run on AMD processors for no good reason. I do know that you're harming performance if you use an implementation for a different micro-architecture rather than one that's correctly tuned. Assuming we're talking GEMM, you don't get the performance just with generic SIMD code. You need to worry about details of caches and instruction latencies, in particular. Consider that you should even avoid AVX512 on some processors. I'm no apologist for Intel, but I suppose people running MKL on AMD would complain that it was slow because it was wrongly tuned. Note that BLIS gets quite close to MKL on Intel anyhow, and the timing variance of a typical HPC application is likely to be greater than the contribution of ~10% from GEMM. I don't know what people do that's MKL-specific, but its major functionality is available in free implementations that will be infinitely faster on POWER and ARM apart from AMD.
      - jchw 7y ago
        Now I can’t claim to be an expert by any means but I feel like you are over complicating this. For one, we know in practice the performance is much better when you force enable AVX2 on AMD with MKL; it was benchmarked. For another, that’s already obvious, because even without micro optimizing we’re still talking about codepaths that are able to better fill the processor and support 256 bit integers. Certainly not everything will always be straightforward but MKL seems like one of the single most obvious libraries for this. Further, Intel has a rich history here. They have been sued and lost over Intel C++ Compiler’s “Cripple AMD” bit. Funny thing is, it's still there. Although now Intel has warnings throughout their pages. Guess removing a strcmp cpuid, “GenuineIntel” was too much work. Let’s be realistic. Intel wants to win the benchmarks. There’s no need to construct an elaborate reality where this is all done out of concern that they might make performance worse on microarchitectures they did not design.
cryptonector 7y ago
If they don't accept AMD optimizations, just fork it.

jchw 7y ago

I wonder if they support allowing AVX2 optimizations on non Intel processors; Intel MKL refuses to. I think for that reason alone I’d never use an upstream Intel library directly...

glangdale 7y ago
I built Hyperscan with my team in Australia (we've since moved on, or been moved on, from Intel). There are no shenanigans to prevent Hyperscan from using AVX2 on non-IA processors, although we didn't have any high performance examples of same when we were last at Intel (2017). The mechanism is trivial to control IIRC and would not be difficult to patch. It is, after all and unlike MKL, open source.
- jchw 7y ago
  Thanks for responding. Please excuse my weariness, then. (I think it’s fair to say it’s pretty well earned at this point, but its certainly not you or your team’s fault.)
- dmbaggett 7y ago
  Kudos for making one of the most impressive pieces of software ever. The level of optimization you achieved is truly rare these days.
  - gnufx 7y ago
    I don't know anything about Hyperscan, but it surprises me that it has a truly rare level of optimization. How does the optimization compare with typical tuned linear algebra, for instance?
    - gnufx 7y ago
      Rather than just downvoting, could someone answer the genuine question about how the optimization that's been done compares with the work on linear algebra which underpins so much? As a specific example, there's libxsmm for small matrices. There's no aspersions cast on Hyperscan at all, just a query about what makes it "truly rare" for the benefit of people who don't have time to study it. It would also be interesting to know how it compares with hardware regex implementations, of which I haven't heard anything recently in connexion with bioinformatics where the interest was.
      - creato 7y ago
        The vast, vast majority of code is not optimized to the level of some linear algebra code. Pointing at MKL or comparable linear algebra packages as an example of why that level of optimization is not "rare" is silly.
      - jonstewart 7y ago
        I might be the only other person to solve multipattern regex searching. It’s easy conceptually, and fiendishly tricksy when you try to get the tests to pass. You want to take advantage of commonality between patterns but the matches must still be independent. The engine I wrote uses a classical interpreter backend, optimized well enough to beat TRE and the old PCRE handily. I’d read about the bit parallel algorithms and play with them a bit, but the pattern sets we’d throw at any given BP algorithm would break it, as any given BP algorithm comes with several caveats. The genius of the HyperScan team is that they threw damn near every algorithm at their pattern sets, including some they invented, and divvied up the patterns into subsets that fit the particular algorithms well. Usually getting the tests to pass with one backend is the last act of any regex author—once it ain’t broke, you’re too exhausted to fix it. Contemplating the complexity that comes with a multi-algorithm backend makes me want to cry. But HyperScan did it. So, to put it in perspective with linear algebra, imagine a library that first spent time analyzing given matrices, and then split them into submatrices and operated on them with different algorithms according to the particular patterns detected therein. That’s kind of insane to contemplate in a linear algebra—it’s really not a domain that compares well at all—but it’s how HyperScan works... and that ignores all the crazy ISA tricks it used.
gnufx 7y ago
I keep seeing complaints about MKL not supporting other architectures. Why do people care? For instance, for linear algebra, use BLIS (like AMD do), or OpenBLAS. Intel actually support BLIS (at least to some extent), and I'm not aware of libxsmm being Intel-only, though I haven't checked. MKL-DNN (whatever it's called now) also appeared to be fairly generic when I looked.
- jchw 7y ago
  My complaint with MKL is that Intel is intentionally harming performance by only allowing AVX2 code to run on Intel’s own processors. It does not matter if there are alternatives because having the ability to avoid something doesn’t excuse bad behavior. That said, if you’re relying on a dependency that uses MKL i guess you are probably SOL without hacks.
  - gnufx 7y ago
    I don't know how MKL works, but it's been reported that you can make it run on AMD processors for no good reason. I do know that you're harming performance if you use an implementation for a different micro-architecture rather than one that's correctly tuned. Assuming we're talking GEMM, you don't get the performance just with generic SIMD code. You need to worry about details of caches and instruction latencies, in particular. Consider that you should even avoid AVX512 on some processors. I'm no apologist for Intel, but I suppose people running MKL on AMD would complain that it was slow because it was wrongly tuned. Note that BLIS gets quite close to MKL on Intel anyhow, and the timing variance of a typical HPC application is likely to be greater than the contribution of ~10% from GEMM. I don't know what people do that's MKL-specific, but its major functionality is available in free implementations that will be infinitely faster on POWER and ARM apart from AMD.
    - jchw 7y ago
      Now I can’t claim to be an expert by any means but I feel like you are over complicating this. For one, we know in practice the performance is much better when you force enable AVX2 on AMD with MKL; it was benchmarked. For another, that’s already obvious, because even without micro optimizing we’re still talking about codepaths that are able to better fill the processor and support 256 bit integers. Certainly not everything will always be straightforward but MKL seems like one of the single most obvious libraries for this. Further, Intel has a rich history here. They have been sued and lost over Intel C++ Compiler’s “Cripple AMD” bit. Funny thing is, it's still there. Although now Intel has warnings throughout their pages. Guess removing a strcmp cpuid, “GenuineIntel” was too much work. Let’s be realistic. Intel wants to win the benchmarks. There’s no need to construct an elaborate reality where this is all done out of concern that they might make performance worse on microarchitectures they did not design.

cryptonector 7y ago

If they don't accept AMD optimizations, just fork it.