3 ms·

This seems overly pessimistic :) We have actually implemented all of those things except first-fault loads in a cross-platform way at reasonable cost in https:/

by janwas 3y ago

This seems overly pessimistic :) We have actually implemented all of those things except first-fault loads in a cross-platform way at reasonable cost in https://github.com/google/highway https://github.com/google/highway. If any of those is the one missing piece that's preventing you from vectorizing, then it's totally worthwhile to emulate it. It may sound horrible to take 4-5 instructions instead of one, but it can still be very worthwhile. FYI for SVE, Arm has requested input and recently clarified that there will only be power of two vector lengths. Happy to discuss.

dzaima 3y ago
Yeah, I've seen it! https://google.github.io/highway/en/master/quick_reference.html#speeding-up-code-for-older-x86-platforms https://google.github.io/highway/en/master/quick_reference.h... In many cases, yeah, emulating things can be fine, but in others, very much not so. If not doing trivial (i.e. autovectorizable) things, I'd imagine you'd still have to be at least somewhat careful about what things will get emulated, and see whether the gain from everything around it makes it worth it for your desired architectures, which is still far from a state of things I'd consider worthy of being standardized. Interesting note on SVE vector lengths though. I'll keep that in mind. Haven't gotten to exploring SVE/RVV much; don't have any personal hardware to optimize for to encourage it.
- janwas 3y ago
  Yes, I appreciated your previous list of 'things it would be better to work around if possible'. But "a large amount of operations will just suck" did sound overly pessimistic :) We are talking about a small percentage of the ~220 ops in Highway. I agree it's important to benchmark the performance win in any case. Perhaps even automatically picking the best instruction set, because it's nontrivial to predict. As to SVE, it can help on AWS Graviton3 - but it's still only 2x256 bit vs 4x128 bit, so whether it's a win depends on whether you can/do use some of the newer instructions that were not yet supported in NEON.
  - dzaima 3y ago
    By "large" I mean compared to the fraction of needs-slow-emulation things present currently in C (which is currently approximately zero, other than using integer/float types unsupported on the target hardware (and maybe multiplication/division), but that's comparatively extremely easy to determine). Maybe my use-case - implementing an array language - is unusually unfit for being done in a generic way, but I just don't see how it (or much of anything outside of very trivial things) would be reasonable to figure out how to do performantly with anything other than either manually using intrinsics, or with custom-written abstractions around such with known worst-case emulation amount (we're doing[0] the latter with a DSL[1] that's basically a basic compile-time code generator; that just leaves unsupported things undefined on any given architecture, and in most occurrences of hitting that I've found ways to write things avoiding expensive emulation). [0]: https://github.com/dzaima/CBQN/tree/master/src/singeli/src https://github.com/dzaima/CBQN/tree/master/src/singeli/src [1]: https://github.com/mlochbaum/Singeli https://github.com/mlochbaum/Singeli
    - janwas 3y ago
      Cool, thanks for sharing, wasn't aware of those. It is natural that C has about zero emulated operators - today's CPUs are compatible with ones likely built with that use-case in mind. That is a low bar for comparison, especially given a large speedup from SIMD :) I agree a custom abstraction is currently the most reasonable approach for SIMD. We're basically doing the same thing, just differing a bit in how much emulation is acceptable, and the language for compile-time code generation (it's nice how compactly you can express things in Singeli).

by janwas 3y ago

dzaima 3y ago

Yeah, I've seen it! https://google.github.io/highway/en/master/quick_reference.html#speeding-up-code-for-older-x86-platforms https://google.github.io/highway/en/master/quick_reference.h... In many cases, yeah, emulating things can be fine, but in others, very much not so. If not doing trivial (i.e. autovectorizable) things, I'd imagine you'd still have to be at least somewhat careful about what things will get emulated, and see whether the gain from everything around it makes it worth it for your desired architectures, which is still far from a state of things I'd consider worthy of being standardized. Interesting note on SVE vector lengths though. I'll keep that in mind. Haven't gotten to exploring SVE/RVV much; don't have any personal hardware to optimize for to encourage it.

janwas 3y ago
Yes, I appreciated your previous list of 'things it would be better to work around if possible'. But "a large amount of operations will just suck" did sound overly pessimistic :) We are talking about a small percentage of the ~220 ops in Highway. I agree it's important to benchmark the performance win in any case. Perhaps even automatically picking the best instruction set, because it's nontrivial to predict. As to SVE, it can help on AWS Graviton3 - but it's still only 2x256 bit vs 4x128 bit, so whether it's a win depends on whether you can/do use some of the newer instructions that were not yet supported in NEON.
- dzaima 3y ago
  By "large" I mean compared to the fraction of needs-slow-emulation things present currently in C (which is currently approximately zero, other than using integer/float types unsupported on the target hardware (and maybe multiplication/division), but that's comparatively extremely easy to determine). Maybe my use-case - implementing an array language - is unusually unfit for being done in a generic way, but I just don't see how it (or much of anything outside of very trivial things) would be reasonable to figure out how to do performantly with anything other than either manually using intrinsics, or with custom-written abstractions around such with known worst-case emulation amount (we're doing[0] the latter with a DSL[1] that's basically a basic compile-time code generator; that just leaves unsupported things undefined on any given architecture, and in most occurrences of hitting that I've found ways to write things avoiding expensive emulation). [0]: https://github.com/dzaima/CBQN/tree/master/src/singeli/src https://github.com/dzaima/CBQN/tree/master/src/singeli/src [1]: https://github.com/mlochbaum/Singeli https://github.com/mlochbaum/Singeli
  - janwas 3y ago
    Cool, thanks for sharing, wasn't aware of those. It is natural that C has about zero emulated operators - today's CPUs are compatible with ones likely built with that use-case in mind. That is a low bar for comparison, especially given a large speedup from SIMD :) I agree a custom abstraction is currently the most reasonable approach for SIMD. We're basically doing the same thing, just differing a bit in how much emulation is acceptable, and the language for compile-time code generation (it's nice how compactly you can express things in Singeli).