3 ms·
Ok, prefetching is a thing that could maybe be standardized as a hint, but even then there could be differences in how many bytes apart your prefetches optimall
by dzaima 3y ago
Ok, prefetching is a thing that could maybe be standardized as a hint, but even then there could be differences in how many bytes apart your prefetches optimally need to be, and what range they'll prefetch (might even be a property of the specific CPU at runtime, at which point things get really hairy to do properly).
My point with SIMD is that pretty much all current things do, at worst, map to one instruction (maybe more for things like large loads/stores) on modern 64-bit architectures. But with SIMD the operations supported by various architectures are so diverse, such that taking the intersection (i.e. preserving things mapping mostly-1-to-1 at worst), you'll have an unusable thing (no non-constant shuffles, 8-bit shifts, float rounding, movemask, fault-only-first loads, compress, limited conversions (for 64-bit floats x86 only has i32↔f64, but NEON only has i64↔f64; and narrowing integers takes a sequence of shuffles on pre-AVX-512 x86)). And if you take the union of architectures, a large amount of operations will just suck on any architecture that's not where it was from. Never mind architectures without SIMD.
Different vector sizes is a comparatively trivial problem to tackle (but then you get to the scalable SIMD sets, i.e. SVE & RISC-V V; and even those differ in an important way - SVE vectors are some multiple of 128 bits long, up to 2048, but rvv also guarantees it being a power of two, and can be up to 2^16 bits, at which point indexes in the vectors don't fit in 8-bit integers).
Sure, the optimizing compiler will optimize things, but that's what it does - optimize things, so its actions should be preferable, and manually writing something equivalent to what the compiler did is still very possible, and often not even that weird (with the discussed exception of vectorization).
- janwas 3y agoThis seems overly pessimistic :) We have actually implemented all of those things except first-fault loads in a cross-platform way at reasonable cost in https://github.com/google/highway https://github.com/google/highway. If any of those is the one missing piece that's preventing you from vectorizing, then it's totally worthwhile to emulate it. It may sound horrible to take 4-5 instructions instead of one, but it can still be very worthwhile. FYI for SVE, Arm has requested input and recently clarified that there will only be power of two vector lengths. Happy to discuss.
- dzaima 3y agoYeah, I've seen it! https://google.github.io/highway/en/master/quick_reference.html#speeding-up-code-for-older-x86-platforms https://google.github.io/highway/en/master/quick_reference.h... In many cases, yeah, emulating things can be fine, but in others, very much not so. If not doing trivial (i.e. autovectorizable) things, I'd imagine you'd still have to be at least somewhat careful about what things will get emulated, and see whether the gain from everything around it makes it worth it for your desired architectures, which is still far from a state of things I'd consider worthy of being standardized. Interesting note on SVE vector lengths though. I'll keep that in mind. Haven't gotten to exploring SVE/RVV much; don't have any personal hardware to optimize for to encourage it.
- janwas 3y agoYes, I appreciated your previous list of 'things it would be better to work around if possible'. But "a large amount of operations will just suck" did sound overly pessimistic :) We are talking about a small percentage of the ~220 ops in Highway. I agree it's important to benchmark the performance win in any case. Perhaps even automatically picking the best instruction set, because it's nontrivial to predict. As to SVE, it can help on AWS Graviton3 - but it's still only 2x256 bit vs 4x128 bit, so whether it's a win depends on whether you can/do use some of the newer instructions that were not yet supported in NEON.
- dzaima 3y agoBy "large" I mean compared to the fraction of needs-slow-emulation things present currently in C (which is currently approximately zero, other than using integer/float types unsupported on the target hardware (and maybe multiplication/division), but that's comparatively extremely easy to determine). Maybe my use-case - implementing an array language - is unusually unfit for being done in a generic way, but I just don't see how it (or much of anything outside of very trivial things) would be reasonable to figure out how to do performantly with anything other than either manually using intrinsics, or with custom-written abstractions around such with known worst-case emulation amount (we're doing[0] the latter with a DSL[1] that's basically a basic compile-time code generator; that just leaves unsupported things undefined on any given architecture, and in most occurrences of hitting that I've found ways to write things avoiding expensive emulation). [0]: https://github.com/dzaima/CBQN/tree/master/src/singeli/src https://github.com/dzaima/CBQN/tree/master/src/singeli/src [1]: https://github.com/mlochbaum/Singeli https://github.com/mlochbaum/Singeli