4 ms·

Cache is a microarchitectural thing. There's no "controlling" it without explicitly knowing the properties of the caches of the running processor (size, latency

by dzaima 3y ago

Cache is a microarchitectural thing. There's no "controlling" it without explicitly knowing the properties of the caches of the running processor (size, latency, associativity, replacement policy, and whatever other quirks the hardware designers decide to add) and building your code around it, at which point nothing will be able to do it in a "standard" way (unless your standard hard-codes all the past, present, and future architectures, which, good luck). SIMD isn't really feasible to fully do in a standard way either. Without specific knowledge of the target, you can easily write SIMD-looking code that ends up slower than a scalar equivalent (e.g. if you do any non-constant shuffle on x86-64 pre-SSSE3 (i.e. the default), you'll get utterly awful results. On ARM NEON, a non-constant i32x4 shuffle is gonna be quite awful too). Threading - <threads.h> is a standard header, no? (granted, it doesn't appear to be used much, but it does exist) And, sure, every statement/expression/operator of yours doesn't end up always literally mapping to a single assembly instruction, but there's pretty much always a way it could, and quite often it does (and in the cases where it doesn't, it (usually) maps to something better than what you wrote). The main thing that doesn't is macros, but those are extremely trivially expandable.

kaba0 3y ago
You can prefetch memory though (but yeah, it indeed is very finicky to get right. So is any sort of decision compilers do, so there is that). Off topic, but the OCaml GC does use prefetch instructions for I believe a 3-4x speedup in an already mature project, so there is that. SIMD is still the panacea of compiler optimizations, you won’t get significant performance improvements from trivial peephole optimizations like replacing * 2 with << 1, and the supposedly low-level C has no way of controlling that. Sure, vector sizes are different between processors, but so is many other things we already generalize/assume. But an optimizing compiler will change up your loops all around, inline your functions, etc, so just because it could correspond to some naive assembly, it won’t unless you compile it with some toy.
- dzaima 3y ago
  Ok, prefetching is a thing that could maybe be standardized as a hint, but even then there could be differences in how many bytes apart your prefetches optimally need to be, and what range they'll prefetch (might even be a property of the specific CPU at runtime, at which point things get really hairy to do properly). My point with SIMD is that pretty much all current things do, at worst, map to one instruction (maybe more for things like large loads/stores) on modern 64-bit architectures. But with SIMD the operations supported by various architectures are so diverse, such that taking the intersection (i.e. preserving things mapping mostly-1-to-1 at worst), you'll have an unusable thing (no non-constant shuffles, 8-bit shifts, float rounding, movemask, fault-only-first loads, compress, limited conversions (for 64-bit floats x86 only has i32↔f64, but NEON only has i64↔f64; and narrowing integers takes a sequence of shuffles on pre-AVX-512 x86)). And if you take the union of architectures, a large amount of operations will just suck on any architecture that's not where it was from. Never mind architectures without SIMD. Different vector sizes is a comparatively trivial problem to tackle (but then you get to the scalable SIMD sets, i.e. SVE & RISC-V V; and even those differ in an important way - SVE vectors are some multiple of 128 bits long, up to 2048, but rvv also guarantees it being a power of two, and can be up to 2^16 bits, at which point indexes in the vectors don't fit in 8-bit integers). Sure, the optimizing compiler will optimize things, but that's what it does - optimize things, so its actions should be preferable, and manually writing something equivalent to what the compiler did is still very possible, and often not even that weird (with the discussed exception of vectorization).
  - janwas 3y ago
    This seems overly pessimistic :) We have actually implemented all of those things except first-fault loads in a cross-platform way at reasonable cost in https://github.com/google/highway https://github.com/google/highway. If any of those is the one missing piece that's preventing you from vectorizing, then it's totally worthwhile to emulate it. It may sound horrible to take 4-5 instructions instead of one, but it can still be very worthwhile. FYI for SVE, Arm has requested input and recently clarified that there will only be power of two vector lengths. Happy to discuss.
    - dzaima 3y ago

by dzaima 3y ago

kaba0 3y ago

You can prefetch memory though (but yeah, it indeed is very finicky to get right. So is any sort of decision compilers do, so there is that). Off topic, but the OCaml GC does use prefetch instructions for I believe a 3-4x speedup in an already mature project, so there is that. SIMD is still the panacea of compiler optimizations, you won’t get significant performance improvements from trivial peephole optimizations like replacing * 2 with << 1, and the supposedly low-level C has no way of controlling that. Sure, vector sizes are different between processors, but so is many other things we already generalize/assume. But an optimizing compiler will change up your loops all around, inline your functions, etc, so just because it could correspond to some naive assembly, it won’t unless you compile it with some toy.

dzaima 3y ago
Ok, prefetching is a thing that could maybe be standardized as a hint, but even then there could be differences in how many bytes apart your prefetches optimally need to be, and what range they'll prefetch (might even be a property of the specific CPU at runtime, at which point things get really hairy to do properly). My point with SIMD is that pretty much all current things do, at worst, map to one instruction (maybe more for things like large loads/stores) on modern 64-bit architectures. But with SIMD the operations supported by various architectures are so diverse, such that taking the intersection (i.e. preserving things mapping mostly-1-to-1 at worst), you'll have an unusable thing (no non-constant shuffles, 8-bit shifts, float rounding, movemask, fault-only-first loads, compress, limited conversions (for 64-bit floats x86 only has i32↔f64, but NEON only has i64↔f64; and narrowing integers takes a sequence of shuffles on pre-AVX-512 x86)). And if you take the union of architectures, a large amount of operations will just suck on any architecture that's not where it was from. Never mind architectures without SIMD. Different vector sizes is a comparatively trivial problem to tackle (but then you get to the scalable SIMD sets, i.e. SVE & RISC-V V; and even those differ in an important way - SVE vectors are some multiple of 128 bits long, up to 2048, but rvv also guarantees it being a power of two, and can be up to 2^16 bits, at which point indexes in the vectors don't fit in 8-bit integers). Sure, the optimizing compiler will optimize things, but that's what it does - optimize things, so its actions should be preferable, and manually writing something equivalent to what the compiler did is still very possible, and often not even that weird (with the discussed exception of vectorization).
- janwas 3y ago
  This seems overly pessimistic :) We have actually implemented all of those things except first-fault loads in a cross-platform way at reasonable cost in https://github.com/google/highway https://github.com/google/highway. If any of those is the one missing piece that's preventing you from vectorizing, then it's totally worthwhile to emulate it. It may sound horrible to take 4-5 instructions instead of one, but it can still be very worthwhile. FYI for SVE, Arm has requested input and recently clarified that there will only be power of two vector lengths. Happy to discuss.
  - dzaima 3y ago