3 ms·
The issue with runtime instruction set detection and dynamic dispatch is that it needs to be at a rather coarse level to be beneficial. Take a simple 4-wide do
by exDM69 3y ago
The issue with runtime instruction set detection and dynamic dispatch is that it needs to be at a rather coarse level to be beneficial.
Take a simple 4-wide dot product for example, on x86_64 you'd have 3-4 different implementations (SSE2, SSE3 w/ hadd, SSE4.2 w/ dpps). But the function itself is just a few clock cycles, and calling it via function pointer will eliminate any gains and you might as well compute it with a scalar loop at that point.
This is further compounded by inhibiting compiler optimizations. You can't use the dot product function in higher level code expecting it to be inlined and further optimized (which is really the key to performance) if it's behind a dynamic dispatch.
A sufficiently smart compiler could maybe propagate the dynamic dispatch above, so that all the dot products get inlined but all code using dot product would get emitted multiple times with different dot product implementations, with the dynamic dispatch only at the top level. This has a slight risk of combinatorial explosion, but there really aren't that many combinations of supported ISAs in real hardware out there.
Another option you can use without any special compiler support is to take all your performance sensitive parts and pack them into a shared object/dll, compile multiple versions with different compiler options and choose the correct dll at runtime. Or even build the entire executable a few times and have some kind of launcher pick the correct one.
- kps 3y agoWhat you say about dynamic dispatch also applies to regular function calls, which is why I'm disappointed that Zig provides no visible distinction at the call site between direct and indirect calls (as K&R C did but ANSI C made optional). I understand the desire for magic-indirection ergonomics; I just don't think the tradeoffs work out the same for code vs data.
- hansvm 3y agoIf the function is comptime-known then you'll get a direct call, and you can mark it near the call site as being required to be comptime-known (e.g., by marking a function argument to another function with the "comptime" tag). To make it super explicit you can make a no-op function like `fn cknown(comptime f: anytype) @TypeOf(f) {return f;}` and then replace stuff like `foo(bar)` with `cknown(foo)(bar)`. It's a tiny bit harder to force an indirect call if that's desired for some reason; I think you'd need to write a slightly longer never inlined helper function to strip the constness from the pointer. It's doable though, just not directly provided by the language.
- kps 3y agoWhat I had in mind is the programmers' view of the code: in K&R C, every function call was visibly either direct or indirect, `f()` or `(*f)()`. My main concern is not actually performance, but comprehensibility: an indirect call is a conditional branch, where the condition can be arbitrarily far away in space and time and is not statically determinable, so it should not be invisible. I appreciate that this doesn't fit Zig's call syntax, which is unlikely to change, so the long-term best case is probably some LSP marking based on the callee type.
- nyberg 3y agohttps://github.com/ziglang/zig/issues/6966 https://github.com/ziglang/zig/issues/6966 was discussed