Great writeup, especially liked the line by line explanation :) We actually did manage to get dynamic dispatch working in Compiler explorer with Matt's help:

Great writeup, especially liked the line by line explanation :) We actually did manage to get dynamic dispatch working in Compiler explorer with Matt's help: https://gcc.godbolt.org/z/KM3ben7ET https://gcc.godbolt.org/z/KM3ben7ET Maybe interesting to also refer to Gemma.cpp's version: https://github.com/google/gemma.cpp/blob/dev/ops/ops-inl.h#L1348 https://github.com/google/gemma.cpp/blob/dev/ops/ops-inl.h#L... We recently added a FastExpMinusOrZero, only valid for inputs <= 0, which is true for softmax.