4 ms·

Surprised its not meaningfully slower. Vulkan and ROCm paths are missing a few optimized versions of the quants they're using.

• GrayShade 15d ago
  I see around 1 tk/s after a while, so perhaps it is.