3 ms·
EDIT: I was completely wrong, I have mostly worked with GGUF and related quantizations that are LUTs, thank you for correcting me.
by jmalicki 4mo ago
EDIT: I was completely wrong, I have mostly worked with GGUF and related quantizations that are LUTs, thank you for correcting me.
- mathisfun123 4mo ago> The quantized integer kernels aren't running true integer multiplication, the quantization is it's own thing, they're basically enums not integers ELI-a-GPU-compiler-engineer-working-at-a-major-vendor (because I am). Ie I can pull up the design docs for our ALUs and literally see that you're wrong.