4 ms·
You want to make multiplication cheap, it's not just about compression
by 0-_-0 6mo ago
You want to make multiplication cheap, it's not just about compression
- mysterydip 6mo agoWouldn’t multiplication just be an 8 bit lookup table? a*b is just lut[a<<4+b]
- 0-_-0 6mo agoA 256 element lookup table is much bigger than a simple multiplier
- kevmo314 6mo agoMultiplication at this resolution is already implemented via lookup tables.
- ineedasername 6mo agoFor FP4, yes... sometimes... it depends. But newer Nvidia architecture eg Blackwell w/ NVFP4 does not, they perform micro block scaling in the core. For older architectures, low quants like FP4 are also often not done native, and instead inflated back to BF16, eg with BnB.
- londons_explore 6mo agoSpecifically, you want to choose 16 values, all of which you can multiply an activation value by using circuitry which is as small as possible.