4 ms·
What does it mean for future hardware if it's not using floating point matrix multiplication units?
by the8472 3y ago
What does it mean for future hardware if it's not using floating point matrix multiplication units?
- kromem 3y agoThis opens the door to very exciting hardware shifts, like to optical computing, where there's already been over a decade of research on ternary optical computing and other parallel research at using optical computing for more efficient neural networks. If this really holds up, it likely means we'll be moving to new dedicated hardware for AI compute much faster than when it was FP.
- cyanydeez 3y agohttps://stackoverflow.com/questions/45373679/why-is-it-faster-to-perform-float-by-float-matrix-multiplication-compared-to-int https://stackoverflow.com/questions/45373679/why-is-it-faste...
- gpderetta 3y agoAs per answer, the reason float is faster than in is because a) hardware companies provide float ALUs than integer ALUs and b) float FMA is a thing, while integer FMA isn't. Both are because currently most HPC-like loads use floats instead of integers, not because of intrinsic hardware reasons.
- KeplerBoy 3y agoIf it's desired integer performance could far exceed float performance, since ALUs need less die area than FPUs. If this paper holds, I'd expect that's where custom accelerators will be heading.
- gpderetta 3y agoOh, I agree, I'm just saying that there is no reason in principle for floats performance to be better than integer. edit: also this might be implementable purely using bitwise vector operations. Would need to check the throughput of those.
- KeplerBoy 3y agoExpect Nvidia to advertise with their TOPS numbers instead of their FLOPS.
- rfoo 3y agoAlready happened years ago. They advertised TOPS for int8/int4 [0], and with 50% sparsity [1]. [0] low-bit CNNs worked pretty well actually. [1] Totally useless marketing snake oil.