3 ms·
Floating point might be expensive on die room, right now I think it's mostly integer, bit manipulation, and compare-and-swap, which is already a lot and may hel
by touisteur 3y ago
Floating point might be expensive on die room, right now I think it's mostly integer, bit manipulation, and compare-and-swap, which is already a lot and may help with quantized network. Curious whether someone implemented FP32/16 or some non-IEEE754 floats in there.
My worry with all these archs is the ease of debug, tracing and grokking whatever the hell is happening. Disaggregation of compute will make programs very hard to write and understand...