3 ms·
Really for me just the mantissa would be fine; no need for exponent bc so much of what I worked on is between 0..1 There was an interesting paper from the Alle
by gumby 4y ago
Really for me just the mantissa would be fine; no need for exponent bc so much of what I worked on is between 0..1
There was an interesting paper from the Allen Institute a few years ago describing a system with 1 bit weights that worked pretty well! Since I read it I've been musing on trying that, though it seems unlikely I will be able to any time soon.
- deleted 4y ago[deleted]
- thfuran 4y agoIf you just have a mantissa, aren't you doing fixed point math?
- gumby 4y agoYes, just looking for a weight in the range 0 <= x < 1. But I want to do large numbers of calculations using the GPU, else I'd use the SIMD int instructions (AVX)
- snickerbockers 4y agoJust do fixed point bruh.
- gumby 4y agoit is, but doesn't give me the hardware affordance I want: https://news.ycombinator.com/item?id=34405604 https://news.ycombinator.com/item?id=34405604